2024-02-26T10:14:41.248 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-26T10:14:41.248 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562 --name vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi --owner scheduled_vshankar@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsstress}} --verbose -- /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/orig.config.yaml 2024-02-26T10:14:41.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T10:14:41.376 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562 branch: wip-vshankar-testing-20240226.064629 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsstress}} email: vshankar@redhat.com first_in_suite: false job_id: '7573562' kernel: client: branch: testing flavor: default kdb: true sha1: 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240226.064629 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240226.064629 sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1879 sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 sleep_before_teardown: 0 subset: 788/1024 suite: fs suite_branch: wip-vshankar-testing-20240226.064629 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMWnHpBAj9JGPCdzouGkLMVA2TWfW5K4EzxQ93ol71TVMNevNgAn5p62vIxtrhOkOkbdZR2C9fdH97znyTefiQQ= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOKLjzfObQmFIIiBw3Da5e09/RMkGdDH2omX9S+dysBhRwk1YZ3KS+AeXmpi9PUvrlWxqeVFv8SPXmQl5UH/oMs= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM2zRjhcwm9/pB7qnx5uaBvrQYhj7wZUT1vvlGVnFTvZrYnHymXYVUMMMhKUJOJ1dI4ssCDg8PwC34vvqF1gBKU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-26_10:07:12 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-26T10:14:41.376 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa; will attempt to use it 2024-02-26T10:14:41.377 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks 2024-02-26T10:14:41.377 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-26T10:14:41.378 INFO:teuthology.task.internal:Checking packages... 2024-02-26T10:14:41.399 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6085b91fefeb74bf4310344a87d81d3217ef7893' 2024-02-26T10:14:41.399 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-26T10:14:41.399 INFO:teuthology.packaging:ref: None 2024-02-26T10:14:41.399 INFO:teuthology.packaging:tag: None 2024-02-26T10:14:41.399 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240226.064629 2024-02-26T10:14:41.399 INFO:teuthology.packaging:sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:14:41.400 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240226.064629 2024-02-26T10:14:41.694 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1686.g6085b91f 2024-02-26T10:14:41.695 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-26T10:14:41.702 INFO:teuthology.task.internal:no buildpackages task found 2024-02-26T10:14:41.702 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-26T10:14:41.708 INFO:teuthology.task.internal:Saving configuration 2024-02-26T10:14:41.727 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-26T10:14:41.733 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T10:14:41.754 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:16.757995', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMWnHpBAj9JGPCdzouGkLMVA2TWfW5K4EzxQ93ol71TVMNevNgAn5p62vIxtrhOkOkbdZR2C9fdH97znyTefiQQ='} 2024-02-26T10:14:41.779 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:16.760825', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOKLjzfObQmFIIiBw3Da5e09/RMkGdDH2omX9S+dysBhRwk1YZ3KS+AeXmpi9PUvrlWxqeVFv8SPXmQl5UH/oMs='} 2024-02-26T10:14:41.803 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:16.759496', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM2zRjhcwm9/pB7qnx5uaBvrQYhj7wZUT1vvlGVnFTvZrYnHymXYVUMMMhKUJOJ1dI4ssCDg8PwC34vvqF1gBKU='} 2024-02-26T10:14:41.803 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-26T10:14:41.809 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-26T10:14:41.809 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-26T10:14:41.809 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-26T10:14:41.809 INFO:teuthology.run_tasks:Running task console_log... 2024-02-26T10:14:41.899 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f88e9d1f160>, signals=[15]) 2024-02-26T10:14:41.900 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-26T10:14:41.906 INFO:teuthology.task.internal:Opening connections... 2024-02-26T10:14:41.906 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T10:14:41.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:14:42.015 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-02-26T10:14:42.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:14:42.101 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-02-26T10:14:42.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:14:42.189 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-26T10:14:42.196 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-02-26T10:14:42.214 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-02-26T10:14:42.214 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-02-26T10:14:42.272 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-02-26T10:14:42.272 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2024-02-26T10:14:42.272 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-02-26T10:14:42.272 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-02-26T10:14:42.272 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2024-02-26T10:14:42.272 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2024-02-26T10:14:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T10:14:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-02-26T10:14:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2024-02-26T10:14:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T10:14:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-02-26T10:14:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T10:14:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T10:14:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T10:14:42.274 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-02-26T10:14:42.315 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-02-26T10:14:42.332 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-02-26T10:14:42.332 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T10:14:42.390 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-02-26T10:14:42.391 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T10:14:42.391 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T10:14:42.391 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T10:14:42.391 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-02-26T10:14:42.415 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-02-26T10:14:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-02-26T10:14:42.432 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-02-26T10:14:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2024-02-26T10:14:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T10:14:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T10:14:42.489 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-02-26T10:14:42.523 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-26T10:14:42.533 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-26T10:14:42.538 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-26T10:14:42.538 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-02-26T10:14:42.540 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-02-26T10:14:42.545 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-02-26T10:14:42.561 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-26T10:14:42.566 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-26T10:14:42.566 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-02-26T10:14:42.597 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-02-26T10:14:42.601 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-02-26T10:14:42.715 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-26T10:14:42.789 INFO:teuthology.run_tasks:Running task kernel... 2024-02-26T10:14:42.803 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '7c0fbba6c508351b48a34af7de7b074eb29ad0f9'} 2024-02-26T10:14:42.803 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-02-26T10:14:42.803 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-02-26T10:14:42.803 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T10:14:42.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:14:42.820 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-02-26T10:14:43.014 DEBUG:teuthology.task.kernel:sha1 for client.0 is 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T10:14:43.014 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "7c0fbba6c508351b48a34af7de7b074eb29ad0f9"... 2024-02-26T10:14:43.014 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-02-26T10:14:43.031 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-361.el9.x86_64 2024-02-26T10:14:43.032 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T10:14:43.032 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-26T10:14:43.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum upgrade -y linux-firmware 2024-02-26T10:14:45.322 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-02-26T10:14:47.189 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.5 MB 00:00 2024-02-26T10:14:48.330 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-02-26T10:14:50.122 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-02-26T10:14:54.898 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.1 MB 00:01 2024-02-26T10:14:56.442 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 15 kB 00:00 2024-02-26T10:14:58.595 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:01 2024-02-26T10:15:03.684 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-02-26T10:15:06.053 INFO:teuthology.orchestra.run.smithi060.stderr:Modular dependency problems: 2024-02-26T10:15:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-02-26T10:15:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 1: conflicting requests 2024-02-26T10:15:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 2: conflicting requests 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 3: conflicting requests 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 4: conflicting requests 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 5: conflicting requests 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 6: conflicting requests 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 7: conflicting requests 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 8: conflicting requests 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 9: conflicting requests 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 10: conflicting requests 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 11: conflicting requests 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 12: conflicting requests 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 13: conflicting requests 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 14: conflicting requests 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 15: conflicting requests 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 16: conflicting requests 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 17: conflicting requests 2024-02-26T10:15:06.055 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 18: conflicting requests 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 19: conflicting requests 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 20: conflicting requests 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 21: conflicting requests 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 22: conflicting requests 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 23: conflicting requests 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.056 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 24: conflicting requests 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 25: conflicting requests 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 26: conflicting requests 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 27: conflicting requests 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 28: conflicting requests 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 29: conflicting requests 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 30: conflicting requests 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.057 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 31: conflicting requests 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 32: conflicting requests 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 33: conflicting requests 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 34: conflicting requests 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 35: conflicting requests 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 36: conflicting requests 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.058 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 38: conflicting requests 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.059 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.060 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.061 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T10:15:06.610 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-02-26T10:15:06.622 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-02-26T10:15:06.622 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-02-26T10:15:06.622 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-02-26T10:15:06.622 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-02-26T10:15:06.622 INFO:teuthology.orchestra.run.smithi060.stdout: linux-firmware noarch 20240111-121.gitb3132c18.el8 CentOS-BaseOS 354 M 2024-02-26T10:15:06.622 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:15:06.623 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-02-26T10:15:06.623 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-02-26T10:15:06.623 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 1 Package 2024-02-26T10:15:06.623 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:15:06.623 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 354 M 2024-02-26T10:15:06.623 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-02-26T10:15:29.192 INFO:teuthology.orchestra.run.smithi060.stdout:linux-firmware-20240111-121.gitb3132c18.el8.noa 16 MB/s | 354 MB 00:22 2024-02-26T10:15:29.193 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-02-26T10:15:29.193 INFO:teuthology.orchestra.run.smithi060.stdout:Total 16 MB/s | 354 MB 00:22 2024-02-26T10:15:29.194 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-02-26T10:15:29.400 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-02-26T10:15:29.400 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-02-26T10:15:31.002 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-02-26T10:15:31.002 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-02-26T10:15:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-02-26T10:15:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T10:15:53.858 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T10:16:01.500 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T10:16:01.501 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T10:16:02.063 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T10:16:02.063 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:16:02.063 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-02-26T10:16:02.063 INFO:teuthology.orchestra.run.smithi060.stdout: linux-firmware-20240111-121.gitb3132c18.el8.noarch 2024-02-26T10:16:02.063 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:16:02.063 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-02-26T10:16:02.361 INFO:teuthology.task.kernel:Downloading kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.0... 2024-02-26T10:16:02.362 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T10:16:06.379 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ 2024-02-26T10:16:06.380 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ --input-file=- 2024-02-26T10:16:06.383 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi060.front.sepia.ceph.com to complete... 2024-02-26T10:19:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26 10:19:57 URL:https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm [335918853/335918853] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-26T10:19:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:FINISHED --2024-02-26 10:19:57-- 2024-02-26T10:19:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:Total wall clock time: 3m 51s 2024-02-26T10:19:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:Downloaded: 1 files, 320M in 3m 48s (1.41 MB/s) 2024-02-26T10:19:57.891 INFO:teuthology.task.kernel:Installing kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.0... 2024-02-26T10:19:57.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-26T10:19:59.923 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying... ######################################## 2024-02-26T10:20:00.014 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing... ######################################## 2024-02-26T10:20:00.099 INFO:teuthology.orchestra.run.smithi060.stdout:Updating / installing... 2024-02-26T10:20:07.928 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-6.8.0_rc4_g7c0fbba6c508-1 ######################################## 2024-02-26T10:20:21.875 INFO:teuthology.orchestra.run.smithi060.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-26T10:20:45.053 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-26T10:20:45.053 DEBUG:teuthology.orchestra.run.smithi060:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-26T10:20:45.120 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:20:45.120 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/System.map 2024-02-26T10:20:45.120 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/config 2024-02-26T10:20:45.120 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel 2024-02-26T10:20:45.120 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-26T10:20:45.121 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm 2024-02-26T10:20:45.122 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ansi_cprng.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/anubis.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/arc4.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_memcpy.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_pq.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_tx.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_xor.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/raid6test.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authenc.ko 2024-02-26T10:20:45.123 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authencesn.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blake2b_generic.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_common.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_generic.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/camellia_generic.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast5_generic.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast6_generic.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast_common.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ccm.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cmac.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crc32_generic.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_engine.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_user.ko 2024-02-26T10:20:45.124 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cts.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/des_generic.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/dh_generic.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/echainiv.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/essiv.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/fcrypt.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/khazad.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4hc.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/md4.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/michael_mic.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcbc.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcrypt.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/rmd160.ko 2024-02-26T10:20:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/seed.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/serpent_generic.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tcrypt.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tea.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_common.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_generic.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/vmac.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/wp512.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xcbc.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xor.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xxhash_generic.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-26T10:20:45.126 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_pad.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/custom_method.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/ec_sys.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/platform_profile.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbs.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbshc.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/video.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/acard-ahci.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ahci_platform.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ata_generic.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/libahci_platform.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_acpi.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ali.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_amd.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_artop.ko 2024-02-26T10:20:45.127 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atiixp.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atp867x.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd640.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd64x.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cypress.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_efar.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt366.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt37x.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it8213.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it821x.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_jmicron.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_marvell.ko 2024-02-26T10:20:45.128 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_mpiix.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_netcell.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ninja32.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87410.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87415.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_oldpiix.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_opti.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_optidma.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pcmcia.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_piccolo.ko 2024-02-26T10:20:45.129 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_rdc.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sch.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_serverworks.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sil680.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sis.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sl82c105.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_triflex.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_via.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pdc_adma.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_inic162x.ko 2024-02-26T10:20:45.130 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_mv.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_nv.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_promise.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_qstor.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil24.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sis.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_svw.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sx4.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_uli.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_via.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_vsc.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay/ks0108.ko 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base 2024-02-26T10:20:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma/bcma.ko 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe/aoe.ko 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/brd.ko 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd/drbd.ko 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/floppy.ko 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/loop.ko 2024-02-26T10:20:45.132 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/nbd.ko 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk/null_blk.ko 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/pktcdvd.ko 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/rbd.ko 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/virtio_blk.ko 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkfront.ko 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram/zram.ko 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char 2024-02-26T10:20:45.133 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hangcheck-timer.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/via-rng.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/lp.ko 2024-02-26T10:20:45.134 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave/mwave.ko 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ppdev.ko 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tlclk.ko 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm.ko 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-26T10:20:45.135 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/uv_mmtimer.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/virtio_console.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp.ko 2024-02-26T10:20:45.136 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-aes.ko 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-sha.ko 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca/dca.ko 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-26T10:20:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/amd64_edac.ko 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/e752x_edac.ko 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/edac_mce_amd.ko 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3000_edac.ko 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3200_edac.ko 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5100_edac.ko 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5400_edac.ko 2024-02-26T10:20:45.138 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7300_edac.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7core_edac.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i82975x_edac.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/ie31200_edac.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/sb_edac.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/x38_edac.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/edd.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-ich.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-appleir.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-aureal.ko 2024-02-26T10:20:45.139 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-axff.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-dr.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elecom.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elo.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-emsff.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gaff.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gt683r.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gyration.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-26T10:20:45.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtekff.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-hyperv.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-icade.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-keytouch.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-kye.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lcpower.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-led.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lenovo.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-multitouch.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-ortek.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-penmount.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-petalynx.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-picolcd.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-pl.ko 2024-02-26T10:20:45.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-plantronics.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-primax.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-rmi.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-common.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-26T10:20:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-saitek.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-samsung.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sjoy.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sony.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-speedlink.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-steelseries.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sunplus.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tivo.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tmff.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-topseed.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-twinhan.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-uclogic.ko 2024-02-26T10:20:45.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-waltop.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-wiimote.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-xinmo.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zpff.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zydacron.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/uhid.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/wacom.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_balloon.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_utils.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_vmbus.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru3.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-26T10:20:45.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7414.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7418.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adc128d818.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1025.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1026.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1029.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1031.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm9240.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ads7828.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7410.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7411.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7462.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7470.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7475.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7x10.ko 2024-02-26T10:20:45.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/amc6821.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/applesmc.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asb100.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asc7621.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/atxp1.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/coretemp.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dme1737.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds1621.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds620.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc1403.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc6w201.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71805f.ko 2024-02-26T10:20:45.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71882fg.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f75375s.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fam15h_power.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fschmd.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g760a.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g762.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl518sm.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl520sm.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5500_temp.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5k_amb.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmaem.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmpex.ko 2024-02-26T10:20:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina209.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina2xx.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/it87.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k10temp.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k8temp.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lineage-pem.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm63.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm73.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm75.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm77.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm78.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm80.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm83.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm85.ko 2024-02-26T10:20:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm87.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm90.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm92.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm93.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95234.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95241.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95245.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc2945.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4151.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4215.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4222.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4245.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4260.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4261.ko 2024-02-26T10:20:45.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max16065.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1619.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1668.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max197.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6639.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6650.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6697.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/mcp3021.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6683.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775-core.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct7802.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87360.ko 2024-02-26T10:20:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87427.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pcf8591.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-26T10:20:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/powr1220.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5627.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5636.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht15.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht21.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/shtc1.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sis5595.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47b397.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m1.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m192.ko 2024-02-26T10:20:45.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/thmc50.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp102.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp103.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp401.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp421.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via-cputemp.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via686a.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt1211.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt8231.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627ehf.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627hf.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83781d.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83791d.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83792d.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83793.ko 2024-02-26T10:20:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83795.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l785ts.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l786ng.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-26T10:20:45.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-26T10:20:45.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-dev.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-mux.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-smbus.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-stub.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_core.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-26T10:20:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-26T10:20:45.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/gameport.ko 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/joydev.ko 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse 2024-02-26T10:20:45.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/appletouch.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/bcm5974.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/cyapatp.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/sermouse.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/altera_ps2.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/arc_ps2.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-26T10:20:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/serio_raw.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/sparse-keymap.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-blinkm.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lm3530.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lp3944.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-ss4200.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-26T10:20:45.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache/bcache.ko 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-bio-prison.ko 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache-smq.ko 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache.ko 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-crypt.ko 2024-02-26T10:20:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-delay.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-flakey.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-log-userspace.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-multipath.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-queue-length.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-raid.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-round-robin.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-service-time.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-switch.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-thin-pool.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-verity.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid0.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid1.ko 2024-02-26T10:20:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid10.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid456.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptbase.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptctl.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptfc.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptlan.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptsas.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptscsih.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptspi.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_ich.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_sch.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/mfd-core.ko 2024-02-26T10:20:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/sm501.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/viperboard.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/vx855.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/wl1273-core.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds9802als.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds990x.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/bh1770glc.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710/cb710.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/at24.ko 2024-02-26T10:20:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/max6875.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/enclosure.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/hpilo.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29003.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29020.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-me.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-txe.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei.ko 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru 2024-02-26T10:20:45.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_7xx1.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_core.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tsl2550.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_balloon.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding/bonding.ko 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53 2024-02-26T10:20:45.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dummy.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/eql.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390 2024-02-26T10:20:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-26T10:20:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-26T10:20:45.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-26T10:20:45.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec 2024-02-26T10:20:45.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dnet.ko 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex 2024-02-26T10:20:45.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ethoc.ko 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/fealnx.ko 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e 2024-02-26T10:20:45.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/jme.ko 2024-02-26T10:20:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-26T10:20:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines 2024-02-26T10:20:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-26T10:20:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro 2024-02-26T10:20:45.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-26T10:20:45.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/geneve.ko 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ifb.ko 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-26T10:20:45.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvlan.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvtap.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mii.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/net_failover.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/netconsole.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/nlmon.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy 2024-02-26T10:20:45.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/amd.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm87xx.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/broadcom.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/cicada.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/davicom.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/dp83640.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/et1011c.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/icplus.ko 2024-02-26T10:20:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/lxt.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/marvell.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/micrel.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/national.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/phylink.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/qsemi.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/realtek.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/smsc.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/ste10Xp.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/vitesse.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_async.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-26T10:20:45.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppoe.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppox.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pptp.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slhc.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slip.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/sungem_phy.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tap.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-26T10:20:45.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_random.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tun.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/veth.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/virtio_net.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan/vxlan.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netfront.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb/ntb.ko 2024-02-26T10:20:45.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme-core.ko 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme.ko 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport.ko 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_cs.ko 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_pc.ko 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_serial.ko 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/xen-pcifront.ko 2024-02-26T10:20:45.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/i82092.ko 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/pd6729.ko 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acerhdf.ko 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-26T10:20:45.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/hdaps.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel_ips.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-26T10:20:45.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi.ko 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-26T10:20:45.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-26T10:20:45.190 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps_parport.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/pps_core.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp_kvm.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-26T10:20:45.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-em3027.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-max6900.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-26T10:20:45.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-x1205.ko 2024-02-26T10:20:45.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-9xxx.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-sas.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-xxxx.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/BusLogic.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/a100u2w.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/advansys.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/am53c974.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-26T10:20:45.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/atp870u.ko 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa/bfa.ko 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ch.ko 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-26T10:20:45.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dc395x.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dmx3191d.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esp_scsi.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-26T10:20:45.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic/fnic.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hpsa.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hptiop.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hv_storvsc.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/initio.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ipr.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ips.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci/isci.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc/libfc.ko 2024-02-26T10:20:45.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas/libsas.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvumi.ko 2024-02-26T10:20:45.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pmcraid.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla1280.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/raid_class.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_debug.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-26T10:20:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ses.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/st.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/stex.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/virtio_scsi.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/wd719x.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb/ssb.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback 2024-02-26T10:20:45.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_file.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_iblock.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_mod.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_pscsi.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_user.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-26T10:20:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-26T10:20:45.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_gsm.ko 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_hdlc.ko 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/nozomi.ko 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/arc_uart.ko 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host 2024-02-26T10:20:45.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_aec.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_cif.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_pci_generic.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_sercos3.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-acm.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usblp.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usbtmc.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-26T10:20:45.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/mdc800.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/microtek.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/adutux.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/appledisplay.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi26.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi62.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ezusb.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/idmouse.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/iowarrior.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ldusb.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/legousbtower.ko 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga 2024-02-26T10:20:45.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usb3503.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usblcd.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/uss720.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/yurex.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy/phy-generic.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/aircable.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ark3116.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ch341.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cp210x.ko 2024-02-26T10:20:45.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cyberjack.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/empeg.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_ti.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipaq.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipw.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ir-usb.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-26T10:20:45.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mct_u232.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7720.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7840.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/navman.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/omninet.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/opticon.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/option.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/oti6858.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/pl2303.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcaux.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcserial.ko 2024-02-26T10:20:45.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/quatech2.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/safe_serial.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/sierra.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ssu100.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/symbolserial.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_debug.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/visor.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/whiteheat.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/uas.ko 2024-02-26T10:20:45.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-karma.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/usb-storage.ko 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip 2024-02-26T10:20:45.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_net.ko 2024-02-26T10:20:45.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_scsi.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/apple_bl.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/lcd.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_balloon.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_mmio.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-26T10:20:45.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_ring.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2482.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2490.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-26T10:20:45.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/wire.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/hpwdt.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/i6300esb.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ib700wdt.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ibmasr.ko 2024-02-26T10:20:45.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it87_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/machzwd.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/nv_tco.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/softdog.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/via_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/wdt_pci.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/xen_wdt.ko 2024-02-26T10:20:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-evtchn.ko 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntalloc.ko 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntdev.ko 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-privcmd.ko 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p/9p.ko 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs/affs.ko 2024-02-26T10:20:45.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs/befs.ko 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/binfmt_misc.ko 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs/btrfs.ko 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles/cachefiles.ko 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph/ceph.ko 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda/coda.ko 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs/cramfs.ko 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm/dlm.ko 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-26T10:20:45.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs/f2fs.ko 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/fat.ko 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/msdos.ko 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/vfat.ko 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/cuse.ko 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/fuse.ko 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2/gfs2.ko 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs/hfs.ko 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus 2024-02-26T10:20:45.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus/hfsplus.ko 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs/isofs.ko 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs/jfs.ko 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd/lockd.ko 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix/minix.ko 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs 2024-02-26T10:20:45.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs/netfs.ko 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfs.ko 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv3.ko 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv4.ko 2024-02-26T10:20:45.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/grace.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/nfs_acl.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd/nfsd.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2/nilfs2.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-celtic.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-centeuro.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-croatian.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-cyrillic.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-gaelic.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-greek.ko 2024-02-26T10:20:45.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-iceland.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-inuit.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-roman.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-romanian.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-turkish.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1250.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1251.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1255.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp737.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp775.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp850.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp852.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp855.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp857.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp860.ko 2024-02-26T10:20:45.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp861.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp862.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp863.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp864.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp865.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp866.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp869.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp874.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp932.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp936.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp949.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp950.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_euc-jp.ko 2024-02-26T10:20:45.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-1.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-13.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-14.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-15.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-2.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-3.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-4.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-5.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-6.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-7.ko 2024-02-26T10:20:45.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-9.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-r.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-ru.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-u.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_utf8.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-26T10:20:45.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-26T10:20:45.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-26T10:20:45.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs 2024-02-26T10:20:45.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs/overlay.ko 2024-02-26T10:20:45.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore 2024-02-26T10:20:45.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore/ramoops.ko 2024-02-26T10:20:45.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs 2024-02-26T10:20:45.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs/reiserfs.ko 2024-02-26T10:20:45.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs 2024-02-26T10:20:45.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs/romfs.ko 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client/cifs.ko 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_arc4.ko 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_md4.ko 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs/squashfs.ko 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv/sysv.ko 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf/udf.ko 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs 2024-02-26T10:20:45.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs/ufs.ko 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs/xfs.ko 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu/rcutorture.ko 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/torture.ko 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/asn1_encoder.ko 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc-itu-t.ko 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc8.ko 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto 2024-02-26T10:20:45.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libarc4.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libdes.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/libcrc32c.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lru_cache.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4_compress.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4hc_compress.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math/cordic.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6/raid6_pq.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_bm.ko 2024-02-26T10:20:45.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_fsm.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_kmp.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd/zstd_compress.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm/hwpoison-inject.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/6lowpan.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_dest.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_fragment.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_hop.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_mobility.ko 2024-02-26T10:20:45.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_routing.ko 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_udp.ko 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/garp.ko 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/mrp.ko 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/p8022.ko 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/psnap.ko 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/stp.ko 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q/8021q.ko 2024-02-26T10:20:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet.ko 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_fd.ko 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_rdma.ko 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_virtio.ko 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv/batman-adv.ko 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/br_netfilter.ko 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/bridge.ko 2024-02-26T10:20:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-26T10:20:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtables.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph 2024-02-26T10:20:45.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph/libceph.ko 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/failover.ko 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/pktgen.ko 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp.ko 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_diag.ko 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv4.ko 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv6.ko 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver 2024-02-26T10:20:45.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver/dns_resolver.ko 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/dsa_core.ko 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_brcm.ko 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_dsa.ko 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_none.ko 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_trailer.ko 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154.ko 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4 2024-02-26T10:20:45.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ah4.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/esp4.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/fou.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/gre.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/inet_diag.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_gre.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_tunnel.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_vti.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipcomp.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipip.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-26T10:20:45.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-26T10:20:45.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_bic.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_dctcp.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_diag.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_highspeed.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_htcp.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_hybla.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_illinois.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_lp.ko 2024-02-26T10:20:45.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_scalable.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_vegas.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_veno.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_westwood.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_yeah.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tunnel4.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_diag.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_tunnel.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ah6.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/esp6.ko 2024-02-26T10:20:45.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/fou6.ko 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_tunnel.ko 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_vti.ko 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ipcomp6.ko 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-26T10:20:45.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-26T10:20:45.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/sit.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/tunnel6.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key/af_key.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_core.ko 2024-02-26T10:20:45.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_eth.ko 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip.ko 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip6.ko 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_netlink.ko 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ppp.ko 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc/llc.ko 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154/mac802154.ko 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls/mpls_gso.ko 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter 2024-02-26T10:20:45.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-26T10:20:45.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-26T10:20:45.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-26T10:20:45.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-26T10:20:45.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-26T10:20:45.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-26T10:20:45.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-26T10:20:45.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs 2024-02-26T10:20:45.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-26T10:20:45.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-26T10:20:45.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conncount.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack.ko 2024-02-26T10:20:45.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-26T10:20:45.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_log_syslog.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_irc.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_sip.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_tables.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_log.ko 2024-02-26T10:20:45.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_chain_nat.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_compat.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_ct.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fib.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_hash.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_limit.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_log.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_masq.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_nat.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_numgen.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_queue.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_quota.ko 2024-02-26T10:20:45.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_redir.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject_inet.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_socket.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_tunnel.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_AUDIT.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CT.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_DSCP.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HL.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HMARK.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LED.ko 2024-02-26T10:20:45.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LOG.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NETMAP.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFLOG.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_RATEEST.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_SECMARK.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TEE.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TPROXY.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TRACE.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_addrtype.ko 2024-02-26T10:20:45.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_bpf.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cgroup.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cluster.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_comment.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connbytes.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlabel.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlimit.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connmark.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_conntrack.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cpu.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dccp.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_devgroup.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dscp.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ecn.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_esp.ko 2024-02-26T10:20:45.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hashlimit.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_helper.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hl.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipcomp.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_iprange.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipvs.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_l2tp.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_length.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_limit.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mac.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mark.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_multiport.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nat.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nfacct.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_osf.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_owner.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_physdev.ko 2024-02-26T10:20:45.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_pkttype.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_policy.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_quota.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_rateest.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_realm.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_recent.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_sctp.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_set.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_socket.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_state.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_statistic.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_string.ko 2024-02-26T10:20:45.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_tcpmss.ko 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_time.ko 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_u32.ko 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink/netlink_diag.ko 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh/nsh.ko 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/openvswitch.ko 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-geneve.ko 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-gre.ko 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-vxlan.ko 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet 2024-02-26T10:20:45.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet/af_packet_diag.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_rdma.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_tcp.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_csum.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_gact.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_mirred.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_nat.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_pedit.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_police.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_simple.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_skbedit.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_vlan.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_basic.ko 2024-02-26T10:20:45.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_bpf.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_flow.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_fw.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_route.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_u32.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_cmp.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_ipset.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_meta.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_nbyte.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_text.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_u32.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_choke.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_codel.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_drr.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_fq.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_gred.ko 2024-02-26T10:20:45.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hfsc.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hhf.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_htb.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_ingress.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio_lib.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_multiq.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_netem.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_pie.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_plug.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_prio.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_qfq.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_red.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfb.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfq.ko 2024-02-26T10:20:45.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_tbf.ko 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_teql.ko 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp.ko 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp_diag.ko 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/sunrpc.ko 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc 2024-02-26T10:20:45.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/diag.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/tipc.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix/unix_diag.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-26T10:20:45.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys/trusted.ko 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib/irqbypass.ko 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias.bin 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.alias.bin 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.bin 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.modinfo 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep.bin 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.devname 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.order 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.softdep 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols 2024-02-26T10:20:45.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols.bin 2024-02-26T10:20:45.263 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T10:20:45.266 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:20:45.267 DEBUG:teuthology.orchestra.run.smithi060:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout:postinstall scriptlet (using /bin/sh): 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout: /usr/bin/kernel-install add 6.8.0-rc4-g7c0fbba6c508 /lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout:for file in vmlinuz System.map config; do 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508"; then 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout: cp "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508" 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout: fi 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout:done 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-26T10:20:45.291 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-26T10:20:45.292 INFO:teuthology.orchestra.run.smithi060.stdout:new-kernel-pkg --remove 6.8.0-rc4-g7c0fbba6c508 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc4-g7c0fbba6c508.img 2024-02-26T10:20:45.292 INFO:teuthology.orchestra.run.smithi060.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-26T10:20:45.292 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-install remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:20:45.292 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-02-26T10:20:45.292 INFO:teuthology.orchestra.run.smithi060.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-26T10:20:45.292 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-26T10:20:45.292 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/update-bootloader --remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:20:45.292 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-02-26T10:20:45.293 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 68.el9 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8264254 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2024-02-26T10:20:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2024-02-26T10:20:45.374 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:20:45.375 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2024-02-26T10:20:45.375 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-26T10:20:45.376 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:20:45.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-26T10:20:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-02-26T10:20:47.387 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-26T10:20:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-02-26T10:20:47.420 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-02-26T10:20:47.435 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.xwGH4UExI5 2024-02-26T10:20:47.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xwGH4UExI5 2024-02-26T10:20:47.507 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.xwGH4UExI5 2024-02-26T10:20:47.649 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.xwGH4UExI5 is 6KB 2024-02-26T10:20:47.698 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.xwGH4UExI5 2024-02-26T10:20:47.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2024-02-26T10:20:47.781 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-26T10:20:47.781 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-26T10:20:47.781 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-26T10:20:47.781 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508.conf 2024-02-26T10:20:47.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:20:47.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-02-26T10:21:17.948 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-26T10:21:17.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T10:21:17.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:21:36.444 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-02-26T10:21:45.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T10:21:45.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:22:45.448 DEBUG:teuthology.orchestra.remote:timed out 2024-02-26T10:22:57.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T10:22:57.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:22:57.948 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-02-26T10:22:58.619 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T10:22:58.619 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc4-g7c0fbba6c508"... 2024-02-26T10:22:58.620 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-02-26T10:22:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:22:58.678 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc4-g7c0fbba6c508 vs 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:22:58.678 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-26T10:22:58.678 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-26T10:22:59.679 DEBUG:teuthology.parallel:result is None 2024-02-26T10:22:59.680 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-26T10:22:59.689 INFO:teuthology.task.internal:Creating test directory... 2024-02-26T10:22:59.689 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T10:22:59.697 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T10:22:59.701 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T10:22:59.748 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-26T10:22:59.755 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-26T10:22:59.765 INFO:teuthology.task.internal:Creating archive directory... 2024-02-26T10:22:59.765 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T10:22:59.771 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T10:22:59.773 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T10:22:59.847 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-26T10:22:59.854 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-26T10:22:59.854 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T10:22:59.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:22:59.871 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T10:22:59.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:22:59.887 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T10:22:59.895 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T10:22:59.913 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T10:22:59.967 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:22:59.986 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:22:59.997 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:23:00.021 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:23:00.022 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:23:00.090 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:23:00.095 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-26T10:23:00.103 INFO:teuthology.task.internal:Configuring sudo... 2024-02-26T10:23:00.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T10:23:00.108 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T10:23:00.110 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T10:23:00.209 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-26T10:23:00.218 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-26T10:23:00.218 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T10:23:00.226 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T10:23:00.228 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T10:23:00.277 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:23:00.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:23:00.473 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:23:00.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:23:00.668 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:23:00.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T10:23:00.768 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:23:00.812 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:23:00.866 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:23:00.948 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:23:00.993 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:23:00.993 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T10:23:01.060 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:23:01.104 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:23:01.166 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:23:01.209 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:23:01.255 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:23:01.255 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T10:23:01.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-02-26T10:23:01.330 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-02-26T10:23:01.332 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-02-26T10:23:01.369 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:23:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:23:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:23:01.953 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-26T10:23:01.962 INFO:teuthology.task.internal:Starting timer... 2024-02-26T10:23:01.963 INFO:teuthology.run_tasks:Running task pcp... 2024-02-26T10:23:01.973 INFO:teuthology.run_tasks:Running task selinux... 2024-02-26T10:23:01.982 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-26T10:23:01.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2024-02-26T10:23:02.103 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2024-02-26T10:23:02.107 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-02-26T10:23:02.176 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-02-26T10:23:02.178 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2024-02-26T10:23:02.227 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2024-02-26T10:23:02.258 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-26T10:23:02.258 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2024-02-26T10:23:02.354 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2024-02-26T10:23:02.355 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-02-26T10:23:02.371 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-02-26T10:23:02.372 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2024-02-26T10:23:02.412 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2024-02-26T10:23:02.413 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi113.front.sepia.ceph.com': 'permissive'} 2024-02-26T10:23:02.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:23:02.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:23:02.493 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:23:02.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:23:02.521 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:23:02.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:23:02.549 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-26T10:23:02.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2024-02-26T10:23:02.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-02-26T10:23:02.673 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2024-02-26T10:23:02.722 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-26T10:23:02.734 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-26T10:23:02.751 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-26T10:23:02.752 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi100.front.sepia.ceph.com,smithi113.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-26T10:29:20.068 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com')] 2024-02-26T10:29:20.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T10:29:20.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:29:20.224 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-02-26T10:29:20.380 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T10:29:20.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-02-26T10:29:20.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:29:20.469 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-02-26T10:29:20.538 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-02-26T10:29:20.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-02-26T10:29:20.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:29:20.634 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-02-26T10:29:20.708 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-02-26T10:29:20.709 INFO:teuthology.run_tasks:Running task clock... 2024-02-26T10:29:20.723 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-26T10:29:20.723 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T10:29:20.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:29:20.732 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T10:29:20.732 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:29:20.734 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T10:29:20.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:29:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T10:29:20.780 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T10:29:20.784 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T10:29:20.802 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T10:29:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-02-26T10:29:20.823 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-02-26T10:29:20.832 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2024-02-26T10:29:20.832 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T10:29:20.839 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T10:29:20.846 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2024-02-26T10:29:20.855 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T10:29:20.864 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T10:29:20.879 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T10:29:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T10:29:20.899 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:29:20.901 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:29:20.902 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-02-26T10:29:20.902 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:20.902 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:20.902 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:20.902 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:20.923 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:29:20.926 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:29:20.926 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-02-26T10:29:20.926 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:20.926 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:20.926 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:20.926 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:20.986 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2024-02-26T10:29:21.030 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2024-02-26T10:29:21.083 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T10:29:21.139 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T10:29:21.262 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:29:21.270 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:29:21.270 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-02-26T10:29:21.270 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:21.270 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:21.270 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:21.270 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:29:21.271 INFO:teuthology.run_tasks:Running task install... 2024-02-26T10:29:21.280 DEBUG:teuthology.task.install:project ceph 2024-02-26T10:29:21.281 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893'}} 2024-02-26T10:29:21.281 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893'} 2024-02-26T10:29:21.281 INFO:teuthology.task.install:Using flavor: default 2024-02-26T10:29:21.287 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-26T10:29:21.287 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-26T10:29:21.288 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'tag': None, 'wait_for_package': False} 2024-02-26T10:29:21.288 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:29:21.290 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'tag': None, 'wait_for_package': False} 2024-02-26T10:29:21.290 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:29:21.291 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'tag': None, 'wait_for_package': False} 2024-02-26T10:29:21.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:29:21.779 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/ 2024-02-26T10:29:21.780 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1686.g6085b91f 2024-02-26T10:29:21.927 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T10:29:21.927 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:29:21.927 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T10:29:22.034 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T10:29:22.034 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6085b91fefeb74bf4310344a87d81d3217ef7893/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T10:29:22.180 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T10:29:22.299 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/ 2024-02-26T10:29:22.299 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1686.g6085b91f 2024-02-26T10:29:22.322 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T10:29:22.412 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2024-02-26T10:29:22.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2024-02-26T10:29:22.624 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T10:29:22.625 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:29:22.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T10:29:22.657 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T10:29:22.658 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6085b91fefeb74bf4310344a87d81d3217ef7893/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T10:29:22.728 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T10:29:22.777 INFO:teuthology.orchestra.run.smithi060.stdout:74 files removed 2024-02-26T10:29:22.810 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T10:29:22.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T10:29:22.877 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-02-26T10:29:22.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-02-26T10:29:22.954 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/ 2024-02-26T10:29:22.954 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1686.g6085b91f 2024-02-26T10:29:23.112 INFO:teuthology.orchestra.run.smithi100.stdout:74 files removed 2024-02-26T10:29:23.138 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T10:29:23.329 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T10:29:23.329 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:29:23.329 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T10:29:23.362 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T10:29:23.362 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6085b91fefeb74bf4310344a87d81d3217ef7893/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T10:29:23.434 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T10:29:23.517 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T10:29:23.588 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2024-02-26T10:29:23.588 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2024-02-26T10:29:23.827 INFO:teuthology.orchestra.run.smithi113.stdout:74 files removed 2024-02-26T10:29:23.853 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T10:29:24.176 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 109 kB/s | 82 kB 00:00 2024-02-26T10:29:24.677 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 22 kB/s | 10 kB 00:00 2024-02-26T10:29:25.115 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 46 kB/s | 82 kB 00:01 2024-02-26T10:29:25.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 45 kB/s | 82 kB 00:01 2024-02-26T10:29:27.124 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 5.2 kB/s | 10 kB 00:01 2024-02-26T10:29:27.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 5.1 kB/s | 10 kB 00:01 2024-02-26T10:29:29.411 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 412 B/s | 1.9 kB 00:04 2024-02-26T10:29:30.121 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-02-26T10:29:31.566 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 444 B/s | 1.9 kB 00:04 2024-02-26T10:29:32.030 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 4.4 MB/s | 2.5 MB 00:00 2024-02-26T10:29:32.720 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - AppStream 7.6 MB/s | 8.1 MB 00:01 2024-02-26T10:29:33.953 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-02-26T10:29:34.778 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.5 MB 00:00 2024-02-26T10:29:36.558 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-02-26T10:29:42.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 132 B/s | 1.9 kB 00:14 2024-02-26T10:29:43.680 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - AppStream 8.2 MB/s | 8.1 MB 00:00 2024-02-26T10:29:45.664 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.5 MB 00:00 2024-02-26T10:29:47.971 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-02-26T10:29:50.849 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-02-26T10:29:55.389 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.1 MB 00:00 2024-02-26T10:29:57.015 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-02-26T10:29:59.050 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-26T10:30:03.682 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 756 kB/s | 19 MB 00:25 2024-02-26T10:30:04.016 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-02-26T10:30:05.101 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 651 kB/s | 19 MB 00:30 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr:Modular dependency problems: 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 1: conflicting requests 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 2: conflicting requests 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 3: conflicting requests 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 4: conflicting requests 2024-02-26T10:30:06.598 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 5: conflicting requests 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 6: conflicting requests 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 7: conflicting requests 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 8: conflicting requests 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 9: conflicting requests 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 10: conflicting requests 2024-02-26T10:30:06.599 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 11: conflicting requests 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 12: conflicting requests 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 13: conflicting requests 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 14: conflicting requests 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 15: conflicting requests 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 16: conflicting requests 2024-02-26T10:30:06.600 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 17: conflicting requests 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 18: conflicting requests 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 19: conflicting requests 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 20: conflicting requests 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 21: conflicting requests 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 22: conflicting requests 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.601 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 23: conflicting requests 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 24: conflicting requests 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 25: conflicting requests 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 26: conflicting requests 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 27: conflicting requests 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 28: conflicting requests 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 29: conflicting requests 2024-02-26T10:30:06.602 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 30: conflicting requests 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 31: conflicting requests 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 32: conflicting requests 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 33: conflicting requests 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 34: conflicting requests 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 35: conflicting requests 2024-02-26T10:30:06.603 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.604 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 36: conflicting requests 2024-02-26T10:30:06.604 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.604 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 38: conflicting requests 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.605 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.606 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.607 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.608 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.608 INFO:teuthology.orchestra.run.smithi113.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T10:30:06.608 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.608 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.608 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.608 INFO:teuthology.orchestra.run.smithi113.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T10:30:06.986 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:30:06.986 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:30:06.999 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T10:30:07.000 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T10:30:07.001 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T10:30:07.002 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T10:30:07.003 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T10:30:07.006 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T10:30:07.007 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T10:30:07.093 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-02-26T10:30:07.121 INFO:teuthology.orchestra.run.smithi113.stdout:========================================================================================= 2024-02-26T10:30:07.121 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-02-26T10:30:07.121 INFO:teuthology.orchestra.run.smithi113.stdout:========================================================================================= 2024-02-26T10:30:07.121 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-02-26T10:30:07.121 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T10:30:07.121 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 6.6 k 2024-02-26T10:30:07.121 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.3 M 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 888 k 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 145 k 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 1.5 M 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 138 k 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 1.8 M 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 7.4 M 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 49 k 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 9.8 M 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 47 M 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 264 k 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 748 k 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 220 k 2024-02-26T10:30:07.122 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 27 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 18 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 31 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 744 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 127 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T10:30:07.123 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 162 k 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 320 k 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 298 k 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 100 k 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 85 k 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 170 k 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T10:30:07.124 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.4 M 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-02-26T10:30:07.125 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 20 M 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 31 k 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 2.1 M 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 248 k 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 4.7 M 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 17 M 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 15 k 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 25 k 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T10:30:07.126 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 163 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 501 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.1 M 2024-02-26T10:30:07.127 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T10:30:07.128 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T10:30:07.129 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T10:30:07.129 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T10:30:07.129 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T10:30:07.130 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T10:30:07.130 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 45 k 2024-02-26T10:30:07.130 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 131 k 2024-02-26T10:30:07.130 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T10:30:07.131 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T10:30:07.132 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T10:30:07.132 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T10:30:07.132 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T10:30:07.132 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T10:30:07.132 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T10:30:07.133 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T10:30:07.133 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T10:30:07.133 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T10:30:07.133 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T10:30:07.133 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T10:30:07.133 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T10:30:07.133 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T10:30:07.133 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T10:30:07.134 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T10:30:07.134 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T10:30:07.134 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T10:30:07.134 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T10:30:07.134 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T10:30:07.134 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T10:30:07.134 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T10:30:07.134 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T10:30:07.135 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T10:30:07.136 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2024-02-26T10:30:07.137 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T10:30:07.137 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:30:07.137 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-02-26T10:30:07.137 INFO:teuthology.orchestra.run.smithi113.stdout:========================================================================================= 2024-02-26T10:30:07.137 INFO:teuthology.orchestra.run.smithi113.stdout:Install 150 Packages 2024-02-26T10:30:07.137 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 16 Packages 2024-02-26T10:30:07.137 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:30:07.138 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 222 M 2024-02-26T10:30:07.138 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-02-26T10:30:08.767 INFO:teuthology.orchestra.run.smithi113.stdout:(1/166): ceph-base-19.0.0-1686.g6085b91f.el9.x8 5.3 MB/s | 5.3 MB 00:01 2024-02-26T10:30:08.900 INFO:teuthology.orchestra.run.smithi113.stdout:(2/166): ceph-fuse-19.0.0-1686.g6085b91f.el9.x8 6.5 MB/s | 888 kB 00:00 2024-02-26T10:30:09.093 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-02-26T10:30:09.192 INFO:teuthology.orchestra.run.smithi113.stdout:(3/166): ceph-19.0.0-1686.g6085b91f.el9.x86_64. 4.5 kB/s | 6.6 kB 00:01 2024-02-26T10:30:09.234 INFO:teuthology.orchestra.run.smithi113.stdout:(4/166): ceph-immutable-object-cache-19.0.0-168 435 kB/s | 145 kB 00:00 2024-02-26T10:30:09.257 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.1 MB 00:01 2024-02-26T10:30:09.601 INFO:teuthology.orchestra.run.smithi113.stdout:(5/166): ceph-common-19.0.0-1686.g6085b91f.el9. 11 MB/s | 20 MB 00:01 2024-02-26T10:30:09.868 INFO:teuthology.orchestra.run.smithi113.stdout:(6/166): ceph-mon-19.0.0-1686.g6085b91f.el9.x86 18 MB/s | 4.7 MB 00:00 2024-02-26T10:30:10.294 INFO:teuthology.orchestra.run.smithi113.stdout:(7/166): ceph-mds-19.0.0-1686.g6085b91f.el9.x86 1.9 MB/s | 2.1 MB 00:01 2024-02-26T10:30:10.519 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 15 kB 00:00 2024-02-26T10:30:10.685 INFO:teuthology.orchestra.run.smithi113.stdout:(8/166): ceph-osd-19.0.0-1686.g6085b91f.el9.x86 21 MB/s | 17 MB 00:00 2024-02-26T10:30:10.735 INFO:teuthology.orchestra.run.smithi113.stdout:(9/166): ceph-selinux-19.0.0-1686.g6085b91f.el9 507 kB/s | 25 kB 00:00 2024-02-26T10:30:10.767 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 15 kB 00:00 2024-02-26T10:30:11.352 INFO:teuthology.orchestra.run.smithi113.stdout:(10/166): ceph-mgr-19.0.0-1686.g6085b91f.el9.x8 720 kB/s | 1.5 MB 00:02 2024-02-26T10:30:11.747 INFO:teuthology.orchestra.run.smithi113.stdout:(11/166): cephfs-mirror-19.0.0-1686.g6085b91f.e 562 kB/s | 220 kB 00:00 2024-02-26T10:30:11.836 INFO:teuthology.orchestra.run.smithi113.stdout:(12/166): cephfs-shell-19.0.0-1686.g6085b91f.el 295 kB/s | 27 kB 00:00 2024-02-26T10:30:11.928 INFO:teuthology.orchestra.run.smithi113.stdout:(13/166): libcephfs-devel-19.0.0-1686.g6085b91f 341 kB/s | 31 kB 00:00 2024-02-26T10:30:12.195 INFO:teuthology.orchestra.run.smithi113.stdout:(14/166): libcephfs2-19.0.0-1686.g6085b91f.el9. 2.7 MB/s | 744 kB 00:00 2024-02-26T10:30:12.304 INFO:teuthology.orchestra.run.smithi113.stdout:(15/166): libcephsqlite-19.0.0-1686.g6085b91f.e 1.5 MB/s | 163 kB 00:00 2024-02-26T10:30:12.412 INFO:teuthology.orchestra.run.smithi113.stdout:(16/166): ceph-radosgw-19.0.0-1686.g6085b91f.el 4.6 MB/s | 9.8 MB 00:02 2024-02-26T10:30:12.438 INFO:teuthology.orchestra.run.smithi113.stdout:(17/166): librados-devel-19.0.0-1686.g6085b91f. 948 kB/s | 127 kB 00:00 2024-02-26T10:30:12.505 INFO:teuthology.orchestra.run.smithi113.stdout:(18/166): libradosstriper1-19.0.0-1686.g6085b91 5.3 MB/s | 501 kB 00:00 2024-02-26T10:30:12.564 INFO:teuthology.orchestra.run.smithi113.stdout:(19/166): python3-ceph-argparse-19.0.0-1686.g60 774 kB/s | 45 kB 00:00 2024-02-26T10:30:12.656 INFO:teuthology.orchestra.run.smithi113.stdout:(20/166): python3-ceph-common-19.0.0-1686.g6085 1.4 MB/s | 131 kB 00:00 2024-02-26T10:30:12.707 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-26T10:30:12.756 INFO:teuthology.orchestra.run.smithi113.stdout:(21/166): python3-cephfs-19.0.0-1686.g6085b91f. 1.6 MB/s | 162 kB 00:00 2024-02-26T10:30:12.865 INFO:teuthology.orchestra.run.smithi113.stdout:(22/166): python3-rados-19.0.0-1686.g6085b91f.e 2.9 MB/s | 320 kB 00:00 2024-02-26T10:30:12.965 INFO:teuthology.orchestra.run.smithi113.stdout:(23/166): python3-rbd-19.0.0-1686.g6085b91f.el9 2.9 MB/s | 298 kB 00:00 2024-02-26T10:30:13.093 INFO:teuthology.orchestra.run.smithi113.stdout:(24/166): python3-rgw-19.0.0-1686.g6085b91f.el9 781 kB/s | 100 kB 00:00 2024-02-26T10:30:13.200 INFO:teuthology.orchestra.run.smithi113.stdout:(25/166): rbd-fuse-19.0.0-1686.g6085b91f.el9.x8 808 kB/s | 85 kB 00:00 2024-02-26T10:30:13.633 INFO:teuthology.orchestra.run.smithi113.stdout:(26/166): ceph-test-19.0.0-1686.g6085b91f.el9.x 16 MB/s | 47 MB 00:02 2024-02-26T10:30:13.716 INFO:teuthology.orchestra.run.smithi113.stdout:(27/166): librgw2-19.0.0-1686.g6085b91f.el9.x86 4.0 MB/s | 5.1 MB 00:01 2024-02-26T10:30:13.750 INFO:teuthology.orchestra.run.smithi113.stdout:(28/166): rbd-nbd-19.0.0-1686.g6085b91f.el9.x86 1.4 MB/s | 170 kB 00:00 2024-02-26T10:30:13.767 INFO:teuthology.orchestra.run.smithi113.stdout:(29/166): ceph-grafana-dashboards-19.0.0-1686.g 628 kB/s | 31 kB 00:00 2024-02-26T10:30:13.800 INFO:teuthology.orchestra.run.smithi113.stdout:(30/166): ceph-mgr-cephadm-19.0.0-1686.g6085b91 2.7 MB/s | 138 kB 00:00 2024-02-26T10:30:13.859 INFO:teuthology.orchestra.run.smithi113.stdout:(31/166): rbd-mirror-19.0.0-1686.g6085b91f.el9. 4.7 MB/s | 3.1 MB 00:00 2024-02-26T10:30:13.966 INFO:teuthology.orchestra.run.smithi113.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1686.g60 2.3 MB/s | 248 kB 00:00 2024-02-26T10:30:14.009 INFO:teuthology.orchestra.run.smithi113.stdout:(33/166): ceph-mgr-dashboard-19.0.0-1686.g6085b 7.4 MB/s | 1.8 MB 00:00 2024-02-26T10:30:14.042 INFO:teuthology.orchestra.run.smithi113.stdout:(34/166): ceph-mgr-rook-19.0.0-1686.g6085b91f.e 655 kB/s | 49 kB 00:00 2024-02-26T10:30:14.045 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 20 MB 00:03 2024-02-26T10:30:14.084 INFO:teuthology.orchestra.run.smithi113.stdout:(35/166): ceph-prometheus-alerts-19.0.0-1686.g6 204 kB/s | 15 kB 00:00 2024-02-26T10:30:14.127 INFO:teuthology.orchestra.run.smithi113.stdout:(36/166): ceph-volume-19.0.0-1686.g6085b91f.el9 3.1 MB/s | 264 kB 00:00 2024-02-26T10:30:14.227 INFO:teuthology.orchestra.run.smithi113.stdout:(37/166): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-02-26T10:30:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:(38/166): cephfs-top-19.0.0-1686.g6085b91f.el9. 146 kB/s | 18 kB 00:00 2024-02-26T10:30:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:(39/166): cephadm-19.0.0-1686.g6085b91f.el9.noa 3.6 MB/s | 748 kB 00:00 2024-02-26T10:30:14.394 INFO:teuthology.orchestra.run.smithi113.stdout:(40/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 105 kB/s | 18 kB 00:00 2024-02-26T10:30:14.453 INFO:teuthology.orchestra.run.smithi113.stdout:(41/166): fuse-2.9.9-15.el9.x86_64.rpm 401 kB/s | 80 kB 00:00 2024-02-26T10:30:14.545 INFO:teuthology.orchestra.run.smithi113.stdout:(42/166): jq-1.6-16.el9.x86_64.rpm 804 kB/s | 187 kB 00:00 2024-02-26T10:30:14.570 INFO:teuthology.orchestra.run.smithi113.stdout:(43/166): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-02-26T10:30:14.646 INFO:teuthology.orchestra.run.smithi113.stdout:(44/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.1 MB/s | 218 kB 00:00 2024-02-26T10:30:14.671 INFO:teuthology.orchestra.run.smithi113.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-02-26T10:30:14.738 INFO:teuthology.orchestra.run.smithi113.stdout:(46/166): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-02-26T10:30:14.763 INFO:teuthology.orchestra.run.smithi113.stdout:(47/166): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-02-26T10:30:14.814 INFO:teuthology.orchestra.run.smithi113.stdout:(48/166): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-02-26T10:30:14.839 INFO:teuthology.orchestra.run.smithi113.stdout:(49/166): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-02-26T10:30:14.873 INFO:teuthology.orchestra.run.smithi113.stdout:(50/166): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-02-26T10:30:14.898 INFO:teuthology.orchestra.run.smithi113.stdout:(51/166): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-02-26T10:30:15.240 INFO:teuthology.orchestra.run.smithi113.stdout:(52/166): boost-program-options-1.75.0-8.el9.x8 313 kB/s | 107 kB 00:00 2024-02-26T10:30:15.282 INFO:teuthology.orchestra.run.smithi113.stdout:(53/166): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 689 kB 00:00 2024-02-26T10:30:15.332 INFO:teuthology.orchestra.run.smithi113.stdout:(54/166): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-02-26T10:30:15.365 INFO:teuthology.orchestra.run.smithi113.stdout:(55/166): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 943 kB 00:00 2024-02-26T10:30:15.391 INFO:teuthology.orchestra.run.smithi113.stdout:(56/166): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-02-26T10:30:15.416 INFO:teuthology.orchestra.run.smithi113.stdout:(57/166): flexiblas-3.0.4-8.el9.x86_64.rpm 389 kB/s | 33 kB 00:00 2024-02-26T10:30:15.467 INFO:teuthology.orchestra.run.smithi113.stdout:(58/166): flexiblas-openblas-openmp-3.0.4-8.el9 241 kB/s | 18 kB 00:00 2024-02-26T10:30:15.567 INFO:teuthology.orchestra.run.smithi113.stdout:(59/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-02-26T10:30:15.592 INFO:teuthology.orchestra.run.smithi113.stdout:(60/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 910 kB/s | 160 kB 00:00 2024-02-26T10:30:15.626 INFO:teuthology.orchestra.run.smithi113.stdout:(61/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 284 kB/s | 45 kB 00:00 2024-02-26T10:30:15.693 INFO:teuthology.orchestra.run.smithi113.stdout:(62/166): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-02-26T10:30:15.726 INFO:teuthology.orchestra.run.smithi113.stdout:(63/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 255 kB/s | 26 kB 00:00 2024-02-26T10:30:15.760 INFO:teuthology.orchestra.run.smithi113.stdout:(64/166): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-02-26T10:30:15.785 INFO:teuthology.orchestra.run.smithi113.stdout:(65/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-02-26T10:30:15.828 INFO:teuthology.orchestra.run.smithi113.stdout:(66/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-02-26T10:30:15.853 INFO:teuthology.orchestra.run.smithi113.stdout:(67/166): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-02-26T10:30:15.886 INFO:teuthology.orchestra.run.smithi113.stdout:(68/166): openblas-0.3.21-2.el9.x86_64.rpm 350 kB/s | 35 kB 00:00 2024-02-26T10:30:16.095 INFO:teuthology.orchestra.run.smithi113.stdout:(69/166): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-02-26T10:30:16.203 INFO:teuthology.orchestra.run.smithi113.stdout:(70/166): postgresql-contrib-13.11-1.el9.x86_64 8.2 MB/s | 905 kB 00:00 2024-02-26T10:30:16.304 INFO:teuthology.orchestra.run.smithi113.stdout:(71/166): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 135 kB 00:00 2024-02-26T10:30:16.396 INFO:teuthology.orchestra.run.smithi113.stdout:(72/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 7.6 MB/s | 4.1 MB 00:00 2024-02-26T10:30:16.563 INFO:teuthology.orchestra.run.smithi113.stdout:(73/166): postgresql-13.11-1.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-02-26T10:30:16.597 INFO:teuthology.orchestra.run.smithi113.stdout:(74/166): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-02-26T10:30:16.672 INFO:teuthology.orchestra.run.smithi113.stdout:(75/166): postgresql-server-13.11-1.el9.x86_64. 16 MB/s | 5.7 MB 00:00 2024-02-26T10:30:16.697 INFO:teuthology.orchestra.run.smithi113.stdout:(76/166): python3-attrs-20.3.0-7.el9.noarch.rpm 654 kB/s | 87 kB 00:00 2024-02-26T10:30:16.781 INFO:teuthology.orchestra.run.smithi113.stdout:(77/166): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-02-26T10:30:16.964 INFO:teuthology.orchestra.run.smithi113.stdout:(78/166): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-02-26T10:30:16.989 INFO:teuthology.orchestra.run.smithi113.stdout:(79/166): python3-jsonpatch-1.21-16.el9.noarch. 123 kB/s | 26 kB 00:00 2024-02-26T10:30:17.015 INFO:teuthology.orchestra.run.smithi113.stdout:(80/166): python3-jinja2-2.11.3-5.el9.noarch.rp 780 kB/s | 248 kB 00:00 2024-02-26T10:30:17.073 INFO:teuthology.orchestra.run.smithi113.stdout:(81/166): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-02-26T10:30:17.099 INFO:teuthology.orchestra.run.smithi113.stdout:(82/166): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-02-26T10:30:17.133 INFO:teuthology.orchestra.run.smithi113.stdout:(83/166): python3-markupsafe-1.1.1-12.el9.x86_6 297 kB/s | 35 kB 00:00 2024-02-26T10:30:17.205 INFO:teuthology.orchestra.run.smithi113.stdout:(84/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-02-26T10:30:17.380 INFO:teuthology.orchestra.run.smithi113.stdout:(85/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-02-26T10:30:17.405 INFO:teuthology.orchestra.run.smithi113.stdout:(86/166): python3-oauthlib-3.1.1-5.el9.noarch.r 815 kB/s | 222 kB 00:00 2024-02-26T10:30:17.439 INFO:teuthology.orchestra.run.smithi113.stdout:(87/166): python3-packaging-20.9-5.el9.noarch.r 330 kB/s | 77 kB 00:00 2024-02-26T10:30:17.498 INFO:teuthology.orchestra.run.smithi113.stdout:(88/166): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-02-26T10:30:17.523 INFO:teuthology.orchestra.run.smithi113.stdout:(89/166): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-02-26T10:30:17.565 INFO:teuthology.orchestra.run.smithi113.stdout:(90/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-02-26T10:30:17.624 INFO:teuthology.orchestra.run.smithi113.stdout:(91/166): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-02-26T10:30:17.649 INFO:teuthology.orchestra.run.smithi113.stdout:(92/166): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-02-26T10:30:17.674 INFO:teuthology.orchestra.run.smithi113.stdout:(93/166): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-02-26T10:30:17.758 INFO:teuthology.orchestra.run.smithi113.stdout:(94/166): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-02-26T10:30:17.783 INFO:teuthology.orchestra.run.smithi113.stdout:(95/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 406 kB/s | 44 kB 00:00 2024-02-26T10:30:17.883 INFO:teuthology.orchestra.run.smithi113.stdout:(96/166): qatlib-23.11.0-1.el9.x86_64.rpm 1.7 MB/s | 216 kB 00:00 2024-02-26T10:30:17.909 INFO:teuthology.orchestra.run.smithi113.stdout:(97/166): qatlib-service-23.11.0-1.el9.x86_64.r 278 kB/s | 35 kB 00:00 2024-02-26T10:30:18.010 INFO:teuthology.orchestra.run.smithi113.stdout:(98/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 356 kB/s | 44 kB 00:00 2024-02-26T10:30:18.151 INFO:teuthology.orchestra.run.smithi113.stdout:(99/166): uuid-1.6.2-55.el9.x86_64.rpm 408 kB/s | 58 kB 00:00 2024-02-26T10:30:18.343 INFO:teuthology.orchestra.run.smithi113.stdout:(100/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-02-26T10:30:18.368 INFO:teuthology.orchestra.run.smithi113.stdout:(101/166): socat-1.7.4.1-5.el9.x86_64.rpm 664 kB/s | 305 kB 00:00 2024-02-26T10:30:18.427 INFO:teuthology.orchestra.run.smithi113.stdout:(102/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 340 kB/s | 94 kB 00:00 2024-02-26T10:30:18.552 INFO:teuthology.orchestra.run.smithi113.stdout:(103/166): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-02-26T10:30:18.585 INFO:teuthology.orchestra.run.smithi113.stdout:(104/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 263 kB/s | 64 kB 00:00 2024-02-26T10:30:18.645 INFO:teuthology.orchestra.run.smithi113.stdout:(105/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 164 kB/s | 45 kB 00:00 2024-02-26T10:30:18.694 INFO:teuthology.orchestra.run.smithi113.stdout:(106/166): protobuf-compiler-3.14.0-13.el9.x86_ 5.9 MB/s | 863 kB 00:00 2024-02-26T10:30:18.801 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-02-26T10:30:19.003 INFO:teuthology.orchestra.run.smithi113.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 63 kB/s | 19 kB 00:00 2024-02-26T10:30:19.103 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-26T10:30:19.204 INFO:teuthology.orchestra.run.smithi113.stdout:(108/166): gperftools-libs-2.9.1-2.el9.x86_64.r 552 kB/s | 309 kB 00:00 2024-02-26T10:30:19.245 INFO:teuthology.orchestra.run.smithi113.stdout:(109/166): abseil-cpp-20211102.0-3.el9.x86_64.r 831 kB/s | 548 kB 00:00 2024-02-26T10:30:19.312 INFO:teuthology.orchestra.run.smithi113.stdout:(110/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-02-26T10:30:19.362 INFO:teuthology.orchestra.run.smithi113.stdout:(111/166): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-02-26T10:30:19.446 INFO:teuthology.orchestra.run.smithi113.stdout:(112/166): libunwind-1.6.2-1.el9.x86_64.rpm 504 kB/s | 67 kB 00:00 2024-02-26T10:30:19.607 INFO:teuthology.orchestra.run.smithi113.stdout:(113/166): luarocks-3.9.2-1.el9.noarch.rpm 619 kB/s | 151 kB 00:00 2024-02-26T10:30:19.772 INFO:teuthology.orchestra.run.smithi113.stdout:(114/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-02-26T10:30:19.881 INFO:teuthology.orchestra.run.smithi113.stdout:(115/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-02-26T10:30:19.923 INFO:teuthology.orchestra.run.smithi113.stdout:(116/166): python3-asyncssh-2.13.2-4.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-02-26T10:30:19.998 INFO:teuthology.orchestra.run.smithi113.stdout:(117/166): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-02-26T10:30:20.032 INFO:teuthology.orchestra.run.smithi113.stdout:(118/166): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-02-26T10:30:20.132 INFO:teuthology.orchestra.run.smithi113.stdout:(119/166): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-02-26T10:30:20.249 INFO:teuthology.orchestra.run.smithi113.stdout:(120/166): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-02-26T10:30:20.274 INFO:teuthology.orchestra.run.smithi113.stdout:(121/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-02-26T10:30:20.358 INFO:teuthology.orchestra.run.smithi113.stdout:(122/166): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-02-26T10:30:20.391 INFO:teuthology.orchestra.run.smithi113.stdout:(123/166): python3-google-auth-2.28.0-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-02-26T10:30:20.500 INFO:teuthology.orchestra.run.smithi113.stdout:(124/166): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-02-26T10:30:20.609 INFO:teuthology.orchestra.run.smithi113.stdout:(125/166): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-02-26T10:30:20.717 INFO:teuthology.orchestra.run.smithi113.stdout:(126/166): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-02-26T10:30:20.835 INFO:teuthology.orchestra.run.smithi113.stdout:(127/166): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-02-26T10:30:20.943 INFO:teuthology.orchestra.run.smithi113.stdout:(128/166): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-02-26T10:30:20.985 INFO:teuthology.orchestra.run.smithi113.stdout:(129/166): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-02-26T10:30:21.052 INFO:teuthology.orchestra.run.smithi113.stdout:(130/166): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-02-26T10:30:21.169 INFO:teuthology.orchestra.run.smithi113.stdout:(131/166): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-02-26T10:30:21.294 INFO:teuthology.orchestra.run.smithi113.stdout:(132/166): python3-more-itertools-8.12.0-2.el9. 629 kB/s | 79 kB 00:00 2024-02-26T10:30:21.328 INFO:teuthology.orchestra.run.smithi113.stdout:(133/166): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-02-26T10:30:21.403 INFO:teuthology.orchestra.run.smithi113.stdout:(134/166): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-02-26T10:30:21.470 INFO:teuthology.orchestra.run.smithi113.stdout:(135/166): libarrow-9.0.0-10.el9.x86_64.rpm 1.8 MB/s | 4.4 MB 00:02 2024-02-26T10:30:21.496 INFO:teuthology.orchestra.run.smithi113.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-02-26T10:30:21.512 INFO:teuthology.orchestra.run.smithi113.stdout:(137/166): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-02-26T10:30:21.596 INFO:teuthology.orchestra.run.smithi113.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 716 kB/s | 90 kB 00:00 2024-02-26T10:30:21.608 INFO:teuthology.orchestra.run.smithi060.stderr:Modular dependency problems: 2024-02-26T10:30:21.608 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-02-26T10:30:21.608 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 1: conflicting requests 2024-02-26T10:30:21.608 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.608 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 2: conflicting requests 2024-02-26T10:30:21.608 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.608 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 3: conflicting requests 2024-02-26T10:30:21.608 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 4: conflicting requests 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 5: conflicting requests 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 6: conflicting requests 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 7: conflicting requests 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 8: conflicting requests 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 9: conflicting requests 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.609 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 10: conflicting requests 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 11: conflicting requests 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 12: conflicting requests 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 13: conflicting requests 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 14: conflicting requests 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 15: conflicting requests 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 16: conflicting requests 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.610 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 17: conflicting requests 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 18: conflicting requests 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 19: conflicting requests 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 20: conflicting requests 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 21: conflicting requests 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 22: conflicting requests 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 23: conflicting requests 2024-02-26T10:30:21.611 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.612 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 24: conflicting requests 2024-02-26T10:30:21.612 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.613 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 25: conflicting requests 2024-02-26T10:30:21.613 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.613 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 26: conflicting requests 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 27: conflicting requests 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 28: conflicting requests 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 29: conflicting requests 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 30: conflicting requests 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 31: conflicting requests 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 32: conflicting requests 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.614 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 33: conflicting requests 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 34: conflicting requests 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 35: conflicting requests 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 36: conflicting requests 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.615 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 38: conflicting requests 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.616 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.617 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.618 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.619 INFO:teuthology.orchestra.run.smithi113.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 240 kB/s | 28 kB 00:00 2024-02-26T10:30:21.638 INFO:teuthology.orchestra.run.smithi113.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 244 kB/s | 31 kB 00:00 2024-02-26T10:30:21.716 INFO:teuthology.orchestra.run.smithi100.stderr:Modular dependency problems: 2024-02-26T10:30:21.716 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 1: conflicting requests 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 2: conflicting requests 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 3: conflicting requests 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 4: conflicting requests 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 5: conflicting requests 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 6: conflicting requests 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.717 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 7: conflicting requests 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 8: conflicting requests 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 9: conflicting requests 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 10: conflicting requests 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 11: conflicting requests 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 12: conflicting requests 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 13: conflicting requests 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.718 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 14: conflicting requests 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 15: conflicting requests 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 16: conflicting requests 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 17: conflicting requests 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 18: conflicting requests 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 19: conflicting requests 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 20: conflicting requests 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.719 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 21: conflicting requests 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 22: conflicting requests 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 23: conflicting requests 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 24: conflicting requests 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 25: conflicting requests 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 26: conflicting requests 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 27: conflicting requests 2024-02-26T10:30:21.720 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 28: conflicting requests 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 29: conflicting requests 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 30: conflicting requests 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 31: conflicting requests 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 32: conflicting requests 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 33: conflicting requests 2024-02-26T10:30:21.721 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 34: conflicting requests 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 35: conflicting requests 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 36: conflicting requests 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.722 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 38: conflicting requests 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.723 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.724 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.724 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.724 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T10:30:21.724 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.724 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.724 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.724 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.724 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.725 INFO:teuthology.orchestra.run.smithi100.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T10:30:21.726 INFO:teuthology.orchestra.run.smithi113.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-02-26T10:30:21.748 INFO:teuthology.orchestra.run.smithi113.stdout:(142/166): python3-rsa-4.9-2.el9.noarch.rpm 441 kB/s | 59 kB 00:00 2024-02-26T10:30:21.772 INFO:teuthology.orchestra.run.smithi113.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 267 kB/s | 36 kB 00:00 2024-02-26T10:30:21.848 INFO:teuthology.orchestra.run.smithi113.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-02-26T10:30:21.882 INFO:teuthology.orchestra.run.smithi113.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-02-26T10:30:21.915 INFO:teuthology.orchestra.run.smithi113.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 630 kB/s | 90 kB 00:00 2024-02-26T10:30:22.024 INFO:teuthology.orchestra.run.smithi113.stdout:(147/166): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-02-26T10:30:22.049 INFO:teuthology.orchestra.run.smithi113.stdout:(148/166): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-02-26T10:30:22.099 INFO:teuthology.orchestra.run.smithi113.stdout:(149/166): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-02-26T10:30:22.116 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:30:22.117 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:30:22.131 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T10:30:22.131 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T10:30:22.132 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T10:30:22.133 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T10:30:22.135 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T10:30:22.138 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T10:30:22.139 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T10:30:22.226 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-26T10:30:22.245 INFO:teuthology.orchestra.run.smithi100.stdout:========================================================================================= 2024-02-26T10:30:22.245 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-26T10:30:22.245 INFO:teuthology.orchestra.run.smithi100.stdout:========================================================================================= 2024-02-26T10:30:22.245 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-26T10:30:22.246 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T10:30:22.246 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 6.6 k 2024-02-26T10:30:22.246 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.3 M 2024-02-26T10:30:22.246 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 888 k 2024-02-26T10:30:22.246 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 145 k 2024-02-26T10:30:22.246 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 1.5 M 2024-02-26T10:30:22.247 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 138 k 2024-02-26T10:30:22.247 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 1.8 M 2024-02-26T10:30:22.247 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 7.4 M 2024-02-26T10:30:22.247 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 49 k 2024-02-26T10:30:22.247 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 9.8 M 2024-02-26T10:30:22.247 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 47 M 2024-02-26T10:30:22.247 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 264 k 2024-02-26T10:30:22.248 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 748 k 2024-02-26T10:30:22.248 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 220 k 2024-02-26T10:30:22.248 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 27 k 2024-02-26T10:30:22.248 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 18 k 2024-02-26T10:30:22.248 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T10:30:22.248 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T10:30:22.248 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T10:30:22.248 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 31 k 2024-02-26T10:30:22.249 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 744 k 2024-02-26T10:30:22.249 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 127 k 2024-02-26T10:30:22.249 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T10:30:22.249 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T10:30:22.249 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T10:30:22.249 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T10:30:22.249 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T10:30:22.250 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T10:30:22.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 162 k 2024-02-26T10:30:22.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 320 k 2024-02-26T10:30:22.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 298 k 2024-02-26T10:30:22.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 100 k 2024-02-26T10:30:22.251 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 85 k 2024-02-26T10:30:22.251 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:30:22.251 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 170 k 2024-02-26T10:30:22.251 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T10:30:22.251 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-02-26T10:30:22.252 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T10:30:22.252 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T10:30:22.252 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T10:30:22.252 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T10:30:22.252 INFO:teuthology.orchestra.run.smithi100.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T10:30:22.252 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T10:30:22.253 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T10:30:22.253 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T10:30:22.253 INFO:teuthology.orchestra.run.smithi100.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T10:30:22.253 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T10:30:22.253 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.4 M 2024-02-26T10:30:22.260 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:30:22.260 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T10:30:22.260 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T10:30:22.261 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T10:30:22.261 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T10:30:22.261 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-26T10:30:22.261 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T10:30:22.261 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 20 M 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 31 k 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 2.1 M 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 248 k 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 4.7 M 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 17 M 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 15 k 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 25 k 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T10:30:22.262 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 163 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T10:30:22.263 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 501 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.1 M 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T10:30:22.264 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 45 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 131 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T10:30:22.265 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T10:30:22.266 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T10:30:22.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T10:30:22.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T10:30:22.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout:========================================================================================= 2024-02-26T10:30:22.270 INFO:teuthology.orchestra.run.smithi100.stdout:Install 150 Packages 2024-02-26T10:30:22.271 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 16 Packages 2024-02-26T10:30:22.271 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:30:22.272 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:30:22.272 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 222 M 2024-02-26T10:30:22.272 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-26T10:30:22.273 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:30:22.287 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T10:30:22.288 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T10:30:22.289 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T10:30:22.290 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T10:30:22.292 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T10:30:22.295 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T10:30:22.297 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T10:30:22.397 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-02-26T10:30:22.424 INFO:teuthology.orchestra.run.smithi060.stdout:========================================================================================= 2024-02-26T10:30:22.424 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-02-26T10:30:22.424 INFO:teuthology.orchestra.run.smithi060.stdout:========================================================================================= 2024-02-26T10:30:22.424 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-02-26T10:30:22.424 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T10:30:22.424 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 6.6 k 2024-02-26T10:30:22.424 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.3 M 2024-02-26T10:30:22.424 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 888 k 2024-02-26T10:30:22.424 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 145 k 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 1.5 M 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 138 k 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 1.8 M 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 7.4 M 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 49 k 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 9.8 M 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 47 M 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 264 k 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 748 k 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 220 k 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 27 k 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 18 k 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T10:30:22.425 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 31 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 744 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 127 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 162 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 320 k 2024-02-26T10:30:22.426 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 298 k 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 100 k 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 85 k 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 170 k 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T10:30:22.427 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.4 M 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T10:30:22.428 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 20 M 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 31 k 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 2.1 M 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 248 k 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 4.7 M 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 17 M 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 15 k 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 25 k 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T10:30:22.429 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 163 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 501 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.1 M 2024-02-26T10:30:22.430 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T10:30:22.431 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T10:30:22.432 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T10:30:22.432 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T10:30:22.432 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T10:30:22.432 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T10:30:22.433 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T10:30:22.433 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 45 k 2024-02-26T10:30:22.433 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 131 k 2024-02-26T10:30:22.433 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T10:30:22.433 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T10:30:22.433 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T10:30:22.433 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T10:30:22.434 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T10:30:22.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T10:30:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T10:30:22.437 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout:========================================================================================= 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout:Install 150 Packages 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 16 Packages 2024-02-26T10:30:22.438 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:30:22.439 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 222 M 2024-02-26T10:30:22.439 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-02-26T10:30:22.466 INFO:teuthology.orchestra.run.smithi113.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-02-26T10:30:23.000 INFO:teuthology.orchestra.run.smithi113.stdout:(151/166): NetworkManager-1.45.91-1.el9.x86_64. 4.3 MB/s | 2.3 MB 00:00 2024-02-26T10:30:23.304 INFO:teuthology.orchestra.run.smithi100.stdout:(1/166): ceph-19.0.0-1686.g6085b91f.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-02-26T10:30:23.500 INFO:teuthology.orchestra.run.smithi113.stdout:(152/166): librados2-19.0.0-1686.g6085b91f.el9. 2.3 MB/s | 3.4 MB 00:01 2024-02-26T10:30:23.642 INFO:teuthology.orchestra.run.smithi113.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 300 kB/s | 42 kB 00:00 2024-02-26T10:30:23.679 INFO:teuthology.orchestra.run.smithi100.stdout:(2/166): ceph-fuse-19.0.0-1686.g6085b91f.el9.x8 2.3 MB/s | 888 kB 00:00 2024-02-26T10:30:23.754 INFO:teuthology.orchestra.run.smithi100.stdout:(3/166): ceph-base-19.0.0-1686.g6085b91f.el9.x8 8.3 MB/s | 5.3 MB 00:00 2024-02-26T10:30:23.780 INFO:teuthology.orchestra.run.smithi100.stdout:(4/166): ceph-immutable-object-cache-19.0.0-168 1.4 MB/s | 145 kB 00:00 2024-02-26T10:30:23.809 INFO:teuthology.orchestra.run.smithi113.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 1.4 MB/s | 247 kB 00:00 2024-02-26T10:30:23.876 INFO:teuthology.orchestra.run.smithi113.stdout:(155/166): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-02-26T10:30:23.903 INFO:teuthology.orchestra.run.smithi060.stdout:(1/166): ceph-19.0.0-1686.g6085b91f.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-02-26T10:30:23.918 INFO:teuthology.orchestra.run.smithi113.stdout:(156/166): NetworkManager-libnm-1.45.91-1.el9.x 2.0 MB/s | 1.8 MB 00:00 2024-02-26T10:30:23.943 INFO:teuthology.orchestra.run.smithi113.stdout:(157/166): elfutils-debuginfod-client-0.190-2.e 545 kB/s | 36 kB 00:00 2024-02-26T10:30:23.988 INFO:teuthology.orchestra.run.smithi100.stdout:(5/166): ceph-mds-19.0.0-1686.g6085b91f.el9.x86 9.1 MB/s | 2.1 MB 00:00 2024-02-26T10:30:23.993 INFO:teuthology.orchestra.run.smithi113.stdout:(158/166): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-02-26T10:30:24.018 INFO:teuthology.orchestra.run.smithi113.stdout:(159/166): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 256 kB 00:00 2024-02-26T10:30:24.061 INFO:teuthology.orchestra.run.smithi113.stdout:(160/166): libacl-2.3.1-4.el9.x86_64.rpm 343 kB/s | 23 kB 00:00 2024-02-26T10:30:24.319 INFO:teuthology.orchestra.run.smithi113.stdout:(161/166): openssl-3.0.7-25.el9.x86_64.rpm 4.1 MB/s | 1.2 MB 00:00 2024-02-26T10:30:24.405 INFO:teuthology.orchestra.run.smithi100.stdout:(6/166): ceph-mgr-19.0.0-1686.g6085b91f.el9.x86 2.4 MB/s | 1.5 MB 00:00 2024-02-26T10:30:24.473 INFO:teuthology.orchestra.run.smithi100.stdout:(7/166): ceph-mon-19.0.0-1686.g6085b91f.el9.x86 9.7 MB/s | 4.7 MB 00:00 2024-02-26T10:30:24.503 INFO:teuthology.orchestra.run.smithi113.stdout:(162/166): librbd1-19.0.0-1686.g6085b91f.el9.x8 1.3 MB/s | 3.1 MB 00:02 2024-02-26T10:30:24.528 INFO:teuthology.orchestra.run.smithi113.stdout:(163/166): xfsdump-3.1.12-4.el9.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2024-02-26T10:30:24.746 INFO:teuthology.orchestra.run.smithi113.stdout:(164/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.4 MB/s | 1.1 MB 00:00 2024-02-26T10:30:24.912 INFO:teuthology.orchestra.run.smithi060.stdout:(2/166): ceph-fuse-19.0.0-1686.g6085b91f.el9.x8 882 kB/s | 888 kB 00:01 2024-02-26T10:30:25.004 INFO:teuthology.orchestra.run.smithi113.stdout:(165/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 156 kB/s | 74 kB 00:00 2024-02-26T10:30:25.062 INFO:teuthology.orchestra.run.smithi060.stdout:(3/166): ceph-immutable-object-cache-19.0.0-168 975 kB/s | 145 kB 00:00 2024-02-26T10:30:25.349 INFO:teuthology.orchestra.run.smithi100.stdout:(8/166): ceph-common-19.0.0-1686.g6085b91f.el9. 8.9 MB/s | 20 MB 00:02 2024-02-26T10:30:25.429 INFO:teuthology.orchestra.run.smithi113.stdout:(166/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 1.6 MB/s | 2.2 MB 00:01 2024-02-26T10:30:25.433 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-02-26T10:30:25.433 INFO:teuthology.orchestra.run.smithi113.stdout:Total 12 MB/s | 222 MB 00:18 2024-02-26T10:30:25.456 INFO:teuthology.orchestra.run.smithi100.stdout:(9/166): ceph-radosgw-19.0.0-1686.g6085b91f.el9 9.9 MB/s | 9.8 MB 00:00 2024-02-26T10:30:25.481 INFO:teuthology.orchestra.run.smithi100.stdout:(10/166): ceph-selinux-19.0.0-1686.g6085b91f.el 191 kB/s | 25 kB 00:00 2024-02-26T10:30:25.521 INFO:teuthology.orchestra.run.smithi060.stdout:(4/166): ceph-base-19.0.0-1686.g6085b91f.el9.x8 2.6 MB/s | 5.3 MB 00:02 2024-02-26T10:30:25.557 INFO:teuthology.orchestra.run.smithi100.stdout:(11/166): cephfs-mirror-19.0.0-1686.g6085b91f.e 2.9 MB/s | 220 kB 00:00 2024-02-26T10:30:25.648 INFO:teuthology.orchestra.run.smithi100.stdout:(12/166): cephfs-shell-19.0.0-1686.g6085b91f.el 295 kB/s | 27 kB 00:00 2024-02-26T10:30:25.732 INFO:teuthology.orchestra.run.smithi100.stdout:(13/166): libcephfs-devel-19.0.0-1686.g6085b91f 374 kB/s | 31 kB 00:00 2024-02-26T10:30:25.891 INFO:teuthology.orchestra.run.smithi100.stdout:(14/166): libcephfs2-19.0.0-1686.g6085b91f.el9. 4.6 MB/s | 744 kB 00:00 2024-02-26T10:30:25.974 INFO:teuthology.orchestra.run.smithi100.stdout:(15/166): libcephsqlite-19.0.0-1686.g6085b91f.e 1.9 MB/s | 163 kB 00:00 2024-02-26T10:30:26.004 INFO:teuthology.orchestra.run.smithi060.stdout:(5/166): ceph-mgr-19.0.0-1686.g6085b91f.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-02-26T10:30:26.049 INFO:teuthology.orchestra.run.smithi100.stdout:(16/166): librados-devel-19.0.0-1686.g6085b91f. 1.7 MB/s | 127 kB 00:00 2024-02-26T10:30:26.150 INFO:teuthology.orchestra.run.smithi100.stdout:(17/166): libradosstriper1-19.0.0-1686.g6085b91 4.9 MB/s | 501 kB 00:00 2024-02-26T10:30:26.263 INFO:teuthology.orchestra.run.smithi060.stdout:(6/166): ceph-mds-19.0.0-1686.g6085b91f.el9.x86 1.8 MB/s | 2.1 MB 00:01 2024-02-26T10:30:26.684 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-02-26T10:30:26.925 INFO:teuthology.orchestra.run.smithi100.stdout:(18/166): librgw2-19.0.0-1686.g6085b91f.el9.x86 6.6 MB/s | 5.1 MB 00:00 2024-02-26T10:30:26.943 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-02-26T10:30:26.943 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-02-26T10:30:27.242 INFO:teuthology.orchestra.run.smithi100.stdout:(19/166): python3-ceph-argparse-19.0.0-1686.g60 142 kB/s | 45 kB 00:00 2024-02-26T10:30:27.317 INFO:teuthology.orchestra.run.smithi100.stdout:(20/166): python3-ceph-common-19.0.0-1686.g6085 1.7 MB/s | 131 kB 00:00 2024-02-26T10:30:27.392 INFO:teuthology.orchestra.run.smithi100.stdout:(21/166): python3-cephfs-19.0.0-1686.g6085b91f. 2.1 MB/s | 162 kB 00:00 2024-02-26T10:30:27.438 INFO:teuthology.orchestra.run.smithi060.stdout:(7/166): ceph-mon-19.0.0-1686.g6085b91f.el9.x86 3.3 MB/s | 4.7 MB 00:01 2024-02-26T10:30:27.476 INFO:teuthology.orchestra.run.smithi100.stdout:(22/166): python3-rados-19.0.0-1686.g6085b91f.e 3.7 MB/s | 320 kB 00:00 2024-02-26T10:30:27.551 INFO:teuthology.orchestra.run.smithi100.stdout:(23/166): python3-rbd-19.0.0-1686.g6085b91f.el9 3.9 MB/s | 298 kB 00:00 2024-02-26T10:30:27.610 INFO:teuthology.orchestra.run.smithi100.stdout:(24/166): python3-rgw-19.0.0-1686.g6085b91f.el9 1.7 MB/s | 100 kB 00:00 2024-02-26T10:30:27.694 INFO:teuthology.orchestra.run.smithi100.stdout:(25/166): rbd-fuse-19.0.0-1686.g6085b91f.el9.x8 1.0 MB/s | 85 kB 00:00 2024-02-26T10:30:28.194 INFO:teuthology.orchestra.run.smithi100.stdout:(26/166): rbd-mirror-19.0.0-1686.g6085b91f.el9. 6.2 MB/s | 3.1 MB 00:00 2024-02-26T10:30:28.277 INFO:teuthology.orchestra.run.smithi100.stdout:(27/166): rbd-nbd-19.0.0-1686.g6085b91f.el9.x86 2.0 MB/s | 170 kB 00:00 2024-02-26T10:30:28.361 INFO:teuthology.orchestra.run.smithi100.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1686.g 377 kB/s | 31 kB 00:00 2024-02-26T10:30:28.912 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-02-26T10:30:28.912 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-02-26T10:30:28.995 INFO:teuthology.orchestra.run.smithi100.stdout:(29/166): ceph-test-19.0.0-1686.g6085b91f.el9.x 13 MB/s | 47 MB 00:03 2024-02-26T10:30:29.020 INFO:teuthology.orchestra.run.smithi100.stdout:(30/166): ceph-mgr-cephadm-19.0.0-1686.g6085b91 209 kB/s | 138 kB 00:00 2024-02-26T10:30:29.204 INFO:teuthology.orchestra.run.smithi100.stdout:(31/166): ceph-mgr-dashboard-19.0.0-1686.g6085b 8.6 MB/s | 1.8 MB 00:00 2024-02-26T10:30:29.287 INFO:teuthology.orchestra.run.smithi100.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1686.g60 2.9 MB/s | 248 kB 00:00 2024-02-26T10:30:29.355 INFO:teuthology.orchestra.run.smithi100.stdout:(33/166): ceph-mgr-rook-19.0.0-1686.g6085b91f.e 740 kB/s | 49 kB 00:00 2024-02-26T10:30:29.646 INFO:teuthology.orchestra.run.smithi060.stdout:(8/166): ceph-osd-19.0.0-1686.g6085b91f.el9.x86 5.0 MB/s | 17 MB 00:03 2024-02-26T10:30:29.713 INFO:teuthology.orchestra.run.smithi100.stdout:(34/166): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-02-26T10:30:29.721 INFO:teuthology.orchestra.run.smithi060.stdout:(9/166): ceph-selinux-19.0.0-1686.g6085b91f.el9 341 kB/s | 25 kB 00:00 2024-02-26T10:30:29.738 INFO:teuthology.orchestra.run.smithi100.stdout:(35/166): ceph-prometheus-alerts-19.0.0-1686.g6 40 kB/s | 15 kB 00:00 2024-02-26T10:30:29.780 INFO:teuthology.orchestra.run.smithi100.stdout:(36/166): ceph-volume-19.0.0-1686.g6085b91f.el9 3.9 MB/s | 264 kB 00:00 2024-02-26T10:30:29.831 INFO:teuthology.orchestra.run.smithi100.stdout:(37/166): cephfs-top-19.0.0-1686.g6085b91f.el9. 367 kB/s | 18 kB 00:00 2024-02-26T10:30:29.864 INFO:teuthology.orchestra.run.smithi100.stdout:(38/166): cephadm-19.0.0-1686.g6085b91f.el9.noa 5.8 MB/s | 748 kB 00:00 2024-02-26T10:30:29.998 INFO:teuthology.orchestra.run.smithi100.stdout:(39/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 105 kB/s | 18 kB 00:00 2024-02-26T10:30:30.106 INFO:teuthology.orchestra.run.smithi100.stdout:(40/166): fuse-2.9.9-15.el9.x86_64.rpm 372 kB/s | 80 kB 00:00 2024-02-26T10:30:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:(41/166): ceph-osd-19.0.0-1686.g6085b91f.el9.x8 2.9 MB/s | 17 MB 00:05 2024-02-26T10:30:30.322 INFO:teuthology.orchestra.run.smithi060.stdout:(10/166): ceph-radosgw-19.0.0-1686.g6085b91f.el 3.4 MB/s | 9.8 MB 00:02 2024-02-26T10:30:30.398 INFO:teuthology.orchestra.run.smithi100.stdout:(42/166): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-02-26T10:30:30.414 INFO:teuthology.orchestra.run.smithi060.stdout:(11/166): cephfs-mirror-19.0.0-1686.g6085b91f.e 2.4 MB/s | 220 kB 00:00 2024-02-26T10:30:30.432 INFO:teuthology.orchestra.run.smithi100.stdout:(43/166): jq-1.6-16.el9.x86_64.rpm 431 kB/s | 187 kB 00:00 2024-02-26T10:30:30.464 INFO:teuthology.orchestra.run.smithi060.stdout:(12/166): cephfs-shell-19.0.0-1686.g6085b91f.el 554 kB/s | 27 kB 00:00 2024-02-26T10:30:30.474 INFO:teuthology.orchestra.run.smithi100.stdout:(44/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 218 kB 00:00 2024-02-26T10:30:30.514 INFO:teuthology.orchestra.run.smithi060.stdout:(13/166): libcephfs-devel-19.0.0-1686.g6085b91f 630 kB/s | 31 kB 00:00 2024-02-26T10:30:30.574 INFO:teuthology.orchestra.run.smithi100.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-02-26T10:30:30.599 INFO:teuthology.orchestra.run.smithi100.stdout:(46/166): libquadmath-11.4.1-3.el9.x86_64.rpm 616 kB/s | 195 kB 00:00 2024-02-26T10:30:30.650 INFO:teuthology.orchestra.run.smithi100.stdout:(47/166): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-02-26T10:30:30.666 INFO:teuthology.orchestra.run.smithi100.stdout:(48/166): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-02-26T10:30:30.692 INFO:teuthology.orchestra.run.smithi100.stdout:(49/166): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-02-26T10:30:30.725 INFO:teuthology.orchestra.run.smithi100.stdout:(50/166): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-02-26T10:30:30.731 INFO:teuthology.orchestra.run.smithi060.stdout:(14/166): libcephfs2-19.0.0-1686.g6085b91f.el9. 3.4 MB/s | 744 kB 00:00 2024-02-26T10:30:30.792 INFO:teuthology.orchestra.run.smithi100.stdout:(51/166): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-02-26T10:30:30.816 INFO:teuthology.orchestra.run.smithi060.stdout:(15/166): libcephsqlite-19.0.0-1686.g6085b91f.e 1.9 MB/s | 163 kB 00:00 2024-02-26T10:30:30.874 INFO:teuthology.orchestra.run.smithi060.stdout:(16/166): librados-devel-19.0.0-1686.g6085b91f. 2.1 MB/s | 127 kB 00:00 2024-02-26T10:30:30.984 INFO:teuthology.orchestra.run.smithi100.stdout:(52/166): boost-program-options-1.75.0-8.el9.x8 558 kB/s | 107 kB 00:00 2024-02-26T10:30:31.068 INFO:teuthology.orchestra.run.smithi100.stdout:(53/166): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-02-26T10:30:31.093 INFO:teuthology.orchestra.run.smithi100.stdout:(54/166): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-02-26T10:30:31.152 INFO:teuthology.orchestra.run.smithi100.stdout:(55/166): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-02-26T10:30:31.177 INFO:teuthology.orchestra.run.smithi100.stdout:(56/166): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-02-26T10:30:31.205 INFO:teuthology.orchestra.run.smithi100.stdout:(57/166): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-02-26T10:30:31.247 INFO:teuthology.orchestra.run.smithi100.stdout:(58/166): flexiblas-openblas-openmp-3.0.4-8.el9 260 kB/s | 18 kB 00:00 2024-02-26T10:30:31.289 INFO:teuthology.orchestra.run.smithi100.stdout:(59/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-02-26T10:30:31.323 INFO:teuthology.orchestra.run.smithi100.stdout:(60/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-02-26T10:30:31.499 INFO:teuthology.orchestra.run.smithi100.stdout:(61/166): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-02-26T10:30:31.524 INFO:teuthology.orchestra.run.smithi100.stdout:(62/166): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-02-26T10:30:31.582 INFO:teuthology.orchestra.run.smithi060.stdout:(17/166): libradosstriper1-19.0.0-1686.g6085b91 708 kB/s | 501 kB 00:00 2024-02-26T10:30:31.583 INFO:teuthology.orchestra.run.smithi100.stdout:(63/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 306 kB/s | 26 kB 00:00 2024-02-26T10:30:31.608 INFO:teuthology.orchestra.run.smithi100.stdout:(64/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-02-26T10:30:31.700 INFO:teuthology.orchestra.run.smithi100.stdout:(65/166): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-02-26T10:30:31.900 INFO:teuthology.orchestra.run.smithi100.stdout:(66/166): openblas-0.3.21-2.el9.x86_64.rpm 176 kB/s | 35 kB 00:00 2024-02-26T10:30:31.933 INFO:teuthology.orchestra.run.smithi100.stdout:(67/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 834 kB/s | 292 kB 00:00 2024-02-26T10:30:32.084 INFO:teuthology.orchestra.run.smithi100.stdout:(68/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-02-26T10:30:32.393 INFO:teuthology.orchestra.run.smithi100.stdout:(69/166): postgresql-13.11-1.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-02-26T10:30:32.635 INFO:teuthology.orchestra.run.smithi100.stdout:(70/166): postgresql-contrib-13.11-1.el9.x86_64 3.7 MB/s | 905 kB 00:00 2024-02-26T10:30:32.701 INFO:teuthology.orchestra.run.smithi100.stdout:(71/166): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 135 kB 00:00 2024-02-26T10:30:32.808 INFO:teuthology.orchestra.run.smithi060.stdout:(18/166): librgw2-19.0.0-1686.g6085b91f.el9.x86 4.2 MB/s | 5.1 MB 00:01 2024-02-26T10:30:32.860 INFO:teuthology.orchestra.run.smithi100.stdout:(72/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 4.5 MB/s | 4.1 MB 00:00 2024-02-26T10:30:32.867 INFO:teuthology.orchestra.run.smithi060.stdout:(19/166): python3-ceph-argparse-19.0.0-1686.g60 788 kB/s | 45 kB 00:00 2024-02-26T10:30:33.069 INFO:teuthology.orchestra.run.smithi100.stdout:(73/166): openblas-openmp-0.3.21-2.el9.x86_64.r 4.1 MB/s | 4.7 MB 00:01 2024-02-26T10:30:33.119 INFO:teuthology.orchestra.run.smithi100.stdout:(74/166): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-02-26T10:30:33.144 INFO:teuthology.orchestra.run.smithi100.stdout:(75/166): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-02-26T10:30:33.200 INFO:teuthology.orchestra.run.smithi060.stdout:(20/166): python3-ceph-common-19.0.0-1686.g6085 395 kB/s | 131 kB 00:00 2024-02-26T10:30:33.252 INFO:teuthology.orchestra.run.smithi100.stdout:(76/166): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-02-26T10:30:33.353 INFO:teuthology.orchestra.run.smithi100.stdout:(77/166): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-02-26T10:30:33.412 INFO:teuthology.orchestra.run.smithi100.stdout:(78/166): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-02-26T10:30:33.470 INFO:teuthology.orchestra.run.smithi100.stdout:(79/166): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-02-26T10:30:33.529 INFO:teuthology.orchestra.run.smithi100.stdout:(80/166): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-02-26T10:30:33.595 INFO:teuthology.orchestra.run.smithi100.stdout:(81/166): python3-markupsafe-1.1.1-12.el9.x86_6 523 kB/s | 35 kB 00:00 2024-02-26T10:30:33.608 INFO:teuthology.orchestra.run.smithi060.stdout:(21/166): python3-cephfs-19.0.0-1686.g6085b91f. 397 kB/s | 162 kB 00:00 2024-02-26T10:30:33.865 INFO:teuthology.orchestra.run.smithi100.stdout:(82/166): postgresql-server-13.11-1.el9.x86_64. 4.9 MB/s | 5.7 MB 00:01 2024-02-26T10:30:33.938 INFO:teuthology.orchestra.run.smithi100.stdout:(83/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-02-26T10:30:33.996 INFO:teuthology.orchestra.run.smithi100.stdout:(84/166): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-02-26T10:30:34.055 INFO:teuthology.orchestra.run.smithi100.stdout:(85/166): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-02-26T10:30:34.122 INFO:teuthology.orchestra.run.smithi100.stdout:(86/166): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-02-26T10:30:34.247 INFO:teuthology.orchestra.run.smithi100.stdout:(87/166): python3-babel-2.9.1-2.el9.noarch.rpm 5.3 MB/s | 6.0 MB 00:01 2024-02-26T10:30:34.272 INFO:teuthology.orchestra.run.smithi100.stdout:(88/166): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-02-26T10:30:34.275 INFO:teuthology.orchestra.run.smithi060.stdout:(22/166): python3-rados-19.0.0-1686.g6085b91f.e 480 kB/s | 320 kB 00:00 2024-02-26T10:30:34.323 INFO:teuthology.orchestra.run.smithi100.stdout:(89/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-02-26T10:30:34.348 INFO:teuthology.orchestra.run.smithi100.stdout:(90/166): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-02-26T10:30:34.382 INFO:teuthology.orchestra.run.smithi100.stdout:(91/166): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-02-26T10:30:34.392 INFO:teuthology.orchestra.run.smithi060.stdout:(23/166): python3-rbd-19.0.0-1686.g6085b91f.el9 2.5 MB/s | 298 kB 00:00 2024-02-26T10:30:34.407 INFO:teuthology.orchestra.run.smithi100.stdout:(92/166): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-02-26T10:30:34.451 INFO:teuthology.orchestra.run.smithi060.stdout:(24/166): python3-rgw-19.0.0-1686.g6085b91f.el9 1.7 MB/s | 100 kB 00:00 2024-02-26T10:30:34.476 INFO:teuthology.orchestra.run.smithi100.stdout:(93/166): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-02-26T10:30:34.510 INFO:teuthology.orchestra.run.smithi060.stdout:(25/166): rbd-fuse-19.0.0-1686.g6085b91f.el9.x8 1.4 MB/s | 85 kB 00:00 2024-02-26T10:30:34.541 INFO:teuthology.orchestra.run.smithi100.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 665 kB/s | 44 kB 00:00 2024-02-26T10:30:34.616 INFO:teuthology.orchestra.run.smithi100.stdout:(95/166): qatlib-23.11.0-1.el9.x86_64.rpm 2.8 MB/s | 216 kB 00:00 2024-02-26T10:30:34.683 INFO:teuthology.orchestra.run.smithi100.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 523 kB/s | 35 kB 00:00 2024-02-26T10:30:34.750 INFO:teuthology.orchestra.run.smithi100.stdout:(97/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 669 kB/s | 44 kB 00:00 2024-02-26T10:30:34.826 INFO:teuthology.orchestra.run.smithi100.stdout:(98/166): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-02-26T10:30:34.892 INFO:teuthology.orchestra.run.smithi100.stdout:(99/166): uuid-1.6.2-55.el9.x86_64.rpm 866 kB/s | 58 kB 00:00 2024-02-26T10:30:34.968 INFO:teuthology.orchestra.run.smithi100.stdout:(100/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-02-26T10:30:35.043 INFO:teuthology.orchestra.run.smithi100.stdout:(101/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 850 kB/s | 64 kB 00:00 2024-02-26T10:30:35.127 INFO:teuthology.orchestra.run.smithi100.stdout:(102/166): python3-numpy-1.20.1-5.el9.x86_64.rp 3.3 MB/s | 5.1 MB 00:01 2024-02-26T10:30:35.144 INFO:teuthology.orchestra.run.smithi100.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-02-26T10:30:35.368 INFO:teuthology.orchestra.run.smithi060.stdout:(26/166): rbd-mirror-19.0.0-1686.g6085b91f.el9. 3.6 MB/s | 3.1 MB 00:00 2024-02-26T10:30:35.427 INFO:teuthology.orchestra.run.smithi060.stdout:(27/166): rbd-nbd-19.0.0-1686.g6085b91f.el9.x86 2.9 MB/s | 170 kB 00:00 2024-02-26T10:30:35.477 INFO:teuthology.orchestra.run.smithi060.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1686.g 638 kB/s | 31 kB 00:00 2024-02-26T10:30:35.502 INFO:teuthology.orchestra.run.smithi100.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 59 kB/s | 22 kB 00:00 2024-02-26T10:30:35.536 INFO:teuthology.orchestra.run.smithi060.stdout:(29/166): ceph-mgr-cephadm-19.0.0-1686.g6085b91 2.3 MB/s | 138 kB 00:00 2024-02-26T10:30:35.600 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-02-26T10:30:35.702 INFO:teuthology.orchestra.run.smithi100.stdout:(105/166): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-02-26T10:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T10:30:35.762 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T10:30:35.815 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T10:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T10:30:35.968 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T10:30:36.009 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T10:30:36.036 INFO:teuthology.orchestra.run.smithi100.stdout:(106/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.8 MB/s | 16 MB 00:01 2024-02-26T10:30:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:(107/166): abseil-cpp-20211102.0-3.el9.x86_64.r 843 kB/s | 548 kB 00:00 2024-02-26T10:30:36.278 INFO:teuthology.orchestra.run.smithi100.stdout:(108/166): gperftools-libs-2.9.1-2.el9.x86_64.r 537 kB/s | 309 kB 00:00 2024-02-26T10:30:36.303 INFO:teuthology.orchestra.run.smithi060.stdout:(30/166): ceph-mgr-dashboard-19.0.0-1686.g6085b 2.3 MB/s | 1.8 MB 00:00 2024-02-26T10:30:36.312 INFO:teuthology.orchestra.run.smithi100.stdout:(109/166): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-02-26T10:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T10:30:36.387 INFO:teuthology.orchestra.run.smithi100.stdout:(110/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 237 kB/s | 26 kB 00:00 2024-02-26T10:30:36.468 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:30:36.505 INFO:teuthology.orchestra.run.smithi100.stdout:(111/166): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-02-26T10:30:36.538 INFO:teuthology.orchestra.run.smithi100.stdout:(112/166): liboath-2.6.7-2.el9.x86_64.rpm 215 kB/s | 49 kB 00:00 2024-02-26T10:30:36.663 INFO:teuthology.orchestra.run.smithi100.stdout:(113/166): libarrow-9.0.0-10.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-02-26T10:30:36.680 INFO:teuthology.orchestra.run.smithi100.stdout:(114/166): luarocks-3.9.2-1.el9.noarch.rpm 862 kB/s | 151 kB 00:00 2024-02-26T10:30:36.732 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:30:36.760 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:30:36.781 INFO:teuthology.orchestra.run.smithi100.stdout:(115/166): python3-asyncssh-2.13.2-4.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-02-26T10:30:36.806 INFO:teuthology.orchestra.run.smithi100.stdout:(116/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 348 kB/s | 43 kB 00:00 2024-02-26T10:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:30:36.889 INFO:teuthology.orchestra.run.smithi100.stdout:(117/166): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-02-26T10:30:36.907 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:30:36.914 INFO:teuthology.orchestra.run.smithi100.stdout:(118/166): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-02-26T10:30:36.945 INFO:teuthology.orchestra.run.smithi060.stdout:(31/166): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-02-26T10:30:36.996 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T10:30:36.998 INFO:teuthology.orchestra.run.smithi100.stdout:(119/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.8 MB/s | 838 kB 00:00 2024-02-26T10:30:37.003 INFO:teuthology.orchestra.run.smithi060.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1686.g60 4.2 MB/s | 248 kB 00:00 2024-02-26T10:30:37.023 INFO:teuthology.orchestra.run.smithi100.stdout:(120/166): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-02-26T10:30:37.054 INFO:teuthology.orchestra.run.smithi060.stdout:(33/166): ceph-mgr-rook-19.0.0-1686.g6085b91f.e 997 kB/s | 49 kB 00:00 2024-02-26T10:30:37.091 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T10:30:37.104 INFO:teuthology.orchestra.run.smithi060.stdout:(34/166): ceph-prometheus-alerts-19.0.0-1686.g6 310 kB/s | 15 kB 00:00 2024-02-26T10:30:37.132 INFO:teuthology.orchestra.run.smithi100.stdout:(121/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-02-26T10:30:37.158 INFO:teuthology.orchestra.run.smithi100.stdout:(122/166): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-02-26T10:30:37.183 INFO:teuthology.orchestra.run.smithi100.stdout:(123/166): python3-colorama-0.4.4-12.el9.noarch 222 kB/s | 35 kB 00:00 2024-02-26T10:30:37.196 INFO:teuthology.orchestra.run.smithi060.stdout:(35/166): ceph-volume-19.0.0-1686.g6085b91f.el9 2.8 MB/s | 264 kB 00:00 2024-02-26T10:30:37.258 INFO:teuthology.orchestra.run.smithi100.stdout:(124/166): python3-google-auth-2.28.0-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-02-26T10:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T10:30:37.296 INFO:teuthology.orchestra.run.smithi060.stdout:(36/166): cephadm-19.0.0-1686.g6085b91f.el9.noa 7.4 MB/s | 748 kB 00:00 2024-02-26T10:30:37.323 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T10:30:37.342 INFO:teuthology.orchestra.run.smithi100.stdout:(125/166): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-02-26T10:30:37.351 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:30:37.367 INFO:teuthology.orchestra.run.smithi100.stdout:(126/166): python3-grpcio-tools-1.46.7-10.el9.x 785 kB/s | 144 kB 00:00 2024-02-26T10:30:37.392 INFO:teuthology.orchestra.run.smithi100.stdout:(127/166): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-02-26T10:30:37.455 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:30:37.459 INFO:teuthology.orchestra.run.smithi100.stdout:(128/166): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-02-26T10:30:37.478 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:30:37.485 INFO:teuthology.orchestra.run.smithi100.stdout:(129/166): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-02-26T10:30:37.510 INFO:teuthology.orchestra.run.smithi100.stdout:(130/166): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-02-26T10:30:37.540 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:30:37.562 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:30:37.577 INFO:teuthology.orchestra.run.smithi100.stdout:(131/166): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-02-26T10:30:37.578 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:30:37.645 INFO:teuthology.orchestra.run.smithi100.stdout:(132/166): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-02-26T10:30:37.686 INFO:teuthology.orchestra.run.smithi100.stdout:(133/166): python3-more-itertools-8.12.0-2.el9. 726 kB/s | 79 kB 00:00 2024-02-26T10:30:37.720 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:30:37.750 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:30:37.753 INFO:teuthology.orchestra.run.smithi100.stdout:(134/166): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-02-26T10:30:37.787 INFO:teuthology.orchestra.run.smithi100.stdout:(135/166): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-02-26T10:30:37.812 INFO:teuthology.orchestra.run.smithi100.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-02-26T10:30:37.853 INFO:teuthology.orchestra.run.smithi100.stdout:(137/166): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-02-26T10:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T10:30:37.895 INFO:teuthology.orchestra.run.smithi100.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-02-26T10:30:37.906 INFO:teuthology.orchestra.run.smithi060.stdout:(37/166): ceph-test-19.0.0-1686.g6085b91f.el9.x 5.8 MB/s | 47 MB 00:08 2024-02-26T10:30:37.920 INFO:teuthology.orchestra.run.smithi100.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-02-26T10:30:37.943 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:30:37.947 INFO:teuthology.orchestra.run.smithi060.stdout:(38/166): cephfs-top-19.0.0-1686.g6085b91f.el9. 28 kB/s | 18 kB 00:00 2024-02-26T10:30:37.954 INFO:teuthology.orchestra.run.smithi100.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-02-26T10:30:38.004 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:30:38.013 INFO:teuthology.orchestra.run.smithi100.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-02-26T10:30:38.038 INFO:teuthology.orchestra.run.smithi100.stdout:(142/166): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-02-26T10:30:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-02-26T10:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:30:38.130 INFO:teuthology.orchestra.run.smithi100.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-02-26T10:30:38.130 INFO:teuthology.orchestra.run.smithi060.stdout:(39/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 78 kB/s | 18 kB 00:00 2024-02-26T10:30:38.138 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:30:38.155 INFO:teuthology.orchestra.run.smithi100.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-02-26T10:30:38.189 INFO:teuthology.orchestra.run.smithi100.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 715 kB/s | 90 kB 00:00 2024-02-26T10:30:38.273 INFO:teuthology.orchestra.run.smithi100.stdout:(147/166): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-02-26T10:30:38.298 INFO:teuthology.orchestra.run.smithi100.stdout:(148/166): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-02-26T10:30:38.323 INFO:teuthology.orchestra.run.smithi100.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-02-26T10:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:30:38.364 INFO:teuthology.orchestra.run.smithi060.stdout:(40/166): fuse-2.9.9-15.el9.x86_64.rpm 193 kB/s | 80 kB 00:00 2024-02-26T10:30:38.448 INFO:teuthology.orchestra.run.smithi100.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-02-26T10:30:38.451 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T10:30:38.546 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T10:30:38.576 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T10:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T10:30:38.683 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T10:30:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T10:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T10:30:38.812 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T10:30:38.839 INFO:teuthology.orchestra.run.smithi060.stdout:(41/166): jq-1.6-16.el9.x86_64.rpm 264 kB/s | 187 kB 00:00 2024-02-26T10:30:38.848 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T10:30:38.907 INFO:teuthology.orchestra.run.smithi100.stdout:(151/166): NetworkManager-1.45.91-1.el9.x86_64. 5.0 MB/s | 2.3 MB 00:00 2024-02-26T10:30:38.915 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T10:30:38.970 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T10:30:38.999 INFO:teuthology.orchestra.run.smithi100.stdout:(152/166): NetworkManager-libnm-1.45.91-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-02-26T10:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T10:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T10:30:39.049 INFO:teuthology.orchestra.run.smithi100.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 848 kB/s | 42 kB 00:00 2024-02-26T10:30:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:(42/166): libquadmath-11.4.1-3.el9.x86_64.rpm 784 kB/s | 195 kB 00:00 2024-02-26T10:30:39.108 INFO:teuthology.orchestra.run.smithi100.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 4.1 MB/s | 247 kB 00:00 2024-02-26T10:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T10:30:39.155 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 36/182 2024-02-26T10:30:39.167 INFO:teuthology.orchestra.run.smithi100.stdout:(155/166): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-02-26T10:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 37/182 2024-02-26T10:30:39.208 INFO:teuthology.orchestra.run.smithi100.stdout:(156/166): elfutils-debuginfod-client-0.190-2.e 874 kB/s | 36 kB 00:00 2024-02-26T10:30:39.236 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 38/182 2024-02-26T10:30:39.267 INFO:teuthology.orchestra.run.smithi100.stdout:(157/166): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-02-26T10:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T10:30:39.315 INFO:teuthology.orchestra.run.smithi060.stdout:(43/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 972 kB/s | 218 kB 00:00 2024-02-26T10:30:39.326 INFO:teuthology.orchestra.run.smithi100.stdout:(158/166): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-02-26T10:30:39.359 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T10:30:39.367 INFO:teuthology.orchestra.run.smithi100.stdout:(159/166): libacl-2.3.1-4.el9.x86_64.rpm 553 kB/s | 23 kB 00:00 2024-02-26T10:30:39.391 INFO:teuthology.orchestra.run.smithi060.stdout:(44/166): libgfortran-11.4.1-3.el9.x86_64.rpm 783 kB/s | 803 kB 00:01 2024-02-26T10:30:39.432 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T10:30:39.468 INFO:teuthology.orchestra.run.smithi100.stdout:(160/166): openssl-3.0.7-25.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-02-26T10:30:39.541 INFO:teuthology.orchestra.run.smithi060.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-02-26T10:30:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T10:30:39.576 INFO:teuthology.orchestra.run.smithi100.stdout:(161/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 20 MB/s | 2.2 MB 00:00 2024-02-26T10:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T10:30:39.626 INFO:teuthology.orchestra.run.smithi100.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2024-02-26T10:30:39.649 INFO:teuthology.orchestra.run.smithi060.stdout:(46/166): python3-ply-3.11-14.el9.noarch.rpm 991 kB/s | 106 kB 00:00 2024-02-26T10:30:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:(47/166): python3-pycparser-2.20-6.el9.noarch.r 707 kB/s | 135 kB 00:00 2024-02-26T10:30:39.868 INFO:teuthology.orchestra.run.smithi100.stdout:(163/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.4 MB/s | 1.1 MB 00:00 2024-02-26T10:30:39.977 INFO:teuthology.orchestra.run.smithi100.stdout:(164/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 686 kB/s | 74 kB 00:00 2024-02-26T10:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T10:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T10:30:40.251 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T10:30:40.266 INFO:teuthology.orchestra.run.smithi060.stdout:(48/166): python3-requests-2.25.1-8.el9.noarch. 294 kB/s | 125 kB 00:00 2024-02-26T10:30:40.280 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T10:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T10:30:40.367 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 49/182 2024-02-26T10:30:40.418 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T10:30:40.550 INFO:teuthology.orchestra.run.smithi060.stdout:(49/166): python3-urllib3-1.26.5-5.el9.noarch.r 759 kB/s | 215 kB 00:00 2024-02-26T10:30:40.552 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T10:30:40.621 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T10:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T10:30:40.748 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T10:30:40.780 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T10:30:40.816 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T10:30:40.819 INFO:teuthology.orchestra.run.smithi100.stdout:(165/166): librados2-19.0.0-1686.g6085b91f.el9. 1.3 MB/s | 3.4 MB 00:02 2024-02-26T10:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T10:30:40.917 INFO:teuthology.orchestra.run.smithi060.stdout:(50/166): python3-cryptography-36.0.1-4.el9.x86 823 kB/s | 1.2 MB 00:01 2024-02-26T10:30:40.932 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T10:30:40.959 INFO:teuthology.orchestra.run.smithi060.stdout:(51/166): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 689 kB 00:00 2024-02-26T10:30:40.975 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T10:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T10:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T10:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T10:30:41.218 INFO:teuthology.orchestra.run.smithi060.stdout:(52/166): ceph-common-19.0.0-1686.g6085b91f.el9 1.1 MB/s | 20 MB 00:17 2024-02-26T10:30:41.220 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T10:30:41.243 INFO:teuthology.orchestra.run.smithi060.stdout:(53/166): boost-program-options-1.75.0-8.el9.x8 378 kB/s | 107 kB 00:00 2024-02-26T10:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T10:30:41.351 INFO:teuthology.orchestra.run.smithi060.stdout:(54/166): elfutils-libelf-devel-0.190-2.el9.x86 162 kB/s | 21 kB 00:00 2024-02-26T10:30:41.358 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T10:30:41.399 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T10:30:41.410 INFO:teuthology.orchestra.run.smithi060.stdout:(55/166): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-02-26T10:30:41.435 INFO:teuthology.orchestra.run.smithi060.stdout:(56/166): flexiblas-3.0.4-8.el9.x86_64.rpm 394 kB/s | 33 kB 00:00 2024-02-26T10:30:41.439 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T10:30:41.464 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T10:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T10:30:41.494 INFO:teuthology.orchestra.run.smithi060.stdout:(57/166): bison-3.7.4-5.el9.x86_64.rpm 1.6 MB/s | 943 kB 00:00 2024-02-26T10:30:41.511 INFO:teuthology.orchestra.run.smithi060.stdout:(58/166): flexiblas-openblas-openmp-3.0.4-8.el9 244 kB/s | 18 kB 00:00 2024-02-26T10:30:41.544 INFO:teuthology.orchestra.run.smithi060.stdout:(59/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 160 kB 00:00 2024-02-26T10:30:41.562 INFO:teuthology.orchestra.run.smithi060.stdout:(60/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 913 kB/s | 45 kB 00:00 2024-02-26T10:30:41.570 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T10:30:41.678 INFO:teuthology.orchestra.run.smithi060.stdout:(61/166): librdkafka-1.6.1-102.el9.x86_64.rpm 4.9 MB/s | 662 kB 00:00 2024-02-26T10:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T10:30:41.729 INFO:teuthology.orchestra.run.smithi060.stdout:(62/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 520 kB/s | 26 kB 00:00 2024-02-26T10:30:41.738 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T10:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T10:30:41.781 INFO:teuthology.orchestra.run.smithi060.stdout:(63/166): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-02-26T10:30:41.800 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T10:30:41.804 INFO:teuthology.orchestra.run.smithi060.stdout:(64/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 635 kB/s | 47 kB 00:00 2024-02-26T10:30:41.840 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T10:30:41.846 INFO:teuthology.orchestra.run.smithi060.stdout:(65/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.5 MB/s | 292 kB 00:00 2024-02-26T10:30:41.871 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T10:30:41.871 INFO:teuthology.orchestra.run.smithi060.stdout:(66/166): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-02-26T10:30:41.897 INFO:teuthology.orchestra.run.smithi060.stdout:(67/166): openblas-0.3.21-2.el9.x86_64.rpm 713 kB/s | 35 kB 00:00 2024-02-26T10:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T10:30:41.936 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T10:30:41.978 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T10:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T10:30:42.070 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T10:30:42.103 INFO:teuthology.orchestra.run.smithi100.stdout:(166/166): librbd1-19.0.0-1686.g6085b91f.el9.x8 849 kB/s | 3.1 MB 00:03 2024-02-26T10:30:42.108 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-26T10:30:42.108 INFO:teuthology.orchestra.run.smithi100.stdout:Total 11 MB/s | 222 MB 00:19 2024-02-26T10:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T10:30:42.201 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T10:30:42.232 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T10:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T10:30:42.378 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T10:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T10:30:42.464 INFO:teuthology.orchestra.run.smithi060.stdout:(68/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 7.3 MB/s | 4.1 MB 00:00 2024-02-26T10:30:42.551 INFO:teuthology.orchestra.run.smithi060.stdout:(69/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-02-26T10:30:42.639 INFO:teuthology.orchestra.run.smithi060.stdout:(70/166): postgresql-13.11-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-02-26T10:30:42.723 INFO:teuthology.orchestra.run.smithi060.stdout:(71/166): postgresql-private-libs-13.11-1.el9.x 1.6 MB/s | 135 kB 00:00 2024-02-26T10:30:42.807 INFO:teuthology.orchestra.run.smithi060.stdout:(72/166): openblas-openmp-0.3.21-2.el9.x86_64.r 5.1 MB/s | 4.7 MB 00:00 2024-02-26T10:30:42.874 INFO:teuthology.orchestra.run.smithi060.stdout:(73/166): postgresql-contrib-13.11-1.el9.x86_64 2.7 MB/s | 905 kB 00:00 2024-02-26T10:30:42.916 INFO:teuthology.orchestra.run.smithi060.stdout:(74/166): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-02-26T10:30:43.074 INFO:teuthology.orchestra.run.smithi060.stdout:(75/166): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-02-26T10:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T10:30:43.192 INFO:teuthology.orchestra.run.smithi060.stdout:(76/166): python3-devel-3.9.17-2.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-02-26T10:30:43.197 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T10:30:43.231 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T10:30:43.323 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-26T10:30:43.329 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T10:30:43.342 INFO:teuthology.orchestra.run.smithi060.stdout:(77/166): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-02-26T10:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T10:30:43.417 INFO:teuthology.orchestra.run.smithi060.stdout:(78/166): python3-jsonpatch-1.21-16.el9.noarch. 345 kB/s | 26 kB 00:00 2024-02-26T10:30:43.492 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T10:30:43.500 INFO:teuthology.orchestra.run.smithi060.stdout:(79/166): python3-jsonpointer-2.0-4.el9.noarch. 232 kB/s | 19 kB 00:00 2024-02-26T10:30:43.525 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T10:30:43.585 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-26T10:30:43.585 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-26T10:30:43.609 INFO:teuthology.orchestra.run.smithi060.stdout:(80/166): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-02-26T10:30:43.659 INFO:teuthology.orchestra.run.smithi060.stdout:(81/166): python3-markupsafe-1.1.1-12.el9.x86_6 707 kB/s | 35 kB 00:00 2024-02-26T10:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T10:30:43.739 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T10:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T10:30:44.449 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T10:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T10:30:44.768 INFO:teuthology.orchestra.run.smithi060.stdout:(82/166): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-02-26T10:30:45.002 INFO:teuthology.orchestra.run.smithi060.stdout:(83/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-02-26T10:30:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:(84/166): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-02-26T10:30:45.286 INFO:teuthology.orchestra.run.smithi060.stdout:(85/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-02-26T10:30:45.311 INFO:teuthology.orchestra.run.smithi060.stdout:(86/166): python3-packaging-20.9-5.el9.noarch.r 582 kB/s | 77 kB 00:00 2024-02-26T10:30:45.353 INFO:teuthology.orchestra.run.smithi060.stdout:(87/166): python3-prettytable-0.7.2-27.el9.noar 640 kB/s | 42 kB 00:00 2024-02-26T10:30:45.372 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-26T10:30:45.373 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-26T10:30:45.436 INFO:teuthology.orchestra.run.smithi060.stdout:(88/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-02-26T10:30:45.520 INFO:teuthology.orchestra.run.smithi060.stdout:(89/166): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-02-26T10:30:45.587 INFO:teuthology.orchestra.run.smithi060.stdout:(90/166): python3-pytz-2021.1-5.el9.noarch.rpm 775 kB/s | 51 kB 00:00 2024-02-26T10:30:45.646 INFO:teuthology.orchestra.run.smithi060.stdout:(91/166): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-02-26T10:30:45.663 INFO:teuthology.orchestra.run.smithi060.stdout:(92/166): python3-requests-oauthlib-1.3.0-12.el 722 kB/s | 54 kB 00:00 2024-02-26T10:30:45.721 INFO:teuthology.orchestra.run.smithi060.stdout:(93/166): python3-toml-0.10.2-6.el9.noarch.rpm 722 kB/s | 42 kB 00:00 2024-02-26T10:30:45.748 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T10:30:45.788 INFO:teuthology.orchestra.run.smithi060.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 669 kB/s | 44 kB 00:00 2024-02-26T10:30:45.844 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T10:30:45.872 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T10:30:45.905 INFO:teuthology.orchestra.run.smithi060.stdout:(95/166): qatlib-23.11.0-1.el9.x86_64.rpm 1.8 MB/s | 216 kB 00:00 2024-02-26T10:30:45.906 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T10:30:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 528 kB/s | 35 kB 00:00 2024-02-26T10:30:46.047 INFO:teuthology.orchestra.run.smithi060.stdout:(97/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 600 kB/s | 44 kB 00:00 2024-02-26T10:30:46.214 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T10:30:46.231 INFO:teuthology.orchestra.run.smithi060.stdout:(98/166): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-02-26T10:30:46.315 INFO:teuthology.orchestra.run.smithi060.stdout:(99/166): uuid-1.6.2-55.el9.x86_64.rpm 701 kB/s | 58 kB 00:00 2024-02-26T10:30:46.415 INFO:teuthology.orchestra.run.smithi060.stdout:(100/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 944 kB/s | 94 kB 00:00 2024-02-26T10:30:46.465 INFO:teuthology.orchestra.run.smithi060.stdout:(101/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-02-26T10:30:46.501 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:30:46.541 INFO:teuthology.orchestra.run.smithi060.stdout:(102/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 606 kB/s | 45 kB 00:00 2024-02-26T10:30:46.575 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:30:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 106/182 2024-02-26T10:30:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T10:30:46.724 INFO:teuthology.orchestra.run.smithi060.stdout:(103/166): lua-devel-5.4.4-4.el9.x86_64.rpm 122 kB/s | 22 kB 00:00 2024-02-26T10:30:46.958 INFO:teuthology.orchestra.run.smithi060.stdout:(104/166): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-02-26T10:30:47.095 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T10:30:47.121 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:30:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:30:47.223 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 110/182 2024-02-26T10:30:47.450 INFO:teuthology.orchestra.run.smithi060.stdout:(105/166): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-02-26T10:30:47.584 INFO:teuthology.orchestra.run.smithi060.stdout:(106/166): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-02-26T10:30:47.642 INFO:teuthology.orchestra.run.smithi060.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 337 kB/s | 19 kB 00:00 2024-02-26T10:30:49.167 INFO:teuthology.orchestra.run.smithi060.stdout:(108/166): libarrow-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-02-26T10:30:49.243 INFO:teuthology.orchestra.run.smithi060.stdout:(109/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 350 kB/s | 26 kB 00:00 2024-02-26T10:30:49.335 INFO:teuthology.orchestra.run.smithi060.stdout:(110/166): liboath-2.6.7-2.el9.x86_64.rpm 534 kB/s | 49 kB 00:00 2024-02-26T10:30:49.385 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:30:49.420 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:30:49.485 INFO:teuthology.orchestra.run.smithi060.stdout:(111/166): libunwind-1.6.2-1.el9.x86_64.rpm 451 kB/s | 67 kB 00:00 2024-02-26T10:30:49.619 INFO:teuthology.orchestra.run.smithi060.stdout:(112/166): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-02-26T10:30:49.927 INFO:teuthology.orchestra.run.smithi060.stdout:(113/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-02-26T10:30:49.953 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:30:49.998 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:30:50.080 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:30:50.080 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T10:30:50.080 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T10:30:50.080 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:30:50.108 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:30:50.194 INFO:teuthology.orchestra.run.smithi060.stdout:(114/166): python3-asyncssh-2.13.2-4.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-02-26T10:30:50.261 INFO:teuthology.orchestra.run.smithi060.stdout:(115/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 660 kB/s | 43 kB 00:00 2024-02-26T10:30:50.311 INFO:teuthology.orchestra.run.smithi060.stdout:(116/166): python3-cachetools-4.2.4-1.el9.noarc 651 kB/s | 32 kB 00:00 2024-02-26T10:30:50.362 INFO:teuthology.orchestra.run.smithi060.stdout:(117/166): python3-certifi-2021.10.8-2.el9.noar 304 kB/s | 15 kB 00:00 2024-02-26T10:30:50.462 INFO:teuthology.orchestra.run.smithi060.stdout:(118/166): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-02-26T10:30:50.654 INFO:teuthology.orchestra.run.smithi060.stdout:(119/166): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-02-26T10:30:50.804 INFO:teuthology.orchestra.run.smithi060.stdout:(120/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-02-26T10:30:50.855 INFO:teuthology.orchestra.run.smithi060.stdout:(121/166): python3-colorama-0.4.4-12.el9.noarch 717 kB/s | 35 kB 00:00 2024-02-26T10:30:50.947 INFO:teuthology.orchestra.run.smithi060.stdout:(122/166): python3-google-auth-2.28.0-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-02-26T10:30:51.789 INFO:teuthology.orchestra.run.smithi060.stdout:(123/166): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-02-26T10:30:51.922 INFO:teuthology.orchestra.run.smithi060.stdout:(124/166): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-02-26T10:30:51.981 INFO:teuthology.orchestra.run.smithi060.stdout:(125/166): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-02-26T10:30:51.985 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-26T10:30:52.073 INFO:teuthology.orchestra.run.smithi060.stdout:(126/166): python3-jaraco-classes-3.2.1-5.el9.n 194 kB/s | 18 kB 00:00 2024-02-26T10:30:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T10:30:52.140 INFO:teuthology.orchestra.run.smithi060.stdout:(127/166): python3-jaraco-collections-3.0.0-8.e 352 kB/s | 23 kB 00:00 2024-02-26T10:30:52.152 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T10:30:52.205 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T10:30:52.239 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T10:30:52.248 INFO:teuthology.orchestra.run.smithi060.stdout:(128/166): python3-jaraco-functools-3.5.0-2.el9 182 kB/s | 19 kB 00:00 2024-02-26T10:30:52.264 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T10:30:52.298 INFO:teuthology.orchestra.run.smithi060.stdout:(129/166): python3-jaraco-text-3.2.0-6.el9.noar 401 kB/s | 20 kB 00:00 2024-02-26T10:30:52.306 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T10:30:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:(130/166): postgresql-server-13.11-1.el9.x86_64 607 kB/s | 5.7 MB 00:09 2024-02-26T10:30:52.600 INFO:teuthology.orchestra.run.smithi060.stdout:(131/166): python3-logutils-0.3.5-21.el9.noarch 254 kB/s | 46 kB 00:00 2024-02-26T10:30:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T10:30:52.691 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:30:52.765 INFO:teuthology.orchestra.run.smithi060.stdout:(132/166): python3-kubernetes-26.1.0-2.el9.noar 2.2 MB/s | 1.0 MB 00:00 2024-02-26T10:30:52.790 INFO:teuthology.orchestra.run.smithi060.stdout:(133/166): python3-more-itertools-8.12.0-2.el9. 417 kB/s | 79 kB 00:00 2024-02-26T10:30:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:30:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:30:52.857 INFO:teuthology.orchestra.run.smithi060.stdout:(134/166): python3-natsort-7.1.1-5.el9.noarch.r 632 kB/s | 58 kB 00:00 2024-02-26T10:30:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:30:52.941 INFO:teuthology.orchestra.run.smithi060.stdout:(135/166): python3-portend-3.1.0-2.el9.noarch.r 199 kB/s | 16 kB 00:00 2024-02-26T10:30:52.954 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:30:52.974 INFO:teuthology.orchestra.run.smithi060.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-02-26T10:30:53.016 INFO:teuthology.orchestra.run.smithi060.stdout:(137/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-02-26T10:30:53.024 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T10:30:53.033 INFO:teuthology.orchestra.run.smithi060.stdout:(138/166): python3-pyperclip-1.8.2-2.el9.noarch 487 kB/s | 28 kB 00:00 2024-02-26T10:30:53.105 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T10:30:53.280 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T10:30:53.309 INFO:teuthology.orchestra.run.smithi060.stdout:(139/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.1 MB/s | 16 MB 00:07 2024-02-26T10:30:53.339 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T10:30:53.342 INFO:teuthology.orchestra.run.smithi060.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 95 kB/s | 31 kB 00:00 2024-02-26T10:30:53.367 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:30:53.368 INFO:teuthology.orchestra.run.smithi060.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 566 kB/s | 188 kB 00:00 2024-02-26T10:30:53.410 INFO:teuthology.orchestra.run.smithi060.stdout:(142/166): python3-tempora-5.0.0-2.el9.noarch.r 540 kB/s | 36 kB 00:00 2024-02-26T10:30:53.435 INFO:teuthology.orchestra.run.smithi060.stdout:(143/166): python3-typing-extensions-4.4.0-2.el 767 kB/s | 51 kB 00:00 2024-02-26T10:30:53.470 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:30:53.493 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:30:53.539 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:30:53.543 INFO:teuthology.orchestra.run.smithi060.stdout:(144/166): python3-rsa-4.9-2.el9.noarch.rpm 253 kB/s | 59 kB 00:00 2024-02-26T10:30:53.561 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:30:53.569 INFO:teuthology.orchestra.run.smithi060.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-02-26T10:30:53.576 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:30:53.594 INFO:teuthology.orchestra.run.smithi060.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 566 kB/s | 90 kB 00:00 2024-02-26T10:30:53.636 INFO:teuthology.orchestra.run.smithi060.stdout:(147/166): python3-zc-lockfile-2.0-10.el9.noarc 303 kB/s | 20 kB 00:00 2024-02-26T10:30:53.719 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:30:53.748 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:30:53.795 INFO:teuthology.orchestra.run.smithi060.stdout:(148/166): re2-20211101-3.el9.x86_64.rpm 960 kB/s | 192 kB 00:00 2024-02-26T10:30:53.828 INFO:teuthology.orchestra.run.smithi060.stdout:(149/166): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2024-02-26T10:30:53.852 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T10:30:53.950 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:30:54.010 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:30:54.078 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:30:54.129 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:30:54.220 INFO:teuthology.orchestra.run.smithi060.stdout:(150/166): librbd1-19.0.0-1686.g6085b91f.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-02-26T10:30:54.351 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:30:54.458 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T10:30:54.543 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T10:30:54.571 INFO:teuthology.orchestra.run.smithi060.stdout:(151/166): thrift-0.15.0-2.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-02-26T10:30:54.574 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T10:30:54.609 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T10:30:54.673 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T10:30:54.708 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T10:30:54.760 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T10:30:54.793 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T10:30:54.821 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T10:30:54.871 INFO:teuthology.orchestra.run.smithi060.stdout:(152/166): librados2-19.0.0-1686.g6085b91f.el9. 3.1 MB/s | 3.4 MB 00:01 2024-02-26T10:30:54.888 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T10:30:54.942 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T10:30:54.973 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T10:30:55.009 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T10:30:55.105 INFO:teuthology.orchestra.run.smithi060.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 182 kB/s | 42 kB 00:00 2024-02-26T10:30:55.118 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T10:30:55.379 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 36/182 2024-02-26T10:30:55.454 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 37/182 2024-02-26T10:30:55.483 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 38/182 2024-02-26T10:30:55.551 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T10:30:55.597 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T10:30:55.670 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T10:30:55.802 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T10:30:55.858 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T10:30:56.113 INFO:teuthology.orchestra.run.smithi060.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 245 kB/s | 247 kB 00:01 2024-02-26T10:30:56.280 INFO:teuthology.orchestra.run.smithi060.stdout:(155/166): acl-2.3.1-4.el9.x86_64.rpm 430 kB/s | 71 kB 00:00 2024-02-26T10:30:56.370 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T10:30:56.397 INFO:teuthology.orchestra.run.smithi060.stdout:(156/166): elfutils-debuginfod-client-0.190-2.e 314 kB/s | 36 kB 00:00 2024-02-26T10:30:56.453 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T10:30:56.482 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T10:30:56.519 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T10:30:56.560 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T10:30:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 49/182 2024-02-26T10:30:56.639 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T10:30:56.722 INFO:teuthology.orchestra.run.smithi060.stdout:(157/166): elfutils-libelf-0.190-2.el9.x86_64.r 594 kB/s | 192 kB 00:00 2024-02-26T10:30:56.784 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T10:30:56.825 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T10:30:56.880 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T10:30:56.944 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T10:30:56.984 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T10:30:57.021 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T10:30:57.067 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T10:30:57.103 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T10:30:57.114 INFO:teuthology.orchestra.run.smithi060.stdout:(158/166): elfutils-libs-0.190-2.el9.x86_64.rpm 654 kB/s | 256 kB 00:00 2024-02-26T10:30:57.146 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T10:30:57.156 INFO:teuthology.orchestra.run.smithi060.stdout:(159/166): libacl-2.3.1-4.el9.x86_64.rpm 568 kB/s | 23 kB 00:00 2024-02-26T10:30:57.188 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T10:30:57.321 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T10:30:57.369 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T10:30:57.399 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T10:30:57.441 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T10:30:57.523 INFO:teuthology.orchestra.run.smithi060.stdout:(160/166): NetworkManager-1.45.91-1.el9.x86_64. 704 kB/s | 2.3 MB 00:03 2024-02-26T10:30:57.535 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T10:30:57.574 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T10:30:57.614 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T10:30:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T10:30:57.665 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T10:30:57.736 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T10:30:57.782 INFO:teuthology.orchestra.run.smithi060.stdout:(161/166): NetworkManager-libnm-1.45.91-1.el9.x 578 kB/s | 1.8 MB 00:03 2024-02-26T10:30:57.856 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T10:30:57.897 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T10:30:57.916 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T10:30:57.942 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T10:30:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T10:30:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T10:30:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T10:30:58.048 INFO:teuthology.orchestra.run.smithi060.stdout:(162/166): openssl-3.0.7-25.el9.x86_64.rpm 1.4 MB/s | 1.2 MB 00:00 2024-02-26T10:30:58.077 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T10:30:58.104 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T10:30:58.115 INFO:teuthology.orchestra.run.smithi060.stdout:(163/166): xfsdump-3.1.12-4.el9.x86_64.rpm 972 kB/s | 323 kB 00:00 2024-02-26T10:30:58.154 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T10:30:58.180 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T10:30:58.267 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T10:30:58.310 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T10:30:58.342 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T10:30:58.374 INFO:teuthology.orchestra.run.smithi060.stdout:(164/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 289 kB/s | 74 kB 00:00 2024-02-26T10:30:58.404 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T10:30:58.486 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T10:30:58.535 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T10:30:59.249 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T10:30:59.299 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T10:30:59.333 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T10:30:59.429 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T10:30:59.451 INFO:teuthology.orchestra.run.smithi060.stdout:(165/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 784 kB/s | 1.1 MB 00:01 2024-02-26T10:30:59.482 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T10:30:59.561 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T10:30:59.593 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T10:30:59.699 INFO:teuthology.orchestra.run.smithi060.stdout:(166/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 1.0 MB/s | 2.2 MB 00:02 2024-02-26T10:30:59.707 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-02-26T10:30:59.707 INFO:teuthology.orchestra.run.smithi060.stdout:Total 5.9 MB/s | 222 MB 00:37 2024-02-26T10:30:59.762 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T10:30:59.782 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T10:30:59.859 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T10:31:00.496 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T10:31:00.621 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T10:31:01.642 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:31:01.643 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2024-02-26T10:31:01.643 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2024-02-26T10:31:01.643 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2024-02-26T10:31:01.643 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2024-02-26T10:31:01.643 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2024-02-26T10:31:01.643 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2024-02-26T10:31:01.643 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2024-02-26T10:31:01.643 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:01.797 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:31:01.812 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:31:01.812 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:01.812 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T10:31:01.812 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:01.830 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T10:31:01.903 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T10:31:01.931 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T10:31:01.981 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T10:31:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T10:31:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:31:02.408 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:31:02.440 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 106/182 2024-02-26T10:31:02.477 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T10:31:02.927 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T10:31:02.956 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:31:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:31:03.049 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 110/182 2024-02-26T10:31:03.248 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:31:03.268 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-02-26T10:31:03.274 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:31:03.274 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:03.274 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T10:31:03.274 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:31:03.274 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:31:03.274 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:03.482 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:31:03.504 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:31:03.504 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:03.504 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T10:31:03.505 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:31:03.505 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:31:03.505 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:03.647 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-02-26T10:31:03.647 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-02-26T10:31:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:31:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:31:03.963 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:03.963 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T10:31:03.964 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:31:03.964 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:31:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:04.079 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T10:31:04.127 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T10:31:04.182 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T10:31:04.364 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T10:31:04.424 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T10:31:04.502 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T10:31:04.590 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T10:31:04.737 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T10:31:04.783 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T10:31:04.816 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T10:31:04.848 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T10:31:04.883 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T10:31:04.929 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T10:31:05.139 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:31:05.179 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:31:05.311 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:31:05.343 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:31:05.374 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 132/182 2024-02-26T10:31:05.435 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 133/182 2024-02-26T10:31:05.458 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:31:05.684 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:31:05.710 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:31:05.717 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:31:05.749 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:31:05.830 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:31:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T10:31:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T10:31:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:31:06.304 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-02-26T10:31:06.305 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-02-26T10:31:06.670 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:31:06.693 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:31:06.793 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:31:06.962 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 137/182 2024-02-26T10:31:06.995 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:31:07.017 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:31:07.017 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:07.017 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T10:31:07.017 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:31:07.017 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:31:07.018 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:07.062 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:31:07.073 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:31:07.171 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 140/182 2024-02-26T10:31:11.539 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 141/182 2024-02-26T10:31:11.606 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 142/182 2024-02-26T10:31:12.453 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T10:31:12.486 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:31:12.508 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:31:12.509 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:12.509 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T10:31:12.509 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:31:12.509 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:31:12.509 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:12.553 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:31:12.575 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:31:12.575 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:12.575 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T10:31:12.575 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:12.620 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:31:12.642 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:31:12.643 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:12.643 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T10:31:12.643 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:31:12.643 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:31:12.643 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:12.946 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:31:12.968 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:31:12.969 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:12.969 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T10:31:12.969 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:31:12.969 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:31:12.969 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:13.020 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 148/182 2024-02-26T10:31:13.061 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 149/182 2024-02-26T10:31:13.093 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T10:31:13.121 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T10:31:13.304 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 152/182 2024-02-26T10:31:13.359 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T10:31:13.391 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 154/182 2024-02-26T10:31:13.432 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T10:31:13.462 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T10:31:13.514 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:31:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T10:31:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-02-26T10:31:13.594 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T10:31:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T10:31:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T10:31:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T10:31:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T10:31:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T10:31:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T10:31:14.214 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:31:14.258 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:31:14.285 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:31:14.331 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T10:31:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T10:31:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:31:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:31:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:31:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:31:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:31:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T10:31:15.163 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 162/182 2024-02-26T10:31:15.193 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T10:31:15.263 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T10:31:15.343 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T10:31:15.383 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T10:31:15.409 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T10:31:15.434 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T10:31:15.459 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T10:31:15.485 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T10:31:15.486 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:31:15.490 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T10:31:15.542 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T10:31:15.557 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:31:15.583 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:31:15.587 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T10:31:15.612 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T10:31:15.638 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T10:31:15.663 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T10:31:15.663 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:31:15.747 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:31:15.754 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:31:15.772 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T10:31:15.778 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:31:15.778 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:31:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:31:15.864 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:31:15.894 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:31:15.958 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:31:15.959 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:31:15.968 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:31:16.025 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:31:16.058 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T10:31:16.085 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T10:31:16.118 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T10:31:16.182 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:31:16.261 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:31:16.305 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:31:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T10:31:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:31:16.922 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:31:17.074 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:31:17.154 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:31:17.435 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:31:17.435 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-02-26T10:31:17.435 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-02-26T10:31:17.435 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-02-26T10:31:17.435 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-02-26T10:31:17.435 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-02-26T10:31:17.436 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-02-26T10:31:17.436 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-02-26T10:31:17.436 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:17.492 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:31:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:31:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:31:17.601 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:17.601 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T10:31:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:17.654 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T10:31:17.779 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T10:31:17.836 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T10:31:17.914 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T10:31:18.029 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T10:31:18.083 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T10:31:18.150 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T10:31:18.195 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T10:31:18.234 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T10:31:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T10:31:18.428 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T10:31:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T10:31:18.521 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T10:31:18.631 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T10:31:18.698 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 36/182 2024-02-26T10:31:18.750 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 37/182 2024-02-26T10:31:18.788 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 38/182 2024-02-26T10:31:18.911 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T10:31:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T10:31:19.072 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:31:19.079 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T10:31:19.097 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:31:19.097 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:19.097 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T10:31:19.097 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:31:19.097 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:31:19.098 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T10:31:19.307 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:31:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T10:31:19.329 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:31:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T10:31:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:31:19.329 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:31:19.329 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:19.784 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:31:19.806 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:31:19.806 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:19.806 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T10:31:19.806 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:31:19.806 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:31:19.806 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:19.914 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T10:31:19.971 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T10:31:20.025 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T10:31:20.062 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T10:31:20.162 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T10:31:20.208 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T10:31:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T10:31:20.251 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T10:31:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T10:31:20.319 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T10:31:20.322 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T10:31:20.374 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 49/182 2024-02-26T10:31:20.407 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T10:31:20.446 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T10:31:20.547 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T10:31:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T10:31:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T10:31:20.656 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T10:31:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T10:31:20.701 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T10:31:20.704 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T10:31:20.729 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T10:31:20.772 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T10:31:20.868 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T10:31:20.913 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T10:31:20.987 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T10:31:21.061 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T10:31:21.113 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T10:31:21.164 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:31:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:31:21.188 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T10:31:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 132/182 2024-02-26T10:31:21.254 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T10:31:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 133/182 2024-02-26T10:31:21.292 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:31:21.298 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 1/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 3/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 4/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 5/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 6/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 7/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 11/182 2024-02-26T10:31:21.299 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 12/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 13/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 14/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 15/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 16/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 17/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 18/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 19/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 20/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 21/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 22/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 23/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 24/182 2024-02-26T10:31:21.300 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 25/182 2024-02-26T10:31:21.301 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 26/182 2024-02-26T10:31:21.301 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 27/182 2024-02-26T10:31:21.301 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 28/182 2024-02-26T10:31:21.301 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 29/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 30/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 31/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 32/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 33/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 34/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 35/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 36/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 37/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 38/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 39/182 2024-02-26T10:31:21.315 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T10:31:21.316 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T10:31:21.317 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T10:31:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T10:31:21.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T10:31:21.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T10:31:21.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T10:31:21.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T10:31:21.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T10:31:21.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T10:31:21.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T10:31:21.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T10:31:21.322 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T10:31:21.323 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 151/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 153/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T10:31:21.324 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T10:31:21.325 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T10:31:21.326 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T10:31:21.326 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T10:31:21.326 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T10:31:21.326 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T10:31:21.326 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T10:31:21.326 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T10:31:21.326 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T10:31:21.326 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T10:31:21.475 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T10:31:21.519 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:31:21.526 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T10:31:21.552 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:31:21.563 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T10:31:21.623 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T10:31:21.779 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T10:31:21.821 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T10:31:21.878 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T10:31:21.912 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T10:31:21.945 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T10:31:22.124 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T10:31:22.312 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T10:31:22.350 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T10:31:22.391 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T10:31:22.443 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T10:31:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:31:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:31:22.538 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T10:31:22.602 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T10:31:22.634 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:31:22.649 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T10:31:22.695 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T10:31:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T10:31:22.798 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 137/182 2024-02-26T10:31:22.821 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:31:22.822 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T10:31:22.843 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:31:22.844 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:22.844 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T10:31:22.844 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:31:22.848 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:31:22.849 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:22.865 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T10:31:22.888 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:31:22.898 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:31:22.989 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 140/182 2024-02-26T10:31:23.018 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T10:31:23.074 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T10:31:23.114 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T10:31:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T10:31:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T10:31:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T10:31:24.251 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T10:31:24.252 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T10:31:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T10:31:24.254 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T10:31:24.254 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T10:31:24.254 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.254 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.254 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.254 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.254 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.254 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.254 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.255 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T10:31:24.256 INFO:teuthology.orchestra.run.smithi113.stdout: jq-1.6-16.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T10:31:24.257 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T10:31:24.258 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T10:31:24.259 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T10:31:24.259 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T10:31:24.259 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T10:31:24.259 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T10:31:24.259 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T10:31:24.260 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T10:31:24.260 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T10:31:24.260 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T10:31:24.260 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T10:31:24.260 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T10:31:24.261 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T10:31:24.262 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T10:31:24.263 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T10:31:24.264 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T10:31:24.264 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T10:31:24.264 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T10:31:24.264 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T10:31:24.264 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T10:31:24.264 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T10:31:24.264 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T10:31:24.265 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.266 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:24.267 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T10:31:24.267 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T10:31:24.267 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T10:31:24.267 INFO:teuthology.orchestra.run.smithi113.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T10:31:24.267 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T10:31:24.267 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T10:31:24.267 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T10:31:24.267 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:31:24.267 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-02-26T10:31:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T10:31:24.510 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T10:31:24.543 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T10:31:24.676 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T10:31:24.739 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T10:31:24.783 DEBUG:teuthology.parallel:result is None 2024-02-26T10:31:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T10:31:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T10:31:25.169 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T10:31:25.200 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T10:31:25.302 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T10:31:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T10:31:26.320 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T10:31:27.374 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 141/182 2024-02-26T10:31:27.448 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 142/182 2024-02-26T10:31:28.007 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T10:31:28.099 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T10:31:28.134 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T10:31:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T10:31:28.301 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T10:31:28.337 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:31:28.359 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:31:28.359 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:28.359 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T10:31:28.359 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:31:28.359 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:31:28.359 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:31:28.442 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:31:28.443 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:28.443 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T10:31:28.443 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:28.488 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:31:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:31:28.510 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:28.510 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T10:31:28.510 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:31:28.510 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:31:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:28.550 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T10:31:28.577 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:31:28.723 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:31:28.758 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 106/182 2024-02-26T10:31:28.813 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T10:31:28.822 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:31:28.845 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:31:28.845 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:28.845 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T10:31:28.845 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:31:28.845 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:31:28.845 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:28.896 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 148/182 2024-02-26T10:31:28.938 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 149/182 2024-02-26T10:31:28.970 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T10:31:28.997 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T10:31:29.177 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 152/182 2024-02-26T10:31:29.227 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T10:31:29.260 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 154/182 2024-02-26T10:31:29.300 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T10:31:29.330 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T10:31:29.346 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T10:31:29.375 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:31:29.436 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T10:31:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:31:29.462 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T10:31:29.909 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:31:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 110/182 2024-02-26T10:31:30.024 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:31:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:31:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:31:30.151 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T10:31:30.974 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 162/182 2024-02-26T10:31:31.074 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T10:31:31.154 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T10:31:31.195 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T10:31:31.220 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T10:31:31.245 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T10:31:31.270 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T10:31:31.305 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T10:31:31.305 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:31:31.393 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:31:31.423 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T10:31:31.451 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T10:31:31.474 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T10:31:31.507 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T10:31:31.507 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:31:31.575 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:31:31.600 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T10:31:31.606 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:31:31.606 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:31:31.786 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:31:31.786 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:31:31.878 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:31:31.902 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T10:31:31.929 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T10:31:31.954 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T10:31:32.017 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:31:32.631 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:31:32.670 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:31:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:31:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:31:33.572 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:31:33.572 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T10:31:33.572 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T10:31:33.573 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:31:33.616 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:31:36.986 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:31:36.986 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 1/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 3/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 4/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 5/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 6/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 7/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 11/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 12/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 13/182 2024-02-26T10:31:36.987 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 14/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 15/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 16/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 17/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 18/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 19/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 20/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 21/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 22/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 23/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 24/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 25/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 26/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 27/182 2024-02-26T10:31:36.988 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 28/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 29/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 30/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 31/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 32/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 33/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 34/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 35/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 36/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 37/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 38/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 39/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T10:31:36.989 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T10:31:36.990 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T10:31:36.991 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T10:31:36.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T10:31:36.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T10:31:36.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T10:31:36.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T10:31:36.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T10:31:36.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T10:31:36.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T10:31:36.992 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T10:31:36.997 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T10:31:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T10:31:36.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T10:31:37.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T10:31:37.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 151/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T10:31:37.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 153/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T10:31:37.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T10:31:37.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T10:31:37.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T10:31:37.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T10:31:37.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T10:31:37.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T10:31:37.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T10:31:37.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T10:31:37.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T10:31:37.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T10:31:37.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T10:31:37.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T10:31:37.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T10:31:37.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T10:31:37.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T10:31:37.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T10:31:37.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T10:31:39.972 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T10:31:39.972 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:39.972 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.973 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.974 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.975 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.976 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.976 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.976 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.976 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.976 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.976 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.976 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.976 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-16.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.977 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.978 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.978 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T10:31:39.978 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T10:31:39.978 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T10:31:39.978 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T10:31:39.978 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T10:31:39.978 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.978 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.978 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T10:31:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T10:31:39.980 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T10:31:39.981 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T10:31:39.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T10:31:39.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T10:31:39.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.985 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:31:39.986 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T10:31:39.986 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T10:31:39.986 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T10:31:39.986 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T10:31:39.986 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T10:31:39.986 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T10:31:39.986 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T10:31:39.987 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-26T10:31:39.987 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-26T10:31:40.421 DEBUG:teuthology.parallel:result is None 2024-02-26T10:31:47.119 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:31:47.119 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2024-02-26T10:31:47.119 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2024-02-26T10:31:47.119 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2024-02-26T10:31:47.119 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2024-02-26T10:31:47.119 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2024-02-26T10:31:47.119 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2024-02-26T10:31:47.119 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2024-02-26T10:31:47.119 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:31:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:31:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:31:47.413 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:47.413 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T10:31:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:31:49.126 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:31:49.211 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:31:49.212 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:49.212 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T10:31:49.212 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:31:49.212 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:31:49.212 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:31:49.473 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:31:49.545 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:31:49.545 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:49.545 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T10:31:49.545 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:31:49.545 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:31:49.545 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:31:50.073 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:31:50.157 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:31:50.157 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:50.158 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T10:31:50.158 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:31:50.158 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:31:50.158 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:31:50.309 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T10:31:50.377 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T10:31:50.458 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T10:31:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T10:31:50.771 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T10:31:50.863 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T10:31:51.001 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T10:31:51.194 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T10:31:51.256 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T10:31:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T10:31:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T10:31:51.384 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T10:31:51.453 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T10:31:52.083 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:31:52.122 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:31:52.176 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 132/182 2024-02-26T10:31:52.289 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 133/182 2024-02-26T10:31:52.315 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:31:52.651 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:31:52.691 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:31:53.875 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:31:53.908 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:31:54.058 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:31:54.291 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 137/182 2024-02-26T10:31:54.327 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:31:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:31:54.419 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:31:54.419 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T10:31:54.419 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:31:54.419 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:31:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:31:54.511 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:31:54.556 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:31:54.666 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 140/182 2024-02-26T10:31:59.691 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 141/182 2024-02-26T10:31:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 142/182 2024-02-26T10:32:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T10:32:00.801 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:32:00.872 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:32:00.873 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:32:00.873 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T10:32:00.873 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:32:00.873 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:32:00.873 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:32:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:32:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:32:00.989 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:32:00.989 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T10:32:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:32:01.052 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:32:01.134 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:32:01.134 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:32:01.135 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T10:32:01.135 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:32:01.135 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:32:01.135 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:32:01.511 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:32:01.581 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:32:01.581 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:32:01.581 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T10:32:01.581 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:32:01.582 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:32:01.582 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:32:01.652 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 148/182 2024-02-26T10:32:01.711 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 149/182 2024-02-26T10:32:01.747 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T10:32:01.785 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T10:32:02.078 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 152/182 2024-02-26T10:32:02.134 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T10:32:02.192 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 154/182 2024-02-26T10:32:02.239 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T10:32:02.278 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T10:32:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:32:02.466 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T10:32:02.502 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T10:32:03.449 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:32:03.508 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:32:03.596 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:32:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T10:32:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 162/182 2024-02-26T10:32:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T10:32:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T10:32:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T10:32:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T10:32:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T10:32:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T10:32:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T10:32:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:32:05.769 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:32:05.818 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T10:32:05.859 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T10:32:05.892 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T10:32:05.925 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T10:32:05.925 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:32:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:32:06.126 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T10:32:06.164 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:32:06.165 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:32:06.501 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:32:06.502 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:32:06.627 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:32:06.663 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T10:32:06.704 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T10:32:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T10:32:06.889 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:32:07.155 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:32:07.156 INFO:teuthology.orchestra.run.smithi060.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-26T10:32:07.156 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:32:12.320 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 1/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 3/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 4/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 5/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 6/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 7/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 11/182 2024-02-26T10:32:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 12/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 13/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 14/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 15/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 16/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 17/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 18/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 19/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 20/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 21/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 22/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 23/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 24/182 2024-02-26T10:32:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 25/182 2024-02-26T10:32:12.338 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 26/182 2024-02-26T10:32:12.338 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 27/182 2024-02-26T10:32:12.338 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 28/182 2024-02-26T10:32:12.338 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 29/182 2024-02-26T10:32:12.338 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 30/182 2024-02-26T10:32:12.338 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 31/182 2024-02-26T10:32:12.338 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 32/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 33/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 34/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 35/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 36/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 37/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 38/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 39/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T10:32:12.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T10:32:12.340 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T10:32:12.341 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T10:32:12.341 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T10:32:12.341 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T10:32:12.341 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T10:32:12.341 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T10:32:12.341 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T10:32:12.341 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T10:32:12.341 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T10:32:12.341 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T10:32:12.342 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T10:32:12.343 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T10:32:12.344 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T10:32:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T10:32:12.346 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T10:32:12.347 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 151/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 153/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T10:32:12.348 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T10:32:12.349 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T10:32:12.350 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T10:32:12.350 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T10:32:12.350 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T10:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T10:32:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T10:32:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: jq-1.6-16.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T10:32:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T10:32:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T10:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T10:32:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T10:32:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T10:32:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T10:32:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T10:32:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T10:32:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T10:32:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T10:32:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T10:32:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T10:32:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T10:32:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:32:14.860 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-02-26T10:32:15.547 DEBUG:teuthology.parallel:result is None 2024-02-26T10:32:15.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:32:16.538 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T10:32:16.597 INFO:teuthology.orchestra.run.smithi060.stdout:19.0.0-1686.g6085b91f.el9 2024-02-26T10:32:16.597 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1686.g6085b91f.el9 2024-02-26T10:32:16.598 INFO:teuthology.task.install:The correct ceph version 19.0.0-1686.g6085b91f is installed. 2024-02-26T10:32:16.600 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:32:17.214 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T10:32:17.239 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-1686.g6085b91f.el9 2024-02-26T10:32:17.240 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1686.g6085b91f.el9 2024-02-26T10:32:17.240 INFO:teuthology.task.install:The correct ceph version 19.0.0-1686.g6085b91f is installed. 2024-02-26T10:32:17.242 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:32:17.430 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T10:32:17.456 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-1686.g6085b91f.el9 2024-02-26T10:32:17.456 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1686.g6085b91f.el9 2024-02-26T10:32:17.456 INFO:teuthology.task.install:The correct ceph version 19.0.0-1686.g6085b91f is installed. 2024-02-26T10:32:17.458 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-26T10:32:17.458 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:32:17.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T10:32:17.551 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:32:17.551 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T10:32:17.586 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:32:17.586 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T10:32:17.621 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-26T10:32:17.621 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:32:17.621 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T10:32:17.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T10:32:17.820 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:32:17.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T10:32:17.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T10:32:17.923 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:32:17.924 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T10:32:17.958 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T10:32:18.027 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-26T10:32:18.027 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:32:18.027 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T10:32:18.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T10:32:18.214 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:32:18.215 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T10:32:18.244 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T10:32:18.311 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:32:18.311 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T10:32:18.339 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T10:32:18.407 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-26T10:32:18.407 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:32:18.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T10:32:18.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T10:32:18.588 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:32:18.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T10:32:18.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T10:32:18.682 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:32:18.682 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T10:32:18.709 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T10:32:18.776 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-26T10:32:18.893 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-02-26T10:32:18.893 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:32:18.894 INFO:tasks.cephadm:Cluster fsid is 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:32:18.894 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-26T10:32:18.894 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.60', 'mon.b': '172.21.15.100', 'mon.c': '172.21.15.113'} 2024-02-26T10:32:18.894 INFO:tasks.cephadm:First mon is mon.a on smithi060 2024-02-26T10:32:18.894 INFO:tasks.cephadm:First mgr is x 2024-02-26T10:32:18.894 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-26T10:32:18.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname $(hostname -s) 2024-02-26T10:32:18.968 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-02-26T10:32:18.995 DEBUG:teuthology.orchestra.run.smithi113:> sudo hostname $(hostname -s) 2024-02-26T10:32:19.022 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-26T10:32:19.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:32:19.292 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77061/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1686-g6085b91f', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1686.g6085b91f'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-26 08:05:30.644350', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240226.064629', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-26T10:32:19.716 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20240226.064629, sha1 6085b91fefeb74bf4310344a87d81d3217ef7893 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:32:19.717 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/x86_64/flavors/default/cephadm 2024-02-26T10:32:19.718 DEBUG:teuthology.orchestra.run.smithi060:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T10:32:20.555 INFO:teuthology.orchestra.run.smithi060.stdout:-rw-r--r--. 1 ubuntu ubuntu 766810 Feb 26 10:32 /home/ubuntu/cephtest/cephadm 2024-02-26T10:32:20.555 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T10:32:21.804 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 766810 Feb 26 10:32 /home/ubuntu/cephtest/cephadm 2024-02-26T10:32:21.804 DEBUG:teuthology.orchestra.run.smithi113:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T10:32:22.279 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r--. 1 ubuntu ubuntu 766810 Feb 26 10:32 /home/ubuntu/cephtest/cephadm 2024-02-26T10:32:22.279 DEBUG:teuthology.orchestra.run.smithi060:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T10:32:22.341 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T10:32:22.360 DEBUG:teuthology.orchestra.run.smithi113:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T10:32:22.395 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 on all hosts... 2024-02-26T10:32:22.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 pull 2024-02-26T10:32:22.403 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 pull 2024-02-26T10:32:22.406 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 pull 2024-02-26T10:32:23.259 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893... 2024-02-26T10:32:23.260 INFO:teuthology.orchestra.run.smithi113.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893... 2024-02-26T10:32:23.761 INFO:teuthology.orchestra.run.smithi060.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893... 2024-02-26T10:32:57.786 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:32:57.787 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1686-g6085b91f (6085b91fefeb74bf4310344a87d81d3217ef7893) squid (dev)", 2024-02-26T10:32:57.787 INFO:teuthology.orchestra.run.smithi060.stdout: "image_id": "07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763", 2024-02-26T10:32:57.787 INFO:teuthology.orchestra.run.smithi060.stdout: "repo_digests": [ 2024-02-26T10:32:57.787 INFO:teuthology.orchestra.run.smithi060.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a" 2024-02-26T10:32:57.787 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:32:57.787 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:33:15.957 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-02-26T10:33:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-1686-g6085b91f (6085b91fefeb74bf4310344a87d81d3217ef7893) squid (dev)", 2024-02-26T10:33:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763", 2024-02-26T10:33:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-02-26T10:33:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a" 2024-02-26T10:33:15.958 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-02-26T10:33:15.959 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-02-26T10:33:16.546 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-02-26T10:33:16.546 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_version": "ceph version 19.0.0-1686-g6085b91f (6085b91fefeb74bf4310344a87d81d3217ef7893) squid (dev)", 2024-02-26T10:33:16.547 INFO:teuthology.orchestra.run.smithi113.stdout: "image_id": "07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763", 2024-02-26T10:33:16.547 INFO:teuthology.orchestra.run.smithi113.stdout: "repo_digests": [ 2024-02-26T10:33:16.547 INFO:teuthology.orchestra.run.smithi113.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a" 2024-02-26T10:33:16.547 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-02-26T10:33:16.547 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-02-26T10:33:16.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph 2024-02-26T10:33:16.683 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-02-26T10:33:16.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph 2024-02-26T10:33:16.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /etc/ceph 2024-02-26T10:33:16.852 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-02-26T10:33:16.883 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /etc/ceph 2024-02-26T10:33:16.911 INFO:tasks.cephadm:Writing seed config... 2024-02-26T10:33:16.913 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-26T10:33:16.914 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-26T10:33:16.914 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-26T10:33:16.914 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-26T10:33:16.914 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-26T10:33:16.914 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-26T10:33:16.915 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-26T10:33:16.915 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-26T10:33:16.915 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-26T10:33:16.915 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-26T10:33:16.915 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-26T10:33:16.915 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-26T10:33:16.915 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-26T10:33:16.915 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-26T10:33:16.915 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-26T10:33:16.916 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-26T10:33:16.916 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-26T10:33:16.916 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-02-26T10:33:16.916 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-26T10:33:16.916 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-26T10:33:16.916 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-26T10:33:16.916 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-26T10:33:16.917 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-26T10:33:16.918 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-26T10:33:16.919 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-02-26T10:33:16.919 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-26T10:33:16.919 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-26T10:33:16.919 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-26T10:33:16.919 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-26T10:33:16.919 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-26T10:33:16.919 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-26T10:33:16.919 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-26T10:33:16.919 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-02-26T10:33:16.920 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:33:16.920 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-26T10:33:16.982 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 51464d50-d492-11ee-95c3-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-26T10:33:16.982 DEBUG:teuthology.orchestra.run.smithi060:mon.a> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.a.service 2024-02-26T10:33:17.029 DEBUG:teuthology.orchestra.run.smithi060:mgr.x> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.x.service 2024-02-26T10:33:17.076 INFO:tasks.cephadm:Bootstrapping... 2024-02-26T10:33:17.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 -v bootstrap --fsid 51464d50-d492-11ee-95c3-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.60 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-26T10:33:17.358 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-02-26T10:33:17.358 INFO:teuthology.orchestra.run.smithi060.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893', '-v', 'bootstrap', '--fsid', '51464d50-d492-11ee-95c3-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.60', '--skip-admin-label'] 2024-02-26T10:33:17.413 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T10:33:17.414 INFO:teuthology.orchestra.run.smithi060.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-26T10:33:17.414 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying podman|docker is present... 2024-02-26T10:33:17.466 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T10:33:17.466 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying lvm2 is present... 2024-02-26T10:33:17.466 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying time synchronization is in place... 2024-02-26T10:33:17.481 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-26T10:33:17.481 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-26T10:33:17.497 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-26T10:33:17.497 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T10:33:17.513 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-02-26T10:33:17.528 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-02-26T10:33:17.528 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-02-26T10:33:17.528 INFO:teuthology.orchestra.run.smithi060.stdout:Repeating the final host check... 2024-02-26T10:33:17.585 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T10:33:17.585 INFO:teuthology.orchestra.run.smithi060.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-26T10:33:17.585 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl is present 2024-02-26T10:33:17.585 INFO:teuthology.orchestra.run.smithi060.stdout:lvcreate is present 2024-02-26T10:33:17.599 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-26T10:33:17.600 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-26T10:33:17.614 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-26T10:33:17.614 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T10:33:17.629 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-02-26T10:33:17.645 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-02-26T10:33:17.645 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-02-26T10:33:17.645 INFO:teuthology.orchestra.run.smithi060.stdout:Host looks OK 2024-02-26T10:33:17.645 INFO:teuthology.orchestra.run.smithi060.stdout:Cluster fsid: 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:33:17.645 INFO:teuthology.orchestra.run.smithi060.stdout:Acquiring lock 140086396014544 on /run/cephadm/51464d50-d492-11ee-95c3-87774f69a715.lock 2024-02-26T10:33:17.646 INFO:teuthology.orchestra.run.smithi060.stdout:Lock 140086396014544 acquired on /run/cephadm/51464d50-d492-11ee-95c3-87774f69a715.lock 2024-02-26T10:33:17.646 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 3300 ... 2024-02-26T10:33:17.647 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 6789 ... 2024-02-26T10:33:17.647 INFO:teuthology.orchestra.run.smithi060.stdout:Base mon IP(s) is [172.21.15.60:3300, 172.21.15.60:6789], mon addrv is [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-02-26T10:33:17.656 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.60 metric 100 2024-02-26T10:33:17.656 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.60 metric 100 2024-02-26T10:33:17.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-26T10:33:17.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-26T10:33:17.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-26T10:33:17.671 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-26T10:33:17.671 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-26T10:33:17.671 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-26T10:33:17.671 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-26T10:33:17.671 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-02-26T10:33:17.671 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-26T10:33:17.672 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-02-26T10:33:17.672 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-02-26T10:33:17.672 INFO:teuthology.orchestra.run.smithi060.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-26T10:33:17.673 INFO:teuthology.orchestra.run.smithi060.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-26T10:33:17.674 INFO:teuthology.orchestra.run.smithi060.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893... 2024-02-26T10:33:18.331 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 2024-02-26T10:33:18.331 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893... 2024-02-26T10:33:18.331 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Getting image source signatures 2024-02-26T10:33:18.331 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-02-26T10:33:18.331 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:eb2989c95feacdc2bcad3201e792ab09d824b02b241a6cd3d81317c998812412 2024-02-26T10:33:18.331 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying config sha256:07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 2024-02-26T10:33:18.331 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-26T10:33:19.530 INFO:teuthology.orchestra.run.smithi060.stdout:ceph: stdout ceph version 19.0.0-1686-g6085b91f (6085b91fefeb74bf4310344a87d81d3217ef7893) squid (dev) 2024-02-26T10:33:19.531 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph version: ceph version 19.0.0-1686-g6085b91f (6085b91fefeb74bf4310344a87d81d3217ef7893) squid (dev) 2024-02-26T10:33:19.531 INFO:teuthology.orchestra.run.smithi060.stdout:Extracting ceph user uid/gid from container image... 2024-02-26T10:33:20.658 INFO:teuthology.orchestra.run.smithi060.stdout:stat: stdout 167 167 2024-02-26T10:33:20.658 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial keys... 2024-02-26T10:33:21.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQDxaNxlYD30BxAAvaB5yTQewNtpAHL+afBZ6g== 2024-02-26T10:33:22.890 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQDyaNxlkQU0ERAAssAju2ZJ19HKHbN0nvL9RA== 2024-02-26T10:33:24.153 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQDzaNxli2kJFxAA0J2Q2aj81tWtwLUoAGfPKw== 2024-02-26T10:33:24.154 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial monmap... 2024-02-26T10:33:25.189 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-26T10:33:25.190 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-26T10:33:25.190 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:33:25.190 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-26T10:33:25.190 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool for a [v2:172.21.15.60:3300,v1:172.21.15.60:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-26T10:33:25.190 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-02-26T10:33:25.190 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: set fsid to 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:33:25.190 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-26T10:33:25.190 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:33:25.190 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mon... 2024-02-26T10:33:26.338 INFO:teuthology.orchestra.run.smithi060.stdout:create mon.a on 2024-02-26T10:33:26.723 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-26T10:33:26.978 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-26T10:33:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-51464d50-d492-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-51464d50-d492-11ee-95c3-87774f69a715.target. 2024-02-26T10:33:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-51464d50-d492-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-51464d50-d492-11ee-95c3-87774f69a715.target. 2024-02-26T10:33:27.679 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.a 2024-02-26T10:33:27.680 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.a.service: Unit ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.a.service not loaded. 2024-02-26T10:33:27.978 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-51464d50-d492-11ee-95c3-87774f69a715.target.wants/ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.a.service → /etc/systemd/system/ceph-51464d50-d492-11ee-95c3-87774f69a715@.service. 2024-02-26T10:33:28.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:28 smithi060 podman[31154]: 2024-02-26 10:33:28.61309997 +0000 UTC m=+0.379478737 container init 6e372e5cb46371946cb48dabe60b7eca8bad46e324436e36152fb3851ac27737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:33:28.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:28 smithi060 podman[31154]: 2024-02-26 10:33:28.623534142 +0000 UTC m=+0.389912910 container start 6e372e5cb46371946cb48dabe60b7eca8bad46e324436e36152fb3851ac27737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:33:28.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:28 smithi060 ceph-mon[31174]: mkfs 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:33:28.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:28 smithi060 ceph-mon[31174]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T10:33:28.920 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T10:33:28.920 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-02-26T10:33:28.940 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T10:33:28.940 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T10:33:28.940 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-02-26T10:33:28.940 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-02-26T10:33:28.941 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon to start... 2024-02-26T10:33:28.941 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon... 2024-02-26T10:33:29.150 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:28 smithi060 bash[31154]: 6e372e5cb46371946cb48dabe60b7eca8bad46e324436e36152fb3851ac27737 2024-02-26T10:33:29.150 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:28 smithi060 systemd[1]: Started Ceph mon.a for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:33:30.015 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:29 smithi060 ceph-mon[31174]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T10:33:30.016 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:29 smithi060 ceph-mon[31174]: monmap e1: 1 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:33:30.016 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:29 smithi060 ceph-mon[31174]: fsmap 2024-02-26T10:33:30.016 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:29 smithi060 ceph-mon[31174]: osdmap e1: 0 total, 0 up, 0 in 2024-02-26T10:33:30.016 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:29 smithi060 ceph-mon[31174]: mgrmap e1: no daemons active 2024-02-26T10:33:30.016 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:29 smithi060 ceph-mon[31174]: from='client.? 172.21.15.60:0/4228494518' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-26T10:33:30.280 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout cluster: 2024-02-26T10:33:30.280 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout id: 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:33:30.280 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-26T10:33:30.280 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:30.280 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout services: 2024-02-26T10:33:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.967681s) 2024-02-26T10:33:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-26T10:33:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-26T10:33:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout data: 2024-02-26T10:33:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-26T10:33:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pgs: 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.569+0000 7f7620400700 1 Processor -- start 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.569+0000 7f7620400700 1 -- start start 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.569+0000 7f7620400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f7618107a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.569+0000 7f7620400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618108040 con 0x7f7618107650 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.570+0000 7f761da00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f7618107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.571+0000 7f761da00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f7618107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48846/0 (socket says 172.21.15.60:48846) 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.571+0000 7f761da00700 1 -- 172.21.15.60:0/3026757908 learned_addr learned my addr 172.21.15.60:0/3026757908 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.571+0000 7f761da00700 1 -- 172.21.15.60:0/3026757908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7618108800 con 0x7f7618107650 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.572+0000 7f761da00700 1 --2- 172.21.15.60:0/3026757908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f7618107a70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f760c0098a0 tx=0x7f760c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3698ba39efa711b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:30.282 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.572+0000 7f7617e00700 1 -- 172.21.15.60:0/3026757908 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f760c004160 con 0x7f7618107650 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.572+0000 7f7617e00700 1 -- 172.21.15.60:0/3026757908 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c004300 con 0x7f7618107650 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.572+0000 7f7617e00700 1 -- 172.21.15.60:0/3026757908 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f760c0036a0 con 0x7f7618107650 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.573+0000 7f7620400700 1 -- 172.21.15.60:0/3026757908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 msgr2=0x7f7618107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.573+0000 7f7620400700 1 --2- 172.21.15.60:0/3026757908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f7618107a70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f760c0098a0 tx=0x7f760c009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.573+0000 7f7620400700 1 -- 172.21.15.60:0/3026757908 shutdown_connections 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.573+0000 7f7620400700 1 --2- 172.21.15.60:0/3026757908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f7618107a70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.573+0000 7f7620400700 1 -- 172.21.15.60:0/3026757908 >> 172.21.15.60:0/3026757908 conn(0x7f7618102cf0 msgr2=0x7f7618105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.573+0000 7f7620400700 1 -- 172.21.15.60:0/3026757908 shutdown_connections 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.573+0000 7f7620400700 1 -- 172.21.15.60:0/3026757908 wait complete. 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.574+0000 7f7620400700 1 Processor -- start 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.574+0000 7f7620400700 1 -- start start 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.575+0000 7f7620400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f761807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:30.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.575+0000 7f7620400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618108d30 con 0x7f7618107650 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.575+0000 7f761da00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f761807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.575+0000 7f761da00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f761807cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48862/0 (socket says 172.21.15.60:48862) 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.575+0000 7f761da00700 1 -- 172.21.15.60:0/4228494518 learned_addr learned my addr 172.21.15.60:0/4228494518 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.575+0000 7f761da00700 1 -- 172.21.15.60:0/4228494518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761807d0b0 con 0x7f7618107650 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.576+0000 7f761da00700 1 --2- 172.21.15.60:0/4228494518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f761807cb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f760c000c00 tx=0x7f760c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.576+0000 7f7616000700 1 -- 172.21.15.60:0/4228494518 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f760c003db0 con 0x7f7618107650 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.576+0000 7f7616000700 1 -- 172.21.15.60:0/4228494518 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c011630 con 0x7f7618107650 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.576+0000 7f7616000700 1 -- 172.21.15.60:0/4228494518 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f760c011950 con 0x7f7618107650 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.576+0000 7f7620400700 1 -- 172.21.15.60:0/4228494518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761807b2a0 con 0x7f7618107650 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.576+0000 7f7620400700 1 -- 172.21.15.60:0/4228494518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761807b7e0 con 0x7f7618107650 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.576+0000 7f7616000700 1 -- 172.21.15.60:0/4228494518 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f760c011af0 con 0x7f7618107650 2024-02-26T10:33:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.576+0000 7f7616000700 1 -- 172.21.15.60:0/4228494518 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f760c01c450 con 0x7f7618107650 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.577+0000 7f7620400700 1 -- 172.21.15.60:0/4228494518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7618068160 con 0x7f7618107650 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.581+0000 7f7616000700 1 -- 172.21.15.60:0/4228494518 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f760c032020 con 0x7f7618107650 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.661+0000 7f7620400700 1 -- 172.21.15.60:0/4228494518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f76181945a0 con 0x7f7618107650 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.662+0000 7f7616000700 1 -- 172.21.15.60:0/4228494518 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f760c004260 con 0x7f7618107650 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.663+0000 7f7620400700 1 -- 172.21.15.60:0/4228494518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 msgr2=0x7f761807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.663+0000 7f7620400700 1 --2- 172.21.15.60:0/4228494518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f761807cb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f760c000c00 tx=0x7f760c004750 comp rx=0 tx=0).stop 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.663+0000 7f7620400700 1 -- 172.21.15.60:0/4228494518 shutdown_connections 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.663+0000 7f7620400700 1 --2- 172.21.15.60:0/4228494518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7618107650 0x7f761807cb70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.663+0000 7f7620400700 1 -- 172.21.15.60:0/4228494518 >> 172.21.15.60:0/4228494518 conn(0x7f7618102cf0 msgr2=0x7f76181919a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.664+0000 7f7620400700 1 -- 172.21.15.60:0/4228494518 shutdown_connections 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:29.664+0000 7f7620400700 1 -- 172.21.15.60:0/4228494518 wait complete. 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:mon is available 2024-02-26T10:33:30.286 INFO:teuthology.orchestra.run.smithi060.stdout:Assimilating anything we can from ceph.conf... 2024-02-26T10:33:31.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:31 smithi060 ceph-mon[31174]: from='client.? 172.21.15.60:0/2782654340' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-26T10:33:31.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:31 smithi060 ceph-mon[31174]: from='client.? 172.21.15.60:0/2782654340' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-26T10:33:31.639 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:31.639 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-02-26T10:33:31.639 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-26T10:33:31.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.931+0000 7f4ac7c00700 1 Processor -- start 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.932+0000 7f4ac7c00700 1 -- start start 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.932+0000 7f4ac7c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.932+0000 7f4ac7c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac010e920 con 0x7f4ac010dfc0 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.933+0000 7f4ac5200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.933+0000 7f4ac5200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac010e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48876/0 (socket says 172.21.15.60:48876) 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.933+0000 7f4ac5200700 1 -- 172.21.15.60:0/2047446736 learned_addr learned my addr 172.21.15.60:0/2047446736 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.933+0000 7f4ac5200700 1 -- 172.21.15.60:0/2047446736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac010eaa0 con 0x7f4ac010dfc0 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.934+0000 7f4ac5200700 1 --2- 172.21.15.60:0/2047446736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac010e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40098a0 tx=0x7f4ab4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fa8c04721484fc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.934+0000 7f4abf400700 1 -- 172.21.15.60:0/2047446736 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ab4004160 con 0x7f4ac010dfc0 2024-02-26T10:33:31.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.934+0000 7f4abf400700 1 -- 172.21.15.60:0/2047446736 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab4004300 con 0x7f4ac010dfc0 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.934+0000 7f4abf400700 1 -- 172.21.15.60:0/2047446736 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ab40036a0 con 0x7f4ac010dfc0 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.935+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2047446736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 msgr2=0x7f4ac010e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.935+0000 7f4ac7c00700 1 --2- 172.21.15.60:0/2047446736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac010e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40098a0 tx=0x7f4ab4009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.935+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2047446736 shutdown_connections 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.935+0000 7f4ac7c00700 1 --2- 172.21.15.60:0/2047446736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac010e3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.935+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2047446736 >> 172.21.15.60:0/2047446736 conn(0x7f4ac007ce70 msgr2=0x7f4ac007f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.935+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2047446736 shutdown_connections 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.935+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2047446736 wait complete. 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.936+0000 7f4ac7c00700 1 Processor -- start 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.936+0000 7f4ac7c00700 1 -- start start 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.937+0000 7f4ac7c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac01a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.937+0000 7f4ac7c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac010f120 con 0x7f4ac010dfc0 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.937+0000 7f4ac5200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac01a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.937+0000 7f4ac5200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac01a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48884/0 (socket says 172.21.15.60:48884) 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.937+0000 7f4ac5200700 1 -- 172.21.15.60:0/2782654340 learned_addr learned my addr 172.21.15.60:0/2782654340 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.938+0000 7f4ac5200700 1 -- 172.21.15.60:0/2782654340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac01a1840 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.938+0000 7f4ac5200700 1 --2- 172.21.15.60:0/2782654340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac01a1300 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4000c00 tx=0x7f4ab4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.938+0000 7f4abd600700 1 -- 172.21.15.60:0/2782654340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ab4003db0 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.938+0000 7f4abd600700 1 -- 172.21.15.60:0/2782654340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab4011630 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.938+0000 7f4abd600700 1 -- 172.21.15.60:0/2782654340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ab4011950 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.938+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2782654340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac01a1b30 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.938+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2782654340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac01a47c0 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.939+0000 7f4abd600700 1 -- 172.21.15.60:0/2782654340 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4ab401a460 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.939+0000 7f4abd600700 1 -- 172.21.15.60:0/2782654340 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4ab401b780 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.940+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2782654340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac0068160 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:30.942+0000 7f4abd600700 1 -- 172.21.15.60:0/2782654340 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4ab4031020 con 0x7f4ac010dfc0 2024-02-26T10:33:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.020+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2782654340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4ac002da00 con 0x7f4ac010dfc0 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.026+0000 7f4abd600700 1 -- 172.21.15.60:0/2782654340 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f4ab401b980 con 0x7f4ac010dfc0 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.027+0000 7f4abd600700 1 -- 172.21.15.60:0/2782654340 <== mon.0 v2:172.21.15.60:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ab401ad00 con 0x7f4ac010dfc0 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.028+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2782654340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 msgr2=0x7f4ac01a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.028+0000 7f4ac7c00700 1 --2- 172.21.15.60:0/2782654340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac01a1300 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4000c00 tx=0x7f4ab4004750 comp rx=0 tx=0).stop 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.028+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2782654340 shutdown_connections 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.028+0000 7f4ac7c00700 1 --2- 172.21.15.60:0/2782654340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ac010dfc0 0x7f4ac01a1300 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.028+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2782654340 >> 172.21.15.60:0/2782654340 conn(0x7f4ac007ce70 msgr2=0x7f4ac0197ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.028+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2782654340 shutdown_connections 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:31.028+0000 7f4ac7c00700 1 -- 172.21.15.60:0/2782654340 wait complete. 2024-02-26T10:33:31.644 INFO:teuthology.orchestra.run.smithi060.stdout:Generating new minimal ceph.conf... 2024-02-26T10:33:32.572 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:32 smithi060 ceph-mon[31174]: from='client.? 172.21.15.60:0/2260137586' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:33:32.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.171+0000 7f62cfe00700 1 Processor -- start 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.171+0000 7f62cfe00700 1 -- start start 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.172+0000 7f62cfe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d010f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.172+0000 7f62cfe00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d010fa20 con 0x7f62d010f030 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.173+0000 7f62cea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d010f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.173+0000 7f62cea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d010f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48890/0 (socket says 172.21.15.60:48890) 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.173+0000 7f62cea00700 1 -- 172.21.15.60:0/1072224919 learned_addr learned my addr 172.21.15.60:0/1072224919 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.173+0000 7f62cea00700 1 -- 172.21.15.60:0/1072224919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d010dcd0 con 0x7f62d010f030 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.174+0000 7f62cea00700 1 --2- 172.21.15.60:0/1072224919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d010f450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098a0 tx=0x7f62c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8712c427a8676e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.174+0000 7f62ce000700 1 -- 172.21.15.60:0/1072224919 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62c0004160 con 0x7f62d010f030 2024-02-26T10:33:32.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.174+0000 7f62ce000700 1 -- 172.21.15.60:0/1072224919 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62c0004300 con 0x7f62d010f030 2024-02-26T10:33:32.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.174+0000 7f62ce000700 1 -- 172.21.15.60:0/1072224919 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62c0011b50 con 0x7f62d010f030 2024-02-26T10:33:32.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.175+0000 7f62cfe00700 1 -- 172.21.15.60:0/1072224919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 msgr2=0x7f62d010f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:32.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.175+0000 7f62cfe00700 1 --2- 172.21.15.60:0/1072224919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d010f450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098a0 tx=0x7f62c0009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:32.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.175+0000 7f62cfe00700 1 -- 172.21.15.60:0/1072224919 shutdown_connections 2024-02-26T10:33:32.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.175+0000 7f62cfe00700 1 --2- 172.21.15.60:0/1072224919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d010f450 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:32.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.175+0000 7f62cfe00700 1 -- 172.21.15.60:0/1072224919 >> 172.21.15.60:0/1072224919 conn(0x7f62d007ce10 msgr2=0x7f62d007f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:32.878 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.175+0000 7f62cfe00700 1 -- 172.21.15.60:0/1072224919 shutdown_connections 2024-02-26T10:33:32.878 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.175+0000 7f62cfe00700 1 -- 172.21.15.60:0/1072224919 wait complete. 2024-02-26T10:33:32.878 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.177+0000 7f62cfe00700 1 Processor -- start 2024-02-26T10:33:32.878 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.177+0000 7f62cfe00700 1 -- start start 2024-02-26T10:33:32.878 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.177+0000 7f62cfe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d01a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:32.878 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.177+0000 7f62cfe00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0112090 con 0x7f62d010f030 2024-02-26T10:33:32.878 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.178+0000 7f62cea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d01a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:32.878 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.178+0000 7f62cea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d01a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48896/0 (socket says 172.21.15.60:48896) 2024-02-26T10:33:32.878 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.178+0000 7f62cea00700 1 -- 172.21.15.60:0/2260137586 learned_addr learned my addr 172.21.15.60:0/2260137586 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:32.879 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.178+0000 7f62cea00700 1 -- 172.21.15.60:0/2260137586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d01a1eb0 con 0x7f62d010f030 2024-02-26T10:33:32.879 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.178+0000 7f62cea00700 1 --2- 172.21.15.60:0/2260137586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d01a1970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62c0000c00 tx=0x7f62c0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:32.879 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.179+0000 7f62c7400700 1 -- 172.21.15.60:0/2260137586 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62c0003b20 con 0x7f62d010f030 2024-02-26T10:33:32.879 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.179+0000 7f62c7400700 1 -- 172.21.15.60:0/2260137586 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62c0003cc0 con 0x7f62d010f030 2024-02-26T10:33:32.879 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.179+0000 7f62c7400700 1 -- 172.21.15.60:0/2260137586 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62c0024910 con 0x7f62d010f030 2024-02-26T10:33:32.879 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.179+0000 7f62cfe00700 1 -- 172.21.15.60:0/2260137586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d01a2140 con 0x7f62d010f030 2024-02-26T10:33:32.880 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.179+0000 7f62cfe00700 1 -- 172.21.15.60:0/2260137586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d01a4e30 con 0x7f62d010f030 2024-02-26T10:33:32.880 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.179+0000 7f62c7400700 1 -- 172.21.15.60:0/2260137586 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f62c0022020 con 0x7f62d010f030 2024-02-26T10:33:32.880 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.179+0000 7f62c7400700 1 -- 172.21.15.60:0/2260137586 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f62c0011e00 con 0x7f62d010f030 2024-02-26T10:33:32.880 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.180+0000 7f62cfe00700 1 -- 172.21.15.60:0/2260137586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d0063380 con 0x7f62d010f030 2024-02-26T10:33:32.880 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.183+0000 7f62c7400700 1 -- 172.21.15.60:0/2260137586 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f62c001acd0 con 0x7f62d010f030 2024-02-26T10:33:32.880 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.262+0000 7f62cfe00700 1 -- 172.21.15.60:0/2260137586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f62d01a5000 con 0x7f62d010f030 2024-02-26T10:33:32.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.263+0000 7f62c7400700 1 -- 172.21.15.60:0/2260137586 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f62c0032030 con 0x7f62d010f030 2024-02-26T10:33:32.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.265+0000 7f62cfe00700 1 -- 172.21.15.60:0/2260137586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 msgr2=0x7f62d01a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:32.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.265+0000 7f62cfe00700 1 --2- 172.21.15.60:0/2260137586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d01a1970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62c0000c00 tx=0x7f62c0011c10 comp rx=0 tx=0).stop 2024-02-26T10:33:32.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.265+0000 7f62cfe00700 1 -- 172.21.15.60:0/2260137586 shutdown_connections 2024-02-26T10:33:32.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.265+0000 7f62cfe00700 1 --2- 172.21.15.60:0/2260137586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d010f030 0x7f62d01a1970 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:32.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.265+0000 7f62cfe00700 1 -- 172.21.15.60:0/2260137586 >> 172.21.15.60:0/2260137586 conn(0x7f62d007ce10 msgr2=0x7f62d007e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:32.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.265+0000 7f62cfe00700 1 -- 172.21.15.60:0/2260137586 shutdown_connections 2024-02-26T10:33:32.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:32.265+0000 7f62cfe00700 1 -- 172.21.15.60:0/2260137586 wait complete. 2024-02-26T10:33:32.883 INFO:teuthology.orchestra.run.smithi060.stdout:Restarting the monitor... 2024-02-26T10:33:33.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:32 smithi060 systemd[1]: Stopping Ceph mon.a for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:33:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31170]: 2024-02-26T10:33:33.367+0000 7f65ed200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:33:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31170]: 2024-02-26T10:33:33.367+0000 7f65ed200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-26T10:33:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:33 smithi060 podman[31419]: 2024-02-26 10:33:33.511431388 +0000 UTC m=+0.274608004 container died 6e372e5cb46371946cb48dabe60b7eca8bad46e324436e36152fb3851ac27737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, io.buildah.version=1.33.5) 2024-02-26T10:33:34.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:33 smithi060 podman[31419]: 2024-02-26 10:33:33.900767039 +0000 UTC m=+0.663943679 container cleanup 6e372e5cb46371946cb48dabe60b7eca8bad46e324436e36152fb3851ac27737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:33:34.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:33 smithi060 bash[31419]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a 2024-02-26T10:33:34.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:34 smithi060 podman[31433]: 2024-02-26 10:33:34.068223162 +0000 UTC m=+0.544127061 container remove 6e372e5cb46371946cb48dabe60b7eca8bad46e324436e36152fb3851ac27737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-26T10:33:34.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:34 smithi060 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-26T10:33:34.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:34 smithi060 systemd[1]: Stopped Ceph mon.a for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:33:34.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:34 smithi060 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.a.service: Consumed 1.958s CPU time. 2024-02-26T10:33:34.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:34 smithi060 systemd[1]: Starting Ceph mon.a for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:33:35.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:34 smithi060 podman[31520]: 2024-02-26 10:33:34.993944166 +0000 UTC m=+0.119774542 container create 0eaaccfac6f94a25e2b61ff8a6c4cf8e1e69ecd095d2caf69584842c4b8ea18c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:33:35.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 podman[31520]: 2024-02-26 10:33:34.915354443 +0000 UTC m=+0.041184817 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:33:35.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 podman[31520]: 2024-02-26 10:33:35.214308787 +0000 UTC m=+0.340139228 container init 0eaaccfac6f94a25e2b61ff8a6c4cf8e1e69ecd095d2caf69584842c4b8ea18c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T10:33:35.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 podman[31520]: 2024-02-26 10:33:35.225536699 +0000 UTC m=+0.351367103 container start 0eaaccfac6f94a25e2b61ff8a6c4cf8e1e69ecd095d2caf69584842c4b8ea18c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-26T10:33:35.361 INFO:teuthology.orchestra.run.smithi060.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-26T10:33:35.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: set uid:gid to 167:167 (ceph:ceph) 2024-02-26T10:33:35.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: ceph version 19.0.0-1686-g6085b91f (6085b91fefeb74bf4310344a87d81d3217ef7893) squid (dev), process ceph-mon, pid 2 2024-02-26T10:33:35.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: pidfile_write: ignore empty --pid-file 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: load: jerasure load: lrc 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: RocksDB version: 7.9.2 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Git sha 0 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Compile date 2024-02-26 07:04:49 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: DB SUMMARY 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: DB Session ID: DNQGZKH746OMXQTOC5J2 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: CURRENT file: CURRENT 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: IDENTITY file: IDENTITY 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99864 ; 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.error_if_exists: 0 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.create_if_missing: 0 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.paranoid_checks: 1 2024-02-26T10:33:35.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.env: 0x558caa555b20 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.fs: PosixFileSystem 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.info_log: 0x558cabdf9920 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_file_opening_threads: 16 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.statistics: (nil) 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.use_fsync: 0 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_log_file_size: 0 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.keep_log_file_num: 1000 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.recycle_log_file_num: 0 2024-02-26T10:33:35.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.allow_fallocate: 1 2024-02-26T10:33:35.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.allow_mmap_reads: 0 2024-02-26T10:33:35.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.allow_mmap_writes: 0 2024-02-26T10:33:35.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.use_direct_reads: 0 2024-02-26T10:33:35.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-26T10:33:35.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.create_missing_column_families: 0 2024-02-26T10:33:35.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.db_log_dir: 2024-02-26T10:33:35.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.wal_dir: 2024-02-26T10:33:35.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-26T10:33:35.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.advise_random_on_open: 1 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.db_write_buffer_size: 0 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.write_buffer_manager: 0x558cabdcd400 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.rate_limiter: (nil) 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-26T10:33:35.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.wal_recovery_mode: 2 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.enable_thread_tracking: 0 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.enable_pipelined_write: 0 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.unordered_write: 0 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.row_cache: None 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.wal_filter: None 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.allow_ingest_behind: 0 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.two_write_queues: 0 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.manual_wal_flush: 0 2024-02-26T10:33:35.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.wal_compression: 0 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.atomic_flush: 0 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.log_readahead_size: 0 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.best_efforts_recovery: 0 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.allow_data_in_errors: 0 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.db_host_id: __hostname__ 2024-02-26T10:33:35.631 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-26T10:33:35.632 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_background_jobs: 2 2024-02-26T10:33:35.632 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_background_compactions: -1 2024-02-26T10:33:35.632 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_subcompactions: 1 2024-02-26T10:33:35.632 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-26T10:33:35.632 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-26T10:33:35.632 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-26T10:33:35.632 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_total_wal_size: 0 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_open_files: -1 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bytes_per_sync: 0 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_readahead_size: 0 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_background_flushes: -1 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Compression algorithms supported: 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: kZSTD supported: 0 2024-02-26T10:33:35.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: kXpressCompression supported: 0 2024-02-26T10:33:35.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: kLZ4HCCompression supported: 1 2024-02-26T10:33:35.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: kZlibCompression supported: 1 2024-02-26T10:33:35.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: kSnappyCompression supported: 1 2024-02-26T10:33:35.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: kLZ4Compression supported: 1 2024-02-26T10:33:35.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: kBZip2Compression supported: 0 2024-02-26T10:33:35.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-26T10:33:35.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-26T10:33:35.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-26T10:33:35.635 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-26T10:33:35.635 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-26T10:33:35.635 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.merge_operator: 2024-02-26T10:33:35.635 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_filter: None 2024-02-26T10:33:35.635 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_filter_factory: None 2024-02-26T10:33:35.635 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.sst_partitioner_factory: None 2024-02-26T10:33:35.635 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558cabdf9a20) 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks: 1 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_top_level_index_and_filter: 1 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_type: 0 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_index_type: 0 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_shortening: 1 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: checksum: 4 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: no_block_cache: 0 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache: 0x558cabdfd090 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_name: BinnedLRUCache 2024-02-26T10:33:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_options: 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: capacity : 536870912 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_shard_bits : 4 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: strict_capacity_limit : 0 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: high_pri_pool_ratio: 0.000 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_compressed: (nil) 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: persistent_cache: (nil) 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size: 4096 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size_deviation: 10 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_restart_interval: 16 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_block_restart_interval: 1 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: metadata_block_size: 4096 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: partition_filters: 0 2024-02-26T10:33:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout: use_delta_encoding: 1 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: filter_policy: bloomfilter 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: whole_key_filtering: 1 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: verify_compression: 0 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: read_amp_bytes_per_bit: 0 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: format_version: 5 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_index_compression: 1 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_align: 0 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: max_auto_readahead_size: 262144 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: prepopulate_block_cache: 0 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: initial_auto_readahead_size: 8192 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_file_reads_for_auto_readahead: 2 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.write_buffer_size: 33554432 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_write_buffer_number: 2 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression: NoCompression 2024-02-26T10:33:35.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression: Disabled 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.prefix_extractor: nullptr 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.num_levels: 7 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-26T10:33:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression_opts.level: 32767 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression_opts.strategy: 0 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression_opts.enabled: false 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.target_file_size_base: 67108864 2024-02-26T10:33:35.640 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-26T10:33:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.arena_block_size: 1048576 2024-02-26T10:33:35.642 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-26T10:33:35.642 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-26T10:33:35.642 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.disable_auto_compactions: 0 2024-02-26T10:33:35.642 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-26T10:33:35.642 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-26T10:33:35.642 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.table_properties_collectors: 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.inplace_update_support: 0 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-26T10:33:35.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.bloom_locality: 0 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.max_successive_merges: 0 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.paranoid_file_checks: 0 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.force_consistency_checks: 1 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.report_bg_io_stats: 0 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.ttl: 2592000 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-26T10:33:35.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-26T10:33:35.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.enable_blob_files: false 2024-02-26T10:33:35.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.min_blob_size: 0 2024-02-26T10:33:35.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.blob_file_size: 268435456 2024-02-26T10:33:35.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-26T10:33:35.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-26T10:33:35.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-26T10:33:35.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-26T10:33:35.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-26T10:33:35.646 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.blob_file_starting_level: 0 2024-02-26T10:33:35.646 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-26T10:33:35.646 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-26T10:33:35.646 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-26T10:33:35.646 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f65f0278-472b-4bf3-b6a5-21d0d29fff41 2024-02-26T10:33:35.646 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708943615284557, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708943615286546, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95041, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 93039, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13801, "raw_average_key_size": 50, "raw_value_size": 85524, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708943615, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f65f0278-472b-4bf3-b6a5-21d0d29fff41", "db_session_id": "DNQGZKH746OMXQTOC5J2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708943615286789, "job": 1, "event": "recovery_finished"} 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558cabeee000 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: DB pointer 0x558cabed8000 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** DB Stats ** 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T10:33:35.647 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: L0 2/0 94.65 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 68.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: Sum 2/0 94.65 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 68.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 68.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-02-26T10:33:35.648 INFO:journalctl@ceph.mon.a.smithi060.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 68.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative compaction: 0.00 GB write, 12.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval compaction: 0.00 GB write, 12.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-26T10:33:35.649 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache BinnedLRUCache@0x558cabdfd090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: starting mon.a rank 0 at public addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] at bind addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mon.a@-1(???) e1 preinit fsid 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mon.a@-1(???).mds e1 new map 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mon.a@-1(???).mds e1 print_map 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout: e1 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout: legacy client fscid: -1 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout: No filesystems configured 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-26T10:33:35.650 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 bash[31520]: 0eaaccfac6f94a25e2b61ff8a6c4cf8e1e69ecd095d2caf69584842c4b8ea18c 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: monmap e1: 1 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: fsmap 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 systemd[1]: Started Ceph mon.a for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: osdmap e1: 0 total, 0 up, 0 in 2024-02-26T10:33:35.651 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:35 smithi060 ceph-mon[31539]: mgrmap e1: no daemons active 2024-02-26T10:33:36.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.931+0000 7f33ab400700 1 Processor -- start 2024-02-26T10:33:36.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.931+0000 7f33ab400700 1 -- start start 2024-02-26T10:33:36.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.932+0000 7f33ab400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a4107a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:36.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.932+0000 7f33ab400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4108060 con 0x7f33a4107670 2024-02-26T10:33:36.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.932+0000 7f33a8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a4107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:36.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.932+0000 7f33a8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a4107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53886/0 (socket says 172.21.15.60:53886) 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.933+0000 7f33a8a00700 1 -- 172.21.15.60:0/2142926705 learned_addr learned my addr 172.21.15.60:0/2142926705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.933+0000 7f33a8a00700 1 -- 172.21.15.60:0/2142926705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4108820 con 0x7f33a4107670 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.933+0000 7f33a8a00700 1 --2- 172.21.15.60:0/2142926705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a4107a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f33980098a0 tx=0x7f3398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5223b766852dd1fb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.934+0000 7f33a3400700 1 -- 172.21.15.60:0/2142926705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3398004160 con 0x7f33a4107670 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.934+0000 7f33a3400700 1 -- 172.21.15.60:0/2142926705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3398004300 con 0x7f33a4107670 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.934+0000 7f33a3400700 1 -- 172.21.15.60:0/2142926705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3398011b50 con 0x7f33a4107670 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.935+0000 7f33ab400700 1 -- 172.21.15.60:0/2142926705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 msgr2=0x7f33a4107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.935+0000 7f33ab400700 1 --2- 172.21.15.60:0/2142926705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a4107a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f33980098a0 tx=0x7f3398009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.935+0000 7f33ab400700 1 -- 172.21.15.60:0/2142926705 shutdown_connections 2024-02-26T10:33:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.935+0000 7f33ab400700 1 --2- 172.21.15.60:0/2142926705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a4107a90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:36.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.935+0000 7f33ab400700 1 -- 172.21.15.60:0/2142926705 >> 172.21.15.60:0/2142926705 conn(0x7f33a4102d50 msgr2=0x7f33a4105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:36.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.935+0000 7f33ab400700 1 -- 172.21.15.60:0/2142926705 shutdown_connections 2024-02-26T10:33:36.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.935+0000 7f33ab400700 1 -- 172.21.15.60:0/2142926705 wait complete. 2024-02-26T10:33:36.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.936+0000 7f33ab400700 1 Processor -- start 2024-02-26T10:33:36.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.936+0000 7f33ab400700 1 -- start start 2024-02-26T10:33:36.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.937+0000 7f33ab400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:36.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.937+0000 7f33ab400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a410bbd0 con 0x7f33a4107670 2024-02-26T10:33:36.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.937+0000 7f33a8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:36.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.937+0000 7f33a8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a407cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53896/0 (socket says 172.21.15.60:53896) 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.937+0000 7f33a8a00700 1 -- 172.21.15.60:0/1848243138 learned_addr learned my addr 172.21.15.60:0/1848243138 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.937+0000 7f33a8a00700 1 -- 172.21.15.60:0/1848243138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a407d0b0 con 0x7f33a4107670 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.938+0000 7f33a8a00700 1 --2- 172.21.15.60:0/1848243138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a407cb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3398011850 tx=0x7f3398011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.938+0000 7f33a1600700 1 -- 172.21.15.60:0/1848243138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3398003af0 con 0x7f33a4107670 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.938+0000 7f33a1600700 1 -- 172.21.15.60:0/1848243138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3398020070 con 0x7f33a4107670 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.938+0000 7f33a1600700 1 -- 172.21.15.60:0/1848243138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33980195a0 con 0x7f33a4107670 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.938+0000 7f33ab400700 1 -- 172.21.15.60:0/1848243138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a407b2a0 con 0x7f33a4107670 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.938+0000 7f33ab400700 1 -- 172.21.15.60:0/1848243138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a407b7e0 con 0x7f33a4107670 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.939+0000 7f33a1600700 1 -- 172.21.15.60:0/1848243138 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3398019be0 con 0x7f33a4107670 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.939+0000 7f33a1600700 1 -- 172.21.15.60:0/1848243138 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3398011e00 con 0x7f33a4107670 2024-02-26T10:33:36.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.939+0000 7f33ab400700 1 -- 172.21.15.60:0/1848243138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a4068160 con 0x7f33a4107670 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:35.943+0000 7f33a1600700 1 -- 172.21.15.60:0/1848243138 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3398003c90 con 0x7f33a4107670 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:36.020+0000 7f33ab400700 1 -- 172.21.15.60:0/1848243138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f33a407bb20 con 0x7f33a4107670 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:36.025+0000 7f33a1600700 1 -- 172.21.15.60:0/1848243138 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3398025460 con 0x7f33a4107670 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:36.026+0000 7f33ab400700 1 -- 172.21.15.60:0/1848243138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 msgr2=0x7f33a407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:36.026+0000 7f33ab400700 1 --2- 172.21.15.60:0/1848243138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a407cb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3398011850 tx=0x7f3398011c10 comp rx=0 tx=0).stop 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:36.026+0000 7f33ab400700 1 -- 172.21.15.60:0/1848243138 shutdown_connections 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:36.026+0000 7f33ab400700 1 --2- 172.21.15.60:0/1848243138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33a4107670 0x7f33a407cb70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:36.026+0000 7f33ab400700 1 -- 172.21.15.60:0/1848243138 >> 172.21.15.60:0/1848243138 conn(0x7f33a4102d50 msgr2=0x7f33a4103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:36.026+0000 7f33ab400700 1 -- 172.21.15.60:0/1848243138 shutdown_connections 2024-02-26T10:33:36.566 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:36.027+0000 7f33ab400700 1 -- 172.21.15.60:0/1848243138 wait complete. 2024-02-26T10:33:36.567 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-26T10:33:36.580 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-26T10:33:36.580 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mgr... 2024-02-26T10:33:36.582 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-26T10:33:36.583 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-26T10:33:37.030 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.x 2024-02-26T10:33:37.031 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.x.service: Unit ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.x.service not loaded. 2024-02-26T10:33:37.288 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-51464d50-d492-11ee-95c3-87774f69a715.target.wants/ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-51464d50-d492-11ee-95c3-87774f69a715@.service. 2024-02-26T10:33:37.319 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:37 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1848243138' entity='client.admin' 2024-02-26T10:33:37.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:37 smithi060 systemd[1]: Starting Ceph mgr.x for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:33:37.938 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:37 smithi060 podman[31755]: 2024-02-26 10:33:37.586884155 +0000 UTC m=+0.033317186 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:33:37.938 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:37 smithi060 podman[31755]: 2024-02-26 10:33:37.687385586 +0000 UTC m=+0.133818654 container create 397be5014c6afccc5174500766bf811ab97d364fb9c40f0badea427ceb944035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD) 2024-02-26T10:33:37.939 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:37 smithi060 podman[31755]: 2024-02-26 10:33:37.924719871 +0000 UTC m=+0.371152959 container init 397be5014c6afccc5174500766bf811ab97d364fb9c40f0badea427ceb944035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-26T10:33:37.939 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:37 smithi060 podman[31755]: 2024-02-26 10:33:37.934737837 +0000 UTC m=+0.381170911 container start 397be5014c6afccc5174500766bf811ab97d364fb9c40f0badea427ceb944035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:33:38.039 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T10:33:38.039 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-02-26T10:33:38.055 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T10:33:38.055 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T10:33:38.055 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-02-26T10:33:38.055 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-02-26T10:33:38.073 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T10:33:38.073 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-02-26T10:33:38.088 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T10:33:38.088 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T10:33:38.089 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-02-26T10:33:38.089 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-26T10:33:38.089 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr to start... 2024-02-26T10:33:38.089 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr... 2024-02-26T10:33:38.237 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:38 smithi060 bash[31755]: 397be5014c6afccc5174500766bf811ab97d364fb9c40f0badea427ceb944035 2024-02-26T10:33:38.237 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:38 smithi060 systemd[1]: Started Ceph mgr.x for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:33:38.747 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:38 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:38.738+0000 7f878b602200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T10:33:39.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:38 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2877582759' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T10:33:39.058 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:38 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:38.877+0000 7f878b602200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T10:33:39.817 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:39 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:39.716+0000 7f878b602200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T10:33:39.828 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T10:33:39.830 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:39.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-02-26T10:33:39.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T10:33:39.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:39.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T10:33:39.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T10:33:39.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:39.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T10:33:39.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T10:33:39.842 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T10:33:28.696653+0000", 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:39.843 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.650+0000 7f79ae600700 1 Processor -- start 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.651+0000 7f79ae600700 1 -- start start 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.651+0000 7f79ae600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a8107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.651+0000 7f79ae600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8108070 con 0x7f79a8107680 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.652+0000 7f79a7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a8107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.652+0000 7f79a7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a8107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53914/0 (socket says 172.21.15.60:53914) 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.652+0000 7f79a7400700 1 -- 172.21.15.60:0/2671165706 learned_addr learned my addr 172.21.15.60:0/2671165706 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.652+0000 7f79a7400700 1 -- 172.21.15.60:0/2671165706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a81087e0 con 0x7f79a8107680 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.653+0000 7f79a7400700 1 --2- 172.21.15.60:0/2671165706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a8107aa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79980098a0 tx=0x7f7998009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2699406d971d5b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.653+0000 7f79a6000700 1 -- 172.21.15.60:0/2671165706 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7998004160 con 0x7f79a8107680 2024-02-26T10:33:39.844 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.653+0000 7f79a6000700 1 -- 172.21.15.60:0/2671165706 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7998004300 con 0x7f79a8107680 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.653+0000 7f79a6000700 1 -- 172.21.15.60:0/2671165706 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7998011b50 con 0x7f79a8107680 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.654+0000 7f79ae600700 1 -- 172.21.15.60:0/2671165706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 msgr2=0x7f79a8107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.654+0000 7f79ae600700 1 --2- 172.21.15.60:0/2671165706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a8107aa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79980098a0 tx=0x7f7998009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.654+0000 7f79ae600700 1 -- 172.21.15.60:0/2671165706 shutdown_connections 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.654+0000 7f79ae600700 1 --2- 172.21.15.60:0/2671165706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a8107aa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.654+0000 7f79ae600700 1 -- 172.21.15.60:0/2671165706 >> 172.21.15.60:0/2671165706 conn(0x7f79a8102cc0 msgr2=0x7f79a81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.654+0000 7f79ae600700 1 -- 172.21.15.60:0/2671165706 shutdown_connections 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.654+0000 7f79ae600700 1 -- 172.21.15.60:0/2671165706 wait complete. 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.655+0000 7f79ae600700 1 Processor -- start 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.655+0000 7f79ae600700 1 -- start start 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.656+0000 7f79ae600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a819d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.656+0000 7f79ae600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a810bb90 con 0x7f79a8107680 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.656+0000 7f79a7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a819d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.656+0000 7f79a7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a819d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53926/0 (socket says 172.21.15.60:53926) 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.656+0000 7f79a7400700 1 -- 172.21.15.60:0/2877582759 learned_addr learned my addr 172.21.15.60:0/2877582759 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.656+0000 7f79a7400700 1 -- 172.21.15.60:0/2877582759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a819dd80 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.657+0000 7f79a7400700 1 --2- 172.21.15.60:0/2877582759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a819d840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7998000c00 tx=0x7f7998011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.657+0000 7f799fe00700 1 -- 172.21.15.60:0/2877582759 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7998003b20 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.657+0000 7f799fe00700 1 -- 172.21.15.60:0/2877582759 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7998003cc0 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.657+0000 7f799fe00700 1 -- 172.21.15.60:0/2877582759 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7998024910 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.657+0000 7f79ae600700 1 -- 172.21.15.60:0/2877582759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a819e010 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.657+0000 7f79ae600700 1 -- 172.21.15.60:0/2877582759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a81a0d00 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.658+0000 7f799fe00700 1 -- 172.21.15.60:0/2877582759 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7998022020 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.658+0000 7f799fe00700 1 -- 172.21.15.60:0/2877582759 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7998011e00 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.658+0000 7f79ae600700 1 -- 172.21.15.60:0/2877582759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a8068160 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.661+0000 7f799fe00700 1 -- 172.21.15.60:0/2877582759 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f799801acd0 con 0x7f79a8107680 2024-02-26T10:33:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.741+0000 7f79ae600700 1 -- 172.21.15.60:0/2877582759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f79a81a0ed0 con 0x7f79a8107680 2024-02-26T10:33:39.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.742+0000 7f799fe00700 1 -- 172.21.15.60:0/2877582759 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f7998032030 con 0x7f79a8107680 2024-02-26T10:33:39.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.744+0000 7f79ae600700 1 -- 172.21.15.60:0/2877582759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 msgr2=0x7f79a819d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:39.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.744+0000 7f79ae600700 1 --2- 172.21.15.60:0/2877582759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a819d840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7998000c00 tx=0x7f7998011c10 comp rx=0 tx=0).stop 2024-02-26T10:33:39.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.744+0000 7f79ae600700 1 -- 172.21.15.60:0/2877582759 shutdown_connections 2024-02-26T10:33:39.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.744+0000 7f79ae600700 1 --2- 172.21.15.60:0/2877582759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a8107680 0x7f79a819d840 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:39.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.744+0000 7f79ae600700 1 -- 172.21.15.60:0/2877582759 >> 172.21.15.60:0/2877582759 conn(0x7f79a8102cc0 msgr2=0x7f79a8103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:39.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.744+0000 7f79ae600700 1 -- 172.21.15.60:0/2877582759 shutdown_connections 2024-02-26T10:33:39.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:38.744+0000 7f79ae600700 1 -- 172.21.15.60:0/2877582759 wait complete. 2024-02-26T10:33:39.847 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (1/15)... 2024-02-26T10:33:40.123 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:39 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:39.844+0000 7f878b602200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T10:33:40.123 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:40.030+0000 7f878b602200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T10:33:40.620 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:40.158+0000 7f878b602200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T10:33:40.620 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:40.270+0000 7f878b602200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T10:33:40.620 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:40.615+0000 7f878b602200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T10:33:40.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:40.762+0000 7f878b602200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T10:33:41.236 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:40.880+0000 7f878b602200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T10:33:41.236 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:41 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:41.007+0000 7f878b602200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T10:33:41.236 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:41 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:41.231+0000 7f878b602200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T10:33:42.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:42 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1796528123' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T10:33:42.796 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:42 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:42.613+0000 7f878b602200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T10:33:42.796 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:42 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:42.791+0000 7f878b602200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T10:33:43.229 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:43.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T10:33:43.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:33:43.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T10:33:43.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T10:33:43.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T10:33:43.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:43.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T10:33:43.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T10:33:43.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:43.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T10:33:43.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:43.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T10:33:43.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T10:33:43.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T10:33:43.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T10:33:43.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:43.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:43.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T10:33:28.696653+0000", 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.422+0000 7f4bc0c00700 1 Processor -- start 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.422+0000 7f4bc0c00700 1 -- start start 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.422+0000 7f4bc0c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc1062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.422+0000 7f4bc0c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc07ae70 con 0x7f4bbc105ec0 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.423+0000 7f4bb9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc1062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.423+0000 7f4bb9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc1062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53928/0 (socket says 172.21.15.60:53928) 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.423+0000 7f4bb9e00700 1 -- 172.21.15.60:0/368346550 learned_addr learned my addr 172.21.15.60:0/368346550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:43.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.424+0000 7f4bb9e00700 1 -- 172.21.15.60:0/368346550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc106820 con 0x7f4bbc105ec0 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.424+0000 7f4bb9e00700 1 --2- 172.21.15.60:0/368346550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc1062e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009e80 tx=0x7f4bac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb214c8f21d05d1f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.424+0000 7f4bb9400700 1 -- 172.21.15.60:0/368346550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bac017070 con 0x7f4bbc105ec0 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.424+0000 7f4bb9400700 1 -- 172.21.15.60:0/368346550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bac00b8c0 con 0x7f4bbc105ec0 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.424+0000 7f4bb9400700 1 -- 172.21.15.60:0/368346550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bac0045c0 con 0x7f4bbc105ec0 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.425+0000 7f4bc0c00700 1 -- 172.21.15.60:0/368346550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 msgr2=0x7f4bbc1062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.425+0000 7f4bc0c00700 1 --2- 172.21.15.60:0/368346550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc1062e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009e80 tx=0x7f4bac00b330 comp rx=0 tx=0).stop 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.425+0000 7f4bc0c00700 1 -- 172.21.15.60:0/368346550 shutdown_connections 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.425+0000 7f4bc0c00700 1 --2- 172.21.15.60:0/368346550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc1062e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.425+0000 7f4bc0c00700 1 -- 172.21.15.60:0/368346550 >> 172.21.15.60:0/368346550 conn(0x7f4bbc101ac0 msgr2=0x7f4bbc103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.425+0000 7f4bc0c00700 1 -- 172.21.15.60:0/368346550 shutdown_connections 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.425+0000 7f4bc0c00700 1 -- 172.21.15.60:0/368346550 wait complete. 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.426+0000 7f4bc0c00700 1 Processor -- start 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.426+0000 7f4bc0c00700 1 -- start start 2024-02-26T10:33:43.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.427+0000 7f4bc0c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.427+0000 7f4bc0c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc1078b0 con 0x7f4bbc105ec0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.427+0000 7f4bb9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.427+0000 7f4bb9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc19d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53936/0 (socket says 172.21.15.60:53936) 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.427+0000 7f4bb9e00700 1 -- 172.21.15.60:0/1796528123 learned_addr learned my addr 172.21.15.60:0/1796528123 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.427+0000 7f4bb9e00700 1 -- 172.21.15.60:0/1796528123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc19dd40 con 0x7f4bbc105ec0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.428+0000 7f4bb9e00700 1 --2- 172.21.15.60:0/1796528123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc19d800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009530 tx=0x7f4bac00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.428+0000 7f4bb3400700 1 -- 172.21.15.60:0/1796528123 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bac020020 con 0x7f4bbc105ec0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.428+0000 7f4bb3400700 1 -- 172.21.15.60:0/1796528123 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bac003b50 con 0x7f4bbc105ec0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.428+0000 7f4bb3400700 1 -- 172.21.15.60:0/1796528123 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bac017040 con 0x7f4bbc105ec0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.428+0000 7f4bc0c00700 1 -- 172.21.15.60:0/1796528123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bbc19dfd0 con 0x7f4bbc105ec0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.428+0000 7f4bc0c00700 1 -- 172.21.15.60:0/1796528123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bbc1a0cc0 con 0x7f4bbc105ec0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.429+0000 7f4bb3400700 1 -- 172.21.15.60:0/1796528123 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4bac023020 con 0x7f4bbc105ec0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.429+0000 7f4bb3400700 1 -- 172.21.15.60:0/1796528123 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4bac00f050 con 0x7f4bbc105ec0 2024-02-26T10:33:43.250 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.429+0000 7f4bc0c00700 1 -- 172.21.15.60:0/1796528123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bbc068160 con 0x7f4bbc105ec0 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.432+0000 7f4bb3400700 1 -- 172.21.15.60:0/1796528123 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4bac02a020 con 0x7f4bbc105ec0 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.513+0000 7f4bc0c00700 1 -- 172.21.15.60:0/1796528123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4bbc02d9c0 con 0x7f4bbc105ec0 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.513+0000 7f4bb3400700 1 -- 172.21.15.60:0/1796528123 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f4bac00b860 con 0x7f4bbc105ec0 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.514+0000 7f4bc0c00700 1 -- 172.21.15.60:0/1796528123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 msgr2=0x7f4bbc19d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.515+0000 7f4bc0c00700 1 --2- 172.21.15.60:0/1796528123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc19d800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009530 tx=0x7f4bac00bd50 comp rx=0 tx=0).stop 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.515+0000 7f4bc0c00700 1 -- 172.21.15.60:0/1796528123 shutdown_connections 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.515+0000 7f4bc0c00700 1 --2- 172.21.15.60:0/1796528123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bbc105ec0 0x7f4bbc19d800 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.515+0000 7f4bc0c00700 1 -- 172.21.15.60:0/1796528123 >> 172.21.15.60:0/1796528123 conn(0x7f4bbc101ac0 msgr2=0x7f4bbc194420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.515+0000 7f4bc0c00700 1 -- 172.21.15.60:0/1796528123 shutdown_connections 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:42.515+0000 7f4bc0c00700 1 -- 172.21.15.60:0/1796528123 wait complete. 2024-02-26T10:33:43.251 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (2/15)... 2024-02-26T10:33:43.617 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:43 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:43.219+0000 7f878b602200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T10:33:43.617 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:43 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:43.347+0000 7f878b602200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T10:33:43.617 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:43 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:43.612+0000 7f878b602200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:33:44.123 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:43 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:43.864+0000 7f878b602200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T10:33:44.580 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:44 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:44.250+0000 7f878b602200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:33:44.580 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:44 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:44.575+0000 7f878b602200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T10:33:45.373 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:44 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:44.888+0000 7f878b602200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T10:33:46.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:45 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/187617855' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T10:33:46.280 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:45 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:45.971+0000 7f878b602200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T10:33:46.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T10:33:46.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T10:33:46.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T10:33:46.557 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T10:33:46.558 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T10:33:46.559 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T10:33:28.696653+0000", 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:46.560 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.808+0000 7fbc83200700 1 Processor -- start 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.808+0000 7fbc83200700 1 -- start start 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.809+0000 7fbc83200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.809+0000 7fbc83200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8407b820 con 0x7fbc8407cd70 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.809+0000 7fbc81e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.809+0000 7fbc81e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8407b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40472/0 (socket says 172.21.15.60:40472) 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.809+0000 7fbc81e00700 1 -- 172.21.15.60:0/3862247681 learned_addr learned my addr 172.21.15.60:0/3862247681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.810+0000 7fbc81e00700 1 -- 172.21.15.60:0/3862247681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8407b9a0 con 0x7fbc8407cd70 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.810+0000 7fbc81e00700 1 --2- 172.21.15.60:0/3862247681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8407b2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc740098a0 tx=0x7fbc74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851f8ef9a14b54b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.810+0000 7fbc81400700 1 -- 172.21.15.60:0/3862247681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc74004160 con 0x7fbc8407cd70 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.810+0000 7fbc81400700 1 -- 172.21.15.60:0/3862247681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc74004300 con 0x7fbc8407cd70 2024-02-26T10:33:46.561 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.810+0000 7fbc81400700 1 -- 172.21.15.60:0/3862247681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc74011b50 con 0x7fbc8407cd70 2024-02-26T10:33:46.562 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.811+0000 7fbc83200700 1 -- 172.21.15.60:0/3862247681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 msgr2=0x7fbc8407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:46.562 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.811+0000 7fbc83200700 1 --2- 172.21.15.60:0/3862247681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8407b2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc740098a0 tx=0x7fbc74009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:46.562 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.811+0000 7fbc83200700 1 -- 172.21.15.60:0/3862247681 shutdown_connections 2024-02-26T10:33:46.562 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.811+0000 7fbc83200700 1 --2- 172.21.15.60:0/3862247681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8407b2e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:46.562 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.811+0000 7fbc83200700 1 -- 172.21.15.60:0/3862247681 >> 172.21.15.60:0/3862247681 conn(0x7fbc84102b80 msgr2=0x7fbc84104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.811+0000 7fbc83200700 1 -- 172.21.15.60:0/3862247681 shutdown_connections 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.811+0000 7fbc83200700 1 -- 172.21.15.60:0/3862247681 wait complete. 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.812+0000 7fbc83200700 1 Processor -- start 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.812+0000 7fbc83200700 1 -- start start 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.813+0000 7fbc83200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.813+0000 7fbc83200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8410d990 con 0x7fbc8407cd70 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.813+0000 7fbc81e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.813+0000 7fbc81e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40478/0 (socket says 172.21.15.60:40478) 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.813+0000 7fbc81e00700 1 -- 172.21.15.60:0/187617855 learned_addr learned my addr 172.21.15.60:0/187617855 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.813+0000 7fbc81e00700 1 -- 172.21.15.60:0/187617855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8419db40 con 0x7fbc8407cd70 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.814+0000 7fbc81e00700 1 --2- 172.21.15.60:0/187617855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8419d600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011b60 tx=0x7fbc74011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.814+0000 7fbc7aa00700 1 -- 172.21.15.60:0/187617855 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc74003b20 con 0x7fbc8407cd70 2024-02-26T10:33:46.563 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.814+0000 7fbc7aa00700 1 -- 172.21.15.60:0/187617855 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc74003cc0 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.814+0000 7fbc7aa00700 1 -- 172.21.15.60:0/187617855 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc74024840 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.814+0000 7fbc83200700 1 -- 172.21.15.60:0/187617855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8419ddd0 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.814+0000 7fbc83200700 1 -- 172.21.15.60:0/187617855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc841a0ac0 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.815+0000 7fbc7aa00700 1 -- 172.21.15.60:0/187617855 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbc74022020 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.815+0000 7fbc7aa00700 1 -- 172.21.15.60:0/187617855 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbc74011e00 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.815+0000 7fbc83200700 1 -- 172.21.15.60:0/187617855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc84063380 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.818+0000 7fbc7aa00700 1 -- 172.21.15.60:0/187617855 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbc74043dd0 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.898+0000 7fbc83200700 1 -- 172.21.15.60:0/187617855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbc841a11a0 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.899+0000 7fbc7aa00700 1 -- 172.21.15.60:0/187617855 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7fbc74032020 con 0x7fbc8407cd70 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.900+0000 7fbc83200700 1 -- 172.21.15.60:0/187617855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 msgr2=0x7fbc8419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.900+0000 7fbc83200700 1 --2- 172.21.15.60:0/187617855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8419d600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011b60 tx=0x7fbc74011c40 comp rx=0 tx=0).stop 2024-02-26T10:33:46.564 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.900+0000 7fbc83200700 1 -- 172.21.15.60:0/187617855 shutdown_connections 2024-02-26T10:33:46.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.900+0000 7fbc83200700 1 --2- 172.21.15.60:0/187617855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8407cd70 0x7fbc8419d600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:46.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.900+0000 7fbc83200700 1 -- 172.21.15.60:0/187617855 >> 172.21.15.60:0/187617855 conn(0x7fbc84102b80 msgr2=0x7fbc84103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:46.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.900+0000 7fbc83200700 1 -- 172.21.15.60:0/187617855 shutdown_connections 2024-02-26T10:33:46.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:45.900+0000 7fbc83200700 1 -- 172.21.15.60:0/187617855 wait complete. 2024-02-26T10:33:46.565 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (3/15)... 2024-02-26T10:33:46.831 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:46 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:46.695+0000 7f878b602200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T10:33:46.832 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:46 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:46.827+0000 7f878b602200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T10:33:47.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:47.119+0000 7f878b602200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: Activating manager daemon x 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: mgrmap e2: x(active, starting, since 0.00937491s) 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: Manager daemon x is now available 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' 2024-02-26T10:33:47.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:47 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T10:33:48.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:48 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' 2024-02-26T10:33:48.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:48 smithi060 ceph-mon[31539]: from='mgr.14100 172.21.15.60:0/1213994352' entity='mgr.x' 2024-02-26T10:33:48.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:48 smithi060 ceph-mon[31539]: mgrmap e3: x(active, since 1.01932s) 2024-02-26T10:33:50.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:50.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:50.293 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T10:33:50.295 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T10:33:50.296 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T10:33:28.696653+0000", 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.161+0000 7faa5a200700 1 Processor -- start 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.161+0000 7faa5a200700 1 -- start start 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.161+0000 7faa5a200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa54109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.161+0000 7faa5a200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5410a080 con 0x7faa54109690 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.162+0000 7faa58e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa54109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.162+0000 7faa58e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa54109ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40572/0 (socket says 172.21.15.60:40572) 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.162+0000 7faa58e00700 1 -- 172.21.15.60:0/4024102624 learned_addr learned my addr 172.21.15.60:0/4024102624 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.162+0000 7faa58e00700 1 -- 172.21.15.60:0/4024102624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5410a840 con 0x7faa54109690 2024-02-26T10:33:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.163+0000 7faa58e00700 1 --2- 172.21.15.60:0/4024102624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa54109ab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faa440098a0 tx=0x7faa44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c82f8ffc34758e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.163+0000 7faa53e00700 1 -- 172.21.15.60:0/4024102624 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa44004160 con 0x7faa54109690 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.163+0000 7faa53e00700 1 -- 172.21.15.60:0/4024102624 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa44004300 con 0x7faa54109690 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.163+0000 7faa53e00700 1 -- 172.21.15.60:0/4024102624 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa44011b50 con 0x7faa54109690 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.164+0000 7faa5a200700 1 -- 172.21.15.60:0/4024102624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 msgr2=0x7faa54109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.164+0000 7faa5a200700 1 --2- 172.21.15.60:0/4024102624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa54109ab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faa440098a0 tx=0x7faa44009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.164+0000 7faa5a200700 1 -- 172.21.15.60:0/4024102624 shutdown_connections 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.164+0000 7faa5a200700 1 --2- 172.21.15.60:0/4024102624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa54109ab0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.164+0000 7faa5a200700 1 -- 172.21.15.60:0/4024102624 >> 172.21.15.60:0/4024102624 conn(0x7faa5407cbf0 msgr2=0x7faa541074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.164+0000 7faa5a200700 1 -- 172.21.15.60:0/4024102624 shutdown_connections 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.164+0000 7faa5a200700 1 -- 172.21.15.60:0/4024102624 wait complete. 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.166+0000 7faa5a200700 1 Processor -- start 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.166+0000 7faa5a200700 1 -- start start 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.166+0000 7faa5a200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa5419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.166+0000 7faa5a200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5410dbf0 con 0x7faa54109690 2024-02-26T10:33:50.298 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.167+0000 7faa58e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa5419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.167+0000 7faa58e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa5419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40580/0 (socket says 172.21.15.60:40580) 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.167+0000 7faa58e00700 1 -- 172.21.15.60:0/3096113311 learned_addr learned my addr 172.21.15.60:0/3096113311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.167+0000 7faa58e00700 1 -- 172.21.15.60:0/3096113311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5419dbc0 con 0x7faa54109690 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.167+0000 7faa58e00700 1 --2- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa5419d680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faa44011850 tx=0x7faa44011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.168+0000 7faa52000700 1 -- 172.21.15.60:0/3096113311 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa44003af0 con 0x7faa54109690 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.168+0000 7faa52000700 1 -- 172.21.15.60:0/3096113311 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa44020070 con 0x7faa54109690 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.168+0000 7faa52000700 1 -- 172.21.15.60:0/3096113311 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa440195a0 con 0x7faa54109690 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.168+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5419de50 con 0x7faa54109690 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.168+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa541a0b40 con 0x7faa54109690 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.169+0000 7faa52000700 1 -- 172.21.15.60:0/3096113311 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 3) v1 ==== 45811+0+0 (secure 0 0 0) 0x7faa44019850 con 0x7faa54109690 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.169+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa54063380 con 0x7faa54109690 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.170+0000 7faa52000700 1 --2- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7faa38038a80 0x7faa3803af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:50.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.170+0000 7faa52000700 1 -- 172.21.15.60:0/3096113311 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faa44011e00 con 0x7faa54109690 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.170+0000 7faa4be00700 1 --2- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7faa38038a80 0x7faa3803af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.170+0000 7faa4be00700 1 --2- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7faa38038a80 0x7faa3803af40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006fd0 tx=0x7faa3c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.176+0000 7faa52000700 1 -- 172.21.15.60:0/3096113311 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa44003c90 con 0x7faa54109690 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.180+0000 7faa52000700 1 -- 172.21.15.60:0/3096113311 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 4) v1 ==== 45917+0+0 (secure 0 0 0) 0x7faa4402bd00 con 0x7faa54109690 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.484+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7faa541a0d10 con 0x7faa54109690 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.485+0000 7faa52000700 1 -- 172.21.15.60:0/3096113311 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7faa541a0d10 con 0x7faa54109690 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7faa38038a80 msgr2=0x7faa3803af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 --2- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7faa38038a80 0x7faa3803af40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006fd0 tx=0x7faa3c006e40 comp rx=0 tx=0).stop 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 msgr2=0x7faa5419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 --2- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa5419d680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faa44011850 tx=0x7faa44011c10 comp rx=0 tx=0).stop 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 shutdown_connections 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 --2- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7faa38038a80 0x7faa3803af40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 --2- 172.21.15.60:0/3096113311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa54109690 0x7faa5419d680 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:50.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 >> 172.21.15.60:0/3096113311 conn(0x7faa5407cbf0 msgr2=0x7faa54106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:50.301 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 shutdown_connections 2024-02-26T10:33:50.301 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:49.489+0000 7faa5a200700 1 -- 172.21.15.60:0/3096113311 wait complete. 2024-02-26T10:33:50.301 INFO:teuthology.orchestra.run.smithi060.stdout:mgr is available 2024-02-26T10:33:50.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:50 smithi060 ceph-mon[31539]: mgrmap e4: x(active, since 2s) 2024-02-26T10:33:50.439 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:50 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3096113311' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T10:33:51.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:51 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3691014852' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-26T10:33:51.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-26T10:33:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.829+0000 7f7ecfc00700 1 Processor -- start 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.829+0000 7f7ecfc00700 1 -- start start 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.830+0000 7f7ecfc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec810e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.830+0000 7f7ecfc00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec810e970 con 0x7f7ec810e010 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.830+0000 7f7ecd200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec810e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:51.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.830+0000 7f7ecd200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec810e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40586/0 (socket says 172.21.15.60:40586) 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.830+0000 7f7ecd200700 1 -- 172.21.15.60:0/1398251435 learned_addr learned my addr 172.21.15.60:0/1398251435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.831+0000 7f7ecd200700 1 -- 172.21.15.60:0/1398251435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec810eaf0 con 0x7f7ec810e010 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.831+0000 7f7ecd200700 1 --2- 172.21.15.60:0/1398251435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec810e430 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0098a0 tx=0x7f7ebc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5352cd2d74eef3b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.831+0000 7f7ec7400700 1 -- 172.21.15.60:0/1398251435 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ebc004160 con 0x7f7ec810e010 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.831+0000 7f7ec7400700 1 -- 172.21.15.60:0/1398251435 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ebc004300 con 0x7f7ec810e010 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.831+0000 7f7ec7400700 1 -- 172.21.15.60:0/1398251435 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ebc011b50 con 0x7f7ec810e010 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.832+0000 7f7ecfc00700 1 -- 172.21.15.60:0/1398251435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 msgr2=0x7f7ec810e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.832+0000 7f7ecfc00700 1 --2- 172.21.15.60:0/1398251435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec810e430 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0098a0 tx=0x7f7ebc009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.832+0000 7f7ecfc00700 1 -- 172.21.15.60:0/1398251435 shutdown_connections 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.832+0000 7f7ecfc00700 1 --2- 172.21.15.60:0/1398251435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec810e430 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:51.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.832+0000 7f7ecfc00700 1 -- 172.21.15.60:0/1398251435 >> 172.21.15.60:0/1398251435 conn(0x7f7ec807ce70 msgr2=0x7f7ec807f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.832+0000 7f7ecfc00700 1 -- 172.21.15.60:0/1398251435 shutdown_connections 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.832+0000 7f7ecfc00700 1 -- 172.21.15.60:0/1398251435 wait complete. 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.834+0000 7f7ecfc00700 1 Processor -- start 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.834+0000 7f7ecfc00700 1 -- start start 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.834+0000 7f7ecfc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec81a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.834+0000 7f7ecfc00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec81120e0 con 0x7f7ec810e010 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.834+0000 7f7ecd200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec81a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.834+0000 7f7ecd200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec81a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40596/0 (socket says 172.21.15.60:40596) 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.834+0000 7f7ecd200700 1 -- 172.21.15.60:0/3691014852 learned_addr learned my addr 172.21.15.60:0/3691014852 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.835+0000 7f7ecd200700 1 -- 172.21.15.60:0/3691014852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec81a2150 con 0x7f7ec810e010 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.835+0000 7f7ecd200700 1 --2- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec81a1c10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc011b60 tx=0x7f7ebc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:51.752 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.835+0000 7f7ec5600700 1 -- 172.21.15.60:0/3691014852 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ebc003970 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.835+0000 7f7ec5600700 1 -- 172.21.15.60:0/3691014852 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ebc003b10 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.835+0000 7f7ec5600700 1 -- 172.21.15.60:0/3691014852 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ebc024610 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.835+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec81a23e0 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.836+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec81a50d0 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.837+0000 7f7ec5600700 1 -- 172.21.15.60:0/3691014852 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 45917+0+0 (secure 0 0 0) 0x7f7ebc01a9f0 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.837+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec8068160 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.837+0000 7f7ec5600700 1 --2- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f7eb0038ba0 0x7f7eb003b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.837+0000 7f7ec5600700 1 -- 172.21.15.60:0/3691014852 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7ebc011e00 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.837+0000 7f7ec7e00700 1 --2- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f7eb0038ba0 0x7f7eb003b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.838+0000 7f7ec7e00700 1 --2- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f7eb0038ba0 0x7f7eb003b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4006fd0 tx=0x7f7eb4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:50.842+0000 7f7ec5600700 1 -- 172.21.15.60:0/3691014852 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7ebc00b3e0 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.054+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f7ec81a52a0 con 0x7f7ec810e010 2024-02-26T10:33:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.056+0000 7f7ec5600700 1 -- 172.21.15.60:0/3691014852 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f7ec81a52a0 con 0x7f7ec810e010 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f7eb0038ba0 msgr2=0x7f7eb003b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 --2- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f7eb0038ba0 0x7f7eb003b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4006fd0 tx=0x7f7eb4006e40 comp rx=0 tx=0).stop 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 msgr2=0x7f7ec81a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 --2- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec81a1c10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc011b60 tx=0x7f7ebc011c40 comp rx=0 tx=0).stop 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 shutdown_connections 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 --2- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f7eb0038ba0 0x7f7eb003b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 --2- 172.21.15.60:0/3691014852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec810e010 0x7f7ec81a1c10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 >> 172.21.15.60:0/3691014852 conn(0x7f7ec807ce70 msgr2=0x7f7ec807da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 shutdown_connections 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:51.059+0000 7f7ecfc00700 1 -- 172.21.15.60:0/3691014852 wait complete. 2024-02-26T10:33:51.754 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling cephadm module... 2024-02-26T10:33:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:52 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1324385991' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-26T10:33:52.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:52 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: ignoring --setuser ceph since I am not root 2024-02-26T10:33:52.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:52 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: ignoring --setgroup ceph since I am not root 2024-02-26T10:33:53.316 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.286+0000 7fc038400700 1 Processor -- start 2024-02-26T10:33:53.317 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.286+0000 7fc038400700 1 -- start start 2024-02-26T10:33:53.317 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.287+0000 7fc038400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc030106340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:53.317 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.287+0000 7fc038400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03007ae30 con 0x7fc030105f20 2024-02-26T10:33:53.317 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.287+0000 7fc035a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc030106340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:53.317 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.287+0000 7fc035a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc030106340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40612/0 (socket says 172.21.15.60:40612) 2024-02-26T10:33:53.317 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.287+0000 7fc035a00700 1 -- 172.21.15.60:0/3643317368 learned_addr learned my addr 172.21.15.60:0/3643317368 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:53.317 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.288+0000 7fc035a00700 1 -- 172.21.15.60:0/3643317368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc030106880 con 0x7fc030105f20 2024-02-26T10:33:53.317 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.288+0000 7fc035a00700 1 --2- 172.21.15.60:0/3643317368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc030106340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0240098a0 tx=0x7fc024009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46479d777648f9bb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:53.317 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.288+0000 7fc02fe00700 1 -- 172.21.15.60:0/3643317368 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc024004160 con 0x7fc030105f20 2024-02-26T10:33:53.318 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.289+0000 7fc02fe00700 1 -- 172.21.15.60:0/3643317368 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc024004300 con 0x7fc030105f20 2024-02-26T10:33:53.318 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.289+0000 7fc02fe00700 1 -- 172.21.15.60:0/3643317368 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc024011b50 con 0x7fc030105f20 2024-02-26T10:33:53.318 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.289+0000 7fc038400700 1 -- 172.21.15.60:0/3643317368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 msgr2=0x7fc030106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:53.318 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.289+0000 7fc038400700 1 --2- 172.21.15.60:0/3643317368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc030106340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0240098a0 tx=0x7fc024009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:53.318 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.289+0000 7fc038400700 1 -- 172.21.15.60:0/3643317368 shutdown_connections 2024-02-26T10:33:53.318 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.289+0000 7fc038400700 1 --2- 172.21.15.60:0/3643317368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc030106340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:53.318 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.289+0000 7fc038400700 1 -- 172.21.15.60:0/3643317368 >> 172.21.15.60:0/3643317368 conn(0x7fc030101b60 msgr2=0x7fc030103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:53.318 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.290+0000 7fc038400700 1 -- 172.21.15.60:0/3643317368 shutdown_connections 2024-02-26T10:33:53.319 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.290+0000 7fc038400700 1 -- 172.21.15.60:0/3643317368 wait complete. 2024-02-26T10:33:53.319 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.291+0000 7fc038400700 1 Processor -- start 2024-02-26T10:33:53.319 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.291+0000 7fc038400700 1 -- start start 2024-02-26T10:33:53.319 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.291+0000 7fc038400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc03019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:53.319 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.291+0000 7fc038400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030107910 con 0x7fc030105f20 2024-02-26T10:33:53.319 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.292+0000 7fc035a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc03019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:53.319 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.292+0000 7fc035a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc03019d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40616/0 (socket says 172.21.15.60:40616) 2024-02-26T10:33:53.319 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.292+0000 7fc035a00700 1 -- 172.21.15.60:0/1324385991 learned_addr learned my addr 172.21.15.60:0/1324385991 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:53.320 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.292+0000 7fc035a00700 1 -- 172.21.15.60:0/1324385991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03019dd50 con 0x7fc030105f20 2024-02-26T10:33:53.320 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.292+0000 7fc035a00700 1 --2- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc03019d810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc024011b60 tx=0x7fc024011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:53.320 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.293+0000 7fc02e000700 1 -- 172.21.15.60:0/1324385991 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc024003970 con 0x7fc030105f20 2024-02-26T10:33:53.320 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.293+0000 7fc02e000700 1 -- 172.21.15.60:0/1324385991 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc024003b10 con 0x7fc030105f20 2024-02-26T10:33:53.320 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.293+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03019dfe0 con 0x7fc030105f20 2024-02-26T10:33:53.320 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.293+0000 7fc02e000700 1 -- 172.21.15.60:0/1324385991 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc024024610 con 0x7fc030105f20 2024-02-26T10:33:53.320 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.293+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0301a0cd0 con 0x7fc030105f20 2024-02-26T10:33:53.320 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.294+0000 7fc02e000700 1 -- 172.21.15.60:0/1324385991 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 45917+0+0 (secure 0 0 0) 0x7fc02401a9f0 con 0x7fc030105f20 2024-02-26T10:33:53.320 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.294+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc030068160 con 0x7fc030105f20 2024-02-26T10:33:53.321 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.295+0000 7fc02e000700 1 --2- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc018038ba0 0x7fc01803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:53.321 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.295+0000 7fc02e000700 1 -- 172.21.15.60:0/1324385991 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc024011e00 con 0x7fc030105f20 2024-02-26T10:33:53.321 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.295+0000 7fc035000700 1 --2- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc018038ba0 0x7fc01803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:53.321 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.296+0000 7fc035000700 1 --2- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc018038ba0 0x7fc01803b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc01c006fd0 tx=0x7fc01c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:53.321 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.300+0000 7fc02e000700 1 -- 172.21.15.60:0/1324385991 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc024019e10 con 0x7fc030105f20 2024-02-26T10:33:53.321 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.584+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc0300008d0 con 0x7fc030105f20 2024-02-26T10:33:53.321 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.637+0000 7fc02e000700 1 -- 172.21.15.60:0/1324385991 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc02404c5b0 con 0x7fc030105f20 2024-02-26T10:33:53.321 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.641+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc018038ba0 msgr2=0x7fc01803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:53.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.641+0000 7fc038400700 1 --2- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc018038ba0 0x7fc01803b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc01c006fd0 tx=0x7fc01c006e40 comp rx=0 tx=0).stop 2024-02-26T10:33:53.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.641+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 msgr2=0x7fc03019d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:53.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.641+0000 7fc02e000700 1 -- 172.21.15.60:0/1324385991 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mgrmap(e 5) v1 ==== 45928+0+0 (secure 0 0 0) 0x7fc024024c40 con 0x7fc030105f20 2024-02-26T10:33:53.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.641+0000 7fc038400700 1 --2- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc03019d810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc024011b60 tx=0x7fc024011c40 comp rx=0 tx=0).stop 2024-02-26T10:33:53.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.643+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 shutdown_connections 2024-02-26T10:33:53.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.643+0000 7fc038400700 1 --2- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc018038ba0 0x7fc01803b060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:53.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.643+0000 7fc038400700 1 --2- 172.21.15.60:0/1324385991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc030105f20 0x7fc03019d810 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:53.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.643+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 >> 172.21.15.60:0/1324385991 conn(0x7fc030101b60 msgr2=0x7fc0301944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:53.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.643+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 shutdown_connections 2024-02-26T10:33:53.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:52.643+0000 7fc038400700 1 -- 172.21.15.60:0/1324385991 wait complete. 2024-02-26T10:33:53.593 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:53 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:53.461+0000 7fab5c0da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T10:33:53.593 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:53 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:53.589+0000 7fab5c0da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T10:33:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:53 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1324385991' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-26T10:33:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:53 smithi060 ceph-mon[31539]: mgrmap e5: x(active, since 5s) 2024-02-26T10:33:54.515 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:54 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:54.416+0000 7fab5c0da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T10:33:54.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T10:33:54.754 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-26T10:33:54.755 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T10:33:54.755 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-26T10:33:54.755 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-26T10:33:54.755 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T10:33:54.756 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.895+0000 7f5cfc000700 1 Processor -- start 2024-02-26T10:33:54.756 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.895+0000 7f5cfc000700 1 -- start start 2024-02-26T10:33:54.756 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.896+0000 7f5cfc000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf4106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:54.756 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.896+0000 7f5cfc000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4106f50 con 0x7f5cf4106560 2024-02-26T10:33:54.757 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.896+0000 7f5cf9600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf4106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:54.757 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.896+0000 7f5cf9600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf4106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34344/0 (socket says 172.21.15.60:34344) 2024-02-26T10:33:54.757 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.896+0000 7f5cf9600700 1 -- 172.21.15.60:0/3757677862 learned_addr learned my addr 172.21.15.60:0/3757677862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:54.757 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.897+0000 7f5cf9600700 1 -- 172.21.15.60:0/3757677862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf4107760 con 0x7f5cf4106560 2024-02-26T10:33:54.757 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.897+0000 7f5cf9600700 1 --2- 172.21.15.60:0/3757677862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf4106980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80098a0 tx=0x7f5ce8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ade5b7aed17b5f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:54.758 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.897+0000 7f5cf3e00700 1 -- 172.21.15.60:0/3757677862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ce8004160 con 0x7f5cf4106560 2024-02-26T10:33:54.758 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.897+0000 7f5cf3e00700 1 -- 172.21.15.60:0/3757677862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ce8004300 con 0x7f5cf4106560 2024-02-26T10:33:54.758 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.897+0000 7f5cf3e00700 1 -- 172.21.15.60:0/3757677862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ce8011b50 con 0x7f5cf4106560 2024-02-26T10:33:54.758 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.898+0000 7f5cfc000700 1 -- 172.21.15.60:0/3757677862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 msgr2=0x7f5cf4106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:54.759 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.898+0000 7f5cfc000700 1 --2- 172.21.15.60:0/3757677862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf4106980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80098a0 tx=0x7f5ce8009bb0 comp rx=0 tx=0).stop 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.898+0000 7f5cfc000700 1 -- 172.21.15.60:0/3757677862 shutdown_connections 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.898+0000 7f5cfc000700 1 --2- 172.21.15.60:0/3757677862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf4106980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.898+0000 7f5cfc000700 1 -- 172.21.15.60:0/3757677862 >> 172.21.15.60:0/3757677862 conn(0x7f5cf4101b60 msgr2=0x7f5cf4103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.898+0000 7f5cfc000700 1 -- 172.21.15.60:0/3757677862 shutdown_connections 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.898+0000 7f5cfc000700 1 -- 172.21.15.60:0/3757677862 wait complete. 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.899+0000 7f5cfc000700 1 Processor -- start 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.899+0000 7f5cfc000700 1 -- start start 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.900+0000 7f5cfc000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.900+0000 7f5cfc000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4079fd0 con 0x7f5cf4106560 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.900+0000 7f5cf9600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.900+0000 7f5cf9600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf419d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34356/0 (socket says 172.21.15.60:34356) 2024-02-26T10:33:54.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.900+0000 7f5cf9600700 1 -- 172.21.15.60:0/869762225 learned_addr learned my addr 172.21.15.60:0/869762225 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.900+0000 7f5cf9600700 1 -- 172.21.15.60:0/869762225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf419ddb0 con 0x7f5cf4106560 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.901+0000 7f5cf9600700 1 --2- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf419d870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000c00 tx=0x7f5ce8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.901+0000 7f5cf2000700 1 -- 172.21.15.60:0/869762225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ce8003b20 con 0x7f5cf4106560 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.901+0000 7f5cf2000700 1 -- 172.21.15.60:0/869762225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ce8003cc0 con 0x7f5cf4106560 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.901+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf419e040 con 0x7f5cf4106560 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.901+0000 7f5cf2000700 1 -- 172.21.15.60:0/869762225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ce8024910 con 0x7f5cf4106560 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.901+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf41a0d30 con 0x7f5cf4106560 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.902+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf4068160 con 0x7f5cf4106560 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.902+0000 7f5cf2000700 1 -- 172.21.15.60:0/869762225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f5ce8022020 con 0x7f5cf4106560 2024-02-26T10:33:54.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.903+0000 7f5cf2000700 1 --2- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f5cdc038ba0 0x7f5cdc03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.903+0000 7f5cf2000700 1 -- 172.21.15.60:0/869762225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5ce8011e00 con 0x7f5cf4106560 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.903+0000 7f5cf8c00700 1 -- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f5cdc038ba0 msgr2=0x7f5cdc03b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2948072642 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.903+0000 7f5cf8c00700 1 --2- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f5cdc038ba0 0x7f5cdc03b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:53.908+0000 7f5cf2000700 1 -- 172.21.15.60:0/869762225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ce8016260 con 0x7f5cf4106560 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.104+0000 7f5cf8c00700 1 -- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f5cdc038ba0 msgr2=0x7f5cdc03b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2948072642 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.104+0000 7f5cf8c00700 1 --2- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f5cdc038ba0 0x7f5cdc03b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.186+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f5cf41a0f00 con 0x7f5cf4106560 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.186+0000 7f5cf2000700 1 -- 172.21.15.60:0/869762225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5cf41a0f00 con 0x7f5cf4106560 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.189+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f5cdc038ba0 msgr2=0x7f5cdc03b060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.189+0000 7f5cfc000700 1 --2- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f5cdc038ba0 0x7f5cdc03b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:54.766 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.189+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 msgr2=0x7f5cf419d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:33:54.767 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.189+0000 7f5cfc000700 1 --2- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf419d870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000c00 tx=0x7f5ce8011c10 comp rx=0 tx=0).stop 2024-02-26T10:33:54.767 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.189+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 shutdown_connections 2024-02-26T10:33:54.767 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.189+0000 7f5cfc000700 1 --2- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7f5cdc038ba0 0x7f5cdc03b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:54.767 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.189+0000 7f5cfc000700 1 --2- 172.21.15.60:0/869762225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cf4106560 0x7f5cf419d870 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:33:54.767 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.189+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 >> 172.21.15.60:0/869762225 conn(0x7f5cf4101b60 msgr2=0x7f5cf4193e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:33:54.767 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.190+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 shutdown_connections 2024-02-26T10:33:54.767 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:54.190+0000 7f5cfc000700 1 -- 172.21.15.60:0/869762225 wait complete. 2024-02-26T10:33:54.767 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-02-26T10:33:54.768 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 5... 2024-02-26T10:33:54.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:33:54 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/869762225' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-26T10:33:54.865 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:54 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:54.545+0000 7fab5c0da200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T10:33:54.865 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:54 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:54.728+0000 7fab5c0da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T10:33:54.865 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:54 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:54.860+0000 7fab5c0da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T10:33:55.123 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:54 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:54.974+0000 7fab5c0da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T10:33:55.476 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:55.323+0000 7fab5c0da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T10:33:55.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:55.472+0000 7fab5c0da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T10:33:55.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:55.591+0000 7fab5c0da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T10:33:55.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:55.718+0000 7fab5c0da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T10:33:56.373 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:55.942+0000 7fab5c0da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T10:33:57.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:57 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:57.263+0000 7fab5c0da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T10:33:57.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:57 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:57.444+0000 7fab5c0da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T10:33:58.007 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:57 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:57.879+0000 7fab5c0da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T10:33:58.007 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:58.003+0000 7fab5c0da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T10:33:58.536 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:58.278+0000 7fab5c0da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:33:58.536 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:58.531+0000 7fab5c0da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T10:33:59.256 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:58.923+0000 7fab5c0da200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:33:59.256 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:59 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:59.251+0000 7fab5c0da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T10:33:59.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:33:59 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:33:59.562+0000 7fab5c0da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T10:34:01.123 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:00.621+0000 7fab5c0da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T10:34:01.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:01.343+0000 7fab5c0da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T10:34:01.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:01.477+0000 7fab5c0da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T10:34:02.123 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:01.767+0000 7fab5c0da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: Active manager daemon x restarted 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: Activating manager daemon x 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: osdmap e2: 0 total, 0 up, 0 in 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: mgrmap e6: x(active, starting, since 0.00509291s) 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: Manager daemon x is now available 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:01 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:02 smithi060 ceph-mon[31539]: Found migration_current of "None". Setting to last migration. 2024-02-26T10:34:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:02 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:02 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:02 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:02 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T10:34:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:02 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T10:34:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:02 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:02 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:03.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:02 smithi060 ceph-mon[31539]: mgrmap e7: x(active, since 1.01549s) 2024-02-26T10:34:03.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T10:34:03.599 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-26T10:34:03.599 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-26T10:34:03.599 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T10:34:03.599 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.346+0000 7fc92b000700 1 Processor -- start 2024-02-26T10:34:03.599 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.346+0000 7fc92b000700 1 -- start start 2024-02-26T10:34:03.599 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.346+0000 7fc92b000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc924106370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.346+0000 7fc92b000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92407ae10 con 0x7fc924105f50 2024-02-26T10:34:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.347+0000 7fc923e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc924106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.347+0000 7fc923e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc924106370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34364/0 (socket says 172.21.15.60:34364) 2024-02-26T10:34:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.347+0000 7fc923e00700 1 -- 172.21.15.60:0/4273282022 learned_addr learned my addr 172.21.15.60:0/4273282022 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.348+0000 7fc923e00700 1 -- 172.21.15.60:0/4273282022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9241068b0 con 0x7fc924105f50 2024-02-26T10:34:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.348+0000 7fc923e00700 1 --2- 172.21.15.60:0/4273282022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc924106370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9140098a0 tx=0x7fc914009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef20278a93bc518 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.348+0000 7fc922a00700 1 -- 172.21.15.60:0/4273282022 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc914004160 con 0x7fc924105f50 2024-02-26T10:34:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.348+0000 7fc922a00700 1 -- 172.21.15.60:0/4273282022 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc914004300 con 0x7fc924105f50 2024-02-26T10:34:03.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.348+0000 7fc922a00700 1 -- 172.21.15.60:0/4273282022 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc914011b50 con 0x7fc924105f50 2024-02-26T10:34:03.602 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.349+0000 7fc92b000700 1 -- 172.21.15.60:0/4273282022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 msgr2=0x7fc924106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:03.602 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.349+0000 7fc92b000700 1 --2- 172.21.15.60:0/4273282022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc924106370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9140098a0 tx=0x7fc914009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:03.602 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.349+0000 7fc92b000700 1 -- 172.21.15.60:0/4273282022 shutdown_connections 2024-02-26T10:34:03.602 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.349+0000 7fc92b000700 1 --2- 172.21.15.60:0/4273282022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc924106370 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:03.602 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.349+0000 7fc92b000700 1 -- 172.21.15.60:0/4273282022 >> 172.21.15.60:0/4273282022 conn(0x7fc924101af0 msgr2=0x7fc924103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.349+0000 7fc92b000700 1 -- 172.21.15.60:0/4273282022 shutdown_connections 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.349+0000 7fc92b000700 1 -- 172.21.15.60:0/4273282022 wait complete. 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.350+0000 7fc92b000700 1 Processor -- start 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.350+0000 7fc92b000700 1 -- start start 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.351+0000 7fc92b000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc92419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.351+0000 7fc92b000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc924107940 con 0x7fc924105f50 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.351+0000 7fc923e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc92419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.351+0000 7fc923e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc92419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34366/0 (socket says 172.21.15.60:34366) 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.351+0000 7fc923e00700 1 -- 172.21.15.60:0/2235813483 learned_addr learned my addr 172.21.15.60:0/2235813483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:03.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.351+0000 7fc923e00700 1 -- 172.21.15.60:0/2235813483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92419dda0 con 0x7fc924105f50 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.352+0000 7fc923e00700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc92419d860 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc914009fb0 tx=0x7fc914011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.352+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9140038b0 con 0x7fc924105f50 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.352+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc914011e50 con 0x7fc924105f50 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.352+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc914019420 con 0x7fc924105f50 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.352+0000 7fc92b000700 1 -- 172.21.15.60:0/2235813483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92419e030 con 0x7fc924105f50 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.352+0000 7fc92b000700 1 -- 172.21.15.60:0/2235813483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9241a0d20 con 0x7fc924105f50 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.353+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 45928+0+0 (secure 0 0 0) 0x7fc914019720 con 0x7fc924105f50 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.354+0000 7fc920c00700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 0x7fc90803b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.354+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 --> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc90803b720 con 0x7fc908038b50 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.354+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc91404ce40 con 0x7fc924105f50 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.354+0000 7fc923400700 1 -- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 msgr2=0x7fc90803b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2948072642 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.354+0000 7fc923400700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 0x7fc90803b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.555+0000 7fc923400700 1 -- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 msgr2=0x7fc90803b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2948072642 2024-02-26T10:34:03.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.555+0000 7fc923400700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 0x7fc90803b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.956+0000 7fc923400700 1 -- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 msgr2=0x7fc90803b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2948072642 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:55.956+0000 7fc923400700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 0x7fc90803b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:56.757+0000 7fc923400700 1 -- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 msgr2=0x7fc90803b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2948072642 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:56.757+0000 7fc923400700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 0x7fc90803b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:58.359+0000 7fc923400700 1 -- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 msgr2=0x7fc90803b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2948072642 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:33:58.359+0000 7fc923400700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 0x7fc90803b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:01.562+0000 7fc923400700 1 -- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 msgr2=0x7fc90803b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2948072642 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:01.562+0000 7fc923400700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 0x7fc90803b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:01.775+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mgrmap(e 6) v1 ==== 45695+0+0 (secure 0 0 0) 0x7fc914019a60 con 0x7fc924105f50 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:01.775+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 msgr2=0x7fc90803b010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:01.775+0000 7fc920c00700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 0x7fc90803b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.780+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 7) v1 ==== 45822+0+0 (secure 0 0 0) 0x7fc91404e350 con 0x7fc924105f50 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.781+0000 7fc920c00700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fc90803c480 0x7fc90803e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.781+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc914017ec0 con 0x7fc90803c480 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.782+0000 7fc923400700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fc90803c480 0x7fc90803e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.786+0000 7fc923400700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fc90803c480 0x7fc90803e870 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc910003a10 tx=0x7fc910009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.788+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 <== mgr.14120 v2:172.21.15.60:6800/603652410 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fc914017ec0 con 0x7fc90803c480 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.796+0000 7fc92b000700 1 -- 172.21.15.60:0/2235813483 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc924068160 con 0x7fc90803c480 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.797+0000 7fc920c00700 1 -- 172.21.15.60:0/2235813483 <== mgr.14120 v2:172.21.15.60:6800/603652410 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc924068160 con 0x7fc90803c480 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 -- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fc90803c480 msgr2=0x7fc90803e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fc90803c480 0x7fc90803e870 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc910003a10 tx=0x7fc910009290 comp rx=0 tx=0).stop 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 -- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 msgr2=0x7fc92419d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc92419d860 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc914009fb0 tx=0x7fc914011bf0 comp rx=0 tx=0).stop 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 -- 172.21.15.60:0/2235813483 shutdown_connections 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fc90803c480 0x7fc90803e870 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc924105f50 0x7fc92419d860 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 --2- 172.21.15.60:0/2235813483 >> [v2:172.21.15.60:6800/2948072642,v1:172.21.15.60:6801/2948072642] conn(0x7fc908038b50 0x7fc90803b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 -- 172.21.15.60:0/2235813483 >> 172.21.15.60:0/2235813483 conn(0x7fc924101af0 msgr2=0x7fc924194500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 -- 172.21.15.60:0/2235813483 shutdown_connections 2024-02-26T10:34:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:02.798+0000 7fc92b000700 1 -- 172.21.15.60:0/2235813483 wait complete. 2024-02-26T10:34:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 5 is available 2024-02-26T10:34:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:Setting orchestrator backend to cephadm... 2024-02-26T10:34:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:03 smithi060 ceph-mon[31539]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-26T10:34:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:03 smithi060 ceph-mon[31539]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-26T10:34:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:03 smithi060 ceph-mon[31539]: [26/Feb/2024:10:34:02] ENGINE Bus STARTING 2024-02-26T10:34:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:03 smithi060 ceph-mon[31539]: [26/Feb/2024:10:34:03] ENGINE Serving on https://172.21.15.60:7150 2024-02-26T10:34:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:03 smithi060 ceph-mon[31539]: [26/Feb/2024:10:34:03] ENGINE Serving on http://172.21.15.60:8765 2024-02-26T10:34:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:03 smithi060 ceph-mon[31539]: [26/Feb/2024:10:34:03] ENGINE Bus STARTED 2024-02-26T10:34:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:03 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:04 smithi060 ceph-mon[31539]: mgrmap e8: x(active, since 2s) 2024-02-26T10:34:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:04 smithi060 ceph-mon[31539]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:04 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:05.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:04 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:05.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.249+0000 7f62f2800700 1 Processor -- start 2024-02-26T10:34:05.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.249+0000 7f62f2800700 1 -- start start 2024-02-26T10:34:05.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.250+0000 7f62f2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:05.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.250+0000 7f62f2800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec108050 con 0x7f62ec107660 2024-02-26T10:34:05.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.250+0000 7f62eb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:05.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.250+0000 7f62eb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44820/0 (socket says 172.21.15.60:44820) 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.250+0000 7f62eb400700 1 -- 172.21.15.60:0/357145882 learned_addr learned my addr 172.21.15.60:0/357145882 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.251+0000 7f62eb400700 1 -- 172.21.15.60:0/357145882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec1087c0 con 0x7f62ec107660 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.251+0000 7f62eb400700 1 --2- 172.21.15.60:0/357145882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec107a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0098a0 tx=0x7f62dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf2f44db140f515 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.252+0000 7f62eaa00700 1 -- 172.21.15.60:0/357145882 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62dc004160 con 0x7f62ec107660 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.252+0000 7f62eaa00700 1 -- 172.21.15.60:0/357145882 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62dc004300 con 0x7f62ec107660 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.252+0000 7f62eaa00700 1 -- 172.21.15.60:0/357145882 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62dc011b50 con 0x7f62ec107660 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.252+0000 7f62f2800700 1 -- 172.21.15.60:0/357145882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 msgr2=0x7f62ec107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.252+0000 7f62f2800700 1 --2- 172.21.15.60:0/357145882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec107a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0098a0 tx=0x7f62dc009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.252+0000 7f62f2800700 1 -- 172.21.15.60:0/357145882 shutdown_connections 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.253+0000 7f62f2800700 1 --2- 172.21.15.60:0/357145882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec107a80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.253+0000 7f62f2800700 1 -- 172.21.15.60:0/357145882 >> 172.21.15.60:0/357145882 conn(0x7f62ec102c80 msgr2=0x7f62ec1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:05.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.253+0000 7f62f2800700 1 -- 172.21.15.60:0/357145882 shutdown_connections 2024-02-26T10:34:05.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.253+0000 7f62f2800700 1 -- 172.21.15.60:0/357145882 wait complete. 2024-02-26T10:34:05.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.254+0000 7f62f2800700 1 Processor -- start 2024-02-26T10:34:05.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.254+0000 7f62f2800700 1 -- start start 2024-02-26T10:34:05.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.254+0000 7f62f2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:05.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.254+0000 7f62f2800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec07b260 con 0x7f62ec107660 2024-02-26T10:34:05.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.255+0000 7f62eb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:05.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.255+0000 7f62eb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec19d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44826/0 (socket says 172.21.15.60:44826) 2024-02-26T10:34:05.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.255+0000 7f62eb400700 1 -- 172.21.15.60:0/1467130978 learned_addr learned my addr 172.21.15.60:0/1467130978 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:05.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.255+0000 7f62eb400700 1 -- 172.21.15.60:0/1467130978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec19dd50 con 0x7f62ec107660 2024-02-26T10:34:05.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.255+0000 7f62eb400700 1 --2- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec19d810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f62dc011670 tx=0x7f62dc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:05.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.256+0000 7f62e8c00700 1 -- 172.21.15.60:0/1467130978 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62dc0039c0 con 0x7f62ec107660 2024-02-26T10:34:05.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.256+0000 7f62e8c00700 1 -- 172.21.15.60:0/1467130978 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62dc011e50 con 0x7f62ec107660 2024-02-26T10:34:05.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.256+0000 7f62e8c00700 1 -- 172.21.15.60:0/1467130978 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62dc01a5c0 con 0x7f62ec107660 2024-02-26T10:34:05.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.256+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec19dfe0 con 0x7f62ec107660 2024-02-26T10:34:05.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.256+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec1a0cd0 con 0x7f62ec107660 2024-02-26T10:34:05.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.257+0000 7f62e8c00700 1 -- 172.21.15.60:0/1467130978 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f62dc024430 con 0x7f62ec107660 2024-02-26T10:34:05.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.257+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec068160 con 0x7f62ec107660 2024-02-26T10:34:05.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.257+0000 7f62e8c00700 1 --2- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f62d0038ba0 0x7f62d003b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:05.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.258+0000 7f62e8c00700 1 -- 172.21.15.60:0/1467130978 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f62dc04cfb0 con 0x7f62ec107660 2024-02-26T10:34:05.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.258+0000 7f62e2a00700 1 --2- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f62d0038ba0 0x7f62d003b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.259+0000 7f62e2a00700 1 --2- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f62d0038ba0 0x7f62d003b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62d4006fd0 tx=0x7f62d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.264+0000 7f62e8c00700 1 -- 172.21.15.60:0/1467130978 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62dc02d910 con 0x7f62ec107660 2024-02-26T10:34:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.507+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f62ec02dba0 con 0x7f62d0038ba0 2024-02-26T10:34:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.516+0000 7f62e8c00700 1 -- 172.21.15.60:0/1467130978 <== mgr.14120 v2:172.21.15.60:6800/603652410 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f62ec02dba0 con 0x7f62d0038ba0 2024-02-26T10:34:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f62d0038ba0 msgr2=0x7f62d003b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 --2- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f62d0038ba0 0x7f62d003b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62d4006fd0 tx=0x7f62d4006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 msgr2=0x7f62ec19d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:05.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 --2- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec19d810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f62dc011670 tx=0x7f62dc011b60 comp rx=0 tx=0).stop 2024-02-26T10:34:05.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 shutdown_connections 2024-02-26T10:34:05.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 --2- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f62d0038ba0 0x7f62d003b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:05.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 --2- 172.21.15.60:0/1467130978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec107660 0x7f62ec19d810 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:05.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 >> 172.21.15.60:0/1467130978 conn(0x7f62ec102c80 msgr2=0x7f62ec194b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:05.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 shutdown_connections 2024-02-26T10:34:05.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:04.520+0000 7f62f2800700 1 -- 172.21.15.60:0/1467130978 wait complete. 2024-02-26T10:34:06.763 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-26T10:34:06.763 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.850+0000 7f59e5400700 1 Processor -- start 2024-02-26T10:34:06.763 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.850+0000 7f59e5400700 1 -- start start 2024-02-26T10:34:06.763 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.851+0000 7f59e5400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e007b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:06.763 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.851+0000 7f59e5400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e007b910 con 0x7f59e007ce90 2024-02-26T10:34:06.763 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.851+0000 7f59de800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:06.763 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.851+0000 7f59de800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e007b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44832/0 (socket says 172.21.15.60:44832) 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.851+0000 7f59de800700 1 -- 172.21.15.60:0/456191664 learned_addr learned my addr 172.21.15.60:0/456191664 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.852+0000 7f59de800700 1 -- 172.21.15.60:0/456191664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e007ba90 con 0x7f59e007ce90 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.852+0000 7f59de800700 1 --2- 172.21.15.60:0/456191664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e007b340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f59d00098a0 tx=0x7f59d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50527290e876a9e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.853+0000 7f59dd400700 1 -- 172.21.15.60:0/456191664 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59d0004160 con 0x7f59e007ce90 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.853+0000 7f59dd400700 1 -- 172.21.15.60:0/456191664 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59d0004300 con 0x7f59e007ce90 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.853+0000 7f59dd400700 1 -- 172.21.15.60:0/456191664 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59d0011b50 con 0x7f59e007ce90 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.853+0000 7f59e5400700 1 -- 172.21.15.60:0/456191664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 msgr2=0x7f59e007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.853+0000 7f59e5400700 1 --2- 172.21.15.60:0/456191664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e007b340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f59d00098a0 tx=0x7f59d0009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.853+0000 7f59e5400700 1 -- 172.21.15.60:0/456191664 shutdown_connections 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.853+0000 7f59e5400700 1 --2- 172.21.15.60:0/456191664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e007b340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.853+0000 7f59e5400700 1 -- 172.21.15.60:0/456191664 >> 172.21.15.60:0/456191664 conn(0x7f59e0102cf0 msgr2=0x7f59e0105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:06.764 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.854+0000 7f59e5400700 1 -- 172.21.15.60:0/456191664 shutdown_connections 2024-02-26T10:34:06.765 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.854+0000 7f59e5400700 1 -- 172.21.15.60:0/456191664 wait complete. 2024-02-26T10:34:06.768 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.855+0000 7f59e5400700 1 Processor -- start 2024-02-26T10:34:06.768 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.855+0000 7f59e5400700 1 -- start start 2024-02-26T10:34:06.768 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.856+0000 7f59e5400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:06.768 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.856+0000 7f59e5400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e010dc00 con 0x7f59e007ce90 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.856+0000 7f59de800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.856+0000 7f59de800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e019d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44842/0 (socket says 172.21.15.60:44842) 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.856+0000 7f59de800700 1 -- 172.21.15.60:0/985610056 learned_addr learned my addr 172.21.15.60:0/985610056 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.856+0000 7f59de800700 1 -- 172.21.15.60:0/985610056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e019dd30 con 0x7f59e007ce90 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.857+0000 7f59de800700 1 --2- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e019d7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009fb0 tx=0x7f59d0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.857+0000 7f59d7400700 1 -- 172.21.15.60:0/985610056 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59d00038b0 con 0x7f59e007ce90 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.857+0000 7f59d7400700 1 -- 172.21.15.60:0/985610056 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59d0011e50 con 0x7f59e007ce90 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.857+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e019dfc0 con 0x7f59e007ce90 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.857+0000 7f59d7400700 1 -- 172.21.15.60:0/985610056 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59d0019420 con 0x7f59e007ce90 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.857+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e01a0cb0 con 0x7f59e007ce90 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.858+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e0068160 con 0x7f59e007ce90 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.858+0000 7f59d7400700 1 -- 172.21.15.60:0/985610056 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f59d0003a50 con 0x7f59e007ce90 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.859+0000 7f59d7400700 1 --2- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f59c4038ba0 0x7f59c403b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:06.769 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.859+0000 7f59d7400700 1 -- 172.21.15.60:0/985610056 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f59d004ccf0 con 0x7f59e007ce90 2024-02-26T10:34:06.770 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.859+0000 7f59dde00700 1 --2- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f59c4038ba0 0x7f59c403b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:06.770 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.860+0000 7f59dde00700 1 --2- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f59c4038ba0 0x7f59c403b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59cc006fd0 tx=0x7f59cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:06.770 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:05.865+0000 7f59d7400700 1 -- 172.21.15.60:0/985610056 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f59d004c640 con 0x7f59e007ce90 2024-02-26T10:34:06.770 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.091+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f59e0104000 con 0x7f59c4038ba0 2024-02-26T10:34:06.770 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.092+0000 7f59d7400700 1 -- 172.21.15.60:0/985610056 <== mgr.14120 v2:172.21.15.60:6800/603652410 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f59e0104000 con 0x7f59c4038ba0 2024-02-26T10:34:06.770 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f59c4038ba0 msgr2=0x7f59c403b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:06.770 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 --2- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f59c4038ba0 0x7f59c403b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59cc006fd0 tx=0x7f59cc006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:06.770 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 msgr2=0x7f59e019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:06.770 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 --2- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e019d7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009fb0 tx=0x7f59d0011bf0 comp rx=0 tx=0).stop 2024-02-26T10:34:06.771 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 shutdown_connections 2024-02-26T10:34:06.771 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 --2- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f59c4038ba0 0x7f59c403b060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:06.771 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 --2- 172.21.15.60:0/985610056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e007ce90 0x7f59e019d7f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:06.771 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 >> 172.21.15.60:0/985610056 conn(0x7f59e0102cf0 msgr2=0x7f59e01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:06.771 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 shutdown_connections 2024-02-26T10:34:06.771 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:06.095+0000 7f59e5400700 1 -- 172.21.15.60:0/985610056 wait complete. 2024-02-26T10:34:06.772 INFO:teuthology.orchestra.run.smithi060.stdout:Generating ssh key... 2024-02-26T10:34:07.079 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:06 smithi060 ceph-mon[31539]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:08.018 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: Generating public/private rsa key pair. 2024-02-26T10:34:08.018 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: Your identification has been saved in /tmp/tmpn2m825ev/key. 2024-02-26T10:34:08.018 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: Your public key has been saved in /tmp/tmpn2m825ev/key.pub. 2024-02-26T10:34:08.018 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: The key fingerprint is: 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: SHA256:++wn4esTqbUQz73NH3U4zo281RMKeAhqnVQDE5Gj2Mg ceph-51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: The key's randomart image is: 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: +---[RSA 3072]----+ 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: | ==o | 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: | oo . | 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: | . + .o. | 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: | E o+ + o . | 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: | o oS* = o.o| 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: | . ..O o+.+=| 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: | .= + ==o+| 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: | .o= o ooo| 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: | o*= ...| 2024-02-26T10:34:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: +----[SHA256]-----+ 2024-02-26T10:34:08.353 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.363+0000 7f8b2e000700 1 Processor -- start 2024-02-26T10:34:08.353 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.363+0000 7f8b2e000700 1 -- start start 2024-02-26T10:34:08.354 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.364+0000 7f8b2e000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b28109b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:08.354 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.364+0000 7f8b2e000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2810a060 con 0x7f8b28109700 2024-02-26T10:34:08.354 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.364+0000 7f8b2cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b28109b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:08.354 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.364+0000 7f8b2cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b28109b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44844/0 (socket says 172.21.15.60:44844) 2024-02-26T10:34:08.355 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.364+0000 7f8b2cc00700 1 -- 172.21.15.60:0/3395886606 learned_addr learned my addr 172.21.15.60:0/3395886606 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:08.355 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.365+0000 7f8b2cc00700 1 -- 172.21.15.60:0/3395886606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2810a1e0 con 0x7f8b28109700 2024-02-26T10:34:08.355 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.365+0000 7f8b2cc00700 1 --2- 172.21.15.60:0/3395886606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b28109b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0098a0 tx=0x7f8b1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71af1b3d77901522 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:08.355 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.365+0000 7f8b27400700 1 -- 172.21.15.60:0/3395886606 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b1c004160 con 0x7f8b28109700 2024-02-26T10:34:08.355 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.366+0000 7f8b27400700 1 -- 172.21.15.60:0/3395886606 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b1c004300 con 0x7f8b28109700 2024-02-26T10:34:08.355 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.366+0000 7f8b27400700 1 -- 172.21.15.60:0/3395886606 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b1c011b50 con 0x7f8b28109700 2024-02-26T10:34:08.357 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.366+0000 7f8b2e000700 1 -- 172.21.15.60:0/3395886606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 msgr2=0x7f8b28109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:08.357 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.366+0000 7f8b2e000700 1 --2- 172.21.15.60:0/3395886606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b28109b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0098a0 tx=0x7f8b1c009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:08.357 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.366+0000 7f8b2e000700 1 -- 172.21.15.60:0/3395886606 shutdown_connections 2024-02-26T10:34:08.357 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.366+0000 7f8b2e000700 1 --2- 172.21.15.60:0/3395886606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b28109b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:08.357 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.366+0000 7f8b2e000700 1 -- 172.21.15.60:0/3395886606 >> 172.21.15.60:0/3395886606 conn(0x7f8b2807cdd0 msgr2=0x7f8b281074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:08.357 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.366+0000 7f8b2e000700 1 -- 172.21.15.60:0/3395886606 shutdown_connections 2024-02-26T10:34:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.366+0000 7f8b2e000700 1 -- 172.21.15.60:0/3395886606 wait complete. 2024-02-26T10:34:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.367+0000 7f8b2e000700 1 Processor -- start 2024-02-26T10:34:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.368+0000 7f8b2e000700 1 -- start start 2024-02-26T10:34:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.368+0000 7f8b2e000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b2819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.368+0000 7f8b2e000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2810db20 con 0x7f8b28109700 2024-02-26T10:34:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.368+0000 7f8b2cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b2819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.368+0000 7f8b2cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b2819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44860/0 (socket says 172.21.15.60:44860) 2024-02-26T10:34:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.368+0000 7f8b2cc00700 1 -- 172.21.15.60:0/3191060516 learned_addr learned my addr 172.21.15.60:0/3191060516 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.369+0000 7f8b2cc00700 1 -- 172.21.15.60:0/3191060516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2819dae0 con 0x7f8b28109700 2024-02-26T10:34:08.360 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.369+0000 7f8b2cc00700 1 --2- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b2819d5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c011b60 tx=0x7f8b1c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.369+0000 7f8b25600700 1 -- 172.21.15.60:0/3191060516 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b1c003b20 con 0x7f8b28109700 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.369+0000 7f8b25600700 1 -- 172.21.15.60:0/3191060516 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b1c003cc0 con 0x7f8b28109700 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.369+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2819dd70 con 0x7f8b28109700 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.369+0000 7f8b25600700 1 -- 172.21.15.60:0/3191060516 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b1c024840 con 0x7f8b28109700 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.370+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b281a0a60 con 0x7f8b28109700 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.371+0000 7f8b25600700 1 -- 172.21.15.60:0/3191060516 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f8b1c022020 con 0x7f8b28109700 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.371+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b28063380 con 0x7f8b28109700 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.371+0000 7f8b25600700 1 --2- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f8b10038bf0 0x7f8b1003b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.371+0000 7f8b25600700 1 -- 172.21.15.60:0/3191060516 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8b1c011e00 con 0x7f8b28109700 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.371+0000 7f8b27e00700 1 --2- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f8b10038bf0 0x7f8b1003b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.372+0000 7f8b27e00700 1 --2- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f8b10038bf0 0x7f8b1003b0b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b14006fd0 tx=0x7f8b14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.376+0000 7f8b25600700 1 -- 172.21.15.60:0/3191060516 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b1c04c4e0 con 0x7f8b28109700 2024-02-26T10:34:08.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.599+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b28107220 con 0x7f8b10038bf0 2024-02-26T10:34:08.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.716+0000 7f8b25600700 1 -- 172.21.15.60:0/3191060516 <== mgr.14120 v2:172.21.15.60:6800/603652410 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8b28107220 con 0x7f8b10038bf0 2024-02-26T10:34:08.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.719+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f8b10038bf0 msgr2=0x7f8b1003b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:08.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.719+0000 7f8b2e000700 1 --2- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f8b10038bf0 0x7f8b1003b0b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b14006fd0 tx=0x7f8b14006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:08.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.719+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 msgr2=0x7f8b2819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:08.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.719+0000 7f8b2e000700 1 --2- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b2819d5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c011b60 tx=0x7f8b1c011c40 comp rx=0 tx=0).stop 2024-02-26T10:34:08.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.719+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 shutdown_connections 2024-02-26T10:34:08.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.719+0000 7f8b2e000700 1 --2- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f8b10038bf0 0x7f8b1003b0b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:08.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.719+0000 7f8b2e000700 1 --2- 172.21.15.60:0/3191060516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b28109700 0x7f8b2819d5a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:08.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.719+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 >> 172.21.15.60:0/3191060516 conn(0x7f8b2807cdd0 msgr2=0x7f8b28106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:08.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.720+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 shutdown_connections 2024-02-26T10:34:08.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:07.720+0000 7f8b2e000700 1 -- 172.21.15.60:0/3191060516 wait complete. 2024-02-26T10:34:08.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:08 smithi060 ceph-mon[31539]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:08.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:08 smithi060 ceph-mon[31539]: Generating ssh key... 2024-02-26T10:34:08.975 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:08 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:08.975 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:08 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:09.787 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDSB0H4V5PocVbKIURPKRENalEbnQ+ZQ0/shFqqWVkZ4+LbAU2qverLCmC3m/Cgp0wHLm6nJ63ZsPs/519ozsZVbmzSWX6fY5rWbNIiNU4y6d7bFjBocnuqHRdqb5OnMpW/LdJTj3ux6eStopm6nW958gGKciXjJBy7NYguurprM1Qxwr8M6JaAs7pGYLFG1RBNjtdWkYfK2BM6dazKaY5EHFf+N1a6s5jxdCyC+t5Dl6iIO5DOnetV22GmfyzW/EVlDUAWg14qFomOcdvNZ8UgkdjfMInQj6yuiTF0Y33LcfmtJSYchXQXlRjEcHfD9/EUxwV3xF4VPRMxX2005L3QnEeLK9iiG23s3Y62UDNkNQBqykdoPZcyleuVNXdB1WRnl5IcAEO2vY7XhF7SZayEmZUac7U2rlSszL4ZPjkUptYsLHxmx6YysV/XR3iSp3Y5itxxP1/CseHJflF0YXhWE287t5vY4INAkMk0SP5+67/3PtltTvWJr/OTx62X3c= ceph-51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:34:09.788 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.969+0000 7f7c20800700 1 Processor -- start 2024-02-26T10:34:09.788 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.969+0000 7f7c20800700 1 -- start start 2024-02-26T10:34:09.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.970+0000 7f7c20800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c1810e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:09.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.970+0000 7f7c20800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1810eb00 con 0x7f7c1810e110 2024-02-26T10:34:09.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.970+0000 7f7c1de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c1810e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:09.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.970+0000 7f7c1de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c1810e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44862/0 (socket says 172.21.15.60:44862) 2024-02-26T10:34:09.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.970+0000 7f7c1de00700 1 -- 172.21.15.60:0/2553426185 learned_addr learned my addr 172.21.15.60:0/2553426185 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:09.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.971+0000 7f7c1de00700 1 -- 172.21.15.60:0/2553426185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1810ede0 con 0x7f7c1810e110 2024-02-26T10:34:09.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.971+0000 7f7c1de00700 1 --2- 172.21.15.60:0/2553426185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c1810e530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0098a0 tx=0x7f7c0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7843590b3a37859 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:09.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.971+0000 7f7c1ca00700 1 -- 172.21.15.60:0/2553426185 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c0c004160 con 0x7f7c1810e110 2024-02-26T10:34:09.799 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.971+0000 7f7c1ca00700 1 -- 172.21.15.60:0/2553426185 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c0c004300 con 0x7f7c1810e110 2024-02-26T10:34:09.799 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.971+0000 7f7c1ca00700 1 -- 172.21.15.60:0/2553426185 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c0c011b50 con 0x7f7c1810e110 2024-02-26T10:34:09.799 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.972+0000 7f7c20800700 1 -- 172.21.15.60:0/2553426185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 msgr2=0x7f7c1810e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:09.799 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.972+0000 7f7c20800700 1 --2- 172.21.15.60:0/2553426185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c1810e530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0098a0 tx=0x7f7c0c009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:09.799 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.973+0000 7f7c20800700 1 -- 172.21.15.60:0/2553426185 shutdown_connections 2024-02-26T10:34:09.799 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.973+0000 7f7c20800700 1 --2- 172.21.15.60:0/2553426185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c1810e530 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:09.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.973+0000 7f7c20800700 1 -- 172.21.15.60:0/2553426185 >> 172.21.15.60:0/2553426185 conn(0x7f7c1807cdd0 msgr2=0x7f7c1807f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:09.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.973+0000 7f7c20800700 1 -- 172.21.15.60:0/2553426185 shutdown_connections 2024-02-26T10:34:09.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.973+0000 7f7c20800700 1 -- 172.21.15.60:0/2553426185 wait complete. 2024-02-26T10:34:09.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.975+0000 7f7c20800700 1 Processor -- start 2024-02-26T10:34:09.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.975+0000 7f7c20800700 1 -- start start 2024-02-26T10:34:09.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.975+0000 7f7c20800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c181a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:09.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.975+0000 7f7c20800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c18112010 con 0x7f7c1810e110 2024-02-26T10:34:09.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.976+0000 7f7c1de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c181a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:09.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.976+0000 7f7c1de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c181a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44868/0 (socket says 172.21.15.60:44868) 2024-02-26T10:34:09.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.976+0000 7f7c1de00700 1 -- 172.21.15.60:0/372195300 learned_addr learned my addr 172.21.15.60:0/372195300 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:09.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.976+0000 7f7c1de00700 1 -- 172.21.15.60:0/372195300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c181a2160 con 0x7f7c1810e110 2024-02-26T10:34:09.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.976+0000 7f7c1de00700 1 --2- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c181a1c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009fb0 tx=0x7f7c0c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:09.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.977+0000 7f7c12a00700 1 -- 172.21.15.60:0/372195300 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c0c0038b0 con 0x7f7c1810e110 2024-02-26T10:34:09.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.977+0000 7f7c12a00700 1 -- 172.21.15.60:0/372195300 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c0c011e50 con 0x7f7c1810e110 2024-02-26T10:34:09.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.977+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c181a23f0 con 0x7f7c1810e110 2024-02-26T10:34:09.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.977+0000 7f7c12a00700 1 -- 172.21.15.60:0/372195300 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c0c019550 con 0x7f7c1810e110 2024-02-26T10:34:09.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.977+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c181a50e0 con 0x7f7c1810e110 2024-02-26T10:34:09.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.978+0000 7f7c12a00700 1 -- 172.21.15.60:0/372195300 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f7c0c003a50 con 0x7f7c1810e110 2024-02-26T10:34:09.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.978+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c18068160 con 0x7f7c1810e110 2024-02-26T10:34:09.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.979+0000 7f7c12a00700 1 --2- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f7c00038c00 0x7f7c0003b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:09.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.979+0000 7f7c12a00700 1 -- 172.21.15.60:0/372195300 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7c0c04cd20 con 0x7f7c1810e110 2024-02-26T10:34:09.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.979+0000 7f7c1d400700 1 --2- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f7c00038c00 0x7f7c0003b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:09.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.980+0000 7f7c1d400700 1 --2- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f7c00038c00 0x7f7c0003b0c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006fd0 tx=0x7f7c08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:09.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:08.984+0000 7f7c12a00700 1 -- 172.21.15.60:0/372195300 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c0c02dc60 con 0x7f7c1810e110 2024-02-26T10:34:09.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.209+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c1807e090 con 0x7f7c00038c00 2024-02-26T10:34:09.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.211+0000 7f7c12a00700 1 -- 172.21.15.60:0/372195300 <== mgr.14120 v2:172.21.15.60:6800/603652410 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f7c1807e090 con 0x7f7c00038c00 2024-02-26T10:34:09.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f7c00038c00 msgr2=0x7f7c0003b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:09.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 --2- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f7c00038c00 0x7f7c0003b0c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006fd0 tx=0x7f7c08006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:09.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 msgr2=0x7f7c181a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:09.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 --2- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c181a1c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009fb0 tx=0x7f7c0c011bf0 comp rx=0 tx=0).stop 2024-02-26T10:34:09.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 shutdown_connections 2024-02-26T10:34:09.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 --2- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f7c00038c00 0x7f7c0003b0c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:09.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 --2- 172.21.15.60:0/372195300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c1810e110 0x7f7c181a1c20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:09.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 >> 172.21.15.60:0/372195300 conn(0x7f7c1807cdd0 msgr2=0x7f7c1807d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:09.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 shutdown_connections 2024-02-26T10:34:09.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:09.214+0000 7f7c20800700 1 -- 172.21.15.60:0/372195300 wait complete. 2024-02-26T10:34:09.805 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-26T10:34:09.806 INFO:teuthology.orchestra.run.smithi060.stdout:Adding key to root@localhost authorized_keys... 2024-02-26T10:34:09.806 INFO:teuthology.orchestra.run.smithi060.stdout:Adding host smithi060... 2024-02-26T10:34:10.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:09 smithi060 ceph-mon[31539]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:10.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:10 smithi060 ceph-mon[31539]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:13.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:12 smithi060 ceph-mon[31539]: Deploying cephadm binary to smithi060 2024-02-26T10:34:18.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Added host 'smithi060' with addr '172.21.15.60' 2024-02-26T10:34:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.407+0000 7f53f8a00700 1 Processor -- start 2024-02-26T10:34:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.408+0000 7f53f8a00700 1 -- start start 2024-02-26T10:34:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.408+0000 7f53f8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f4107a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.408+0000 7f53f8a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f4108030 con 0x7f53f4107640 2024-02-26T10:34:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.408+0000 7f53f1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f4107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.409+0000 7f53f1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f4107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44874/0 (socket says 172.21.15.60:44874) 2024-02-26T10:34:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.409+0000 7f53f1e00700 1 -- 172.21.15.60:0/2831346039 learned_addr learned my addr 172.21.15.60:0/2831346039 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.409+0000 7f53f1e00700 1 -- 172.21.15.60:0/2831346039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f41087f0 con 0x7f53f4107640 2024-02-26T10:34:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.409+0000 7f53f1e00700 1 --2- 172.21.15.60:0/2831346039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f4107a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53e40098a0 tx=0x7f53e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1d16e24f1f95f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.410+0000 7f53f0a00700 1 -- 172.21.15.60:0/2831346039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53e4004160 con 0x7f53f4107640 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.410+0000 7f53f0a00700 1 -- 172.21.15.60:0/2831346039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53e4004300 con 0x7f53f4107640 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.410+0000 7f53f0a00700 1 -- 172.21.15.60:0/2831346039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53e4011b50 con 0x7f53f4107640 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.411+0000 7f53f8a00700 1 -- 172.21.15.60:0/2831346039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 msgr2=0x7f53f4107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.411+0000 7f53f8a00700 1 --2- 172.21.15.60:0/2831346039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f4107a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53e40098a0 tx=0x7f53e4009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.411+0000 7f53f8a00700 1 -- 172.21.15.60:0/2831346039 shutdown_connections 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.411+0000 7f53f8a00700 1 --2- 172.21.15.60:0/2831346039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f4107a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.411+0000 7f53f8a00700 1 -- 172.21.15.60:0/2831346039 >> 172.21.15.60:0/2831346039 conn(0x7f53f4102cc0 msgr2=0x7f53f41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.411+0000 7f53f8a00700 1 -- 172.21.15.60:0/2831346039 shutdown_connections 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.411+0000 7f53f8a00700 1 -- 172.21.15.60:0/2831346039 wait complete. 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.412+0000 7f53f8a00700 1 Processor -- start 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.412+0000 7f53f8a00700 1 -- start start 2024-02-26T10:34:18.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.412+0000 7f53f8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.412+0000 7f53f8a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f410bba0 con 0x7f53f4107640 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.413+0000 7f53f1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.413+0000 7f53f1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f419d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44888/0 (socket says 172.21.15.60:44888) 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.413+0000 7f53f1e00700 1 -- 172.21.15.60:0/1000193961 learned_addr learned my addr 172.21.15.60:0/1000193961 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.413+0000 7f53f1e00700 1 -- 172.21.15.60:0/1000193961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f419dd40 con 0x7f53f4107640 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.414+0000 7f53f1e00700 1 --2- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f419d800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000c00 tx=0x7f53e4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.414+0000 7f53eaa00700 1 -- 172.21.15.60:0/1000193961 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53e4003b20 con 0x7f53f4107640 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.414+0000 7f53eaa00700 1 -- 172.21.15.60:0/1000193961 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53e4003cc0 con 0x7f53f4107640 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.414+0000 7f53eaa00700 1 -- 172.21.15.60:0/1000193961 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53e4024910 con 0x7f53f4107640 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.414+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f419dfd0 con 0x7f53f4107640 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.414+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f41a0cc0 con 0x7f53f4107640 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.415+0000 7f53eaa00700 1 -- 172.21.15.60:0/1000193961 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f53e4022020 con 0x7f53f4107640 2024-02-26T10:34:18.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.415+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f4068160 con 0x7f53f4107640 2024-02-26T10:34:18.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.416+0000 7f53eaa00700 1 --2- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f53d8038c50 0x7f53d803b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:18.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.416+0000 7f53eaa00700 1 -- 172.21.15.60:0/1000193961 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f53e4011e00 con 0x7f53f4107640 2024-02-26T10:34:18.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.416+0000 7f53f1400700 1 --2- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f53d8038c50 0x7f53d803b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:18.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.417+0000 7f53f1400700 1 --2- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f53d8038c50 0x7f53d803b110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:18.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.422+0000 7f53eaa00700 1 -- 172.21.15.60:0/1000193961 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53e4016260 con 0x7f53f4107640 2024-02-26T10:34:18.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:10.648+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}) v1 -- 0x7f53f4103e80 con 0x7f53d8038c50 2024-02-26T10:34:18.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.429+0000 7f53eaa00700 1 -- 172.21.15.60:0/1000193961 <== mgr.14120 v2:172.21.15.60:6800/603652410 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f53f4103e80 con 0x7f53d8038c50 2024-02-26T10:34:18.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.433+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f53d8038c50 msgr2=0x7f53d803b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:18.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.433+0000 7f53f8a00700 1 --2- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f53d8038c50 0x7f53d803b110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:18.118 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.433+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 msgr2=0x7f53f419d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:18.119 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.433+0000 7f53f8a00700 1 --2- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f419d800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000c00 tx=0x7f53e4011c10 comp rx=0 tx=0).stop 2024-02-26T10:34:18.119 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.433+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 shutdown_connections 2024-02-26T10:34:18.119 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.433+0000 7f53f8a00700 1 --2- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f53d8038c50 0x7f53d803b110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:18.119 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.433+0000 7f53f8a00700 1 --2- 172.21.15.60:0/1000193961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f4107640 0x7f53f419d800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:18.119 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.433+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 >> 172.21.15.60:0/1000193961 conn(0x7f53f4102cc0 msgr2=0x7f53f4103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:18.119 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.434+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 shutdown_connections 2024-02-26T10:34:18.120 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:17.434+0000 7f53f8a00700 1 -- 172.21.15.60:0/1000193961 wait complete. 2024-02-26T10:34:18.120 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mon service... 2024-02-26T10:34:18.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:18 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:18.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:18 smithi060 ceph-mon[31539]: Added host smithi060 2024-02-26T10:34:18.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:18 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:19.909 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-26T10:34:19.909 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.793+0000 7fed35000700 1 Processor -- start 2024-02-26T10:34:19.909 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.793+0000 7fed35000700 1 -- start start 2024-02-26T10:34:19.909 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.794+0000 7fed35000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed30107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:19.909 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.794+0000 7fed35000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed30108050 con 0x7fed30107660 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.794+0000 7fed2de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed30107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.794+0000 7fed2de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed30107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49372/0 (socket says 172.21.15.60:49372) 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.794+0000 7fed2de00700 1 -- 172.21.15.60:0/2606622633 learned_addr learned my addr 172.21.15.60:0/2606622633 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.795+0000 7fed2de00700 1 -- 172.21.15.60:0/2606622633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed301087c0 con 0x7fed30107660 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.795+0000 7fed2de00700 1 --2- 172.21.15.60:0/2606622633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed30107a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fed200098a0 tx=0x7fed20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e83b11e35b535dd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.795+0000 7fed2ca00700 1 -- 172.21.15.60:0/2606622633 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed20004160 con 0x7fed30107660 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.796+0000 7fed2ca00700 1 -- 172.21.15.60:0/2606622633 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed20004300 con 0x7fed30107660 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.796+0000 7fed2ca00700 1 -- 172.21.15.60:0/2606622633 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed20011b50 con 0x7fed30107660 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.796+0000 7fed35000700 1 -- 172.21.15.60:0/2606622633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 msgr2=0x7fed30107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:19.910 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.796+0000 7fed35000700 1 --2- 172.21.15.60:0/2606622633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed30107a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fed200098a0 tx=0x7fed20009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.796+0000 7fed35000700 1 -- 172.21.15.60:0/2606622633 shutdown_connections 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.796+0000 7fed35000700 1 --2- 172.21.15.60:0/2606622633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed30107a80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.796+0000 7fed35000700 1 -- 172.21.15.60:0/2606622633 >> 172.21.15.60:0/2606622633 conn(0x7fed30102c80 msgr2=0x7fed301050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.796+0000 7fed35000700 1 -- 172.21.15.60:0/2606622633 shutdown_connections 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.797+0000 7fed35000700 1 -- 172.21.15.60:0/2606622633 wait complete. 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.798+0000 7fed35000700 1 Processor -- start 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.798+0000 7fed35000700 1 -- start start 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.798+0000 7fed35000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed3019d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.798+0000 7fed35000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3010bb70 con 0x7fed30107660 2024-02-26T10:34:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.798+0000 7fed2de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed3019d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.799+0000 7fed2de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed3019d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49378/0 (socket says 172.21.15.60:49378) 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.799+0000 7fed2de00700 1 -- 172.21.15.60:0/4112858370 learned_addr learned my addr 172.21.15.60:0/4112858370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.799+0000 7fed2de00700 1 -- 172.21.15.60:0/4112858370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3019dd60 con 0x7fed30107660 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.799+0000 7fed2de00700 1 --2- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed3019d820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed20000c00 tx=0x7fed20011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.799+0000 7fed26a00700 1 -- 172.21.15.60:0/4112858370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed20003b20 con 0x7fed30107660 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.799+0000 7fed26a00700 1 -- 172.21.15.60:0/4112858370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed20003cc0 con 0x7fed30107660 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.799+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3019dff0 con 0x7fed30107660 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.799+0000 7fed26a00700 1 -- 172.21.15.60:0/4112858370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed20024910 con 0x7fed30107660 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.800+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed301a0ce0 con 0x7fed30107660 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.801+0000 7fed26a00700 1 -- 172.21.15.60:0/4112858370 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7fed20022020 con 0x7fed30107660 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.801+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed30068160 con 0x7fed30107660 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.801+0000 7fed26a00700 1 --2- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fed14038bf0 0x7fed1403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.801+0000 7fed26a00700 1 -- 172.21.15.60:0/4112858370 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fed20011e00 con 0x7fed30107660 2024-02-26T10:34:19.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.802+0000 7fed2d400700 1 --2- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fed14038bf0 0x7fed1403b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.802+0000 7fed2d400700 1 --2- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fed14038bf0 0x7fed1403b0b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed1c006fd0 tx=0x7fed1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:18.807+0000 7fed26a00700 1 -- 172.21.15.60:0/4112858370 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed2004c490 con 0x7fed30107660 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.060+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fed30103e60 con 0x7fed14038bf0 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.069+0000 7fed26a00700 1 -- 172.21.15.60:0/4112858370 <== mgr.14120 v2:172.21.15.60:6800/603652410 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fed30103e60 con 0x7fed14038bf0 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.072+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fed14038bf0 msgr2=0x7fed1403b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.072+0000 7fed35000700 1 --2- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fed14038bf0 0x7fed1403b0b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed1c006fd0 tx=0x7fed1c006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.073+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 msgr2=0x7fed3019d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.073+0000 7fed35000700 1 --2- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed3019d820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed20000c00 tx=0x7fed20011c10 comp rx=0 tx=0).stop 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.073+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 shutdown_connections 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.073+0000 7fed35000700 1 --2- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fed14038bf0 0x7fed1403b0b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.073+0000 7fed35000700 1 --2- 172.21.15.60:0/4112858370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed30107660 0x7fed3019d820 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.073+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 >> 172.21.15.60:0/4112858370 conn(0x7fed30102c80 msgr2=0x7fed30103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.073+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 shutdown_connections 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:19.073+0000 7fed35000700 1 -- 172.21.15.60:0/4112858370 wait complete. 2024-02-26T10:34:19.913 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mgr service... 2024-02-26T10:34:20.147 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:20 smithi060 ceph-mon[31539]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:20.148 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:20 smithi060 ceph-mon[31539]: Saving service mon spec with placement count:5 2024-02-26T10:34:20.148 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:20 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:22.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:22 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:22.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:22 smithi060 ceph-mon[31539]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:22.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:22 smithi060 ceph-mon[31539]: Saving service mgr spec with placement count:2 2024-02-26T10:34:22.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:22 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:22.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:22 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:22.361 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.353+0000 7efc9d600700 1 Processor -- start 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.353+0000 7efc9d600700 1 -- start start 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.353+0000 7efc9d600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.353+0000 7efc9d600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9807b800 con 0x7efc9807ce10 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.354+0000 7efc96a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.354+0000 7efc96a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9807b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49388/0 (socket says 172.21.15.60:49388) 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.354+0000 7efc96a00700 1 -- 172.21.15.60:0/3288585131 learned_addr learned my addr 172.21.15.60:0/3288585131 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.354+0000 7efc96a00700 1 -- 172.21.15.60:0/3288585131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9807b980 con 0x7efc9807ce10 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.355+0000 7efc96a00700 1 --2- 172.21.15.60:0/3288585131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9807b2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efc880098a0 tx=0x7efc88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367f87e9036a3f8d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.355+0000 7efc95600700 1 -- 172.21.15.60:0/3288585131 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc88004160 con 0x7efc9807ce10 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.355+0000 7efc95600700 1 -- 172.21.15.60:0/3288585131 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc88004300 con 0x7efc9807ce10 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.355+0000 7efc95600700 1 -- 172.21.15.60:0/3288585131 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc88011b20 con 0x7efc9807ce10 2024-02-26T10:34:22.362 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.356+0000 7efc9d600700 1 -- 172.21.15.60:0/3288585131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 msgr2=0x7efc9807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.356+0000 7efc9d600700 1 --2- 172.21.15.60:0/3288585131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9807b2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efc880098a0 tx=0x7efc88009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.356+0000 7efc9d600700 1 -- 172.21.15.60:0/3288585131 shutdown_connections 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.356+0000 7efc9d600700 1 --2- 172.21.15.60:0/3288585131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9807b2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.356+0000 7efc9d600700 1 -- 172.21.15.60:0/3288585131 >> 172.21.15.60:0/3288585131 conn(0x7efc98102cc0 msgr2=0x7efc981050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.356+0000 7efc9d600700 1 -- 172.21.15.60:0/3288585131 shutdown_connections 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.356+0000 7efc9d600700 1 -- 172.21.15.60:0/3288585131 wait complete. 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.357+0000 7efc9d600700 1 Processor -- start 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.357+0000 7efc9d600700 1 -- start start 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.358+0000 7efc9d600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.358+0000 7efc9d600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9810dc00 con 0x7efc9807ce10 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.358+0000 7efc96a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.358+0000 7efc96a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49392/0 (socket says 172.21.15.60:49392) 2024-02-26T10:34:22.363 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.358+0000 7efc96a00700 1 -- 172.21.15.60:0/350881668 learned_addr learned my addr 172.21.15.60:0/350881668 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.359+0000 7efc96a00700 1 -- 172.21.15.60:0/350881668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9819dbe0 con 0x7efc9807ce10 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.359+0000 7efc96a00700 1 --2- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9819d6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc880117e0 tx=0x7efc88011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.359+0000 7efc8f400700 1 -- 172.21.15.60:0/350881668 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc88003ba0 con 0x7efc9807ce10 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.359+0000 7efc8f400700 1 -- 172.21.15.60:0/350881668 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc8801a910 con 0x7efc9807ce10 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.359+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9819de70 con 0x7efc9807ce10 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.359+0000 7efc8f400700 1 -- 172.21.15.60:0/350881668 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc880248f0 con 0x7efc9807ce10 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.359+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9819e2d0 con 0x7efc9807ce10 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.361+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc98068160 con 0x7efc9807ce10 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.361+0000 7efc8f400700 1 -- 172.21.15.60:0/350881668 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7efc88022020 con 0x7efc9807ce10 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.361+0000 7efc8f400700 1 --2- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7efc7c038ba0 0x7efc7c03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.361+0000 7efc8f400700 1 -- 172.21.15.60:0/350881668 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efc8804d400 con 0x7efc9807ce10 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.362+0000 7efc96000700 1 --2- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7efc7c038ba0 0x7efc7c03b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.363+0000 7efc96000700 1 --2- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7efc7c038ba0 0x7efc7c03b060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efc84006fd0 tx=0x7efc84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:22.364 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.368+0000 7efc8f400700 1 -- 172.21.15.60:0/350881668 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc8801ab30 con 0x7efc9807ce10 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.606+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7efc98103e50 con 0x7efc7c038ba0 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.616+0000 7efc8f400700 1 -- 172.21.15.60:0/350881668 <== mgr.14120 v2:172.21.15.60:6800/603652410 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efc98103e50 con 0x7efc7c038ba0 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.619+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7efc7c038ba0 msgr2=0x7efc7c03b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.619+0000 7efc9d600700 1 --2- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7efc7c038ba0 0x7efc7c03b060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efc84006fd0 tx=0x7efc84006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.619+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 msgr2=0x7efc9819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.619+0000 7efc9d600700 1 --2- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9819d6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc880117e0 tx=0x7efc88011ba0 comp rx=0 tx=0).stop 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.619+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 shutdown_connections 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.619+0000 7efc9d600700 1 --2- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7efc7c038ba0 0x7efc7c03b060 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.619+0000 7efc9d600700 1 --2- 172.21.15.60:0/350881668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9807ce10 0x7efc9819d6a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.619+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 >> 172.21.15.60:0/350881668 conn(0x7efc98102cc0 msgr2=0x7efc98103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.619+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 shutdown_connections 2024-02-26T10:34:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:21.620+0000 7efc9d600700 1 -- 172.21.15.60:0/350881668 wait complete. 2024-02-26T10:34:23.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:23 smithi060 ceph-mon[31539]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:34:24.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.914+0000 7fa629e00700 1 Processor -- start 2024-02-26T10:34:24.322 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.914+0000 7fa629e00700 1 -- start start 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.914+0000 7fa629e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa624107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.914+0000 7fa629e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624108070 con 0x7fa624107680 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.915+0000 7fa622a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa624107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.915+0000 7fa622a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa624107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49408/0 (socket says 172.21.15.60:49408) 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.915+0000 7fa622a00700 1 -- 172.21.15.60:0/3155921880 learned_addr learned my addr 172.21.15.60:0/3155921880 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.915+0000 7fa622a00700 1 -- 172.21.15.60:0/3155921880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6241087e0 con 0x7fa624107680 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.916+0000 7fa622a00700 1 --2- 172.21.15.60:0/3155921880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa624107aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098a0 tx=0x7fa614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d2573945716646 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.916+0000 7fa622000700 1 -- 172.21.15.60:0/3155921880 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa614004160 con 0x7fa624107680 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.916+0000 7fa622000700 1 -- 172.21.15.60:0/3155921880 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa614004300 con 0x7fa624107680 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.916+0000 7fa622000700 1 -- 172.21.15.60:0/3155921880 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa614011b20 con 0x7fa624107680 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.917+0000 7fa629e00700 1 -- 172.21.15.60:0/3155921880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 msgr2=0x7fa624107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.917+0000 7fa629e00700 1 --2- 172.21.15.60:0/3155921880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa624107aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098a0 tx=0x7fa614009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:24.323 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.917+0000 7fa629e00700 1 -- 172.21.15.60:0/3155921880 shutdown_connections 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.917+0000 7fa629e00700 1 --2- 172.21.15.60:0/3155921880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa624107aa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.917+0000 7fa629e00700 1 -- 172.21.15.60:0/3155921880 >> 172.21.15.60:0/3155921880 conn(0x7fa624102cc0 msgr2=0x7fa6241050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.917+0000 7fa629e00700 1 -- 172.21.15.60:0/3155921880 shutdown_connections 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.917+0000 7fa629e00700 1 -- 172.21.15.60:0/3155921880 wait complete. 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.918+0000 7fa629e00700 1 Processor -- start 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.919+0000 7fa629e00700 1 -- start start 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.919+0000 7fa629e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa62419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.919+0000 7fa629e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62410bb90 con 0x7fa624107680 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.919+0000 7fa622a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa62419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.919+0000 7fa622a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa62419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49416/0 (socket says 172.21.15.60:49416) 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.919+0000 7fa622a00700 1 -- 172.21.15.60:0/361952037 learned_addr learned my addr 172.21.15.60:0/361952037 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.920+0000 7fa622a00700 1 -- 172.21.15.60:0/361952037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62419dce0 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.920+0000 7fa622a00700 1 --2- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa62419d7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa6140117e0 tx=0x7fa614011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.921+0000 7fa628a00700 1 -- 172.21.15.60:0/361952037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa614003bf0 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.921+0000 7fa628a00700 1 -- 172.21.15.60:0/361952037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa61401a910 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.921+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62419df70 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.921+0000 7fa628a00700 1 -- 172.21.15.60:0/361952037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa614024970 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.921+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62419e3d0 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.922+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa624068160 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.922+0000 7fa628a00700 1 -- 172.21.15.60:0/361952037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7fa614022020 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.923+0000 7fa628a00700 1 --2- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fa608038ba0 0x7fa60803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.923+0000 7fa628a00700 1 -- 172.21.15.60:0/361952037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa61404d280 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.923+0000 7fa61a000700 1 --2- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fa608038ba0 0x7fa60803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.924+0000 7fa61a000700 1 --2- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fa608038ba0 0x7fa60803b060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa60c006fd0 tx=0x7fa60c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:22.932+0000 7fa628a00700 1 -- 172.21.15.60:0/361952037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa614017ca0 con 0x7fa624107680 2024-02-26T10:34:24.325 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.165+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fa6241a0c60 con 0x7fa624107680 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.168+0000 7fa628a00700 1 -- 172.21.15.60:0/361952037 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fa61404cbd0 con 0x7fa624107680 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fa608038ba0 msgr2=0x7fa60803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 --2- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fa608038ba0 0x7fa60803b060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa60c006fd0 tx=0x7fa60c006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 msgr2=0x7fa62419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 --2- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa62419d7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa6140117e0 tx=0x7fa614011ba0 comp rx=0 tx=0).stop 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 shutdown_connections 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 --2- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7fa608038ba0 0x7fa60803b060 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 --2- 172.21.15.60:0/361952037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624107680 0x7fa62419d7a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 >> 172.21.15.60:0/361952037 conn(0x7fa624102cc0 msgr2=0x7fa624103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 shutdown_connections 2024-02-26T10:34:24.326 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:23.172+0000 7fa629e00700 1 -- 172.21.15.60:0/361952037 wait complete. 2024-02-26T10:34:24.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:24 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/361952037' entity='client.admin' 2024-02-26T10:34:25.856 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:25 smithi060 ceph-mon[31539]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:34:25.856 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:25 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:25.856 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:25 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/36749859' entity='client.admin' 2024-02-26T10:34:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.966+0000 7f479ba00700 1 Processor -- start 2024-02-26T10:34:25.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.966+0000 7f479ba00700 1 -- start start 2024-02-26T10:34:25.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.966+0000 7f479ba00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f4794107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:25.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.966+0000 7f479ba00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794108070 con 0x7f4794107680 2024-02-26T10:34:25.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.967+0000 7f4799000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f4794107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:25.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.967+0000 7f4799000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f4794107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45624/0 (socket says 172.21.15.60:45624) 2024-02-26T10:34:25.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.967+0000 7f4799000700 1 -- 172.21.15.60:0/2723977288 learned_addr learned my addr 172.21.15.60:0/2723977288 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:25.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.967+0000 7f4799000700 1 -- 172.21.15.60:0/2723977288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47941087e0 con 0x7f4794107680 2024-02-26T10:34:25.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.968+0000 7f4799000700 1 --2- 172.21.15.60:0/2723977288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f4794107aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4788009e80 tx=0x7f478800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5da0ba2651ae40 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:25.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.968+0000 7f4793400700 1 -- 172.21.15.60:0/2723977288 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4788017070 con 0x7f4794107680 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.968+0000 7f4793400700 1 -- 172.21.15.60:0/2723977288 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f478800b8c0 con 0x7f4794107680 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.968+0000 7f4793400700 1 -- 172.21.15.60:0/2723977288 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47880045c0 con 0x7f4794107680 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.969+0000 7f479ba00700 1 -- 172.21.15.60:0/2723977288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 msgr2=0x7f4794107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.969+0000 7f479ba00700 1 --2- 172.21.15.60:0/2723977288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f4794107aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4788009e80 tx=0x7f478800b330 comp rx=0 tx=0).stop 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.969+0000 7f479ba00700 1 -- 172.21.15.60:0/2723977288 shutdown_connections 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.969+0000 7f479ba00700 1 --2- 172.21.15.60:0/2723977288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f4794107aa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.969+0000 7f479ba00700 1 -- 172.21.15.60:0/2723977288 >> 172.21.15.60:0/2723977288 conn(0x7f4794102cc0 msgr2=0x7f47941050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.969+0000 7f479ba00700 1 -- 172.21.15.60:0/2723977288 shutdown_connections 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.969+0000 7f479ba00700 1 -- 172.21.15.60:0/2723977288 wait complete. 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.971+0000 7f479ba00700 1 Processor -- start 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.971+0000 7f479ba00700 1 -- start start 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.971+0000 7f479ba00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f479419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.971+0000 7f479ba00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479410bb90 con 0x7f4794107680 2024-02-26T10:34:25.874 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.971+0000 7f4799000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f479419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.971+0000 7f4799000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f479419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45632/0 (socket says 172.21.15.60:45632) 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.971+0000 7f4799000700 1 -- 172.21.15.60:0/36749859 learned_addr learned my addr 172.21.15.60:0/36749859 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.972+0000 7f4799000700 1 -- 172.21.15.60:0/36749859 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479419dd10 con 0x7f4794107680 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.972+0000 7f4799000700 1 --2- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f479419d7d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4788009fd0 tx=0x7f4788003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.973+0000 7f4791600700 1 -- 172.21.15.60:0/36749859 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4788020020 con 0x7f4794107680 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.973+0000 7f4791600700 1 -- 172.21.15.60:0/36749859 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f478800bba0 con 0x7f4794107680 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.973+0000 7f4791600700 1 -- 172.21.15.60:0/36749859 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4788017040 con 0x7f4794107680 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.973+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479419dfa0 con 0x7f4794107680 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.973+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47941a0c90 con 0x7f4794107680 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.974+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4794068160 con 0x7f4794107680 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.974+0000 7f4791600700 1 -- 172.21.15.60:0/36749859 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f4788023020 con 0x7f4794107680 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.975+0000 7f4791600700 1 --2- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f477c038bf0 0x7f477c03b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:25.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.975+0000 7f4791600700 1 -- 172.21.15.60:0/36749859 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f478800f050 con 0x7f4794107680 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.976+0000 7f4793e00700 1 --2- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f477c038bf0 0x7f477c03b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.976+0000 7f4793e00700 1 --2- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f477c038bf0 0x7f477c03b0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4780006fd0 tx=0x7f4780006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:24.981+0000 7f4791600700 1 -- 172.21.15.60:0/36749859 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f478802a020 con 0x7f4794107680 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.209+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f47941a12d0 con 0x7f4794107680 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.213+0000 7f4791600700 1 -- 172.21.15.60:0/36749859 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f478802d350 con 0x7f4794107680 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.216+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f477c038bf0 msgr2=0x7f477c03b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.216+0000 7f479ba00700 1 --2- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f477c038bf0 0x7f477c03b0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4780006fd0 tx=0x7f4780006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.216+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 msgr2=0x7f479419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.216+0000 7f479ba00700 1 --2- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f479419d7d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4788009fd0 tx=0x7f4788003d00 comp rx=0 tx=0).stop 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.217+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 shutdown_connections 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.217+0000 7f479ba00700 1 --2- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f477c038bf0 0x7f477c03b0b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.217+0000 7f479ba00700 1 --2- 172.21.15.60:0/36749859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4794107680 0x7f479419d7d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:25.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.217+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 >> 172.21.15.60:0/36749859 conn(0x7f4794102cc0 msgr2=0x7f4794103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:25.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.217+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 shutdown_connections 2024-02-26T10:34:25.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:25.217+0000 7f479ba00700 1 -- 172.21.15.60:0/36749859 wait complete. 2024-02-26T10:34:25.877 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling the dashboard module... 2024-02-26T10:34:27.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:27 smithi060 ceph-mon[31539]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:34:27.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:27 smithi060 ceph-mon[31539]: from='mgr.14120 172.21.15.60:0/1263307206' entity='mgr.x' 2024-02-26T10:34:27.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:27 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4128701023' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-26T10:34:27.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:27 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: ignoring --setuser ceph since I am not root 2024-02-26T10:34:27.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:27 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: ignoring --setgroup ceph since I am not root 2024-02-26T10:34:28.157 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.459+0000 7ff546400700 1 Processor -- start 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.459+0000 7ff546400700 1 -- start start 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.459+0000 7ff546400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff54010a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.459+0000 7ff546400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54007add0 con 0x7ff540108170 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.460+0000 7ff53f400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff54010a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.460+0000 7ff53f400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff54010a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45644/0 (socket says 172.21.15.60:45644) 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.460+0000 7ff53f400700 1 -- 172.21.15.60:0/4142071559 learned_addr learned my addr 172.21.15.60:0/4142071559 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.460+0000 7ff53f400700 1 -- 172.21.15.60:0/4142071559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54010aaa0 con 0x7ff540108170 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.461+0000 7ff53f400700 1 --2- 172.21.15.60:0/4142071559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff54010a560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff5300098a0 tx=0x7ff530009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3310e9126bc632 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.461+0000 7ff53e000700 1 -- 172.21.15.60:0/4142071559 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff530004160 con 0x7ff540108170 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.461+0000 7ff53e000700 1 -- 172.21.15.60:0/4142071559 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff530004300 con 0x7ff540108170 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.461+0000 7ff53e000700 1 -- 172.21.15.60:0/4142071559 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff530011b20 con 0x7ff540108170 2024-02-26T10:34:28.163 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.462+0000 7ff546400700 1 -- 172.21.15.60:0/4142071559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 msgr2=0x7ff54010a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:28.164 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.462+0000 7ff546400700 1 --2- 172.21.15.60:0/4142071559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff54010a560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff5300098a0 tx=0x7ff530009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:28.164 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.462+0000 7ff546400700 1 -- 172.21.15.60:0/4142071559 shutdown_connections 2024-02-26T10:34:28.164 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.462+0000 7ff546400700 1 --2- 172.21.15.60:0/4142071559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff54010a560 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:28.164 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.462+0000 7ff546400700 1 -- 172.21.15.60:0/4142071559 >> 172.21.15.60:0/4142071559 conn(0x7ff540101ac0 msgr2=0x7ff540103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:28.164 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.462+0000 7ff546400700 1 -- 172.21.15.60:0/4142071559 shutdown_connections 2024-02-26T10:34:28.164 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.462+0000 7ff546400700 1 -- 172.21.15.60:0/4142071559 wait complete. 2024-02-26T10:34:28.164 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.463+0000 7ff546400700 1 Processor -- start 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.464+0000 7ff546400700 1 -- start start 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.464+0000 7ff546400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff5401079f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.464+0000 7ff546400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54010ba40 con 0x7ff540108170 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.464+0000 7ff53f400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff5401079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.464+0000 7ff53f400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff5401079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45650/0 (socket says 172.21.15.60:45650) 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.464+0000 7ff53f400700 1 -- 172.21.15.60:0/4128701023 learned_addr learned my addr 172.21.15.60:0/4128701023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.465+0000 7ff53f400700 1 -- 172.21.15.60:0/4128701023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540107f30 con 0x7ff540108170 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.465+0000 7ff53f400700 1 --2- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff5401079f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff530011b60 tx=0x7ff530011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.465+0000 7ff537e00700 1 -- 172.21.15.60:0/4128701023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff530003b60 con 0x7ff540108170 2024-02-26T10:34:28.165 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.465+0000 7ff537e00700 1 -- 172.21.15.60:0/4128701023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff53001a910 con 0x7ff540108170 2024-02-26T10:34:28.166 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.465+0000 7ff537e00700 1 -- 172.21.15.60:0/4128701023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff530024930 con 0x7ff540108170 2024-02-26T10:34:28.166 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.465+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff540106120 con 0x7ff540108170 2024-02-26T10:34:28.166 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.466+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5401065e0 con 0x7ff540108170 2024-02-26T10:34:28.166 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.467+0000 7ff537e00700 1 -- 172.21.15.60:0/4128701023 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7ff530022020 con 0x7ff540108170 2024-02-26T10:34:28.166 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.467+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff540068160 con 0x7ff540108170 2024-02-26T10:34:28.166 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.467+0000 7ff537e00700 1 --2- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7ff524038ba0 0x7ff52403b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:28.166 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.467+0000 7ff537e00700 1 -- 172.21.15.60:0/4128701023 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff53004cc10 con 0x7ff540108170 2024-02-26T10:34:28.166 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.467+0000 7ff53ea00700 1 --2- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7ff524038ba0 0x7ff52403b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:28.166 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.468+0000 7ff53ea00700 1 --2- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7ff524038ba0 0x7ff52403b060 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff52c006fd0 tx=0x7ff52c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.473+0000 7ff537e00700 1 -- 172.21.15.60:0/4128701023 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff53000b3e0 con 0x7ff540108170 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:26.769+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7ff540106920 con 0x7ff540108170 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.486+0000 7ff537e00700 1 -- 172.21.15.60:0/4128701023 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7ff540106920 con 0x7ff540108170 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.489+0000 7ff537e00700 1 -- 172.21.15.60:0/4128701023 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mgrmap(e 9) v1 ==== 45941+0+0 (secure 0 0 0) 0x7ff53001a430 con 0x7ff540108170 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.490+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7ff524038ba0 msgr2=0x7ff52403b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.490+0000 7ff546400700 1 --2- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7ff524038ba0 0x7ff52403b060 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff52c006fd0 tx=0x7ff52c006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.491+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 msgr2=0x7ff5401079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.491+0000 7ff546400700 1 --2- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff5401079f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff530011b60 tx=0x7ff530011c40 comp rx=0 tx=0).stop 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.491+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 shutdown_connections 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.491+0000 7ff546400700 1 --2- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7ff524038ba0 0x7ff52403b060 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.491+0000 7ff546400700 1 --2- 172.21.15.60:0/4128701023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff540108170 0x7ff5401079f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.491+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 >> 172.21.15.60:0/4128701023 conn(0x7ff540101ac0 msgr2=0x7ff5401025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.491+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 shutdown_connections 2024-02-26T10:34:28.167 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:27.491+0000 7ff546400700 1 -- 172.21.15.60:0/4128701023 wait complete. 2024-02-26T10:34:28.499 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:28 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:28.388+0000 7f65c78be200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T10:34:28.755 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:28 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:28.520+0000 7f65c78be200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T10:34:28.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:28 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4128701023' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-26T10:34:28.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:28 smithi060 ceph-mon[31539]: mgrmap e9: x(active, since 25s) 2024-02-26T10:34:29.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:29.370+0000 7f65c78be200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T10:34:29.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:29.500+0000 7f65c78be200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T10:34:30.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:29 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4255819466' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-26T10:34:30.124 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:29.686+0000 7f65c78be200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T10:34:30.124 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:29.818+0000 7f65c78be200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T10:34:30.124 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:29.932+0000 7f65c78be200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T10:34:30.431 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:30.280+0000 7f65c78be200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T10:34:30.539 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T10:34:30.539 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-26T10:34:30.539 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.478+0000 7f92d0a00700 1 Processor -- start 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.479+0000 7f92d0a00700 1 -- start start 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.479+0000 7f92d0a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc106340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.479+0000 7f92d0a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc07ae10 con 0x7f92cc105f20 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.480+0000 7f92c9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc106340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.480+0000 7f92c9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc106340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45678/0 (socket says 172.21.15.60:45678) 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.480+0000 7f92c9e00700 1 -- 172.21.15.60:0/398999562 learned_addr learned my addr 172.21.15.60:0/398999562 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.480+0000 7f92c9e00700 1 -- 172.21.15.60:0/398999562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc106880 con 0x7f92cc105f20 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.480+0000 7f92c9e00700 1 --2- 172.21.15.60:0/398999562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc106340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0098a0 tx=0x7f92bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae88a0da5dd1d3e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.481+0000 7f92c8a00700 1 -- 172.21.15.60:0/398999562 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92bc004160 con 0x7f92cc105f20 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.481+0000 7f92c8a00700 1 -- 172.21.15.60:0/398999562 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92bc004300 con 0x7f92cc105f20 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.481+0000 7f92c8a00700 1 -- 172.21.15.60:0/398999562 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92bc011b20 con 0x7f92cc105f20 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.481+0000 7f92d0a00700 1 -- 172.21.15.60:0/398999562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 msgr2=0x7f92cc106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.481+0000 7f92d0a00700 1 --2- 172.21.15.60:0/398999562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc106340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0098a0 tx=0x7f92bc009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.482+0000 7f92d0a00700 1 -- 172.21.15.60:0/398999562 shutdown_connections 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.482+0000 7f92d0a00700 1 --2- 172.21.15.60:0/398999562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc106340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.482+0000 7f92d0a00700 1 -- 172.21.15.60:0/398999562 >> 172.21.15.60:0/398999562 conn(0x7f92cc101ac0 msgr2=0x7f92cc103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.482+0000 7f92d0a00700 1 -- 172.21.15.60:0/398999562 shutdown_connections 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.482+0000 7f92d0a00700 1 -- 172.21.15.60:0/398999562 wait complete. 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.483+0000 7f92d0a00700 1 Processor -- start 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.483+0000 7f92d0a00700 1 -- start start 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.484+0000 7f92d0a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:30.541 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.484+0000 7f92d0a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc107950 con 0x7f92cc105f20 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.484+0000 7f92c9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.484+0000 7f92c9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc19d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45680/0 (socket says 172.21.15.60:45680) 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.484+0000 7f92c9e00700 1 -- 172.21.15.60:0/4255819466 learned_addr learned my addr 172.21.15.60:0/4255819466 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.484+0000 7f92c9e00700 1 -- 172.21.15.60:0/4255819466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc19dd20 con 0x7f92cc105f20 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.485+0000 7f92c9e00700 1 --2- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc19d7e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0116a0 tx=0x7f92bc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.485+0000 7f92c2a00700 1 -- 172.21.15.60:0/4255819466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92bc003a40 con 0x7f92cc105f20 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.485+0000 7f92c2a00700 1 -- 172.21.15.60:0/4255819466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92bc003be0 con 0x7f92cc105f20 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.485+0000 7f92c2a00700 1 -- 172.21.15.60:0/4255819466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92bc01a750 con 0x7f92cc105f20 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.485+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc19dfb0 con 0x7f92cc105f20 2024-02-26T10:34:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.485+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc19e410 con 0x7f92cc105f20 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.486+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc068160 con 0x7f92cc105f20 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.487+0000 7f92c2a00700 1 -- 172.21.15.60:0/4255819466 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) v1 ==== 45941+0+0 (secure 0 0 0) 0x7f92bc024aa0 con 0x7f92cc105f20 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.487+0000 7f92c2a00700 1 --2- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f92b0038c00 0x7f92b003b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.487+0000 7f92c2a00700 1 -- 172.21.15.60:0/4255819466 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f92bc04d0a0 con 0x7f92cc105f20 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.487+0000 7f92c9400700 1 -- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f92b0038c00 msgr2=0x7f92b003b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/603652410 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.487+0000 7f92c9400700 1 --2- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f92b0038c00 0x7f92b003b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.492+0000 7f92c2a00700 1 -- 172.21.15.60:0/4255819466 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92bc011e20 con 0x7f92cc105f20 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.688+0000 7f92c9400700 1 -- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f92b0038c00 msgr2=0x7f92b003b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/603652410 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.688+0000 7f92c9400700 1 --2- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f92b0038c00 0x7f92b003b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.768+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f92cc062960 con 0x7f92cc105f20 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.769+0000 7f92c2a00700 1 -- 172.21.15.60:0/4255819466 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f92bc04c9f0 con 0x7f92cc105f20 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.771+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f92b0038c00 msgr2=0x7f92b003b0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T10:34:30.543 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.771+0000 7f92d0a00700 1 --2- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f92b0038c00 0x7f92b003b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:30.544 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.772+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 msgr2=0x7f92cc19d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:30.544 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.772+0000 7f92d0a00700 1 --2- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc19d7e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0116a0 tx=0x7f92bc011ba0 comp rx=0 tx=0).stop 2024-02-26T10:34:30.544 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.772+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 shutdown_connections 2024-02-26T10:34:30.544 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.772+0000 7f92d0a00700 1 --2- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f92b0038c00 0x7f92b003b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:30.544 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.772+0000 7f92d0a00700 1 --2- 172.21.15.60:0/4255819466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92cc105f20 0x7f92cc19d7e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:30.544 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.772+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 >> 172.21.15.60:0/4255819466 conn(0x7f92cc101ac0 msgr2=0x7f92cc194480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:30.544 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.772+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 shutdown_connections 2024-02-26T10:34:30.544 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:29.772+0000 7f92d0a00700 1 -- 172.21.15.60:0/4255819466 wait complete. 2024-02-26T10:34:30.545 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-02-26T10:34:30.545 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 9... 2024-02-26T10:34:30.687 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:30.427+0000 7f65c78be200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T10:34:30.687 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:30.549+0000 7f65c78be200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T10:34:30.687 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:30.682+0000 7f65c78be200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T10:34:31.038 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:30.916+0000 7f65c78be200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T10:34:32.610 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:32.250+0000 7f65c78be200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T10:34:32.610 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:32.428+0000 7f65c78be200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T10:34:33.123 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:32.856+0000 7f65c78be200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T10:34:33.123 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:32.979+0000 7f65c78be200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T10:34:33.503 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:33.246+0000 7f65c78be200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:34:33.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:33.498+0000 7f65c78be200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T10:34:34.211 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:33.886+0000 7f65c78be200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:34:34.529 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:34 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:34.212+0000 7f65c78be200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T10:34:34.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:34 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:34.524+0000 7f65c78be200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T10:34:35.873 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:35 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:35.586+0000 7f65c78be200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T10:34:36.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:36.305+0000 7f65c78be200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T10:34:36.623 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:36.439+0000 7f65c78be200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T10:34:37.123 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:34:36.730+0000 7f65c78be200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T10:34:37.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: Active manager daemon x restarted 2024-02-26T10:34:37.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: Activating manager daemon x 2024-02-26T10:34:37.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: osdmap e3: 0 total, 0 up, 0 in 2024-02-26T10:34:37.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: mgrmap e10: x(active, starting, since 0.00953362s) 2024-02-26T10:34:37.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:34:37.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T10:34:37.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T10:34:37.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T10:34:37.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T10:34:37.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:36 smithi060 ceph-mon[31539]: Manager daemon x is now available 2024-02-26T10:34:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T10:34:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T10:34:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:37 smithi060 ceph-mon[31539]: [26/Feb/2024:10:34:37] ENGINE Bus STARTING 2024-02-26T10:34:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:37 smithi060 ceph-mon[31539]: [26/Feb/2024:10:34:37] ENGINE Serving on http://172.21.15.60:8765 2024-02-26T10:34:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:37 smithi060 ceph-mon[31539]: [26/Feb/2024:10:34:37] ENGINE Serving on https://172.21.15.60:7150 2024-02-26T10:34:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:37 smithi060 ceph-mon[31539]: [26/Feb/2024:10:34:37] ENGINE Bus STARTED 2024-02-26T10:34:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:37 smithi060 ceph-mon[31539]: mgrmap e11: x(active, since 1.01198s) 2024-02-26T10:34:38.499 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T10:34:38.499 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-26T10:34:38.499 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-26T10:34:38.500 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T10:34:38.500 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.193+0000 7f5d28600700 1 Processor -- start 2024-02-26T10:34:38.500 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.193+0000 7f5d28600700 1 -- start start 2024-02-26T10:34:38.500 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.194+0000 7f5d28600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2010a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:38.500 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.194+0000 7f5d28600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2007ae70 con 0x7f5d201080d0 2024-02-26T10:34:38.500 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.194+0000 7f5d25c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2010a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:38.500 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.194+0000 7f5d25c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2010a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45688/0 (socket says 172.21.15.60:45688) 2024-02-26T10:34:38.500 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.194+0000 7f5d25c00700 1 -- 172.21.15.60:0/1641334231 learned_addr learned my addr 172.21.15.60:0/1641334231 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:38.500 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.195+0000 7f5d25c00700 1 -- 172.21.15.60:0/1641334231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2010aa00 con 0x7f5d201080d0 2024-02-26T10:34:38.501 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.195+0000 7f5d25c00700 1 --2- 172.21.15.60:0/1641334231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2010a4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d140098a0 tx=0x7f5d14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f070d7333ec15828 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:38.501 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.195+0000 7f5d1fe00700 1 -- 172.21.15.60:0/1641334231 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d14004160 con 0x7f5d201080d0 2024-02-26T10:34:38.501 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.195+0000 7f5d1fe00700 1 -- 172.21.15.60:0/1641334231 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d14004300 con 0x7f5d201080d0 2024-02-26T10:34:38.501 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.195+0000 7f5d1fe00700 1 -- 172.21.15.60:0/1641334231 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d14011b20 con 0x7f5d201080d0 2024-02-26T10:34:38.501 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.196+0000 7f5d28600700 1 -- 172.21.15.60:0/1641334231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 msgr2=0x7f5d2010a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:38.501 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.196+0000 7f5d28600700 1 --2- 172.21.15.60:0/1641334231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2010a4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d140098a0 tx=0x7f5d14009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:38.501 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.196+0000 7f5d28600700 1 -- 172.21.15.60:0/1641334231 shutdown_connections 2024-02-26T10:34:38.501 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.196+0000 7f5d28600700 1 --2- 172.21.15.60:0/1641334231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2010a4c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:38.501 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.196+0000 7f5d28600700 1 -- 172.21.15.60:0/1641334231 >> 172.21.15.60:0/1641334231 conn(0x7f5d20101ac0 msgr2=0x7f5d20103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.196+0000 7f5d28600700 1 -- 172.21.15.60:0/1641334231 shutdown_connections 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.196+0000 7f5d28600700 1 -- 172.21.15.60:0/1641334231 wait complete. 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.197+0000 7f5d28600700 1 Processor -- start 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.198+0000 7f5d28600700 1 -- start start 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.198+0000 7f5d28600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.198+0000 7f5d28600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2010ba60 con 0x7f5d201080d0 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.198+0000 7f5d25c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.198+0000 7f5d25c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2019d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45692/0 (socket says 172.21.15.60:45692) 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.198+0000 7f5d25c00700 1 -- 172.21.15.60:0/4103540951 learned_addr learned my addr 172.21.15.60:0/4103540951 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.198+0000 7f5d25c00700 1 -- 172.21.15.60:0/4103540951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2019dd50 con 0x7f5d201080d0 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.199+0000 7f5d25c00700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2019d810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d14011b60 tx=0x7f5d14011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.199+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d14003b60 con 0x7f5d201080d0 2024-02-26T10:34:38.502 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.199+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d1401a910 con 0x7f5d201080d0 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.199+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d14024930 con 0x7f5d201080d0 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.199+0000 7f5d28600700 1 -- 172.21.15.60:0/4103540951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2019dfe0 con 0x7f5d201080d0 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.199+0000 7f5d28600700 1 -- 172.21.15.60:0/4103540951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2019e440 con 0x7f5d201080d0 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.201+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) v1 ==== 45941+0+0 (secure 0 0 0) 0x7f5d14022020 con 0x7f5d201080d0 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.201+0000 7f5d1e000700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 0x7f5d0803b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.201+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 --> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d0803b820 con 0x7f5d08038c50 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.201+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5d1404cd10 con 0x7f5d201080d0 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.202+0000 7f5d25200700 1 -- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 msgr2=0x7f5d0803b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/603652410 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.202+0000 7f5d25200700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 0x7f5d0803b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.403+0000 7f5d25200700 1 -- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 msgr2=0x7f5d0803b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/603652410 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.403+0000 7f5d25200700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 0x7f5d0803b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.804+0000 7f5d25200700 1 -- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 msgr2=0x7f5d0803b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/603652410 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:31.804+0000 7f5d25200700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 0x7f5d0803b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T10:34:38.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:32.605+0000 7f5d25200700 1 -- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 msgr2=0x7f5d0803b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/603652410 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:32.605+0000 7f5d25200700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 0x7f5d0803b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:34.207+0000 7f5d25200700 1 -- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 msgr2=0x7f5d0803b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/603652410 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:34.207+0000 7f5d25200700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 0x7f5d0803b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:36.738+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mgrmap(e 10) v1 ==== 45708+0+0 (secure 0 0 0) 0x7f5d14024ad0 con 0x7f5d201080d0 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:36.739+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 msgr2=0x7f5d0803b110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:36.739+0000 7f5d1e000700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 0x7f5d0803b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.742+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 11) v1 ==== 45835+0+0 (secure 0 0 0) 0x7f5d1404e4e0 con 0x7f5d201080d0 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.742+0000 7f5d1e000700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5d0803c740 0x7f5d0803eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.742+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d14012ea0 con 0x7f5d0803c740 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.743+0000 7f5d25200700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5d0803c740 0x7f5d0803eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.744+0000 7f5d25200700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5d0803c740 0x7f5d0803eb30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c003a10 tx=0x7f5d0c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:38.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.746+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f5d14012ea0 con 0x7f5d0803c740 2024-02-26T10:34:38.505 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.757+0000 7f5d28600700 1 -- 172.21.15.60:0/4103540951 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f5d20068160 con 0x7f5d0803c740 2024-02-26T10:34:38.505 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.757+0000 7f5d1e000700 1 -- 172.21.15.60:0/4103540951 <== mgr.14152 v2:172.21.15.60:6800/3493132094 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f5d20068160 con 0x7f5d0803c740 2024-02-26T10:34:38.505 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.758+0000 7f5d28600700 1 -- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5d0803c740 msgr2=0x7f5d0803eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:38.505 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.758+0000 7f5d28600700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5d0803c740 0x7f5d0803eb30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c003a10 tx=0x7f5d0c009270 comp rx=0 tx=0).stop 2024-02-26T10:34:38.505 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.758+0000 7f5d28600700 1 -- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 msgr2=0x7f5d2019d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:38.505 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.758+0000 7f5d28600700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2019d810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d14011b60 tx=0x7f5d14011c40 comp rx=0 tx=0).stop 2024-02-26T10:34:38.506 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.758+0000 7f5d28600700 1 -- 172.21.15.60:0/4103540951 shutdown_connections 2024-02-26T10:34:38.506 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.758+0000 7f5d28600700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5d0803c740 0x7f5d0803eb30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:38.506 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.758+0000 7f5d28600700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d201080d0 0x7f5d2019d810 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:38.506 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.758+0000 7f5d28600700 1 --2- 172.21.15.60:0/4103540951 >> [v2:172.21.15.60:6800/603652410,v1:172.21.15.60:6801/603652410] conn(0x7f5d08038c50 0x7f5d0803b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:38.506 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.758+0000 7f5d28600700 1 -- 172.21.15.60:0/4103540951 >> 172.21.15.60:0/4103540951 conn(0x7f5d20101ac0 msgr2=0x7f5d20102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:38.506 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.759+0000 7f5d28600700 1 -- 172.21.15.60:0/4103540951 shutdown_connections 2024-02-26T10:34:38.506 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:37.759+0000 7f5d28600700 1 -- 172.21.15.60:0/4103540951 wait complete. 2024-02-26T10:34:38.506 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 9 is available 2024-02-26T10:34:38.506 INFO:teuthology.orchestra.run.smithi060.stdout:Generating a dashboard self-signed certificate... 2024-02-26T10:34:39.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:38 smithi060 ceph-mon[31539]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-26T10:34:39.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:38 smithi060 ceph-mon[31539]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-26T10:34:40.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:39 smithi060 ceph-mon[31539]: mgrmap e12: x(active, since 2s) 2024-02-26T10:34:40.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:39 smithi060 ceph-mon[31539]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:40.458 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-26T10:34:40.458 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.262+0000 7fc01da00700 1 Processor -- start 2024-02-26T10:34:40.458 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.263+0000 7fc01da00700 1 -- start start 2024-02-26T10:34:40.458 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.263+0000 7fc01da00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc018107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:40.458 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.263+0000 7fc01da00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018108050 con 0x7fc018107660 2024-02-26T10:34:40.458 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.264+0000 7fc016a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc018107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:40.458 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.264+0000 7fc016a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc018107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48842/0 (socket says 172.21.15.60:48842) 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.264+0000 7fc016a00700 1 -- 172.21.15.60:0/455275745 learned_addr learned my addr 172.21.15.60:0/455275745 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.264+0000 7fc016a00700 1 -- 172.21.15.60:0/455275745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0181087c0 con 0x7fc018107660 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.265+0000 7fc016a00700 1 --2- 172.21.15.60:0/455275745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc018107a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0080098a0 tx=0x7fc008009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc3aa14469a2328 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.265+0000 7fc015600700 1 -- 172.21.15.60:0/455275745 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc008004160 con 0x7fc018107660 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.265+0000 7fc015600700 1 -- 172.21.15.60:0/455275745 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc008004300 con 0x7fc018107660 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.265+0000 7fc015600700 1 -- 172.21.15.60:0/455275745 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc008011b20 con 0x7fc018107660 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.266+0000 7fc01da00700 1 -- 172.21.15.60:0/455275745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 msgr2=0x7fc018107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.266+0000 7fc01da00700 1 --2- 172.21.15.60:0/455275745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc018107a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0080098a0 tx=0x7fc008009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.266+0000 7fc01da00700 1 -- 172.21.15.60:0/455275745 shutdown_connections 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.266+0000 7fc01da00700 1 --2- 172.21.15.60:0/455275745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc018107a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.266+0000 7fc01da00700 1 -- 172.21.15.60:0/455275745 >> 172.21.15.60:0/455275745 conn(0x7fc018102c80 msgr2=0x7fc0181050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.266+0000 7fc01da00700 1 -- 172.21.15.60:0/455275745 shutdown_connections 2024-02-26T10:34:40.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.266+0000 7fc01da00700 1 -- 172.21.15.60:0/455275745 wait complete. 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.267+0000 7fc01da00700 1 Processor -- start 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.267+0000 7fc01da00700 1 -- start start 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.268+0000 7fc01da00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc01819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.268+0000 7fc01da00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01810bb70 con 0x7fc018107660 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.268+0000 7fc016a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc01819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.268+0000 7fc016a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc01819b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48850/0 (socket says 172.21.15.60:48850) 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.268+0000 7fc016a00700 1 -- 172.21.15.60:0/3075745408 learned_addr learned my addr 172.21.15.60:0/3075745408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.268+0000 7fc016a00700 1 -- 172.21.15.60:0/3075745408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01819ba70 con 0x7fc018107660 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.269+0000 7fc016a00700 1 --2- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc01819b530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc0080117e0 tx=0x7fc008011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.269+0000 7fc00f400700 1 -- 172.21.15.60:0/3075745408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc008003bf0 con 0x7fc018107660 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.269+0000 7fc00f400700 1 -- 172.21.15.60:0/3075745408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc00801a910 con 0x7fc018107660 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.269+0000 7fc00f400700 1 -- 172.21.15.60:0/3075745408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc008024970 con 0x7fc018107660 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.269+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01819bd00 con 0x7fc018107660 2024-02-26T10:34:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.269+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01819c160 con 0x7fc018107660 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.270+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc018068160 con 0x7fc018107660 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.271+0000 7fc00f400700 1 -- 172.21.15.60:0/3075745408 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) v1 ==== 45941+0+0 (secure 0 0 0) 0x7fc008022020 con 0x7fc018107660 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.271+0000 7fc00f400700 1 --2- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbffc038c50 0x7fbffc03b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.271+0000 7fc00f400700 1 -- 172.21.15.60:0/3075745408 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc00804d030 con 0x7fc018107660 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.271+0000 7fc016000700 1 --2- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbffc038c50 0x7fbffc03b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.272+0000 7fc016000700 1 --2- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbffc038c50 0x7fbffc03b110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc004006fd0 tx=0x7fc004006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.276+0000 7fc00f400700 1 -- 172.21.15.60:0/3075745408 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc00801ac30 con 0x7fc018107660 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.510+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc018103e00 con 0x7fbffc038c50 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.809+0000 7fc00f400700 1 -- 172.21.15.60:0/3075745408 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc018103e00 con 0x7fbffc038c50 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.813+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbffc038c50 msgr2=0x7fbffc03b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.813+0000 7fc01da00700 1 --2- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbffc038c50 0x7fbffc03b110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc004006fd0 tx=0x7fc004006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.813+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 msgr2=0x7fc01819b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.813+0000 7fc01da00700 1 --2- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc01819b530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc0080117e0 tx=0x7fc008011ba0 comp rx=0 tx=0).stop 2024-02-26T10:34:40.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.813+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 shutdown_connections 2024-02-26T10:34:40.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.813+0000 7fc01da00700 1 --2- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbffc038c50 0x7fbffc03b110 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:40.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.813+0000 7fc01da00700 1 --2- 172.21.15.60:0/3075745408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc018107660 0x7fc01819b530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:40.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.813+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 >> 172.21.15.60:0/3075745408 conn(0x7fc018102c80 msgr2=0x7fc0181036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:40.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.814+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 shutdown_connections 2024-02-26T10:34:40.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:39.814+0000 7fc01da00700 1 -- 172.21.15.60:0/3075745408 wait complete. 2024-02-26T10:34:40.462 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial admin user... 2024-02-26T10:34:41.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:41.001 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:41.001 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:42.173 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$6QpmpB9v1FlHelPeBl9bIeWxjRkRM9HVs5vSNAzONLNQhy2e70oSy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708943681, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-26T10:34:42.173 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.996+0000 7f3f97000700 1 Processor -- start 2024-02-26T10:34:42.173 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.996+0000 7f3f97000700 1 -- start start 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.996+0000 7f3f97000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f90106360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.996+0000 7f3f97000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9007ae30 con 0x7f3f90105f40 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.997+0000 7f3f8fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f90106360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.997+0000 7f3f8fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f90106360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48858/0 (socket says 172.21.15.60:48858) 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.997+0000 7f3f8fe00700 1 -- 172.21.15.60:0/3635086789 learned_addr learned my addr 172.21.15.60:0/3635086789 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.997+0000 7f3f8fe00700 1 -- 172.21.15.60:0/3635086789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f901068a0 con 0x7f3f90105f40 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.998+0000 7f3f8fe00700 1 --2- 172.21.15.60:0/3635086789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f90106360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f800098a0 tx=0x7f3f80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696f05e7ca1edfcf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.998+0000 7f3f8ea00700 1 -- 172.21.15.60:0/3635086789 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f80004160 con 0x7f3f90105f40 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.998+0000 7f3f8ea00700 1 -- 172.21.15.60:0/3635086789 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f80004300 con 0x7f3f90105f40 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.998+0000 7f3f8ea00700 1 -- 172.21.15.60:0/3635086789 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f80011b20 con 0x7f3f90105f40 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.999+0000 7f3f97000700 1 -- 172.21.15.60:0/3635086789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 msgr2=0x7f3f90106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.999+0000 7f3f97000700 1 --2- 172.21.15.60:0/3635086789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f90106360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f800098a0 tx=0x7f3f80009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:42.174 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.999+0000 7f3f97000700 1 -- 172.21.15.60:0/3635086789 shutdown_connections 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.999+0000 7f3f97000700 1 --2- 172.21.15.60:0/3635086789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f90106360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:40.999+0000 7f3f97000700 1 -- 172.21.15.60:0/3635086789 >> 172.21.15.60:0/3635086789 conn(0x7f3f90101b80 msgr2=0x7f3f90103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.000+0000 7f3f97000700 1 -- 172.21.15.60:0/3635086789 shutdown_connections 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.000+0000 7f3f97000700 1 -- 172.21.15.60:0/3635086789 wait complete. 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.001+0000 7f3f97000700 1 Processor -- start 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.001+0000 7f3f97000700 1 -- start start 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.001+0000 7f3f97000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f9019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.001+0000 7f3f97000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90107970 con 0x7f3f90105f40 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.002+0000 7f3f8fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f9019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.002+0000 7f3f8fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f9019d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48864/0 (socket says 172.21.15.60:48864) 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.002+0000 7f3f8fe00700 1 -- 172.21.15.60:0/782567080 learned_addr learned my addr 172.21.15.60:0/782567080 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.002+0000 7f3f8fe00700 1 -- 172.21.15.60:0/782567080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9019dd00 con 0x7f3f90105f40 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.002+0000 7f3f8fe00700 1 --2- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f9019d7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f80011b60 tx=0x7f3f80011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.003+0000 7f3f8cc00700 1 -- 172.21.15.60:0/782567080 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f80003b60 con 0x7f3f90105f40 2024-02-26T10:34:42.175 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.003+0000 7f3f8cc00700 1 -- 172.21.15.60:0/782567080 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f8001a910 con 0x7f3f90105f40 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.003+0000 7f3f8cc00700 1 -- 172.21.15.60:0/782567080 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f80024930 con 0x7f3f90105f40 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.003+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9019df90 con 0x7f3f90105f40 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.003+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9019e3f0 con 0x7f3f90105f40 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.004+0000 7f3f8cc00700 1 -- 172.21.15.60:0/782567080 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) v1 ==== 45941+0+0 (secure 0 0 0) 0x7f3f80022020 con 0x7f3f90105f40 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.004+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f90068160 con 0x7f3f90105f40 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.005+0000 7f3f8cc00700 1 --2- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3f74038c50 0x7f3f7403b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.005+0000 7f3f8cc00700 1 -- 172.21.15.60:0/782567080 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3f8004ce10 con 0x7f3f90105f40 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.005+0000 7f3f8f400700 1 --2- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3f74038c50 0x7f3f7403b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.006+0000 7f3f8f400700 1 --2- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3f74038c50 0x7f3f7403b110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.011+0000 7f3f8cc00700 1 -- 172.21.15.60:0/782567080 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f8001a4e0 con 0x7f3f90105f40 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.258+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3f90197260 con 0x7f3f74038c50 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.511+0000 7f3f8cc00700 1 -- 172.21.15.60:0/782567080 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f3f90197260 con 0x7f3f74038c50 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.514+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3f74038c50 msgr2=0x7f3f7403b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:42.176 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.514+0000 7f3f97000700 1 --2- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3f74038c50 0x7f3f7403b110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:42.177 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.514+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 msgr2=0x7f3f9019d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:42.177 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.514+0000 7f3f97000700 1 --2- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f9019d7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f80011b60 tx=0x7f3f80011c40 comp rx=0 tx=0).stop 2024-02-26T10:34:42.177 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.514+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 shutdown_connections 2024-02-26T10:34:42.177 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.514+0000 7f3f97000700 1 --2- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3f74038c50 0x7f3f7403b110 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:42.177 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.514+0000 7f3f97000700 1 --2- 172.21.15.60:0/782567080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f90105f40 0x7f3f9019d7c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:42.177 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.514+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 >> 172.21.15.60:0/782567080 conn(0x7f3f90101b80 msgr2=0x7f3f901943a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:42.177 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.515+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 shutdown_connections 2024-02-26T10:34:42.177 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:41.515+0000 7f3f97000700 1 -- 172.21.15.60:0/782567080 wait complete. 2024-02-26T10:34:42.177 INFO:teuthology.orchestra.run.smithi060.stdout:Fetching dashboard port number... 2024-02-26T10:34:42.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:42 smithi060 ceph-mon[31539]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:42.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:42 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:42.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:42 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:43.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:43 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2542863426' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-26T10:34:44.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 8443 2024-02-26T10:34:44.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.770+0000 7f03cde00700 1 Processor -- start 2024-02-26T10:34:44.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.771+0000 7f03cde00700 1 -- start start 2024-02-26T10:34:44.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.771+0000 7f03cde00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c8107a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:44.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.771+0000 7f03cde00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8108040 con 0x7f03c8107650 2024-02-26T10:34:44.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.772+0000 7f03c6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c8107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:44.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.772+0000 7f03c6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c8107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48874/0 (socket says 172.21.15.60:48874) 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.772+0000 7f03c6a00700 1 -- 172.21.15.60:0/3440014959 learned_addr learned my addr 172.21.15.60:0/3440014959 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.772+0000 7f03c6a00700 1 -- 172.21.15.60:0/3440014959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c8108800 con 0x7f03c8107650 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.773+0000 7f03c6a00700 1 --2- 172.21.15.60:0/3440014959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c8107a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03b80098a0 tx=0x7f03b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cb892974094e49 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.773+0000 7f03c5600700 1 -- 172.21.15.60:0/3440014959 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03b8004160 con 0x7f03c8107650 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.773+0000 7f03c5600700 1 -- 172.21.15.60:0/3440014959 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03b8004300 con 0x7f03c8107650 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.773+0000 7f03c5600700 1 -- 172.21.15.60:0/3440014959 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03b8011b20 con 0x7f03c8107650 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.774+0000 7f03cde00700 1 -- 172.21.15.60:0/3440014959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 msgr2=0x7f03c8107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.774+0000 7f03cde00700 1 --2- 172.21.15.60:0/3440014959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c8107a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03b80098a0 tx=0x7f03b8009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.774+0000 7f03cde00700 1 -- 172.21.15.60:0/3440014959 shutdown_connections 2024-02-26T10:34:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.774+0000 7f03cde00700 1 --2- 172.21.15.60:0/3440014959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c8107a70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.774+0000 7f03cde00700 1 -- 172.21.15.60:0/3440014959 >> 172.21.15.60:0/3440014959 conn(0x7f03c8102cf0 msgr2=0x7f03c8105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.774+0000 7f03cde00700 1 -- 172.21.15.60:0/3440014959 shutdown_connections 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.774+0000 7f03cde00700 1 -- 172.21.15.60:0/3440014959 wait complete. 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.775+0000 7f03cde00700 1 Processor -- start 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.775+0000 7f03cde00700 1 -- start start 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.776+0000 7f03cde00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.776+0000 7f03cde00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c810bbb0 con 0x7f03c8107650 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.776+0000 7f03c6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.776+0000 7f03c6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c807cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48888/0 (socket says 172.21.15.60:48888) 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.776+0000 7f03c6a00700 1 -- 172.21.15.60:0/2542863426 learned_addr learned my addr 172.21.15.60:0/2542863426 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.777+0000 7f03c6a00700 1 -- 172.21.15.60:0/2542863426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c807d0b0 con 0x7f03c8107650 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.777+0000 7f03c6a00700 1 --2- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c807cb70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f03b8000c00 tx=0x7f03b8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.777+0000 7f03cca00700 1 -- 172.21.15.60:0/2542863426 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03b8003b60 con 0x7f03c8107650 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.777+0000 7f03cca00700 1 -- 172.21.15.60:0/2542863426 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03b801a910 con 0x7f03c8107650 2024-02-26T10:34:44.646 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.777+0000 7f03cca00700 1 -- 172.21.15.60:0/2542863426 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03b8024930 con 0x7f03c8107650 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.777+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c807b2a0 con 0x7f03c8107650 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.778+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c807b760 con 0x7f03c8107650 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.779+0000 7f03cca00700 1 -- 172.21.15.60:0/2542863426 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) v1 ==== 45941+0+0 (secure 0 0 0) 0x7f03b8022020 con 0x7f03c8107650 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.779+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c8068160 con 0x7f03c8107650 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.779+0000 7f03cca00700 1 --2- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03ac038bf0 0x7f03ac03b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.779+0000 7f03cca00700 1 -- 172.21.15.60:0/2542863426 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f03b804cda0 con 0x7f03c8107650 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.780+0000 7f03c6000700 1 --2- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03ac038bf0 0x7f03ac03b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.780+0000 7f03c6000700 1 --2- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03ac038bf0 0x7f03ac03b0b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03b4006fd0 tx=0x7f03b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:42.790+0000 7f03cca00700 1 -- 172.21.15.60:0/2542863426 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f03b802ad60 con 0x7f03c8107650 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.039+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f03c807b9e0 con 0x7f03c8107650 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.040+0000 7f03cca00700 1 -- 172.21.15.60:0/2542863426 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f03b8021d60 con 0x7f03c8107650 2024-02-26T10:34:44.647 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.043+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03ac038bf0 msgr2=0x7f03ac03b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:44.648 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.043+0000 7f03cde00700 1 --2- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03ac038bf0 0x7f03ac03b0b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03b4006fd0 tx=0x7f03b4006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:44.648 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.043+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 msgr2=0x7f03c807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:44.648 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.043+0000 7f03cde00700 1 --2- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c807cb70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f03b8000c00 tx=0x7f03b8011c10 comp rx=0 tx=0).stop 2024-02-26T10:34:44.648 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.044+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 shutdown_connections 2024-02-26T10:34:44.648 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.044+0000 7f03cde00700 1 --2- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03ac038bf0 0x7f03ac03b0b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:44.648 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.044+0000 7f03cde00700 1 --2- 172.21.15.60:0/2542863426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03c8107650 0x7f03c807cb70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:44.648 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.044+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 >> 172.21.15.60:0/2542863426 conn(0x7f03c8102cf0 msgr2=0x7f03c81038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:44.648 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.044+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 shutdown_connections 2024-02-26T10:34:44.648 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:43.044+0000 7f03cde00700 1 -- 172.21.15.60:0/2542863426 wait complete. 2024-02-26T10:34:44.673 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T10:34:44.674 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-02-26T10:34:44.700 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T10:34:44.700 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T10:34:44.700 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-02-26T10:34:44.700 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-26T10:34:44.707 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph Dashboard is now available at: 2024-02-26T10:34:44.707 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:44.707 INFO:teuthology.orchestra.run.smithi060.stdout: URL: https://smithi060.front.sepia.ceph.com:8443/ 2024-02-26T10:34:44.707 INFO:teuthology.orchestra.run.smithi060.stdout: User: admin 2024-02-26T10:34:44.707 INFO:teuthology.orchestra.run.smithi060.stdout: Password: q40pn44x5s 2024-02-26T10:34:44.707 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:44.708 INFO:teuthology.orchestra.run.smithi060.stdout:Saving cluster configuration to /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config directory 2024-02-26T10:34:44.710 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling autotune for osd_memory_target 2024-02-26T10:34:44.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:44 smithi060 ceph-mon[31539]: mgrmap e13: x(active, since 6s) 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.319+0000 7f042e200700 1 Processor -- start 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.319+0000 7f042e200700 1 -- start start 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.320+0000 7f042e200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f042810f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.320+0000 7f042e200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042810f980 con 0x7f042810f020 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.320+0000 7f042ce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f042810f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.320+0000 7f042ce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f042810f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60626/0 (socket says 172.21.15.60:60626) 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.320+0000 7f042ce00700 1 -- 172.21.15.60:0/2089008635 learned_addr learned my addr 172.21.15.60:0/2089008635 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.321+0000 7f042ce00700 1 -- 172.21.15.60:0/2089008635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042810fb00 con 0x7f042810f020 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.321+0000 7f042ce00700 1 --2- 172.21.15.60:0/2089008635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f042810f440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f041c0098a0 tx=0x7f041c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b6166f7c1edcf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:46.218 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.321+0000 7f0427400700 1 -- 172.21.15.60:0/2089008635 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f041c004160 con 0x7f042810f020 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.321+0000 7f0427400700 1 -- 172.21.15.60:0/2089008635 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f041c004300 con 0x7f042810f020 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.321+0000 7f0427400700 1 -- 172.21.15.60:0/2089008635 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f041c011b20 con 0x7f042810f020 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.322+0000 7f042e200700 1 -- 172.21.15.60:0/2089008635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 msgr2=0x7f042810f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.322+0000 7f042e200700 1 --2- 172.21.15.60:0/2089008635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f042810f440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f041c0098a0 tx=0x7f041c009bb0 comp rx=0 tx=0).stop 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.322+0000 7f042e200700 1 -- 172.21.15.60:0/2089008635 shutdown_connections 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.322+0000 7f042e200700 1 --2- 172.21.15.60:0/2089008635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f042810f440 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.322+0000 7f042e200700 1 -- 172.21.15.60:0/2089008635 >> 172.21.15.60:0/2089008635 conn(0x7f042807ce10 msgr2=0x7f042807f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.323+0000 7f042e200700 1 -- 172.21.15.60:0/2089008635 shutdown_connections 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.323+0000 7f042e200700 1 -- 172.21.15.60:0/2089008635 wait complete. 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.324+0000 7f042e200700 1 Processor -- start 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.324+0000 7f042e200700 1 -- start start 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.324+0000 7f042e200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f04281a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.324+0000 7f042e200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0428111f10 con 0x7f042810f020 2024-02-26T10:34:46.219 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.325+0000 7f042ce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f04281a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.325+0000 7f042ce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f04281a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60640/0 (socket says 172.21.15.60:60640) 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.325+0000 7f042ce00700 1 -- 172.21.15.60:0/442748691 learned_addr learned my addr 172.21.15.60:0/442748691 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.325+0000 7f042ce00700 1 -- 172.21.15.60:0/442748691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04281a1f20 con 0x7f042810f020 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.325+0000 7f042ce00700 1 --2- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f04281a19e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f041c000c00 tx=0x7f041c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.326+0000 7f0425600700 1 -- 172.21.15.60:0/442748691 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f041c003b60 con 0x7f042810f020 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.326+0000 7f0425600700 1 -- 172.21.15.60:0/442748691 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f041c01a910 con 0x7f042810f020 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.326+0000 7f0425600700 1 -- 172.21.15.60:0/442748691 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f041c024930 con 0x7f042810f020 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.326+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04281a21b0 con 0x7f042810f020 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.326+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04281a2610 con 0x7f042810f020 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.327+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0428063380 con 0x7f042810f020 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.327+0000 7f0425600700 1 -- 172.21.15.60:0/442748691 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f041c022020 con 0x7f042810f020 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.327+0000 7f0425600700 1 --2- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0410038c90 0x7f041003b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:46.220 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.328+0000 7f0425600700 1 -- 172.21.15.60:0/442748691 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f041c04cf30 con 0x7f042810f020 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.328+0000 7f0427e00700 1 --2- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0410038c90 0x7f041003b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.328+0000 7f0427e00700 1 --2- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0410038c90 0x7f041003b150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0414006fd0 tx=0x7f0414006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.333+0000 7f0425600700 1 -- 172.21.15.60:0/442748691 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f041c01ab70 con 0x7f042810f020 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.554+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f04281a50a0 con 0x7f042810f020 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.555+0000 7f0425600700 1 -- 172.21.15.60:0/442748691 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f041c02d3f0 con 0x7f042810f020 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0410038c90 msgr2=0x7f041003b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 --2- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0410038c90 0x7f041003b150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0414006fd0 tx=0x7f0414006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 msgr2=0x7f04281a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 --2- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f04281a19e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f041c000c00 tx=0x7f041c011c10 comp rx=0 tx=0).stop 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 shutdown_connections 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 --2- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0410038c90 0x7f041003b150 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 --2- 172.21.15.60:0/442748691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042810f020 0x7f04281a19e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:46.221 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 >> 172.21.15.60:0/442748691 conn(0x7f042807ce10 msgr2=0x7f042807e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:46.222 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 shutdown_connections 2024-02-26T10:34:46.222 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:45.559+0000 7f042e200700 1 -- 172.21.15.60:0/442748691 wait complete. 2024-02-26T10:34:47.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.801+0000 7f90de400700 1 Processor -- start 2024-02-26T10:34:47.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.802+0000 7f90de400700 1 -- start start 2024-02-26T10:34:47.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.802+0000 7f90de400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d8107a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:47.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.802+0000 7f90de400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8108040 con 0x7f90d8107650 2024-02-26T10:34:47.854 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.803+0000 7f90d7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d8107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:47.854 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.803+0000 7f90d7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d8107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60654/0 (socket says 172.21.15.60:60654) 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.803+0000 7f90d7400700 1 -- 172.21.15.60:0/645250145 learned_addr learned my addr 172.21.15.60:0/645250145 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.803+0000 7f90d7400700 1 -- 172.21.15.60:0/645250145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d8108800 con 0x7f90d8107650 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.804+0000 7f90d7400700 1 --2- 172.21.15.60:0/645250145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d8107a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009e80 tx=0x7f90c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65ed8644028c2cf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.804+0000 7f90d6000700 1 -- 172.21.15.60:0/645250145 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90c8017070 con 0x7f90d8107650 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.804+0000 7f90d6000700 1 -- 172.21.15.60:0/645250145 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90c800b8c0 con 0x7f90d8107650 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.804+0000 7f90d6000700 1 -- 172.21.15.60:0/645250145 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90c80045c0 con 0x7f90d8107650 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.805+0000 7f90de400700 1 -- 172.21.15.60:0/645250145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 msgr2=0x7f90d8107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.805+0000 7f90de400700 1 --2- 172.21.15.60:0/645250145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d8107a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009e80 tx=0x7f90c800b330 comp rx=0 tx=0).stop 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.805+0000 7f90de400700 1 -- 172.21.15.60:0/645250145 shutdown_connections 2024-02-26T10:34:47.855 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.805+0000 7f90de400700 1 --2- 172.21.15.60:0/645250145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d8107a70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.805+0000 7f90de400700 1 -- 172.21.15.60:0/645250145 >> 172.21.15.60:0/645250145 conn(0x7f90d8102cf0 msgr2=0x7f90d8105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.805+0000 7f90de400700 1 -- 172.21.15.60:0/645250145 shutdown_connections 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.805+0000 7f90de400700 1 -- 172.21.15.60:0/645250145 wait complete. 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.806+0000 7f90de400700 1 Processor -- start 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.807+0000 7f90de400700 1 -- start start 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.807+0000 7f90de400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.807+0000 7f90de400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d810bbb0 con 0x7f90d8107650 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.808+0000 7f90d7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.808+0000 7f90d7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d819b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60668/0 (socket says 172.21.15.60:60668) 2024-02-26T10:34:47.859 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.808+0000 7f90d7400700 1 -- 172.21.15.60:0/949834041 learned_addr learned my addr 172.21.15.60:0/949834041 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.808+0000 7f90d7400700 1 -- 172.21.15.60:0/949834041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d819bad0 con 0x7f90d8107650 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.808+0000 7f90d7400700 1 --2- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d819b590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009fd0 tx=0x7f90c8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.808+0000 7f90cfe00700 1 -- 172.21.15.60:0/949834041 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90c8020020 con 0x7f90d8107650 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.809+0000 7f90cfe00700 1 -- 172.21.15.60:0/949834041 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90c800bba0 con 0x7f90d8107650 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.809+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d819bd60 con 0x7f90d8107650 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.809+0000 7f90cfe00700 1 -- 172.21.15.60:0/949834041 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90c8017040 con 0x7f90d8107650 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.809+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d819ea50 con 0x7f90d8107650 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.810+0000 7f90cfe00700 1 -- 172.21.15.60:0/949834041 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f90c8023020 con 0x7f90d8107650 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.810+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d8068160 con 0x7f90d8107650 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.810+0000 7f90cfe00700 1 --2- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90bc038cf0 0x7f90bc03b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.810+0000 7f90cfe00700 1 -- 172.21.15.60:0/949834041 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f90c800f050 con 0x7f90d8107650 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.811+0000 7f90d6a00700 1 --2- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90bc038cf0 0x7f90bc03b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:47.860 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.811+0000 7f90d6a00700 1 --2- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90bc038cf0 0x7f90bc03b1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f90c4006fd0 tx=0x7f90c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:46.816+0000 7f90cfe00700 1 -- 172.21.15.60:0/949834041 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f90c802a020 con 0x7f90d8107650 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.161+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f90d819ec20 con 0x7f90d8107650 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.166+0000 7f90cfe00700 1 -- 172.21.15.60:0/949834041 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f90d819ec20 con 0x7f90d8107650 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.169+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90bc038cf0 msgr2=0x7f90bc03b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.169+0000 7f90de400700 1 --2- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90bc038cf0 0x7f90bc03b1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f90c4006fd0 tx=0x7f90c4006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.169+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 msgr2=0x7f90d819b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.169+0000 7f90de400700 1 --2- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d819b590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009fd0 tx=0x7f90c8003d00 comp rx=0 tx=0).stop 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.169+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 shutdown_connections 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.169+0000 7f90de400700 1 --2- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90bc038cf0 0x7f90bc03b1b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.169+0000 7f90de400700 1 --2- 172.21.15.60:0/949834041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d8107650 0x7f90d819b590 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:47.862 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.169+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 >> 172.21.15.60:0/949834041 conn(0x7f90d8102cf0 msgr2=0x7f90d8103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:47.862 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.170+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 shutdown_connections 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T10:34:47.170+0000 7f90de400700 1 -- 172.21.15.60:0/949834041 wait complete. 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout:Or, if you are only running a single cluster on this host: 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: ceph telemetry on 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:47.863 INFO:teuthology.orchestra.run.smithi060.stdout:For more information see: 2024-02-26T10:34:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-26T10:34:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:34:47.864 INFO:teuthology.orchestra.run.smithi060.stdout:Bootstrap complete. 2024-02-26T10:34:47.958 INFO:tasks.cephadm:Fetching config... 2024-02-26T10:34:47.959 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:34:47.959 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-26T10:34:48.010 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-26T10:34:48.011 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:34:48.011 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-26T10:34:48.096 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-26T10:34:48.096 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:34:48.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-26T10:34:48.226 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-26T10:34:48.226 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:34:48.226 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-26T10:34:48.280 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-26T10:34:48.280 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDSB0H4V5PocVbKIURPKRENalEbnQ+ZQ0/shFqqWVkZ4+LbAU2qverLCmC3m/Cgp0wHLm6nJ63ZsPs/519ozsZVbmzSWX6fY5rWbNIiNU4y6d7bFjBocnuqHRdqb5OnMpW/LdJTj3ux6eStopm6nW958gGKciXjJBy7NYguurprM1Qxwr8M6JaAs7pGYLFG1RBNjtdWkYfK2BM6dazKaY5EHFf+N1a6s5jxdCyC+t5Dl6iIO5DOnetV22GmfyzW/EVlDUAWg14qFomOcdvNZ8UgkdjfMInQj6yuiTF0Y33LcfmtJSYchXQXlRjEcHfD9/EUxwV3xF4VPRMxX2005L3QnEeLK9iiG23s3Y62UDNkNQBqykdoPZcyleuVNXdB1WRnl5IcAEO2vY7XhF7SZayEmZUac7U2rlSszL4ZPjkUptYsLHxmx6YysV/XR3iSp3Y5itxxP1/CseHJflF0YXhWE287t5vY4INAkMk0SP5+67/3PtltTvWJr/OTx62X3c= ceph-51464d50-d492-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T10:34:48.405 INFO:teuthology.orchestra.run.smithi060.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDSB0H4V5PocVbKIURPKRENalEbnQ+ZQ0/shFqqWVkZ4+LbAU2qverLCmC3m/Cgp0wHLm6nJ63ZsPs/519ozsZVbmzSWX6fY5rWbNIiNU4y6d7bFjBocnuqHRdqb5OnMpW/LdJTj3ux6eStopm6nW958gGKciXjJBy7NYguurprM1Qxwr8M6JaAs7pGYLFG1RBNjtdWkYfK2BM6dazKaY5EHFf+N1a6s5jxdCyC+t5Dl6iIO5DOnetV22GmfyzW/EVlDUAWg14qFomOcdvNZ8UgkdjfMInQj6yuiTF0Y33LcfmtJSYchXQXlRjEcHfD9/EUxwV3xF4VPRMxX2005L3QnEeLK9iiG23s3Y62UDNkNQBqykdoPZcyleuVNXdB1WRnl5IcAEO2vY7XhF7SZayEmZUac7U2rlSszL4ZPjkUptYsLHxmx6YysV/XR3iSp3Y5itxxP1/CseHJflF0YXhWE287t5vY4INAkMk0SP5+67/3PtltTvWJr/OTx62X3c= ceph-51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:34:48.456 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:48 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/949834041' entity='client.admin' 2024-02-26T10:34:48.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDSB0H4V5PocVbKIURPKRENalEbnQ+ZQ0/shFqqWVkZ4+LbAU2qverLCmC3m/Cgp0wHLm6nJ63ZsPs/519ozsZVbmzSWX6fY5rWbNIiNU4y6d7bFjBocnuqHRdqb5OnMpW/LdJTj3ux6eStopm6nW958gGKciXjJBy7NYguurprM1Qxwr8M6JaAs7pGYLFG1RBNjtdWkYfK2BM6dazKaY5EHFf+N1a6s5jxdCyC+t5Dl6iIO5DOnetV22GmfyzW/EVlDUAWg14qFomOcdvNZ8UgkdjfMInQj6yuiTF0Y33LcfmtJSYchXQXlRjEcHfD9/EUxwV3xF4VPRMxX2005L3QnEeLK9iiG23s3Y62UDNkNQBqykdoPZcyleuVNXdB1WRnl5IcAEO2vY7XhF7SZayEmZUac7U2rlSszL4ZPjkUptYsLHxmx6YysV/XR3iSp3Y5itxxP1/CseHJflF0YXhWE287t5vY4INAkMk0SP5+67/3PtltTvWJr/OTx62X3c= ceph-51464d50-d492-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T10:34:48.521 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDSB0H4V5PocVbKIURPKRENalEbnQ+ZQ0/shFqqWVkZ4+LbAU2qverLCmC3m/Cgp0wHLm6nJ63ZsPs/519ozsZVbmzSWX6fY5rWbNIiNU4y6d7bFjBocnuqHRdqb5OnMpW/LdJTj3ux6eStopm6nW958gGKciXjJBy7NYguurprM1Qxwr8M6JaAs7pGYLFG1RBNjtdWkYfK2BM6dazKaY5EHFf+N1a6s5jxdCyC+t5Dl6iIO5DOnetV22GmfyzW/EVlDUAWg14qFomOcdvNZ8UgkdjfMInQj6yuiTF0Y33LcfmtJSYchXQXlRjEcHfD9/EUxwV3xF4VPRMxX2005L3QnEeLK9iiG23s3Y62UDNkNQBqykdoPZcyleuVNXdB1WRnl5IcAEO2vY7XhF7SZayEmZUac7U2rlSszL4ZPjkUptYsLHxmx6YysV/XR3iSp3Y5itxxP1/CseHJflF0YXhWE287t5vY4INAkMk0SP5+67/3PtltTvWJr/OTx62X3c= ceph-51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:34:48.536 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDSB0H4V5PocVbKIURPKRENalEbnQ+ZQ0/shFqqWVkZ4+LbAU2qverLCmC3m/Cgp0wHLm6nJ63ZsPs/519ozsZVbmzSWX6fY5rWbNIiNU4y6d7bFjBocnuqHRdqb5OnMpW/LdJTj3ux6eStopm6nW958gGKciXjJBy7NYguurprM1Qxwr8M6JaAs7pGYLFG1RBNjtdWkYfK2BM6dazKaY5EHFf+N1a6s5jxdCyC+t5Dl6iIO5DOnetV22GmfyzW/EVlDUAWg14qFomOcdvNZ8UgkdjfMInQj6yuiTF0Y33LcfmtJSYchXQXlRjEcHfD9/EUxwV3xF4VPRMxX2005L3QnEeLK9iiG23s3Y62UDNkNQBqykdoPZcyleuVNXdB1WRnl5IcAEO2vY7XhF7SZayEmZUac7U2rlSszL4ZPjkUptYsLHxmx6YysV/XR3iSp3Y5itxxP1/CseHJflF0YXhWE287t5vY4INAkMk0SP5+67/3PtltTvWJr/OTx62X3c= ceph-51464d50-d492-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T10:34:48.582 INFO:teuthology.orchestra.run.smithi113.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDSB0H4V5PocVbKIURPKRENalEbnQ+ZQ0/shFqqWVkZ4+LbAU2qverLCmC3m/Cgp0wHLm6nJ63ZsPs/519ozsZVbmzSWX6fY5rWbNIiNU4y6d7bFjBocnuqHRdqb5OnMpW/LdJTj3ux6eStopm6nW958gGKciXjJBy7NYguurprM1Qxwr8M6JaAs7pGYLFG1RBNjtdWkYfK2BM6dazKaY5EHFf+N1a6s5jxdCyC+t5Dl6iIO5DOnetV22GmfyzW/EVlDUAWg14qFomOcdvNZ8UgkdjfMInQj6yuiTF0Y33LcfmtJSYchXQXlRjEcHfD9/EUxwV3xF4VPRMxX2005L3QnEeLK9iiG23s3Y62UDNkNQBqykdoPZcyleuVNXdB1WRnl5IcAEO2vY7XhF7SZayEmZUac7U2rlSszL4ZPjkUptYsLHxmx6YysV/XR3iSp3Y5itxxP1/CseHJflF0YXhWE287t5vY4INAkMk0SP5+67/3PtltTvWJr/OTx62X3c= ceph-51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:34:48.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-26T10:34:49.052 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:34:51.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.711+0000 7fc418a00700 1 -- 172.21.15.60:0/2047237439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 msgr2=0x7fc414100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:51.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.711+0000 7fc418a00700 1 --2- 172.21.15.60:0/2047237439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 0x7fc414100aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc404009910 tx=0x7fc404009c20 comp rx=0 tx=0).stop 2024-02-26T10:34:51.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.712+0000 7fc418a00700 1 -- 172.21.15.60:0/2047237439 shutdown_connections 2024-02-26T10:34:51.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.712+0000 7fc418a00700 1 --2- 172.21.15.60:0/2047237439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 0x7fc414100aa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:51.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.712+0000 7fc418a00700 1 -- 172.21.15.60:0/2047237439 >> 172.21.15.60:0/2047237439 conn(0x7fc4140fbc20 msgr2=0x7fc4140fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:51.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.712+0000 7fc418a00700 1 -- 172.21.15.60:0/2047237439 shutdown_connections 2024-02-26T10:34:51.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.712+0000 7fc418a00700 1 -- 172.21.15.60:0/2047237439 wait complete. 2024-02-26T10:34:51.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.713+0000 7fc418a00700 1 Processor -- start 2024-02-26T10:34:51.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.713+0000 7fc418a00700 1 -- start start 2024-02-26T10:34:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.713+0000 7fc418a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 0x7fc4141990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.713+0000 7fc418a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414106710 con 0x7fc414100680 2024-02-26T10:34:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.714+0000 7fc411e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 0x7fc4141990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.714+0000 7fc411e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 0x7fc4141990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60688/0 (socket says 172.21.15.60:60688) 2024-02-26T10:34:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.714+0000 7fc411e00700 1 -- 172.21.15.60:0/2419903057 learned_addr learned my addr 172.21.15.60:0/2419903057 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:51.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.714+0000 7fc411e00700 1 -- 172.21.15.60:0/2419903057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4141995f0 con 0x7fc414100680 2024-02-26T10:34:51.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.715+0000 7fc411e00700 1 --2- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 0x7fc4141990b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc404005850 tx=0x7fc404005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:51.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.715+0000 7fc40aa00700 1 -- 172.21.15.60:0/2419903057 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc404026070 con 0x7fc414100680 2024-02-26T10:34:51.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.715+0000 7fc40aa00700 1 -- 172.21.15.60:0/2419903057 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4040044a0 con 0x7fc414100680 2024-02-26T10:34:51.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.715+0000 7fc40aa00700 1 -- 172.21.15.60:0/2419903057 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc404003ab0 con 0x7fc414100680 2024-02-26T10:34:51.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.715+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc414199880 con 0x7fc414100680 2024-02-26T10:34:51.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.715+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41419c570 con 0x7fc414100680 2024-02-26T10:34:51.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.716+0000 7fc40aa00700 1 -- 172.21.15.60:0/2419903057 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fc404029070 con 0x7fc414100680 2024-02-26T10:34:51.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.716+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc414106e90 con 0x7fc414100680 2024-02-26T10:34:51.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.717+0000 7fc40aa00700 1 --2- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3f8038c40 0x7fc3f803b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:51.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.717+0000 7fc40aa00700 1 -- 172.21.15.60:0/2419903057 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc40401e070 con 0x7fc414100680 2024-02-26T10:34:51.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.717+0000 7fc411400700 1 --2- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3f8038c40 0x7fc3f803b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.718+0000 7fc411400700 1 --2- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3f8038c40 0x7fc3f803b100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.723+0000 7fc40aa00700 1 -- 172.21.15.60:0/2419903057 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc404013070 con 0x7fc414100680 2024-02-26T10:34:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.950+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fc41419c990 con 0x7fc414100680 2024-02-26T10:34:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.954+0000 7fc40aa00700 1 -- 172.21.15.60:0/2419903057 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc4040300c0 con 0x7fc414100680 2024-02-26T10:34:51.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.957+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3f8038c40 msgr2=0x7fc3f803b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:51.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.957+0000 7fc418a00700 1 --2- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3f8038c40 0x7fc3f803b100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:51.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.957+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 msgr2=0x7fc4141990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:51.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.957+0000 7fc418a00700 1 --2- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 0x7fc4141990b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc404005850 tx=0x7fc404005030 comp rx=0 tx=0).stop 2024-02-26T10:34:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.957+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 shutdown_connections 2024-02-26T10:34:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.957+0000 7fc418a00700 1 --2- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3f8038c40 0x7fc3f803b100 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.957+0000 7fc418a00700 1 --2- 172.21.15.60:0/2419903057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc414100680 0x7fc4141990b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.957+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 >> 172.21.15.60:0/2419903057 conn(0x7fc4140fbc20 msgr2=0x7fc414069140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.958+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 shutdown_connections 2024-02-26T10:34:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:51.958+0000 7fc418a00700 1 -- 172.21.15.60:0/2419903057 wait complete. 2024-02-26T10:34:52.348 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:52.348 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:52.348 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:34:52.348 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:52.348 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:52.348 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:52.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:52 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2419903057' entity='client.admin' 2024-02-26T10:34:52.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:52.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:52.801 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-26T10:34:52.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-26T10:34:53.228 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:34:55.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.832+0000 7f0ed1000700 1 -- 172.21.15.60:0/1807707467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc107f10 msgr2=0x7f0ecc108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:55.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.832+0000 7f0ed1000700 1 --2- 172.21.15.60:0/1807707467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc107f10 0x7f0ecc108330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009910 tx=0x7f0ebc009c20 comp rx=0 tx=0).stop 2024-02-26T10:34:55.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.832+0000 7f0ed1000700 1 -- 172.21.15.60:0/1807707467 shutdown_connections 2024-02-26T10:34:55.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.832+0000 7f0ed1000700 1 --2- 172.21.15.60:0/1807707467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc107f10 0x7f0ecc108330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:55.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.832+0000 7f0ed1000700 1 -- 172.21.15.60:0/1807707467 >> 172.21.15.60:0/1807707467 conn(0x7f0ecc076bd0 msgr2=0x7f0ecc079010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:55.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.832+0000 7f0ed1000700 1 -- 172.21.15.60:0/1807707467 shutdown_connections 2024-02-26T10:34:55.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.832+0000 7f0ed1000700 1 -- 172.21.15.60:0/1807707467 wait complete. 2024-02-26T10:34:55.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.833+0000 7f0ed1000700 1 Processor -- start 2024-02-26T10:34:55.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.833+0000 7f0ed1000700 1 -- start start 2024-02-26T10:34:55.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.834+0000 7f0ed1000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc19d570 0x7f0ecc19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:55.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.834+0000 7f0ed1000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc10dc50 con 0x7f0ecc19d570 2024-02-26T10:34:55.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.834+0000 7f0ecb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc19d570 0x7f0ecc19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:55.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.834+0000 7f0ecb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc19d570 0x7f0ecc19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45512/0 (socket says 172.21.15.60:45512) 2024-02-26T10:34:55.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.834+0000 7f0ecb400700 1 -- 172.21.15.60:0/1653748019 learned_addr learned my addr 172.21.15.60:0/1653748019 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:55.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.834+0000 7f0ecb400700 1 -- 172.21.15.60:0/1653748019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc19ded0 con 0x7f0ecc19d570 2024-02-26T10:34:55.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.835+0000 7f0ecb400700 1 --2- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc19d570 0x7f0ecc19d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc000c00 tx=0x7f0ebc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:55.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.835+0000 7f0ec3e00700 1 -- 172.21.15.60:0/1653748019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc026070 con 0x7f0ecc19d570 2024-02-26T10:34:55.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.835+0000 7f0ec3e00700 1 -- 172.21.15.60:0/1653748019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc00bc30 con 0x7f0ecc19d570 2024-02-26T10:34:55.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.835+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc19e160 con 0x7f0ecc19d570 2024-02-26T10:34:55.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.835+0000 7f0ec3e00700 1 -- 172.21.15.60:0/1653748019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc003910 con 0x7f0ecc19d570 2024-02-26T10:34:55.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.835+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc1a0c40 con 0x7f0ecc19d570 2024-02-26T10:34:55.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.836+0000 7f0ec3e00700 1 -- 172.21.15.60:0/1653748019 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f0ebc029070 con 0x7f0ecc19d570 2024-02-26T10:34:55.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.836+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc10e3d0 con 0x7f0ecc19d570 2024-02-26T10:34:55.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.837+0000 7f0ec3e00700 1 --2- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0eb0038ca0 0x7f0eb003b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:55.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.837+0000 7f0ec3e00700 1 -- 172.21.15.60:0/1653748019 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0ebc01e070 con 0x7f0ecc19d570 2024-02-26T10:34:55.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.837+0000 7f0ecaa00700 1 --2- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0eb0038ca0 0x7f0eb003b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:55.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.838+0000 7f0ecaa00700 1 --2- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0eb0038ca0 0x7f0eb003b160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8006fd0 tx=0x7f0eb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:55.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:55.843+0000 7f0ec3e00700 1 -- 172.21.15.60:0/1653748019 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ebc013070 con 0x7f0ecc19d570 2024-02-26T10:34:56.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.072+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ecc02d900 con 0x7f0eb0038ca0 2024-02-26T10:34:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.079+0000 7f0ec3e00700 1 -- 172.21.15.60:0/1653748019 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0ecc02d900 con 0x7f0eb0038ca0 2024-02-26T10:34:56.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0eb0038ca0 msgr2=0x7f0eb003b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:56.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 --2- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0eb0038ca0 0x7f0eb003b160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8006fd0 tx=0x7f0eb8006e40 comp rx=0 tx=0).stop 2024-02-26T10:34:56.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc19d570 msgr2=0x7f0ecc19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:56.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 --2- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc19d570 0x7f0ecc19d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc000c00 tx=0x7f0ebc005030 comp rx=0 tx=0).stop 2024-02-26T10:34:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 shutdown_connections 2024-02-26T10:34:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 --2- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0eb0038ca0 0x7f0eb003b160 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 --2- 172.21.15.60:0/1653748019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ecc19d570 0x7f0ecc19d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 >> 172.21.15.60:0/1653748019 conn(0x7f0ecc076bd0 msgr2=0x7f0ecc0777a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 shutdown_connections 2024-02-26T10:34:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:56.082+0000 7f0ed1000700 1 -- 172.21.15.60:0/1653748019 wait complete. 2024-02-26T10:34:56.810 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi100 2024-02-26T10:34:56.811 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:34:56.811 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.conf 2024-02-26T10:34:56.829 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:34:56.830 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:34:56.887 INFO:tasks.cephadm:Adding host smithi100 to orchestrator... 2024-02-26T10:34:56.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch host add smithi100 2024-02-26T10:34:57.083 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:57 smithi060 ceph-mon[31539]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:34:57.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:57.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:34:57.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:34:57.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:34:57.084 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:57 smithi060 ceph-mon[31539]: Updating smithi060:/etc/ceph/ceph.conf 2024-02-26T10:34:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:34:58.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:58 smithi060 ceph-mon[31539]: Updating smithi060:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:34:58.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:58 smithi060 ceph-mon[31539]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:34:58.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:58 smithi060 ceph-mon[31539]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:34:59.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.314+0000 7fa88be00700 1 -- 172.21.15.60:0/2936750987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 msgr2=0x7fa88c074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:34:59.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.314+0000 7fa88be00700 1 --2- 172.21.15.60:0/2936750987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 0x7fa88c074500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009910 tx=0x7fa87c009c20 comp rx=0 tx=0).stop 2024-02-26T10:34:59.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.314+0000 7fa88be00700 1 -- 172.21.15.60:0/2936750987 shutdown_connections 2024-02-26T10:34:59.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.314+0000 7fa88be00700 1 --2- 172.21.15.60:0/2936750987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 0x7fa88c074500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:34:59.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.314+0000 7fa88be00700 1 -- 172.21.15.60:0/2936750987 >> 172.21.15.60:0/2936750987 conn(0x7fa88c0fd050 msgr2=0x7fa88c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:34:59.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.314+0000 7fa88be00700 1 -- 172.21.15.60:0/2936750987 shutdown_connections 2024-02-26T10:34:59.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.314+0000 7fa88be00700 1 -- 172.21.15.60:0/2936750987 wait complete. 2024-02-26T10:34:59.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.315+0000 7fa88be00700 1 Processor -- start 2024-02-26T10:34:59.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.315+0000 7fa88be00700 1 -- start start 2024-02-26T10:34:59.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.316+0000 7fa88be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 0x7fa88c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:59.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.316+0000 7fa88be00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c10de00 con 0x7fa88c074120 2024-02-26T10:34:59.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.316+0000 7fa88aa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 0x7fa88c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:59.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.316+0000 7fa88aa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 0x7fa88c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45526/0 (socket says 172.21.15.60:45526) 2024-02-26T10:34:59.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.316+0000 7fa88aa00700 1 -- 172.21.15.60:0/3685716634 learned_addr learned my addr 172.21.15.60:0/3685716634 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:34:59.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.316+0000 7fa88aa00700 1 -- 172.21.15.60:0/3685716634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c1997e0 con 0x7fa88c074120 2024-02-26T10:34:59.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.317+0000 7fa88aa00700 1 --2- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 0x7fa88c1992a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa87c006010 tx=0x7fa87c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:59.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.317+0000 7fa883400700 1 -- 172.21.15.60:0/3685716634 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa87c026070 con 0x7fa88c074120 2024-02-26T10:34:59.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.317+0000 7fa883400700 1 -- 172.21.15.60:0/3685716634 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa87c00bc30 con 0x7fa88c074120 2024-02-26T10:34:59.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.317+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c199a70 con 0x7fa88c074120 2024-02-26T10:34:59.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.317+0000 7fa883400700 1 -- 172.21.15.60:0/3685716634 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa87c003ab0 con 0x7fa88c074120 2024-02-26T10:34:59.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.317+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c199ed0 con 0x7fa88c074120 2024-02-26T10:34:59.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.318+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c0757d0 con 0x7fa88c074120 2024-02-26T10:34:59.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.318+0000 7fa883400700 1 -- 172.21.15.60:0/3685716634 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fa87c029070 con 0x7fa88c074120 2024-02-26T10:34:59.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.319+0000 7fa883400700 1 --2- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa870038c40 0x7fa87003b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:34:59.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.319+0000 7fa883400700 1 -- 172.21.15.60:0/3685716634 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa87c01e070 con 0x7fa88c074120 2024-02-26T10:34:59.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.319+0000 7fa883e00700 1 --2- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa870038c40 0x7fa87003b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:34:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.320+0000 7fa883e00700 1 --2- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa870038c40 0x7fa87003b100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa874006fd0 tx=0x7fa874006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:34:59.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.324+0000 7fa883400700 1 -- 172.21.15.60:0/3685716634 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa87c013070 con 0x7fa88c074120 2024-02-26T10:34:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:34:59.553+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}) v1 -- 0x7fa88c19a670 con 0x7fa870038c40 2024-02-26T10:34:59.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:59 smithi060 ceph-mon[31539]: Updating smithi060:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T10:34:59.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:59.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:34:59.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:34:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:00 smithi060 ceph-mon[31539]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:00 smithi060 ceph-mon[31539]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:35:02.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:01 smithi060 ceph-mon[31539]: Deploying cephadm binary to smithi100 2024-02-26T10:35:03.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:02 smithi060 ceph-mon[31539]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:04.770 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi100' with addr '172.21.15.100' 2024-02-26T10:35:04.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.765+0000 7fa883400700 1 -- 172.21.15.60:0/3685716634 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa88c19a670 con 0x7fa870038c40 2024-02-26T10:35:04.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.768+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa870038c40 msgr2=0x7fa87003b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:04.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.769+0000 7fa88be00700 1 --2- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa870038c40 0x7fa87003b100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa874006fd0 tx=0x7fa874006e40 comp rx=0 tx=0).stop 2024-02-26T10:35:04.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.769+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 msgr2=0x7fa88c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:04.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.769+0000 7fa88be00700 1 --2- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 0x7fa88c1992a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa87c006010 tx=0x7fa87c005050 comp rx=0 tx=0).stop 2024-02-26T10:35:04.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.769+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 shutdown_connections 2024-02-26T10:35:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.769+0000 7fa88be00700 1 --2- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa870038c40 0x7fa87003b100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.769+0000 7fa88be00700 1 --2- 172.21.15.60:0/3685716634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c074120 0x7fa88c1992a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.769+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 >> 172.21.15.60:0/3685716634 conn(0x7fa88c0fd050 msgr2=0x7fa88c105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.769+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 shutdown_connections 2024-02-26T10:35:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:04.769+0000 7fa88be00700 1 -- 172.21.15.60:0/3685716634 wait complete. 2024-02-26T10:35:05.076 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:04 smithi060 ceph-mon[31539]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:05.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch host ls --format=json 2024-02-26T10:35:05.933 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:35:06.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:05 smithi060 ceph-mon[31539]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:06.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:06.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:05 smithi060 ceph-mon[31539]: Added host smithi100 2024-02-26T10:35:06.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:06.085 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:07.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:06 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:07.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.836+0000 7fc79ba00700 1 -- 172.21.15.60:0/2430913172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 msgr2=0x7fc79410e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.836+0000 7fc79ba00700 1 --2- 172.21.15.60:0/2430913172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 0x7fc79410e220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc788009910 tx=0x7fc788009c20 comp rx=0 tx=0).stop 2024-02-26T10:35:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.836+0000 7fc79ba00700 1 -- 172.21.15.60:0/2430913172 shutdown_connections 2024-02-26T10:35:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.836+0000 7fc79ba00700 1 --2- 172.21.15.60:0/2430913172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 0x7fc79410e220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.836+0000 7fc79ba00700 1 -- 172.21.15.60:0/2430913172 >> 172.21.15.60:0/2430913172 conn(0x7fc794077580 msgr2=0x7fc7940799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.836+0000 7fc79ba00700 1 -- 172.21.15.60:0/2430913172 shutdown_connections 2024-02-26T10:35:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.836+0000 7fc79ba00700 1 -- 172.21.15.60:0/2430913172 wait complete. 2024-02-26T10:35:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.837+0000 7fc79ba00700 1 Processor -- start 2024-02-26T10:35:07.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.838+0000 7fc79ba00700 1 -- start start 2024-02-26T10:35:07.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.838+0000 7fc79ba00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 0x7fc7941a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:07.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.838+0000 7fc79ba00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794112350 con 0x7fc79410de40 2024-02-26T10:35:07.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.838+0000 7fc799000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 0x7fc7941a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:07.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.838+0000 7fc799000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 0x7fc7941a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45002/0 (socket says 172.21.15.60:45002) 2024-02-26T10:35:07.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.838+0000 7fc799000700 1 -- 172.21.15.60:0/527318233 learned_addr learned my addr 172.21.15.60:0/527318233 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:35:07.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.839+0000 7fc799000700 1 -- 172.21.15.60:0/527318233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7941a3960 con 0x7fc79410de40 2024-02-26T10:35:07.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.839+0000 7fc799000700 1 --2- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 0x7fc7941a3420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc788006010 tx=0x7fc788005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:07.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.839+0000 7fc791600700 1 -- 172.21.15.60:0/527318233 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc788026070 con 0x7fc79410de40 2024-02-26T10:35:07.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.839+0000 7fc791600700 1 -- 172.21.15.60:0/527318233 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc78800bc30 con 0x7fc79410de40 2024-02-26T10:35:07.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.839+0000 7fc791600700 1 -- 172.21.15.60:0/527318233 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7880039d0 con 0x7fc79410de40 2024-02-26T10:35:07.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.839+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7941a3c50 con 0x7fc79410de40 2024-02-26T10:35:07.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.840+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79419d5e0 con 0x7fc79410de40 2024-02-26T10:35:07.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.841+0000 7fc791600700 1 -- 172.21.15.60:0/527318233 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fc788029070 con 0x7fc79410de40 2024-02-26T10:35:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.841+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7941094b0 con 0x7fc79410de40 2024-02-26T10:35:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.841+0000 7fc791600700 1 --2- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc77c038c40 0x7fc77c03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.841+0000 7fc791600700 1 -- 172.21.15.60:0/527318233 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc78801e070 con 0x7fc79410de40 2024-02-26T10:35:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.842+0000 7fc793e00700 1 --2- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc77c038c40 0x7fc77c03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.842+0000 7fc793e00700 1 --2- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc77c038c40 0x7fc77c03b100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc780006fd0 tx=0x7fc780006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:07.847+0000 7fc791600700 1 -- 172.21.15.60:0/527318233 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc788013070 con 0x7fc79410de40 2024-02-26T10:35:08.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.098+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc79419eb10 con 0x7fc77c038c40 2024-02-26T10:35:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.100+0000 7fc791600700 1 -- 172.21.15.60:0/527318233 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fc79419eb10 con 0x7fc77c038c40 2024-02-26T10:35:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:35:08.104 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}] 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.103+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc77c038c40 msgr2=0x7fc77c03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.103+0000 7fc79ba00700 1 --2- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc77c038c40 0x7fc77c03b100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc780006fd0 tx=0x7fc780006e40 comp rx=0 tx=0).stop 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.103+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 msgr2=0x7fc7941a3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.103+0000 7fc79ba00700 1 --2- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 0x7fc7941a3420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc788006010 tx=0x7fc788005030 comp rx=0 tx=0).stop 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.103+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 shutdown_connections 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.103+0000 7fc79ba00700 1 --2- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc77c038c40 0x7fc77c03b100 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.103+0000 7fc79ba00700 1 --2- 172.21.15.60:0/527318233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc79410de40 0x7fc7941a3420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.103+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 >> 172.21.15.60:0/527318233 conn(0x7fc794077580 msgr2=0x7fc794078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.104+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 shutdown_connections 2024-02-26T10:35:08.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:08.104+0000 7fc79ba00700 1 -- 172.21.15.60:0/527318233 wait complete. 2024-02-26T10:35:08.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:08 smithi060 ceph-mon[31539]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:08.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:08.824 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi113 2024-02-26T10:35:08.824 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:35:08.824 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/etc/ceph/ceph.conf 2024-02-26T10:35:08.843 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:35:08.843 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:35:08.901 INFO:tasks.cephadm:Adding host smithi113 to orchestrator... 2024-02-26T10:35:08.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch host add smithi113 2024-02-26T10:35:09.376 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:35:09.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:09 smithi060 ceph-mon[31539]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:35:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:10 smithi060 ceph-mon[31539]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:11.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.487+0000 7fc788600700 1 -- 172.21.15.60:0/500644906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 msgr2=0x7fc780074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:11.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.487+0000 7fc788600700 1 --2- 172.21.15.60:0/500644906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 0x7fc780074500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc774009910 tx=0x7fc774009c20 comp rx=0 tx=0).stop 2024-02-26T10:35:11.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.487+0000 7fc788600700 1 -- 172.21.15.60:0/500644906 shutdown_connections 2024-02-26T10:35:11.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.487+0000 7fc788600700 1 --2- 172.21.15.60:0/500644906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 0x7fc780074500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:11.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.487+0000 7fc788600700 1 -- 172.21.15.60:0/500644906 >> 172.21.15.60:0/500644906 conn(0x7fc7800fd200 msgr2=0x7fc7800ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:11.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.487+0000 7fc788600700 1 -- 172.21.15.60:0/500644906 shutdown_connections 2024-02-26T10:35:11.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.487+0000 7fc788600700 1 -- 172.21.15.60:0/500644906 wait complete. 2024-02-26T10:35:11.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.489+0000 7fc788600700 1 Processor -- start 2024-02-26T10:35:11.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.489+0000 7fc788600700 1 -- start start 2024-02-26T10:35:11.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.489+0000 7fc788600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 0x7fc7801994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:11.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.489+0000 7fc788600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78010df90 con 0x7fc780074120 2024-02-26T10:35:11.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.490+0000 7fc785c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 0x7fc7801994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:11.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.490+0000 7fc785c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 0x7fc7801994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45018/0 (socket says 172.21.15.60:45018) 2024-02-26T10:35:11.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.490+0000 7fc785c00700 1 -- 172.21.15.60:0/1986331351 learned_addr learned my addr 172.21.15.60:0/1986331351 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:35:11.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.490+0000 7fc785c00700 1 -- 172.21.15.60:0/1986331351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780199a00 con 0x7fc780074120 2024-02-26T10:35:11.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.491+0000 7fc785c00700 1 --2- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 0x7fc7801994c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc774005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:11.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.491+0000 7fc77e000700 1 -- 172.21.15.60:0/1986331351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc774026070 con 0x7fc780074120 2024-02-26T10:35:11.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.491+0000 7fc77e000700 1 -- 172.21.15.60:0/1986331351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77400bc30 con 0x7fc780074120 2024-02-26T10:35:11.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.491+0000 7fc77e000700 1 -- 172.21.15.60:0/1986331351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7740039a0 con 0x7fc780074120 2024-02-26T10:35:11.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.491+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc780199c90 con 0x7fc780074120 2024-02-26T10:35:11.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.491+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78019a0f0 con 0x7fc780074120 2024-02-26T10:35:11.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.493+0000 7fc77e000700 1 -- 172.21.15.60:0/1986331351 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fc774029070 con 0x7fc780074120 2024-02-26T10:35:11.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.493+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7800757d0 con 0x7fc780074120 2024-02-26T10:35:11.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.493+0000 7fc77e000700 1 --2- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc768038ca0 0x7fc76803b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:11.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.493+0000 7fc77e000700 1 -- 172.21.15.60:0/1986331351 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc77401e070 con 0x7fc780074120 2024-02-26T10:35:11.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.494+0000 7fc785200700 1 --2- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc768038ca0 0x7fc76803b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:11.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.494+0000 7fc785200700 1 --2- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc768038ca0 0x7fc76803b160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc76c006fd0 tx=0x7fc76c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:11.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.499+0000 7fc77e000700 1 -- 172.21.15.60:0/1986331351 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc774013070 con 0x7fc780074120 2024-02-26T10:35:11.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:11.755+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi113", "target": ["mon-mgr", ""]}) v1 -- 0x7fc78019a820 con 0x7fc768038ca0 2024-02-26T10:35:11.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:11.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:11.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:11.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:11.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:35:11.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:11.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:35:11.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:11 smithi060 ceph-mon[31539]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-26T10:35:12.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:12 smithi060 ceph-mon[31539]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:12.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:12 smithi060 ceph-mon[31539]: Updating smithi100:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:12.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:12 smithi060 ceph-mon[31539]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:35:12.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:12 smithi060 ceph-mon[31539]: Updating smithi100:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T10:35:12.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:12.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:12.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:13.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:13 smithi060 ceph-mon[31539]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi113", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:35:13.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:13 smithi060 ceph-mon[31539]: Deploying cephadm binary to smithi113 2024-02-26T10:35:14.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:14 smithi060 ceph-mon[31539]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.767+0000 7fc77e000700 1 -- 172.21.15.60:0/1986331351 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc78019a820 con 0x7fc768038ca0 2024-02-26T10:35:16.773 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi113' with addr '172.21.15.113' 2024-02-26T10:35:16.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc768038ca0 msgr2=0x7fc76803b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:16.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 --2- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc768038ca0 0x7fc76803b160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc76c006fd0 tx=0x7fc76c006e40 comp rx=0 tx=0).stop 2024-02-26T10:35:16.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 msgr2=0x7fc7801994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:16.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 --2- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 0x7fc7801994c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc774005030 comp rx=0 tx=0).stop 2024-02-26T10:35:16.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 shutdown_connections 2024-02-26T10:35:16.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 --2- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc768038ca0 0x7fc76803b160 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:16.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 --2- 172.21.15.60:0/1986331351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780074120 0x7fc7801994c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:16.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 >> 172.21.15.60:0/1986331351 conn(0x7fc7800fd200 msgr2=0x7fc7801059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:16.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 shutdown_connections 2024-02-26T10:35:16.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:16.772+0000 7fc788600700 1 -- 172.21.15.60:0/1986331351 wait complete. 2024-02-26T10:35:16.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:16 smithi060 ceph-mon[31539]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:17.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch host ls --format=json 2024-02-26T10:35:17.999 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:35:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:17 smithi060 ceph-mon[31539]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:17 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:18.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:17 smithi060 ceph-mon[31539]: Added host smithi113 2024-02-26T10:35:18.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:17 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:18.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:17 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:18 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.985+0000 7f10fb400700 1 -- 172.21.15.60:0/89659416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 msgr2=0x7f10f4109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.985+0000 7f10fb400700 1 --2- 172.21.15.60:0/89659416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 0x7f10f4109d30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009910 tx=0x7f10e8009c20 comp rx=0 tx=0).stop 2024-02-26T10:35:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.985+0000 7f10fb400700 1 -- 172.21.15.60:0/89659416 shutdown_connections 2024-02-26T10:35:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.985+0000 7f10fb400700 1 --2- 172.21.15.60:0/89659416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 0x7f10f4109d30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.986+0000 7f10fb400700 1 -- 172.21.15.60:0/89659416 >> 172.21.15.60:0/89659416 conn(0x7f10f40ff460 msgr2=0x7f10f4101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.986+0000 7f10fb400700 1 -- 172.21.15.60:0/89659416 shutdown_connections 2024-02-26T10:35:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.986+0000 7f10fb400700 1 -- 172.21.15.60:0/89659416 wait complete. 2024-02-26T10:35:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.987+0000 7f10fb400700 1 Processor -- start 2024-02-26T10:35:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.987+0000 7f10fb400700 1 -- start start 2024-02-26T10:35:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.988+0000 7f10fb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 0x7f10f41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.988+0000 7f10fb400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f410df90 con 0x7f10f4109950 2024-02-26T10:35:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.988+0000 7f10f8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 0x7f10f41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.988+0000 7f10f8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 0x7f10f41994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46218/0 (socket says 172.21.15.60:46218) 2024-02-26T10:35:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.988+0000 7f10f8a00700 1 -- 172.21.15.60:0/2917239069 learned_addr learned my addr 172.21.15.60:0/2917239069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:35:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.989+0000 7f10f8a00700 1 -- 172.21.15.60:0/2917239069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f4199a20 con 0x7f10f4109950 2024-02-26T10:35:19.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.989+0000 7f10f8a00700 1 --2- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 0x7f10f41994e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10e8006010 tx=0x7f10e8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:19.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.989+0000 7f10f1600700 1 -- 172.21.15.60:0/2917239069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e8026070 con 0x7f10f4109950 2024-02-26T10:35:19.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.989+0000 7f10f1600700 1 -- 172.21.15.60:0/2917239069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10e800bbd0 con 0x7f10f4109950 2024-02-26T10:35:19.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.989+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f4199d10 con 0x7f10f4109950 2024-02-26T10:35:19.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.989+0000 7f10f1600700 1 -- 172.21.15.60:0/2917239069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e8003910 con 0x7f10f4109950 2024-02-26T10:35:19.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.989+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f419ddb0 con 0x7f10f4109950 2024-02-26T10:35:19.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.991+0000 7f10f1600700 1 -- 172.21.15.60:0/2917239069 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f10e8029070 con 0x7f10f4109950 2024-02-26T10:35:19.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.991+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f4105090 con 0x7f10f4109950 2024-02-26T10:35:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.991+0000 7f10f1600700 1 --2- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f10dc038ca0 0x7f10dc03b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.991+0000 7f10f1600700 1 -- 172.21.15.60:0/2917239069 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f10e801e070 con 0x7f10f4109950 2024-02-26T10:35:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.992+0000 7f10f3e00700 1 --2- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f10dc038ca0 0x7f10dc03b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:19.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.992+0000 7f10f3e00700 1 --2- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f10dc038ca0 0x7f10dc03b160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f10e0006fd0 tx=0x7f10e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:20.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:19.997+0000 7f10f1600700 1 -- 172.21.15.60:0/2917239069 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f10e8013070 con 0x7f10f4109950 2024-02-26T10:35:20.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.226+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10f419a840 con 0x7f10dc038ca0 2024-02-26T10:35:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.229+0000 7f10f1600700 1 -- 172.21.15.60:0/2917239069 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f10f419a840 con 0x7f10dc038ca0 2024-02-26T10:35:20.233 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:35:20.233 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.113", "hostname": "smithi113", "labels": [], "status": ""}] 2024-02-26T10:35:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.232+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f10dc038ca0 msgr2=0x7f10dc03b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.232+0000 7f10fb400700 1 --2- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f10dc038ca0 0x7f10dc03b160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f10e0006fd0 tx=0x7f10e0006e40 comp rx=0 tx=0).stop 2024-02-26T10:35:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.232+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 msgr2=0x7f10f41994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.232+0000 7f10fb400700 1 --2- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 0x7f10f41994e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10e8006010 tx=0x7f10e8005030 comp rx=0 tx=0).stop 2024-02-26T10:35:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.232+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 shutdown_connections 2024-02-26T10:35:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.232+0000 7f10fb400700 1 --2- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f10dc038ca0 0x7f10dc03b160 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.232+0000 7f10fb400700 1 --2- 172.21.15.60:0/2917239069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10f4109950 0x7f10f41994e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.232+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 >> 172.21.15.60:0/2917239069 conn(0x7f10f40ff460 msgr2=0x7f10f410b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.233+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 shutdown_connections 2024-02-26T10:35:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:20.233+0000 7f10fb400700 1 -- 172.21.15.60:0/2917239069 wait complete. 2024-02-26T10:35:20.843 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:20 smithi060 ceph-mon[31539]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:20.843 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:20.912 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-26T10:35:20.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd crush tunables default 2024-02-26T10:35:21.379 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:35:21.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:21 smithi060 ceph-mon[31539]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:35:22.583 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:22 smithi060 ceph-mon[31539]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.367+0000 7f7dcce00700 1 -- 172.21.15.60:0/2238861613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 msgr2=0x7f7dc8109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.367+0000 7f7dcce00700 1 --2- 172.21.15.60:0/2238861613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 0x7f7dc8109d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009910 tx=0x7f7db8009c20 comp rx=0 tx=0).stop 2024-02-26T10:35:23.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.367+0000 7f7dcce00700 1 -- 172.21.15.60:0/2238861613 shutdown_connections 2024-02-26T10:35:23.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.367+0000 7f7dcce00700 1 --2- 172.21.15.60:0/2238861613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 0x7f7dc8109d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:23.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.367+0000 7f7dcce00700 1 -- 172.21.15.60:0/2238861613 >> 172.21.15.60:0/2238861613 conn(0x7f7dc80ff470 msgr2=0x7f7dc8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:23.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.367+0000 7f7dcce00700 1 -- 172.21.15.60:0/2238861613 shutdown_connections 2024-02-26T10:35:23.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.367+0000 7f7dcce00700 1 -- 172.21.15.60:0/2238861613 wait complete. 2024-02-26T10:35:23.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.368+0000 7f7dcce00700 1 Processor -- start 2024-02-26T10:35:23.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.369+0000 7f7dcce00700 1 -- start start 2024-02-26T10:35:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.369+0000 7f7dcce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 0x7f7dc81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.369+0000 7f7dcce00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc810dfa0 con 0x7f7dc8109960 2024-02-26T10:35:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.370+0000 7f7dc5e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 0x7f7dc81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.370+0000 7f7dc5e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 0x7f7dc81994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46240/0 (socket says 172.21.15.60:46240) 2024-02-26T10:35:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.370+0000 7f7dc5e00700 1 -- 172.21.15.60:0/1996614657 learned_addr learned my addr 172.21.15.60:0/1996614657 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:35:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.370+0000 7f7dc5e00700 1 -- 172.21.15.60:0/1996614657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8199a10 con 0x7f7dc8109960 2024-02-26T10:35:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.370+0000 7f7dc5e00700 1 --2- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 0x7f7dc81994d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7db8005850 tx=0x7f7db8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.371+0000 7f7dbea00700 1 -- 172.21.15.60:0/1996614657 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7db8026070 con 0x7f7dc8109960 2024-02-26T10:35:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.371+0000 7f7dbea00700 1 -- 172.21.15.60:0/1996614657 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7db800bbd0 con 0x7f7dc8109960 2024-02-26T10:35:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.371+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc8199d00 con 0x7f7dc8109960 2024-02-26T10:35:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.371+0000 7f7dbea00700 1 -- 172.21.15.60:0/1996614657 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7db8003970 con 0x7f7dc8109960 2024-02-26T10:35:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.371+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc819dda0 con 0x7f7dc8109960 2024-02-26T10:35:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.372+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc81050a0 con 0x7f7dc8109960 2024-02-26T10:35:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.373+0000 7f7dbea00700 1 -- 172.21.15.60:0/1996614657 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f7db8029070 con 0x7f7dc8109960 2024-02-26T10:35:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.373+0000 7f7dbea00700 1 --2- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7dac038c40 0x7f7dac03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.373+0000 7f7dbea00700 1 -- 172.21.15.60:0/1996614657 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7db801e070 con 0x7f7dc8109960 2024-02-26T10:35:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.374+0000 7f7dc5400700 1 --2- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7dac038c40 0x7f7dac03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:23.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.374+0000 7f7dc5400700 1 --2- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7dac038c40 0x7f7dac03b100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7db4006fd0 tx=0x7f7db4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.379+0000 7f7dbea00700 1 -- 172.21.15.60:0/1996614657 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7db8013070 con 0x7f7dc8109960 2024-02-26T10:35:23.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.611+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7dc80633b0 con 0x7f7dc8109960 2024-02-26T10:35:23.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.738+0000 7f7dbea00700 1 -- 172.21.15.60:0/1996614657 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7dc80633b0 con 0x7f7dc8109960 2024-02-26T10:35:23.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.742+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7dac038c40 msgr2=0x7f7dac03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:23.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.742+0000 7f7dcce00700 1 --2- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7dac038c40 0x7f7dac03b100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7db4006fd0 tx=0x7f7db4006e40 comp rx=0 tx=0).stop 2024-02-26T10:35:23.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.742+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 msgr2=0x7f7dc81994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:23.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.742+0000 7f7dcce00700 1 --2- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 0x7f7dc81994d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7db8005850 tx=0x7f7db8005030 comp rx=0 tx=0).stop 2024-02-26T10:35:23.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.742+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 shutdown_connections 2024-02-26T10:35:23.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.742+0000 7f7dcce00700 1 --2- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7dac038c40 0x7f7dac03b100 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:23.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.742+0000 7f7dcce00700 1 --2- 172.21.15.60:0/1996614657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dc8109960 0x7f7dc81994d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:23.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.743+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 >> 172.21.15.60:0/1996614657 conn(0x7f7dc80ff470 msgr2=0x7f7dc810b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:23.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.743+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 shutdown_connections 2024-02-26T10:35:23.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:23.743+0000 7f7dcce00700 1 -- 172.21.15.60:0/1996614657 wait complete. 2024-02-26T10:35:23.749 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-02-26T10:35:23.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:23.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi113", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: Updating smithi113:/etc/ceph/ceph.conf 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: Updating smithi113:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1996614657' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-26T10:35:23.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:23 smithi060 ceph-mon[31539]: Updating smithi113:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T10:35:24.445 INFO:tasks.cephadm:Adding mon.a on smithi060 2024-02-26T10:35:24.445 INFO:tasks.cephadm:Adding mon.b on smithi100 2024-02-26T10:35:24.445 INFO:tasks.cephadm:Adding mon.c on smithi113 2024-02-26T10:35:24.446 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch apply mon '3;smithi060:172.21.15.60=a;smithi100:172.21.15.100=b;smithi113:172.21.15.113=c' 2024-02-26T10:35:24.668 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:25.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:24 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1996614657' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-26T10:35:25.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:24 smithi060 ceph-mon[31539]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:25.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:25.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:25.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:26.025 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:25 smithi060 ceph-mon[31539]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:26.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.079+0000 7f55d2433700 1 -- 172.21.15.113:0/465036762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 msgr2=0x7f55cc107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.079+0000 7f55d2433700 1 --2- 172.21.15.113:0/465036762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 0x7f55cc107b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009910 tx=0x7f55bc009c20 comp rx=0 tx=0).stop 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.079+0000 7f55d2433700 1 -- 172.21.15.113:0/465036762 shutdown_connections 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.079+0000 7f55d2433700 1 --2- 172.21.15.113:0/465036762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 0x7f55cc107b80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.079+0000 7f55d2433700 1 -- 172.21.15.113:0/465036762 >> 172.21.15.113:0/465036762 conn(0x7f55cc0fb870 msgr2=0x7f55cc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.079+0000 7f55d2433700 1 -- 172.21.15.113:0/465036762 shutdown_connections 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.079+0000 7f55d2433700 1 -- 172.21.15.113:0/465036762 wait complete. 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.080+0000 7f55d2433700 1 Processor -- start 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.080+0000 7f55d2433700 1 -- start start 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.080+0000 7f55d2433700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 0x7f55cc1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:26.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.080+0000 7f55d2433700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc10bda0 con 0x7f55cc1077a0 2024-02-26T10:35:26.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.081+0000 7f55cbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 0x7f55cc1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:26.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.081+0000 7f55cbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 0x7f55cc1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.113:47740/0 (socket says 172.21.15.113:47740) 2024-02-26T10:35:26.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.081+0000 7f55cbfff700 1 -- 172.21.15.113:0/250673670 learned_addr learned my addr 172.21.15.113:0/250673670 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:35:26.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.082+0000 7f55cbfff700 1 -- 172.21.15.113:0/250673670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc197720 con 0x7f55cc1077a0 2024-02-26T10:35:26.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.082+0000 7f55cbfff700 1 --2- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 0x7f55cc1971e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005850 tx=0x7f55bc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:26.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.082+0000 7f55c97fa700 1 -- 172.21.15.113:0/250673670 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55bc026070 con 0x7f55cc1077a0 2024-02-26T10:35:26.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.082+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc1979b0 con 0x7f55cc1077a0 2024-02-26T10:35:26.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.082+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc19bab0 con 0x7f55cc1077a0 2024-02-26T10:35:26.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.082+0000 7f55c97fa700 1 -- 172.21.15.113:0/250673670 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55bc00b750 con 0x7f55cc1077a0 2024-02-26T10:35:26.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.082+0000 7f55c97fa700 1 -- 172.21.15.113:0/250673670 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55bc030060 con 0x7f55cc1077a0 2024-02-26T10:35:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.083+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b8005330 con 0x7f55cc1077a0 2024-02-26T10:35:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.084+0000 7f55c97fa700 1 -- 172.21.15.113:0/250673670 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f55bc029070 con 0x7f55cc1077a0 2024-02-26T10:35:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.084+0000 7f55c97fa700 1 --2- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f55b4038bf0 0x7f55b403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.084+0000 7f55c97fa700 1 -- 172.21.15.113:0/250673670 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f55bc01e070 con 0x7f55cc1077a0 2024-02-26T10:35:26.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.085+0000 7f55cb7fe700 1 --2- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f55b4038bf0 0x7f55b403b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:26.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.086+0000 7f55cb7fe700 1 --2- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f55b4038bf0 0x7f55b403b0b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f55c0006fd0 tx=0x7f55c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:26.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.100+0000 7f55c97fa700 1 -- 172.21.15.113:0/250673670 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55bc013070 con 0x7f55cc1077a0 2024-02-26T10:35:26.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.300+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi100:172.21.15.100=b;smithi113:172.21.15.113=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f55b8000c80 con 0x7f55b4038bf0 2024-02-26T10:35:26.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.315+0000 7f55c97fa700 1 -- 172.21.15.113:0/250673670 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f55b8000c80 con 0x7f55b4038bf0 2024-02-26T10:35:26.318 INFO:teuthology.orchestra.run.smithi113.stdout:Scheduled mon update... 2024-02-26T10:35:26.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f55b4038bf0 msgr2=0x7f55b403b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:26.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 --2- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f55b4038bf0 0x7f55b403b0b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f55c0006fd0 tx=0x7f55c0006e40 comp rx=0 tx=0).stop 2024-02-26T10:35:26.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 msgr2=0x7f55cc1971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:26.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 --2- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 0x7f55cc1971e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005850 tx=0x7f55bc005030 comp rx=0 tx=0).stop 2024-02-26T10:35:26.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 shutdown_connections 2024-02-26T10:35:26.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 --2- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f55b4038bf0 0x7f55b403b0b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:26.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 --2- 172.21.15.113:0/250673670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55cc1077a0 0x7f55cc1971e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:26.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 >> 172.21.15.113:0/250673670 conn(0x7f55cc0fb870 msgr2=0x7f55cc0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:26.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 shutdown_connections 2024-02-26T10:35:26.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:26.317+0000 7f55d2433700 1 -- 172.21.15.113:0/250673670 wait complete. 2024-02-26T10:35:26.976 DEBUG:teuthology.orchestra.run.smithi100:mon.b> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.b.service 2024-02-26T10:35:26.979 DEBUG:teuthology.orchestra.run.smithi113:mon.c> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.c.service 2024-02-26T10:35:26.981 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T10:35:26.981 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T10:35:27.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi100:172.21.15.100=b;smithi113:172.21.15.113=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:35:27.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi100:172.21.15.100=b;smithi113:172.21.15.113=c;count:3 2024-02-26T10:35:27.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:27.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:27.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:27.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:35:27.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:27.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:27.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:27.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:27 smithi060 ceph-mon[31539]: Deploying daemon mon.c on smithi113 2024-02-26T10:35:27.708 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:28.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:28 smithi060 ceph-mon[31539]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.652+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1448758305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 msgr2=0x7f0dd0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.652+0000 7f0dd5d95700 1 --2- 172.21.15.113:0/1448758305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 0x7f0dd0106a60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009910 tx=0x7f0dc0009c20 comp rx=0 tx=0).stop 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.652+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1448758305 shutdown_connections 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.652+0000 7f0dd5d95700 1 --2- 172.21.15.113:0/1448758305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 0x7f0dd0106a60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.652+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1448758305 >> 172.21.15.113:0/1448758305 conn(0x7f0dd0077250 msgr2=0x7f0dd00fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.652+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1448758305 shutdown_connections 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.652+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1448758305 wait complete. 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.653+0000 7f0dd5d95700 1 Processor -- start 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.653+0000 7f0dd5d95700 1 -- start start 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.653+0000 7f0dd5d95700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 0x7f0dd0194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.653+0000 7f0dd5d95700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0109b30 con 0x7f0dd0106680 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.654+0000 7f0dcf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 0x7f0dd0194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.654+0000 7f0dcf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 0x7f0dd0194f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.113:45280/0 (socket says 172.21.15.113:45280) 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.654+0000 7f0dcf7fe700 1 -- 172.21.15.113:0/1117972853 learned_addr learned my addr 172.21.15.113:0/1117972853 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.654+0000 7f0dcf7fe700 1 -- 172.21.15.113:0/1117972853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd01954b0 con 0x7f0dd0106680 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.655+0000 7f0dcf7fe700 1 --2- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 0x7f0dd0194f70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0004f80 tx=0x7f0dc0005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.655+0000 7f0dcd7fa700 1 -- 172.21.15.113:0/1117972853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0dc0026070 con 0x7f0dd0106680 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.655+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd01957a0 con 0x7f0dd0106680 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.655+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd0199840 con 0x7f0dd0106680 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.655+0000 7f0dcd7fa700 1 -- 172.21.15.113:0/1117972853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0dc0004740 con 0x7f0dd0106680 2024-02-26T10:35:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.655+0000 7f0dcd7fa700 1 -- 172.21.15.113:0/1117972853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0dc0030060 con 0x7f0dd0106680 2024-02-26T10:35:29.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.656+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db0005330 con 0x7f0dd0106680 2024-02-26T10:35:29.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.657+0000 7f0dcd7fa700 1 -- 172.21.15.113:0/1117972853 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f0dc0029070 con 0x7f0dd0106680 2024-02-26T10:35:29.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.657+0000 7f0dcd7fa700 1 --2- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dbc03ae90 0x7f0dbc03d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:29.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.657+0000 7f0dcd7fa700 1 -- 172.21.15.113:0/1117972853 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0dc001e070 con 0x7f0dd0106680 2024-02-26T10:35:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.658+0000 7f0dc6ffd700 1 --2- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dbc03ae90 0x7f0dbc03d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.659+0000 7f0dc6ffd700 1 --2- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dbc03ae90 0x7f0dbc03d350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0db8006fd0 tx=0x7f0db8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:29.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.671+0000 7f0dcd7fa700 1 -- 172.21.15.113:0/1117972853 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0dc0013070 con 0x7f0dd0106680 2024-02-26T10:35:29.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.950+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0db0005620 con 0x7f0dd0106680 2024-02-26T10:35:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.951+0000 7f0dcd7fa700 1 -- 172.21.15.113:0/1117972853 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f0dc0021830 con 0x7f0dd0106680 2024-02-26T10:35:29.955 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:35:29.955 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":1,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","modified":"2024-02-26T10:33:24.659423Z","created":"2024-02-26T10:33:24.659423Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-26T10:35:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dbc03ae90 msgr2=0x7f0dbc03d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 --2- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dbc03ae90 0x7f0dbc03d350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0db8006fd0 tx=0x7f0db8006e40 comp rx=0 tx=0).stop 2024-02-26T10:35:29.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 msgr2=0x7f0dd0194f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 --2- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 0x7f0dd0194f70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0004f80 tx=0x7f0dc0005060 comp rx=0 tx=0).stop 2024-02-26T10:35:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 shutdown_connections 2024-02-26T10:35:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 --2- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dbc03ae90 0x7f0dbc03d350 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 --2- 172.21.15.113:0/1117972853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dd0106680 0x7f0dd0194f70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 >> 172.21.15.113:0/1117972853 conn(0x7f0dd0077250 msgr2=0x7f0dd00fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 shutdown_connections 2024-02-26T10:35:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:29.953+0000 7f0dd5d95700 1 -- 172.21.15.113:0/1117972853 wait complete. 2024-02-26T10:35:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:dumped monmap epoch 1 2024-02-26T10:35:30.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:30 smithi060 ceph-mon[31539]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:30.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:30 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/1117972853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T10:35:31.538 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T10:35:31.538 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T10:35:31.854 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:35:32.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:32 smithi060 ceph-mon[31539]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:33.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.636+0000 7ff75f3de700 1 -- 172.21.15.113:0/2212399060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 msgr2=0x7ff758107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.636+0000 7ff75f3de700 1 --2- 172.21.15.113:0/2212399060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 0x7ff758107d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff754009910 tx=0x7ff754009c20 comp rx=0 tx=0).stop 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.636+0000 7ff75f3de700 1 -- 172.21.15.113:0/2212399060 shutdown_connections 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.636+0000 7ff75f3de700 1 --2- 172.21.15.113:0/2212399060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 0x7ff758107d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.636+0000 7ff75f3de700 1 -- 172.21.15.113:0/2212399060 >> 172.21.15.113:0/2212399060 conn(0x7ff758076d70 msgr2=0x7ff758077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.636+0000 7ff75f3de700 1 -- 172.21.15.113:0/2212399060 shutdown_connections 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.636+0000 7ff75f3de700 1 -- 172.21.15.113:0/2212399060 wait complete. 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.637+0000 7ff75f3de700 1 Processor -- start 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.637+0000 7ff75f3de700 1 -- start start 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.637+0000 7ff75f3de700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 0x7ff758197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.637+0000 7ff75f3de700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75810bf80 con 0x7ff7581079a0 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.638+0000 7ff75d17a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 0x7ff758197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:33.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.638+0000 7ff75d17a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 0x7ff758197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.113:45300/0 (socket says 172.21.15.113:45300) 2024-02-26T10:35:33.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.638+0000 7ff75d17a700 1 -- 172.21.15.113:0/523191667 learned_addr learned my addr 172.21.15.113:0/523191667 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:35:33.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.638+0000 7ff75d17a700 1 -- 172.21.15.113:0/523191667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff758197880 con 0x7ff7581079a0 2024-02-26T10:35:33.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.639+0000 7ff75d17a700 1 --2- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 0x7ff758197340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff754006010 tx=0x7ff754005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:33.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.639+0000 7ff74e7fc700 1 -- 172.21.15.113:0/523191667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff754026070 con 0x7ff7581079a0 2024-02-26T10:35:33.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.639+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff758197b10 con 0x7ff7581079a0 2024-02-26T10:35:33.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.639+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff758197f70 con 0x7ff7581079a0 2024-02-26T10:35:33.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.639+0000 7ff74e7fc700 1 -- 172.21.15.113:0/523191667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff75400bc30 con 0x7ff7581079a0 2024-02-26T10:35:33.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.639+0000 7ff74e7fc700 1 -- 172.21.15.113:0/523191667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff754003950 con 0x7ff7581079a0 2024-02-26T10:35:33.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.640+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c005330 con 0x7ff7581079a0 2024-02-26T10:35:33.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.640+0000 7ff74e7fc700 1 -- 172.21.15.113:0/523191667 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7ff754029070 con 0x7ff7581079a0 2024-02-26T10:35:33.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.641+0000 7ff74e7fc700 1 --2- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff744038ca0 0x7ff74403b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:33.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.641+0000 7ff74e7fc700 1 -- 172.21.15.113:0/523191667 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff75401e070 con 0x7ff7581079a0 2024-02-26T10:35:33.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.641+0000 7ff75c979700 1 --2- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff744038ca0 0x7ff74403b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:33.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.642+0000 7ff75c979700 1 --2- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff744038ca0 0x7ff74403b160 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff748006fd0 tx=0x7ff748006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:33.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.656+0000 7ff74e7fc700 1 -- 172.21.15.113:0/523191667 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff754013070 con 0x7ff7581079a0 2024-02-26T10:35:33.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.790+0000 7ff74e7fc700 1 -- 172.21.15.113:0/523191667 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ff754021410 con 0x7ff7581079a0 2024-02-26T10:35:33.794 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:33 smithi113 ceph-mon[36126]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-26T10:35:33.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:33.936+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff73c0058f0 con 0x7ff7581079a0 2024-02-26T10:35:34.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:33 smithi113 bash[36059]: 800ead2bfef6bd25db134bbab8cad5d48ea37cdeaace6c3f9ec5184f9d7a9af9 2024-02-26T10:35:34.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:33 smithi113 systemd[1]: Started Ceph mon.c for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:35:38.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.826+0000 7ff74e7fc700 1 -- 172.21.15.113:0/523191667 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7ff75400e3e0 con 0x7ff7581079a0 2024-02-26T10:35:38.831 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:35:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":2,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","modified":"2024-02-26T10:35:33.789753Z","created":"2024-02-26T10:33:24.659423Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T10:35:38.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff744038ca0 msgr2=0x7ff74403b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:38.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 --2- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff744038ca0 0x7ff74403b160 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff748006fd0 tx=0x7ff748006e40 comp rx=0 tx=0).stop 2024-02-26T10:35:38.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 msgr2=0x7ff758197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:38.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 --2- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 0x7ff758197340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff754006010 tx=0x7ff754005030 comp rx=0 tx=0).stop 2024-02-26T10:35:38.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 shutdown_connections 2024-02-26T10:35:38.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 --2- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff744038ca0 0x7ff74403b160 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 --2- 172.21.15.113:0/523191667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7581079a0 0x7ff758197340 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 >> 172.21.15.113:0/523191667 conn(0x7ff758076d70 msgr2=0x7ff7580fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 shutdown_connections 2024-02-26T10:35:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:38.829+0000 7ff75f3de700 1 -- 172.21.15.113:0/523191667 wait complete. 2024-02-26T10:35:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:dumped monmap epoch 2 2024-02-26T10:35:39.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: mon.a calling monitor election 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='client.? 172.21.15.113:0/523191667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: mon.c calling monitor election 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: monmap e2: 2 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: fsmap 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: mgrmap e13: x(active, since 62s) 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: overall HEALTH_OK 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:39.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:39.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:35:39.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: mon.a calling monitor election 2024-02-26T10:35:39.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/523191667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: mon.c calling monitor election 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: monmap e2: 2 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: fsmap 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:39.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: mgrmap e13: x(active, since 62s) 2024-02-26T10:35:39.376 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: overall HEALTH_OK 2024-02-26T10:35:39.376 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:39.376 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:39.376 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:39.376 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:39.376 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:40.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:39 smithi113 ceph-mon[36126]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:40.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:39 smithi113 ceph-mon[36126]: Deploying daemon mon.b on smithi100 2024-02-26T10:35:40.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:39 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:40.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:39 smithi060 ceph-mon[31539]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:40.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:39 smithi060 ceph-mon[31539]: Deploying daemon mon.b on smithi100 2024-02-26T10:35:40.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:39 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:40.420 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T10:35:40.420 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T10:35:40.663 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:35:42.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.048+0000 7f6d1913f700 1 -- 172.21.15.113:0/2118102588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d14107830 msgr2=0x7f6d14107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.049+0000 7f6d1913f700 1 --2- 172.21.15.113:0/2118102588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d14107830 0x7f6d14107c10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00b0a0 tx=0x7f6d0c00b3b0 comp rx=0 tx=0).stop 2024-02-26T10:35:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.049+0000 7f6d1913f700 1 -- 172.21.15.113:0/2118102588 shutdown_connections 2024-02-26T10:35:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.049+0000 7f6d1913f700 1 --2- 172.21.15.113:0/2118102588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d14107830 0x7f6d14107c10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.049+0000 7f6d1913f700 1 -- 172.21.15.113:0/2118102588 >> 172.21.15.113:0/2118102588 conn(0x7f6d140fd320 msgr2=0x7f6d140ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.049+0000 7f6d1913f700 1 -- 172.21.15.113:0/2118102588 shutdown_connections 2024-02-26T10:35:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.049+0000 7f6d1913f700 1 -- 172.21.15.113:0/2118102588 wait complete. 2024-02-26T10:35:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.049+0000 7f6d1913f700 1 Processor -- start 2024-02-26T10:35:42.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d1913f700 1 -- start start 2024-02-26T10:35:42.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d1913f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 0x7f6d14197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d1913f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d14198030 0x7f6d1419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d1913f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1410be70 con 0x7f6d14198030 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d1913f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1410bcf0 con 0x7f6d14197710 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d12d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 0x7f6d14197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d12d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 0x7f6d14197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36566/0 (socket says 172.21.15.113:36566) 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d12d9d700 1 -- 172.21.15.113:0/1363556312 learned_addr learned my addr 172.21.15.113:0/1363556312 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d12d9d700 1 -- 172.21.15.113:0/1363556312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 msgr2=0x7f6d14197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d12d9d700 1 -- 172.21.15.113:0/1363556312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 msgr2=0x7f6d14197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d12d9d700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 0x7f6d14197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d12d9d700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 0x7f6d14197af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.050+0000 7f6d1259c700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d14198030 0x7f6d1419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.051+0000 7f6d1259c700 1 -- 172.21.15.113:0/1363556312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 msgr2=0x7f6d14197af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T10:35:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.051+0000 7f6d1259c700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 0x7f6d14197af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.051+0000 7f6d1259c700 1 -- 172.21.15.113:0/1363556312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1419c5a0 con 0x7f6d14198030 2024-02-26T10:35:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.052+0000 7f6d1259c700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d14198030 0x7f6d1419bfe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b4d0 tx=0x7f6d0800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.052+0000 7f6cfbfff700 1 -- 172.21.15.113:0/1363556312 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6d08011630 con 0x7f6d14198030 2024-02-26T10:35:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.052+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1419c890 con 0x7f6d14198030 2024-02-26T10:35:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.052+0000 7f6cfbfff700 1 -- 172.21.15.113:0/1363556312 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d08011cb0 con 0x7f6d14198030 2024-02-26T10:35:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.052+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1419ce20 con 0x7f6d14198030 2024-02-26T10:35:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.052+0000 7f6cfbfff700 1 -- 172.21.15.113:0/1363556312 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6d08010e60 con 0x7f6d14198030 2024-02-26T10:35:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.053+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d14102f70 con 0x7f6d14198030 2024-02-26T10:35:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.053+0000 7f6cfbfff700 1 -- 172.21.15.113:0/1363556312 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f6d08018020 con 0x7f6d14198030 2024-02-26T10:35:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.054+0000 7f6cfbfff700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6cfc038990 0x7f6cfc03ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.054+0000 7f6cfbfff700 1 -- 172.21.15.113:0/1363556312 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d0804b970 con 0x7f6d14198030 2024-02-26T10:35:42.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.054+0000 7f6d12d9d700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6cfc038990 0x7f6cfc03ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:42.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.055+0000 7f6d12d9d700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6cfc038990 0x7f6cfc03ae50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009fd0 tx=0x7f6d0c00b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:42.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.069+0000 7f6cfbfff700 1 -- 172.21.15.113:0/1363556312 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6d0801f020 con 0x7f6d14198030 2024-02-26T10:35:42.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:41 smithi113 ceph-mon[36126]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.341+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6d141a23f0 con 0x7f6d14198030 2024-02-26T10:35:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.342+0000 7f6cfbfff700 1 -- 172.21.15.113:0/1363556312 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f6d08010510 con 0x7f6d14198030 2024-02-26T10:35:42.346 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:35:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":2,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","modified":"2024-02-26T10:35:33.789753Z","created":"2024-02-26T10:33:24.659423Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T10:35:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6cfc038990 msgr2=0x7f6cfc03ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6cfc038990 0x7f6cfc03ae50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009fd0 tx=0x7f6d0c00b7f0 comp rx=0 tx=0).stop 2024-02-26T10:35:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d14198030 msgr2=0x7f6d1419bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d14198030 0x7f6d1419bfe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b4d0 tx=0x7f6d0800b890 comp rx=0 tx=0).stop 2024-02-26T10:35:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 shutdown_connections 2024-02-26T10:35:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6cfc038990 0x7f6cfc03ae50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d14197710 0x7f6d14197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 --2- 172.21.15.113:0/1363556312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d14198030 0x7f6d1419bfe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 >> 172.21.15.113:0/1363556312 conn(0x7f6d140fd320 msgr2=0x7f6d14109460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 shutdown_connections 2024-02-26T10:35:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:42.344+0000 7f6d1913f700 1 -- 172.21.15.113:0/1363556312 wait complete. 2024-02-26T10:35:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:dumped monmap epoch 2 2024-02-26T10:35:42.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 systemd[1]: Starting Ceph mon.b for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:35:42.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:41 smithi060 ceph-mon[31539]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 podman[36032]: 2024-02-26 10:35:42.530328552 +0000 UTC m=+0.107821036 container create 2aaee5209894be7a125093914b4ad9ebc3bd1fe85273ad8a3c63455225a4bf20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True) 2024-02-26T10:35:42.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 podman[36032]: 2024-02-26 10:35:42.440099476 +0000 UTC m=+0.017591970 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:35:42.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 podman[36032]: 2024-02-26 10:35:42.724129322 +0000 UTC m=+0.301621810 container init 2aaee5209894be7a125093914b4ad9ebc3bd1fe85273ad8a3c63455225a4bf20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:35:42.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 podman[36032]: 2024-02-26 10:35:42.727178848 +0000 UTC m=+0.304671333 container start 2aaee5209894be7a125093914b4ad9ebc3bd1fe85273ad8a3c63455225a4bf20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-26T10:35:42.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: set uid:gid to 167:167 (ceph:ceph) 2024-02-26T10:35:42.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: ceph version 19.0.0-1686-g6085b91f (6085b91fefeb74bf4310344a87d81d3217ef7893) squid (dev), process ceph-mon, pid 2 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pidfile_write: ignore empty --pid-file 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: load: jerasure load: lrc 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: RocksDB version: 7.9.2 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Git sha 0 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Compile date 2024-02-26 07:04:49 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: DB SUMMARY 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: DB Session ID: S0AV9KVA968M1S70TDWF 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: CURRENT file: CURRENT 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: IDENTITY file: IDENTITY 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.error_if_exists: 0 2024-02-26T10:35:42.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.create_if_missing: 0 2024-02-26T10:35:42.901 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.paranoid_checks: 1 2024-02-26T10:35:42.901 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-26T10:35:42.901 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-26T10:35:42.901 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-26T10:35:42.901 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.env: 0x5612fb472b20 2024-02-26T10:35:42.901 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.fs: PosixFileSystem 2024-02-26T10:35:42.901 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.info_log: 0x5612fc5538e0 2024-02-26T10:35:42.901 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_file_opening_threads: 16 2024-02-26T10:35:42.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.statistics: (nil) 2024-02-26T10:35:42.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.use_fsync: 0 2024-02-26T10:35:42.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_log_file_size: 0 2024-02-26T10:35:42.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-26T10:35:42.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-26T10:35:42.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.keep_log_file_num: 1000 2024-02-26T10:35:42.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.recycle_log_file_num: 0 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.allow_fallocate: 1 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.allow_mmap_reads: 0 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.allow_mmap_writes: 0 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.use_direct_reads: 0 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.create_missing_column_families: 0 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.db_log_dir: 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.wal_dir: 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-26T10:35:42.903 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.advise_random_on_open: 1 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.db_write_buffer_size: 0 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.write_buffer_manager: 0x5612fc527400 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.rate_limiter: (nil) 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.wal_recovery_mode: 2 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.enable_thread_tracking: 0 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.enable_pipelined_write: 0 2024-02-26T10:35:42.904 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.unordered_write: 0 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.row_cache: None 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.wal_filter: None 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.allow_ingest_behind: 0 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.two_write_queues: 0 2024-02-26T10:35:42.905 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.manual_wal_flush: 0 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.wal_compression: 0 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.atomic_flush: 0 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.log_readahead_size: 0 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.best_efforts_recovery: 0 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.allow_data_in_errors: 0 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.db_host_id: __hostname__ 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-26T10:35:42.906 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_background_jobs: 2 2024-02-26T10:35:42.907 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_background_compactions: -1 2024-02-26T10:35:42.907 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_subcompactions: 1 2024-02-26T10:35:42.907 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-26T10:35:42.907 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-26T10:35:42.907 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-26T10:35:42.907 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_total_wal_size: 0 2024-02-26T10:35:42.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-26T10:35:42.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-26T10:35:42.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-26T10:35:42.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-26T10:35:42.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_open_files: -1 2024-02-26T10:35:42.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bytes_per_sync: 0 2024-02-26T10:35:42.909 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-26T10:35:42.909 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-26T10:35:42.909 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_readahead_size: 0 2024-02-26T10:35:42.909 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_background_flushes: -1 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Compression algorithms supported: 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: kZSTD supported: 0 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: kXpressCompression supported: 0 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: kLZ4HCCompression supported: 1 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: kZlibCompression supported: 1 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: kSnappyCompression supported: 1 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: kLZ4Compression supported: 1 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: kBZip2Compression supported: 0 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-26T10:35:42.910 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.merge_operator: 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_filter: None 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_filter_factory: None 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.sst_partitioner_factory: None 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5612fc5539e0) 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout: index_type: 0 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout: data_block_index_type: 0 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout: index_shortening: 1 2024-02-26T10:35:42.911 INFO:journalctl@ceph.mon.b.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: checksum: 4 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: no_block_cache: 0 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache: 0x5612fc557090 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache_options: 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: capacity : 536870912 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: num_shard_bits : 4 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: strict_capacity_limit : 0 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache_compressed: (nil) 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: persistent_cache: (nil) 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_size: 4096 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_size_deviation: 10 2024-02-26T10:35:42.912 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_restart_interval: 16 2024-02-26T10:35:42.913 INFO:journalctl@ceph.mon.b.smithi100.stdout: index_block_restart_interval: 1 2024-02-26T10:35:42.913 INFO:journalctl@ceph.mon.b.smithi100.stdout: metadata_block_size: 4096 2024-02-26T10:35:42.913 INFO:journalctl@ceph.mon.b.smithi100.stdout: partition_filters: 0 2024-02-26T10:35:42.913 INFO:journalctl@ceph.mon.b.smithi100.stdout: use_delta_encoding: 1 2024-02-26T10:35:42.913 INFO:journalctl@ceph.mon.b.smithi100.stdout: filter_policy: bloomfilter 2024-02-26T10:35:42.913 INFO:journalctl@ceph.mon.b.smithi100.stdout: whole_key_filtering: 1 2024-02-26T10:35:42.914 INFO:journalctl@ceph.mon.b.smithi100.stdout: verify_compression: 0 2024-02-26T10:35:42.914 INFO:journalctl@ceph.mon.b.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-02-26T10:35:42.914 INFO:journalctl@ceph.mon.b.smithi100.stdout: format_version: 5 2024-02-26T10:35:42.915 INFO:journalctl@ceph.mon.b.smithi100.stdout: enable_index_compression: 1 2024-02-26T10:35:42.915 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_align: 0 2024-02-26T10:35:42.915 INFO:journalctl@ceph.mon.b.smithi100.stdout: max_auto_readahead_size: 262144 2024-02-26T10:35:42.915 INFO:journalctl@ceph.mon.b.smithi100.stdout: prepopulate_block_cache: 0 2024-02-26T10:35:42.915 INFO:journalctl@ceph.mon.b.smithi100.stdout: initial_auto_readahead_size: 8192 2024-02-26T10:35:42.915 INFO:journalctl@ceph.mon.b.smithi100.stdout: num_file_reads_for_auto_readahead: 2 2024-02-26T10:35:42.915 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.write_buffer_size: 33554432 2024-02-26T10:35:42.916 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_write_buffer_number: 2 2024-02-26T10:35:42.916 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression: NoCompression 2024-02-26T10:35:42.916 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression: Disabled 2024-02-26T10:35:42.916 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.prefix_extractor: nullptr 2024-02-26T10:35:42.916 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-26T10:35:42.916 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.num_levels: 7 2024-02-26T10:35:42.917 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-26T10:35:42.917 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-26T10:35:42.917 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-26T10:35:42.917 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-26T10:35:42.917 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-26T10:35:42.918 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-26T10:35:42.918 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-26T10:35:42.918 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-26T10:35:42.918 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-26T10:35:42.918 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-26T10:35:42.919 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-26T10:35:42.919 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-26T10:35:42.919 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-26T10:35:42.919 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression_opts.level: 32767 2024-02-26T10:35:42.919 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression_opts.strategy: 0 2024-02-26T10:35:42.919 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression_opts.enabled: false 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.target_file_size_base: 67108864 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-26T10:35:42.920 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-26T10:35:42.921 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.arena_block_size: 1048576 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.disable_auto_compactions: 0 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.table_properties_collectors: 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.inplace_update_support: 0 2024-02-26T10:35:42.922 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.bloom_locality: 0 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.max_successive_merges: 0 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.paranoid_file_checks: 0 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.force_consistency_checks: 1 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.report_bg_io_stats: 0 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.ttl: 2592000 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-26T10:35:42.923 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-26T10:35:42.925 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.enable_blob_files: false 2024-02-26T10:35:42.925 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.min_blob_size: 0 2024-02-26T10:35:42.926 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.blob_file_size: 268435456 2024-02-26T10:35:42.926 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-26T10:35:42.926 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-26T10:35:42.926 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-26T10:35:42.926 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-26T10:35:42.926 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-26T10:35:42.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.blob_file_starting_level: 0 2024-02-26T10:35:42.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-26T10:35:42.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-26T10:35:42.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-26T10:35:42.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d04adfb5-c614-4ff5-b65b-41e727ff4843 2024-02-26T10:35:42.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708943742756812, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708943742757227, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708943742, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d04adfb5-c614-4ff5-b65b-41e727ff4843", "db_session_id": "S0AV9KVA968M1S70TDWF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708943742757307, "job": 1, "event": "recovery_finished"} 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5612fc648000 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: DB pointer 0x5612fc632000 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** DB Stats ** 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T10:35:42.928 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** Compaction Stats [default] ** 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-26T10:35:42.929 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** Compaction Stats [default] ** 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-26T10:35:42.930 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout: Block cache BinnedLRUCache@0x5612fc557090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: using public_addr v2:172.21.15.100:0/0 -> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: starting mon.b rank -1 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(???) e0 preinit fsid 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).mds e1 new map 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-26T10:35:42.931 INFO:journalctl@ceph.mon.b.smithi100.stdout: e1 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout: legacy client fscid: -1 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout: No filesystems configured 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T10:35:42.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Added host smithi100 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Updating smithi100:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Updating smithi100:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.934 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi113", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Deploying cephadm binary to smithi113 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Added host smithi113 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.935 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi113", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Updating smithi113:/etc/ceph/ceph.conf 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Updating smithi113:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1996614657' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Updating smithi113:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1996614657' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-26T10:35:42.936 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi100:172.21.15.100=b;smithi113:172.21.15.113=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi100:172.21.15.100=b;smithi113:172.21.15.113=c;count:3 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:42.937 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Deploying daemon mon.c on smithi113 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/1117972853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.a calling monitor election 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/523191667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.c calling monitor election 2024-02-26T10:35:42.938 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: monmap e2: 2 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: fsmap 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mgrmap e13: x(active, since 62s) 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: overall HEALTH_OK 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:42.939 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:42.940 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.940 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: Deploying daemon mon.b on smithi100 2024-02-26T10:35:42.940 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:42.940 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:42.940 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 ceph-mon[36051]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-26T10:35:42.940 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 bash[36032]: 2aaee5209894be7a125093914b4ad9ebc3bd1fe85273ad8a3c63455225a4bf20 2024-02-26T10:35:42.940 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:42 smithi100 systemd[1]: Started Ceph mon.b for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:35:43.960 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T10:35:43.960 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T10:35:44.203 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:35:48.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:48.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: mon.c calling monitor election 2024-02-26T10:35:48.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: mon.a calling monitor election 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:35:48.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: fsmap 2024-02-26T10:35:48.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:48.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: mgrmap e13: x(active, since 71s) 2024-02-26T10:35:48.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T10:35:48.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T10:35:48.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T10:35:48.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-02-26T10:35:48.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:48.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:48.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: mon.c calling monitor election 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: mon.a calling monitor election 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: fsmap 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: mgrmap e13: x(active, since 71s) 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:48.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:48.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.765+0000 7f927e7fc700 1 -- 172.21.15.113:0/2491218186 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92740116d0 con 0x7f92680035c0 2024-02-26T10:35:48.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.766+0000 7f9285cf3700 1 -- 172.21.15.113:0/2491218186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92680035c0 msgr2=0x7f9268005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:48.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.766+0000 7f9285cf3700 1 --2- 172.21.15.113:0/2491218186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92680035c0 0x7f9268005a50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9280078fa0 tx=0x7f927400feb0 comp rx=0 tx=0).stop 2024-02-26T10:35:48.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.766+0000 7f9285cf3700 1 -- 172.21.15.113:0/2491218186 shutdown_connections 2024-02-26T10:35:48.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.766+0000 7f9285cf3700 1 --2- 172.21.15.113:0/2491218186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92680035c0 0x7f9268005a50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:48.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.766+0000 7f9285cf3700 1 -- 172.21.15.113:0/2491218186 >> 172.21.15.113:0/2491218186 conn(0x7f9280076e40 msgr2=0x7f9280077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:48.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.766+0000 7f9285cf3700 1 -- 172.21.15.113:0/2491218186 shutdown_connections 2024-02-26T10:35:48.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.766+0000 7f9285cf3700 1 -- 172.21.15.113:0/2491218186 wait complete. 2024-02-26T10:35:48.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f9285cf3700 1 Processor -- start 2024-02-26T10:35:48.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f9285cf3700 1 -- start start 2024-02-26T10:35:48.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f9285cf3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92680035c0 0x7f92801a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:48.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f9285cf3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92801a3c00 0x7f928019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:48.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f9285cf3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928019ddc0 0x7f928019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:48.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f9285cf3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928010ff70 con 0x7f92680035c0 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f9285cf3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928010fdf0 con 0x7f92801a3c00 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f9285cf3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92801100f0 con 0x7f928019ddc0 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f927ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928019ddc0 0x7f928019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f927ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928019ddc0 0x7f928019e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50734/0 (socket says 172.21.15.113:50734) 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f927ffff700 1 -- 172.21.15.113:0/1244349889 learned_addr learned my addr 172.21.15.113:0/1244349889 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.767+0000 7f927effd700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92801a3c00 0x7f928019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927effd700 1 -- 172.21.15.113:0/1244349889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92801a3c00 msgr2=0x7f928019d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927effd700 1 -- 172.21.15.113:0/1244349889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92801a3c00 msgr2=0x7f928019d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927effd700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92801a3c00 0x7f928019d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927effd700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92801a3c00 0x7f928019d880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927f7fe700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92680035c0 0x7f92801a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:48.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927ffff700 1 -- 172.21.15.113:0/1244349889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92801a3c00 msgr2=0x7f928019d880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927ffff700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92801a3c00 0x7f928019d880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927ffff700 1 -- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92680035c0 msgr2=0x7f92801a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927ffff700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92680035c0 0x7f92801a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927ffff700 1 -- 172.21.15.113:0/1244349889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92801aa590 con 0x7f928019ddc0 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927ffff700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928019ddc0 0x7f928019e270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f927800ba50 tx=0x7f927800be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927cff9700 1 -- 172.21.15.113:0/1244349889 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278019070 con 0x7f928019ddc0 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927cff9700 1 -- 172.21.15.113:0/1244349889 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9278014440 con 0x7f928019ddc0 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92801aa7c0 con 0x7f928019ddc0 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f927cff9700 1 -- 172.21.15.113:0/1244349889 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278007590 con 0x7f928019ddc0 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.768+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92801aad00 con 0x7f928019ddc0 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.769+0000 7f927cff9700 1 -- 172.21.15.113:0/1244349889 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f927801c070 con 0x7f928019ddc0 2024-02-26T10:35:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.769+0000 7f927cff9700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f926c0390d0 0x7f926c03b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:48.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.769+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9280063380 con 0x7f928019ddc0 2024-02-26T10:35:48.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.769+0000 7f927cff9700 1 -- 172.21.15.113:0/1244349889 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f927804b200 con 0x7f928019ddc0 2024-02-26T10:35:48.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.770+0000 7f927f7fe700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f926c0390d0 0x7f926c03b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:48.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.771+0000 7f927f7fe700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f926c0390d0 0x7f926c03b590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f927400f8d0 tx=0x7f9274005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:48.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:48.775+0000 7f927cff9700 1 -- 172.21.15.113:0/1244349889 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9278014910 con 0x7f928019ddc0 2024-02-26T10:35:49.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.058+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f92801aaff0 con 0x7f928019ddc0 2024-02-26T10:35:49.063 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:49.063 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:49.063 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:49.063 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:49.063 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-02-26T10:35:49.063 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":3,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","modified":"2024-02-26T10:35:42.779510Z","created":"2024-02-26T10:33:24.659423Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T10:35:49.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.058+0000 7f927cff9700 1 -- 172.21.15.113:0/1244349889 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7f92780145f0 con 0x7f928019ddc0 2024-02-26T10:35:49.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f926c0390d0 msgr2=0x7f926c03b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:49.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f926c0390d0 0x7f926c03b590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f927400f8d0 tx=0x7f9274005a90 comp rx=0 tx=0).stop 2024-02-26T10:35:49.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928019ddc0 msgr2=0x7f928019e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928019ddc0 0x7f928019e270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f927800ba50 tx=0x7f927800be10 comp rx=0 tx=0).stop 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 shutdown_connections 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f926c0390d0 0x7f926c03b590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928019ddc0 0x7f928019e270 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92680035c0 0x7f92801a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 --2- 172.21.15.113:0/1244349889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92801a3c00 0x7f928019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 >> 172.21.15.113:0/1244349889 conn(0x7f9280076e40 msgr2=0x7f92800ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 shutdown_connections 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:49.060+0000 7f9285cf3700 1 -- 172.21.15.113:0/1244349889 wait complete. 2024-02-26T10:35:49.065 INFO:teuthology.orchestra.run.smithi113.stderr:dumped monmap epoch 3 2024-02-26T10:35:49.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:49.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:49.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:49.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: mon.c calling monitor election 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: mon.a calling monitor election 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:50.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: fsmap 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: mgrmap e13: x(active, since 71s) 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-02-26T10:35:50.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:50.138 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:50.138 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:50.138 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:50.138 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:50.138 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:50.156 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-26T10:35:50.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph config generate-minimal-conf 2024-02-26T10:35:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:35:51.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: mon.b calling monitor election 2024-02-26T10:35:51.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: Updating smithi060:/etc/ceph/ceph.conf 2024-02-26T10:35:51.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-26T10:35:51.026 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: Updating smithi113:/etc/ceph/ceph.conf 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: mon.b calling monitor election 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: mon.c calling monitor election 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: mon.a calling monitor election 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: fsmap 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: mgrmap e13: x(active, since 73s) 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: Cluster is now healthy 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: overall HEALTH_OK 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: Updating smithi100:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: Updating smithi113:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:51.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:50 smithi060 ceph-mon[31539]: Updating smithi060:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:51.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: mon.b calling monitor election 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: Updating smithi060:/etc/ceph/ceph.conf 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: Updating smithi113:/etc/ceph/ceph.conf 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: mon.b calling monitor election 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: mon.c calling monitor election 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: mon.a calling monitor election 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: fsmap 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: mgrmap e13: x(active, since 73s) 2024-02-26T10:35:51.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T10:35:51.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: Cluster is now healthy 2024-02-26T10:35:51.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: overall HEALTH_OK 2024-02-26T10:35:51.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: Updating smithi100:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:51.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: Updating smithi113:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:51.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:50 smithi113 ceph-mon[36126]: Updating smithi060:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:51.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: mon.b calling monitor election 2024-02-26T10:35:51.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: Updating smithi060:/etc/ceph/ceph.conf 2024-02-26T10:35:51.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-26T10:35:51.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: Updating smithi113:/etc/ceph/ceph.conf 2024-02-26T10:35:51.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: mon.b calling monitor election 2024-02-26T10:35:51.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: mon.c calling monitor election 2024-02-26T10:35:51.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: mon.a calling monitor election 2024-02-26T10:35:51.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: fsmap 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: mgrmap e13: x(active, since 73s) 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: Cluster is now healthy 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: overall HEALTH_OK 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: Updating smithi100:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: Updating smithi113:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:51.137 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:50 smithi100 ceph-mon[36051]: Updating smithi060:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T10:35:52.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:52.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:52.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:51 smithi060 ceph-mon[31539]: Reconfiguring daemon mon.a on smithi060 2024-02-26T10:35:52.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:52.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:52.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T10:35:52.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:52.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T10:35:52.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:52.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:51 smithi113 ceph-mon[36126]: Reconfiguring daemon mon.a on smithi060 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T10:35:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:52.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T10:35:52.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:52.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:51 smithi100 ceph-mon[36051]: Reconfiguring daemon mon.a on smithi060 2024-02-26T10:35:52.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.550+0000 7f9a44200700 1 -- 172.21.15.60:0/3130534593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c074180 msgr2=0x7f9a3c074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:52.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.550+0000 7f9a44200700 1 --2- 172.21.15.60:0/3130534593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c074180 0x7f9a3c074560 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009910 tx=0x7f9a30009c20 comp rx=0 tx=0).stop 2024-02-26T10:35:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.550+0000 7f9a44200700 1 -- 172.21.15.60:0/3130534593 shutdown_connections 2024-02-26T10:35:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.550+0000 7f9a44200700 1 --2- 172.21.15.60:0/3130534593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c074180 0x7f9a3c074560 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.550+0000 7f9a44200700 1 -- 172.21.15.60:0/3130534593 >> 172.21.15.60:0/3130534593 conn(0x7f9a3c0fd230 msgr2=0x7f9a3c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.550+0000 7f9a44200700 1 -- 172.21.15.60:0/3130534593 shutdown_connections 2024-02-26T10:35:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.550+0000 7f9a44200700 1 -- 172.21.15.60:0/3130534593 wait complete. 2024-02-26T10:35:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.550+0000 7f9a44200700 1 Processor -- start 2024-02-26T10:35:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.550+0000 7f9a44200700 1 -- start start 2024-02-26T10:35:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a44200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a3c074180 0x7f9a3c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a44200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c199c40 0x7f9a3c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a44200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c19e510 0x7f9a3c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a44200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c10dfb0 con 0x7f9a3c199c40 2024-02-26T10:35:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a44200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c10de30 con 0x7f9a3c19e510 2024-02-26T10:35:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a44200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c10e130 con 0x7f9a3c074180 2024-02-26T10:35:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a40e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c199c40 0x7f9a3c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a40e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c199c40 0x7f9a3c19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52518/0 (socket says 172.21.15.60:52518) 2024-02-26T10:35:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a40e00700 1 -- 172.21.15.60:0/3118967420 learned_addr learned my addr 172.21.15.60:0/3118967420 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:35:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a41800700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a3c074180 0x7f9a3c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.551+0000 7f9a42200700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c19e510 0x7f9a3c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a40e00700 1 -- 172.21.15.60:0/3118967420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a3c074180 msgr2=0x7f9a3c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a40e00700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a3c074180 0x7f9a3c199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a40e00700 1 -- 172.21.15.60:0/3118967420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c19e510 msgr2=0x7f9a3c19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a40e00700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c19e510 0x7f9a3c19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a40e00700 1 -- 172.21.15.60:0/3118967420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c1a6280 con 0x7f9a3c199c40 2024-02-26T10:35:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a40e00700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c199c40 0x7f9a3c19dfd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800b780 tx=0x7f9a2800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a3a000700 1 -- 172.21.15.60:0/3118967420 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28016070 con 0x7f9a3c199c40 2024-02-26T10:35:52.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a3a000700 1 -- 172.21.15.60:0/3118967420 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a280036a0 con 0x7f9a3c199c40 2024-02-26T10:35:52.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a3a000700 1 -- 172.21.15.60:0/3118967420 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28011af0 con 0x7f9a3c199c40 2024-02-26T10:35:52.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c1a64b0 con 0x7f9a3c199c40 2024-02-26T10:35:52.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.552+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c1a69a0 con 0x7f9a3c199c40 2024-02-26T10:35:52.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.554+0000 7f9a3a000700 1 -- 172.21.15.60:0/3118967420 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f9a28019070 con 0x7f9a3c199c40 2024-02-26T10:35:52.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.554+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c063380 con 0x7f9a3c199c40 2024-02-26T10:35:52.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.554+0000 7f9a3a000700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9a240390d0 0x7f9a2403b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:52.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.554+0000 7f9a3a000700 1 -- 172.21.15.60:0/3118967420 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a2804a770 con 0x7f9a3c199c40 2024-02-26T10:35:52.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.554+0000 7f9a41800700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9a240390d0 0x7f9a2403b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:52.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.555+0000 7f9a41800700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9a240390d0 0x7f9a2403b590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a30006010 tx=0x7f9a300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:52.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.560+0000 7f9a3a000700 1 -- 172.21.15.60:0/3118967420 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a28010380 con 0x7f9a3c199c40 2024-02-26T10:35:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.792+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9a3c1a6cf0 con 0x7f9a3c199c40 2024-02-26T10:35:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.793+0000 7f9a3a000700 1 -- 172.21.15.60:0/3118967420 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f9a28014120 con 0x7f9a3c199c40 2024-02-26T10:35:52.797 INFO:teuthology.orchestra.run.smithi060.stdout:# minimal ceph.conf for 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:35:52.797 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-02-26T10:35:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:35:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: mon_host = [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] 2024-02-26T10:35:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9a240390d0 msgr2=0x7f9a2403b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9a240390d0 0x7f9a2403b590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a30006010 tx=0x7f9a300058e0 comp rx=0 tx=0).stop 2024-02-26T10:35:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c199c40 msgr2=0x7f9a3c19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c199c40 0x7f9a3c19dfd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800b780 tx=0x7f9a2800ba90 comp rx=0 tx=0).stop 2024-02-26T10:35:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 shutdown_connections 2024-02-26T10:35:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9a240390d0 0x7f9a2403b590 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a3c19e510 0x7f9a3c19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a3c074180 0x7f9a3c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 --2- 172.21.15.60:0/3118967420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a3c199c40 0x7f9a3c19dfd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 >> 172.21.15.60:0/3118967420 conn(0x7f9a3c0fd230 msgr2=0x7f9a3c105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.796+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 shutdown_connections 2024-02-26T10:35:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:35:52.797+0000 7f9a44200700 1 -- 172.21.15.60:0/3118967420 wait complete. 2024-02-26T10:35:53.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:52 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:53.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:52 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3118967420' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:52 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3118967420' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:53.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:52 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:53.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:52 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3118967420' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:53.843 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-26T10:35:53.843 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:35:53.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T10:35:53.942 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:35:53.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:35:54.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:53 smithi060 ceph-mon[31539]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:54.070 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:35:54.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T10:35:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:53 smithi113 ceph-mon[36126]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:54.100 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:53 smithi100 ceph-mon[36051]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:54.105 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:35:54.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:35:54.176 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:35:54.176 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T10:35:54.210 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:35:54.210 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T10:35:54.280 INFO:tasks.cephadm:Adding mgr.x on smithi060 2024-02-26T10:35:54.280 INFO:tasks.cephadm:Adding mgr.y on smithi100 2024-02-26T10:35:54.280 INFO:tasks.cephadm:Adding mgr.z on smithi113 2024-02-26T10:35:54.281 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch apply mgr '3;smithi060=x;smithi100=y;smithi113=z' 2024-02-26T10:35:54.554 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:35:55.808 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:55.808 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:55.808 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:55 smithi113 ceph-mon[36126]: Reconfiguring mon.b (monmap changed)... 2024-02-26T10:35:55.808 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:55.808 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T10:35:55.808 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:55.808 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:55 smithi113 ceph-mon[36126]: Reconfiguring daemon mon.b on smithi100 2024-02-26T10:35:55.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:55.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:55.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:55 smithi060 ceph-mon[31539]: Reconfiguring mon.b (monmap changed)... 2024-02-26T10:35:55.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:55.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T10:35:55.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:55.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:55 smithi060 ceph-mon[31539]: Reconfiguring daemon mon.b on smithi100 2024-02-26T10:35:55.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:55.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:55.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:55 smithi100 ceph-mon[36051]: Reconfiguring mon.b (monmap changed)... 2024-02-26T10:35:55.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:55.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T10:35:55.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:55.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:55 smithi100 ceph-mon[36051]: Reconfiguring daemon mon.b on smithi100 2024-02-26T10:35:56.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.116+0000 7f619b7fe700 1 -- 172.21.15.113:0/3364638717 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61900056d0 con 0x7f619c102ea0 2024-02-26T10:35:56.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.117+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3364638717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f619c102ea0 msgr2=0x7f619c1032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:56.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.117+0000 7f61a2ea7700 1 --2- 172.21.15.113:0/3364638717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f619c102ea0 0x7f619c1032c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6190009910 tx=0x7f6190009c20 comp rx=0 tx=0).stop 2024-02-26T10:35:56.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.117+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3364638717 shutdown_connections 2024-02-26T10:35:56.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.117+0000 7f61a2ea7700 1 --2- 172.21.15.113:0/3364638717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f619c102ea0 0x7f619c1032c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:56.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.117+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3364638717 >> 172.21.15.113:0/3364638717 conn(0x7f619c0fb4a0 msgr2=0x7f619c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:56.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.117+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3364638717 shutdown_connections 2024-02-26T10:35:56.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.117+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3364638717 wait complete. 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.117+0000 7f61a2ea7700 1 Processor -- start 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.117+0000 7f61a2ea7700 1 -- start start 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a2ea7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f619c102ea0 0x7f619c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a2ea7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f619c197540 0x7f619c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a2ea7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619c1979c0 0x7f619c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a2ea7700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c107860 con 0x7f619c197540 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a2ea7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c1076e0 con 0x7f619c102ea0 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a2ea7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c1079e0 con 0x7f619c1979c0 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a0c43700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f619c102ea0 0x7f619c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a1444700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619c1979c0 0x7f619c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a0c43700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f619c102ea0 0x7f619c197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.113:41224/0 (socket says 172.21.15.113:41224) 2024-02-26T10:35:56.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a1444700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619c1979c0 0x7f619c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50746/0 (socket says 172.21.15.113:50746) 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a0c43700 1 -- 172.21.15.113:0/3830736403 learned_addr learned my addr 172.21.15.113:0/3830736403 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f619bfff700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f619c197540 0x7f619c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a1444700 1 -- 172.21.15.113:0/3830736403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f619c102ea0 msgr2=0x7f619c197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a1444700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f619c102ea0 0x7f619c197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a1444700 1 -- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f619c197540 msgr2=0x7f619c19c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a1444700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f619c197540 0x7f619c19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a1444700 1 -- 172.21.15.113:0/3830736403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c19f410 con 0x7f619c1979c0 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.118+0000 7f61a1444700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619c1979c0 0x7f619c19eb40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f619400bd50 tx=0x7f619400bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.119+0000 7f6199ffb700 1 -- 172.21.15.113:0/3830736403 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194019070 con 0x7f619c1979c0 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.119+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c19f700 con 0x7f619c1979c0 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.119+0000 7f6199ffb700 1 -- 172.21.15.113:0/3830736403 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f619400cc90 con 0x7f619c1979c0 2024-02-26T10:35:56.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.119+0000 7f6199ffb700 1 -- 172.21.15.113:0/3830736403 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194012740 con 0x7f619c1979c0 2024-02-26T10:35:56.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.119+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c19fcc0 con 0x7f619c1979c0 2024-02-26T10:35:56.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.119+0000 7f6199ffb700 1 -- 172.21.15.113:0/3830736403 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f619401c070 con 0x7f619c1979c0 2024-02-26T10:35:56.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.120+0000 7f6199ffb700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f618c0390d0 0x7f618c03b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:35:56.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.120+0000 7f6199ffb700 1 -- 172.21.15.113:0/3830736403 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f619404b3d0 con 0x7f619c1979c0 2024-02-26T10:35:56.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.120+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c063380 con 0x7f619c1979c0 2024-02-26T10:35:56.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.120+0000 7f61a0c43700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f618c0390d0 0x7f618c03b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:35:56.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.121+0000 7f61a0c43700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f618c0390d0 0x7f618c03b590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6190005200 tx=0x7f619001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:35:56.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.125+0000 7f6199ffb700 1 -- 172.21.15.113:0/3830736403 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6194007290 con 0x7f619c1979c0 2024-02-26T10:35:56.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.341+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi100=y;smithi113=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f619c062190 con 0x7f618c0390d0 2024-02-26T10:35:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.355+0000 7f6199ffb700 1 -- 172.21.15.113:0/3830736403 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f619c062190 con 0x7f618c0390d0 2024-02-26T10:35:56.360 INFO:teuthology.orchestra.run.smithi113.stdout:Scheduled mgr update... 2024-02-26T10:35:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.357+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f618c0390d0 msgr2=0x7f618c03b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.357+0000 7f61a2ea7700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f618c0390d0 0x7f618c03b590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6190005200 tx=0x7f619001a040 comp rx=0 tx=0).stop 2024-02-26T10:35:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.357+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619c1979c0 msgr2=0x7f619c19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:35:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.357+0000 7f61a2ea7700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619c1979c0 0x7f619c19eb40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f619400bd50 tx=0x7f619400bd80 comp rx=0 tx=0).stop 2024-02-26T10:35:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.358+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 shutdown_connections 2024-02-26T10:35:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.358+0000 7f61a2ea7700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f618c0390d0 0x7f618c03b590 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.358+0000 7f61a2ea7700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619c1979c0 0x7f619c19eb40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.358+0000 7f61a2ea7700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f619c102ea0 0x7f619c197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.358+0000 7f61a2ea7700 1 --2- 172.21.15.113:0/3830736403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f619c197540 0x7f619c19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:35:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.358+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 >> 172.21.15.113:0/3830736403 conn(0x7f619c0fb4a0 msgr2=0x7f619c0fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:35:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.358+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 shutdown_connections 2024-02-26T10:35:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:35:56.358+0000 7f61a2ea7700 1 -- 172.21.15.113:0/3830736403 wait complete. 2024-02-26T10:35:56.713 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:56 smithi113 ceph-mon[36126]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:56.713 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:56.713 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:56.713 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:56.713 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T10:35:56.713 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:56.713 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:56.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:56 smithi060 ceph-mon[31539]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:56.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:56.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:56.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:56.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T10:35:56.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:56.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:56.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:56 smithi100 ceph-mon[36051]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:56.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:56.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:56.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T10:35:56.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T10:35:56.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:56.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:56.967 DEBUG:teuthology.orchestra.run.smithi100:mgr.y> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.y.service 2024-02-26T10:35:56.970 DEBUG:teuthology.orchestra.run.smithi113:mgr.z> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.z.service 2024-02-26T10:35:56.972 INFO:tasks.cephadm:Deploying OSDs... 2024-02-26T10:35:56.972 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:35:56.972 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T10:35:57.034 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T10:35:57.035 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-02-26T10:35:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T10:35:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:57.089 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-02-26T10:35:57.089 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:57.089 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:57.089 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 10:34:52.763497329 +0000 2024-02-26T10:35:57.090 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 10:28:45.810795135 +0000 2024-02-26T10:35:57.090 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 10:28:45.810795135 +0000 2024-02-26T10:35:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 10:28:45.810795135 +0000 2024-02-26T10:35:57.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T10:35:57.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-02-26T10:35:57.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-02-26T10:35:57.190 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000254468 s, 2.0 MB/s 2024-02-26T10:35:57.195 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T10:35:57.256 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-02-26T10:35:57.309 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T10:35:57.309 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:57.309 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-02-26T10:35:57.309 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:57.309 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:57.309 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 10:34:52.854496406 +0000 2024-02-26T10:35:57.310 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 10:28:46.398789129 +0000 2024-02-26T10:35:57.310 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 10:28:46.398789129 +0000 2024-02-26T10:35:57.310 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 10:28:46.398789129 +0000 2024-02-26T10:35:57.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T10:35:57.405 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-02-26T10:35:57.406 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-02-26T10:35:57.406 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000274513 s, 1.9 MB/s 2024-02-26T10:35:57.410 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T10:35:57.470 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-02-26T10:35:57.525 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T10:35:57.525 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-02-26T10:35:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 10:34:52.914495798 +0000 2024-02-26T10:35:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 10:28:46.973783256 +0000 2024-02-26T10:35:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 10:28:46.973783256 +0000 2024-02-26T10:35:57.526 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 10:28:46.973783256 +0000 2024-02-26T10:35:57.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T10:35:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-02-26T10:35:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-02-26T10:35:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000271996 s, 1.9 MB/s 2024-02-26T10:35:57.624 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T10:35:57.660 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:57 smithi100 ceph-mon[36051]: Reconfiguring mon.c (monmap changed)... 2024-02-26T10:35:57.660 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:57 smithi100 ceph-mon[36051]: Reconfiguring daemon mon.c on smithi113 2024-02-26T10:35:57.660 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:57 smithi100 ceph-mon[36051]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi100=y;smithi113=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:35:57.660 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:57 smithi100 ceph-mon[36051]: Saving service mgr spec with placement smithi060=x;smithi100=y;smithi113=z;count:3 2024-02-26T10:35:57.685 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-02-26T10:35:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T10:35:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:57.738 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-02-26T10:35:57.738 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:57.739 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:57.739 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 10:34:52.976495169 +0000 2024-02-26T10:35:57.739 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 10:28:47.667776168 +0000 2024-02-26T10:35:57.739 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 10:28:47.667776168 +0000 2024-02-26T10:35:57.739 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 10:28:47.667776168 +0000 2024-02-26T10:35:57.739 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T10:35:57.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:57 smithi060 ceph-mon[31539]: Reconfiguring mon.c (monmap changed)... 2024-02-26T10:35:57.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:57 smithi060 ceph-mon[31539]: Reconfiguring daemon mon.c on smithi113 2024-02-26T10:35:57.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:57 smithi060 ceph-mon[31539]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi100=y;smithi113=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:35:57.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:57 smithi060 ceph-mon[31539]: Saving service mgr spec with placement smithi060=x;smithi100=y;smithi113=z;count:3 2024-02-26T10:35:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-02-26T10:35:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-02-26T10:35:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000275008 s, 1.9 MB/s 2024-02-26T10:35:57.840 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T10:35:57.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:57 smithi113 ceph-mon[36126]: Reconfiguring mon.c (monmap changed)... 2024-02-26T10:35:57.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:57 smithi113 ceph-mon[36126]: Reconfiguring daemon mon.c on smithi113 2024-02-26T10:35:57.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:57 smithi113 ceph-mon[36126]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi100=y;smithi113=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:35:57.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:57 smithi113 ceph-mon[36126]: Saving service mgr spec with placement smithi060=x;smithi100=y;smithi113=z;count:3 2024-02-26T10:35:57.901 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-26T10:35:57.901 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T10:35:57.920 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T10:35:57.920 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-02-26T10:35:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T10:35:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:57.975 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-02-26T10:35:57.975 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:57.976 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:57.976 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-02-26 10:35:42.190404616 +0000 2024-02-26T10:35:57.976 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-02-26 10:27:33.432166454 +0000 2024-02-26T10:35:57.976 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-02-26 10:27:33.432166454 +0000 2024-02-26T10:35:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-02-26 10:27:33.432166454 +0000 2024-02-26T10:35:57.976 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T10:35:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-02-26T10:35:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-02-26T10:35:58.042 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000184589 s, 2.8 MB/s 2024-02-26T10:35:58.043 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T10:35:58.099 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-02-26T10:35:58.156 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T10:35:58.156 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:58.156 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-02-26T10:35:58.156 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:58.156 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:58.156 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-02-26 10:35:42.190404616 +0000 2024-02-26T10:35:58.157 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-02-26 10:27:33.778157148 +0000 2024-02-26T10:35:58.157 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-02-26 10:27:33.778157148 +0000 2024-02-26T10:35:58.157 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-02-26 10:27:33.778157148 +0000 2024-02-26T10:35:58.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T10:35:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-02-26T10:35:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-02-26T10:35:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000208243 s, 2.5 MB/s 2024-02-26T10:35:58.222 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T10:35:58.277 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-02-26T10:35:58.331 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T10:35:58.332 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:58.332 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-02-26T10:35:58.332 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:58.332 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:58.332 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-02-26 10:35:42.191404590 +0000 2024-02-26T10:35:58.332 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-02-26 10:27:34.118148003 +0000 2024-02-26T10:35:58.332 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-02-26 10:27:34.118148003 +0000 2024-02-26T10:35:58.332 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-02-26 10:27:34.118148003 +0000 2024-02-26T10:35:58.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T10:35:58.397 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-02-26T10:35:58.397 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-02-26T10:35:58.397 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000213382 s, 2.4 MB/s 2024-02-26T10:35:58.398 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T10:35:58.453 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-02-26T10:35:58.508 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T10:35:58.508 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:58.508 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-02-26T10:35:58.508 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:58.508 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:58.509 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-02-26 10:35:42.191404590 +0000 2024-02-26T10:35:58.509 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-02-26 10:27:34.456138912 +0000 2024-02-26T10:35:58.509 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-02-26 10:27:34.456138912 +0000 2024-02-26T10:35:58.509 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-02-26 10:27:34.456138912 +0000 2024-02-26T10:35:58.509 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T10:35:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-02-26T10:35:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-02-26T10:35:58.574 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000186996 s, 2.7 MB/s 2024-02-26T10:35:58.575 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T10:35:58.630 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-02-26T10:35:58.631 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T10:35:58.646 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T10:35:58.646 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-02-26T10:35:58.700 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T10:35:58.700 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:58.700 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-02-26T10:35:58.700 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:58.700 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:58.701 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-02-26 10:35:31.768896558 +0000 2024-02-26T10:35:58.701 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-02-26 10:27:33.430411460 +0000 2024-02-26T10:35:58.701 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-02-26 10:27:33.430411460 +0000 2024-02-26T10:35:58.701 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-02-26 10:27:33.430411460 +0000 2024-02-26T10:35:58.701 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T10:35:58.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-02-26T10:35:58.767 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-02-26T10:35:58.767 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000215823 s, 2.4 MB/s 2024-02-26T10:35:58.771 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T10:35:58.824 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-02-26T10:35:58.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T10:35:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:58.857 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T10:35:58.857 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:58.857 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-02-26T10:35:58.857 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:58.857 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:58.857 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-02-26 10:35:31.769896535 +0000 2024-02-26T10:35:58.857 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-02-26 10:27:33.803402691 +0000 2024-02-26T10:35:58.857 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-02-26 10:27:33.803402691 +0000 2024-02-26T10:35:58.858 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-02-26 10:27:33.803402691 +0000 2024-02-26T10:35:58.858 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T10:35:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:35:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T10:35:58.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T10:35:58.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T10:35:58.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T10:35:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T10:35:58.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:35:58.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-02-26T10:35:58.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-02-26T10:35:58.923 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000178224 s, 2.9 MB/s 2024-02-26T10:35:58.924 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T10:35:58.980 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-02-26T10:35:59.035 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T10:35:59.035 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:59.035 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-02-26T10:35:59.035 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:59.035 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:59.035 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-02-26 10:35:31.769896535 +0000 2024-02-26T10:35:59.035 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-02-26 10:27:34.157394368 +0000 2024-02-26T10:35:59.035 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-02-26 10:27:34.157394368 +0000 2024-02-26T10:35:59.035 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-02-26 10:27:34.157394368 +0000 2024-02-26T10:35:59.036 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T10:35:59.100 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-02-26T10:35:59.100 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-02-26T10:35:59.100 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000177047 s, 2.9 MB/s 2024-02-26T10:35:59.101 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T10:35:59.156 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-02-26T10:35:59.210 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T10:35:59.211 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:35:59.211 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-02-26T10:35:59.211 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:35:59.211 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:35:59.211 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-02-26 10:35:31.770896513 +0000 2024-02-26T10:35:59.211 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-02-26 10:27:34.496386398 +0000 2024-02-26T10:35:59.211 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-02-26 10:27:34.496386398 +0000 2024-02-26T10:35:59.211 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-02-26 10:27:34.496386398 +0000 2024-02-26T10:35:59.211 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T10:35:59.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-02-26T10:35:59.276 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-02-26T10:35:59.276 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000199345 s, 2.6 MB/s 2024-02-26T10:35:59.277 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T10:35:59.333 INFO:tasks.cephadm:Deploying osd.0 on smithi060 with /dev/vg_nvme/lv_4... 2024-02-26T10:35:59.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T10:35:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:35:59.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:35:59 smithi113 ceph-mon[36126]: Deploying daemon mgr.z on smithi113 2024-02-26T10:35:59.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:35:59 smithi060 ceph-mon[31539]: Deploying daemon mgr.z on smithi113 2024-02-26T10:35:59.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:35:59 smithi100 ceph-mon[36051]: Deploying daemon mgr.z on smithi113 2024-02-26T10:36:00.318 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:00 smithi113 systemd[1]: Starting Ceph mgr.z for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:36:00.577 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:00 smithi113 podman[37995]: 2024-02-26 10:36:00.217253658 +0000 UTC m=+0.017974710 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:36:00.578 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:00 smithi113 podman[37995]: 2024-02-26 10:36:00.326800649 +0000 UTC m=+0.127521693 container create 9242054c1bbf1531bd27a90e2b14ee73551c86d451115a0e0523b424e8fd005d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-26T10:36:00.578 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:00 smithi113 podman[37995]: 2024-02-26 10:36:00.554288432 +0000 UTC m=+0.355009467 container init 9242054c1bbf1531bd27a90e2b14ee73551c86d451115a0e0523b424e8fd005d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-26T10:36:00.578 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:00 smithi113 podman[37995]: 2024-02-26 10:36:00.556801503 +0000 UTC m=+0.357522556 container start 9242054c1bbf1531bd27a90e2b14ee73551c86d451115a0e0523b424e8fd005d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:36:00.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:00 smithi113 ceph-mon[36126]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:00.841 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:00 smithi113 ceph-mgr[38014]: -- 172.21.15.113:0/2622608130 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d5a54e5a0 con 0x555d5a522c00 2024-02-26T10:36:00.842 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:00 smithi113 bash[37995]: 9242054c1bbf1531bd27a90e2b14ee73551c86d451115a0e0523b424e8fd005d 2024-02-26T10:36:00.842 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:00 smithi113 systemd[1]: Started Ceph mgr.z for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:36:00.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:00 smithi060 ceph-mon[31539]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:00.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:00 smithi100 ceph-mon[36051]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:01.591 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:01.093+0000 7fd29fff3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T10:36:01.591 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:01.184+0000 7fd29fff3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T10:36:02.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:02.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:02.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:02.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-mon[36126]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:02.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T10:36:02.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T10:36:02.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T10:36:02.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:02.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-mon[36126]: Deploying daemon mgr.y on smithi100 2024-02-26T10:36:02.050 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:01.815+0000 7fd29fff3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T10:36:02.050 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:01.906+0000 7fd29fff3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T10:36:02.050 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:02.044+0000 7fd29fff3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T10:36:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:01 smithi060 ceph-mon[31539]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T10:36:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T10:36:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T10:36:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:01 smithi060 ceph-mon[31539]: Deploying daemon mgr.y on smithi100 2024-02-26T10:36:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:01 smithi100 ceph-mon[36051]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T10:36:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T10:36:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T10:36:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:02.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:01 smithi100 ceph-mon[36051]: Deploying daemon mgr.y on smithi100 2024-02-26T10:36:02.341 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:02.137+0000 7fd29fff3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T10:36:02.341 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:02.218+0000 7fd29fff3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T10:36:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T10:36:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T10:36:02.665 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-02-26T10:36:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-02-26T10:36:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0392678 s, 267 MB/s 2024-02-26T10:36:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-02-26T10:36:02.768 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:02.483+0000 7fd29fff3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T10:36:02.769 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:02.589+0000 7fd29fff3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T10:36:02.769 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:02.673+0000 7fd29fff3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T10:36:02.769 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:02.764+0000 7fd29fff3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T10:36:02.843 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:02 smithi100 systemd[1]: Starting Ceph mgr.y for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:36:03.091 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:02.937+0000 7fd29fff3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T10:36:03.112 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:02 smithi100 podman[37429]: 2024-02-26 10:36:02.742031681 +0000 UTC m=+0.017293370 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:36:03.113 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:02 smithi100 podman[37429]: 2024-02-26 10:36:02.848544069 +0000 UTC m=+0.123805764 container create 1cf2df4ef95ea0c9a740693db0195dcebb8e59719a576ed625a121fd8296a308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD) 2024-02-26T10:36:03.113 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:03 smithi100 podman[37429]: 2024-02-26 10:36:03.050646475 +0000 UTC m=+0.325908155 container init 1cf2df4ef95ea0c9a740693db0195dcebb8e59719a576ed625a121fd8296a308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:36:03.113 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:03 smithi100 podman[37429]: 2024-02-26 10:36:03.053179925 +0000 UTC m=+0.328441602 container start 1cf2df4ef95ea0c9a740693db0195dcebb8e59719a576ed625a121fd8296a308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-26T10:36:03.113 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:03 smithi100 ceph-mgr[37449]: -- 172.21.15.100:0/3922403420 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5638ecc225a0 con 0x5638ecbf7000 2024-02-26T10:36:03.113 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:03 smithi100 bash[37429]: 1cf2df4ef95ea0c9a740693db0195dcebb8e59719a576ed625a121fd8296a308 2024-02-26T10:36:03.384 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:03 smithi100 systemd[1]: Started Ceph mgr.y for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:36:03.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_4 2024-02-26T10:36:03.885 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:03 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:03.601+0000 7f4f467c6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T10:36:03.885 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:03 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:03.700+0000 7f4f467c6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T10:36:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:36:04.186 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:03 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:03.918+0000 7fd29fff3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T10:36:04.186 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:04.068+0000 7fd29fff3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T10:36:04.442 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:04 smithi060 ceph-mon[31539]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:04.442 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:04 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.442 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:04 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.443 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:04 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.443 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:04 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.443 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:04 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:04.471 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:04.423+0000 7fd29fff3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T10:36:04.471 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-mon[36126]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:04.471 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.471 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.471 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.471 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.472 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:04.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-mon[36051]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:04.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:04.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:04.477 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:04.377+0000 7f4f467c6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T10:36:04.477 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:04.472+0000 7f4f467c6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T10:36:04.805 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:04.615+0000 7f4f467c6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T10:36:04.805 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:04.709+0000 7f4f467c6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T10:36:04.805 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:04.800+0000 7f4f467c6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T10:36:04.841 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:04.517+0000 7fd29fff3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T10:36:04.841 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:04.708+0000 7fd29fff3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:36:05.194 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:04.908+0000 7fd29fff3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T10:36:05.355 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:05.064+0000 7f4f467c6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T10:36:05.355 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:05.173+0000 7f4f467c6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T10:36:05.355 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:05.258+0000 7f4f467c6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T10:36:05.355 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:05.350+0000 7f4f467c6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T10:36:05.449 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:05.191+0000 7fd29fff3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:36:05.450 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:05.446+0000 7fd29fff3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T10:36:05.635 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:05.523+0000 7f4f467c6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T10:36:05.841 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:05.676+0000 7fd29fff3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T10:36:05.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.951+0000 7fe820e00700 1 -- 172.21.15.60:0/1239961455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c102710 msgr2=0x7fe81c104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:05.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.951+0000 7fe820e00700 1 --2- 172.21.15.60:0/1239961455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c102710 0x7fe81c104b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009990 tx=0x7fe80c009ca0 comp rx=0 tx=0).stop 2024-02-26T10:36:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.951+0000 7fe820e00700 1 -- 172.21.15.60:0/1239961455 shutdown_connections 2024-02-26T10:36:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.951+0000 7fe820e00700 1 --2- 172.21.15.60:0/1239961455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c107970 0x7fe81c109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.951+0000 7fe820e00700 1 --2- 172.21.15.60:0/1239961455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c102710 0x7fe81c104b00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.951+0000 7fe820e00700 1 --2- 172.21.15.60:0/1239961455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81c105040 0x7fe81c107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.951+0000 7fe820e00700 1 -- 172.21.15.60:0/1239961455 >> 172.21.15.60:0/1239961455 conn(0x7fe81c0fc120 msgr2=0x7fe81c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:36:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.951+0000 7fe820e00700 1 -- 172.21.15.60:0/1239961455 shutdown_connections 2024-02-26T10:36:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.951+0000 7fe820e00700 1 -- 172.21.15.60:0/1239961455 wait complete. 2024-02-26T10:36:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.952+0000 7fe820e00700 1 Processor -- start 2024-02-26T10:36:05.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe820e00700 1 -- start start 2024-02-26T10:36:05.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe820e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c102710 0x7fe81c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:36:05.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe820e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c105040 0x7fe81c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:36:05.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe820e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81c107970 0x7fe81c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:36:05.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe820e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c10d8a0 con 0x7fe81c102710 2024-02-26T10:36:05.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe820e00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c10d720 con 0x7fe81c105040 2024-02-26T10:36:05.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe820e00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c10da20 con 0x7fe81c107970 2024-02-26T10:36:05.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe81aa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c105040 0x7fe81c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe81be00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81c107970 0x7fe81c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe81aa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c105040 0x7fe81c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53224/0 (socket says 172.21.15.60:53224) 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe81b400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c102710 0x7fe81c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe81be00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81c107970 0x7fe81c19f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:45400/0 (socket says 172.21.15.60:45400) 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.953+0000 7fe81aa00700 1 -- 172.21.15.60:0/2488168997 learned_addr learned my addr 172.21.15.60:0/2488168997 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.954+0000 7fe81aa00700 1 -- 172.21.15.60:0/2488168997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81c107970 msgr2=0x7fe81c19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.954+0000 7fe81aa00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81c107970 0x7fe81c19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.954+0000 7fe81aa00700 1 -- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c102710 msgr2=0x7fe81c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.954+0000 7fe81aa00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c102710 0x7fe81c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.954+0000 7fe81aa00700 1 -- 172.21.15.60:0/2488168997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c19fae0 con 0x7fe81c105040 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.954+0000 7fe81be00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81c107970 0x7fe81c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.954+0000 7fe81b400700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c102710 0x7fe81c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:36:05.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.954+0000 7fe81aa00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c105040 0x7fe81c1983c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe80800e9a0 tx=0x7fe80800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:36:05.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.954+0000 7fe813e00700 1 -- 172.21.15.60:0/2488168997 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80800ca10 con 0x7fe81c105040 2024-02-26T10:36:05.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.955+0000 7fe813e00700 1 -- 172.21.15.60:0/2488168997 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe80800cbb0 con 0x7fe81c105040 2024-02-26T10:36:05.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.955+0000 7fe813e00700 1 -- 172.21.15.60:0/2488168997 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808018710 con 0x7fe81c105040 2024-02-26T10:36:05.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.955+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c19fdd0 con 0x7fe81c105040 2024-02-26T10:36:05.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.955+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c1a0230 con 0x7fe81c105040 2024-02-26T10:36:05.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.956+0000 7fe813e00700 1 -- 172.21.15.60:0/2488168997 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fe8080188b0 con 0x7fe81c105040 2024-02-26T10:36:05.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.956+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c068a10 con 0x7fe81c105040 2024-02-26T10:36:05.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.957+0000 7fe813e00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe800038e80 0x7fe80003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:36:05.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.957+0000 7fe813e00700 1 -- 172.21.15.60:0/2488168997 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe80804c000 con 0x7fe81c105040 2024-02-26T10:36:05.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.957+0000 7fe81b400700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe800038e80 0x7fe80003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:36:05.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.958+0000 7fe81b400700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe800038e80 0x7fe80003b340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00b2e0 tx=0x7fe80c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:36:05.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:05.963+0000 7fe813e00700 1 -- 172.21.15.60:0/2488168997 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe808020c50 con 0x7fe81c105040 2024-02-26T10:36:06.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:06.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:06.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:06.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T10:36:06.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T10:36:06.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T10:36:06.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:06.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:05 smithi060 ceph-mon[31539]: Reconfiguring daemon mgr.x on smithi060 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T10:36:06.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:06.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:05 smithi100 ceph-mon[36051]: Reconfiguring daemon mgr.x on smithi060 2024-02-26T10:36:06.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:06.199+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe81c062190 con 0x7fe800038e80 2024-02-26T10:36:06.238 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:06.239 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:05 smithi113 ceph-mon[36126]: Reconfiguring daemon mgr.x on smithi060 2024-02-26T10:36:06.591 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:06 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:06.478+0000 7fd29fff3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T10:36:06.859 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:06.481+0000 7f4f467c6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T10:36:06.860 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:06.611+0000 7f4f467c6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T10:36:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:06 smithi060 ceph-mon[31539]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:36:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:06 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:36:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:06 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:36:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:06 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:06 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:07.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:06 smithi100 ceph-mon[36051]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:36:07.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:06 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:36:07.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:06 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:36:07.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:06 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:07.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:06 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:07.135 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:06.951+0000 7f4f467c6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T10:36:07.135 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:07.039+0000 7f4f467c6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T10:36:07.140 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:06 smithi113 ceph-mon[36126]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:36:07.141 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:06 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:36:07.141 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:06 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:36:07.141 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:06 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:07.141 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:06 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:07.141 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:07 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:07.041+0000 7fd29fff3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T10:36:07.141 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:07 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:07.137+0000 7fd29fff3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T10:36:07.591 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:36:07 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z[38010]: 2024-02-26T10:36:07.351+0000 7fd29fff3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T10:36:07.635 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:07.232+0000 7f4f467c6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:36:07.635 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:07.417+0000 7f4f467c6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T10:36:07.993 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:07.714+0000 7f4f467c6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:36:07.993 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:07.988+0000 7f4f467c6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T10:36:08.365 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:08.226+0000 7f4f467c6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T10:36:08.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:08.381+0000 7fe813e00700 1 -- 172.21.15.60:0/2488168997 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 14) v1 ==== 91342+0+0 (secure 0 0 0) 0x7fe808010430 con 0x7fe81c105040 2024-02-26T10:36:08.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:08.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:08.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:08.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: Standby manager daemon z started 2024-02-26T10:36:08.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T10:36:08.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T10:36:08.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T10:36:08.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T10:36:08.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:08.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:08.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:08.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:08.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:08.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:08.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:08.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: Standby manager daemon z started 2024-02-26T10:36:08.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T10:36:08.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T10:36:08.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T10:36:08.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T10:36:08.636 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:08.636 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:08.636 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:08.636 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:08 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:08.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:08.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:08.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:08.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: Standby manager daemon z started 2024-02-26T10:36:08.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T10:36:08.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T10:36:08.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T10:36:08.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.? 172.21.15.113:0/3229622919' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T10:36:08.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:08.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:08.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:08.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:08 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:09.385 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:09.031+0000 7f4f467c6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T10:36:09.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:09 smithi060 ceph-mon[31539]: mgrmap e14: x(active, since 91s), standbys: z 2024-02-26T10:36:09.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:09 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T10:36:09.698 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:09 smithi100 ceph-mon[36051]: mgrmap e14: x(active, since 91s), standbys: z 2024-02-26T10:36:09.698 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:09 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T10:36:09.698 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:09.596+0000 7f4f467c6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T10:36:09.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:09 smithi113 ceph-mon[36126]: mgrmap e14: x(active, since 91s), standbys: z 2024-02-26T10:36:09.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:09 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T10:36:10.135 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:09.693+0000 7f4f467c6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T10:36:10.135 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:36:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y[37445]: 2024-02-26T10:36:09.907+0000 7f4f467c6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T10:36:10.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:10.405+0000 7fe813e00700 1 -- 172.21.15.60:0/2488168997 <== mon.2 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7fe808018ba0 con 0x7fe81c105040 2024-02-26T10:36:10.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:10 smithi113 ceph-mon[36126]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:10.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:10 smithi113 ceph-mon[36126]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T10:36:10.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:10 smithi113 ceph-mon[36126]: Standby manager daemon y started 2024-02-26T10:36:10.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:10 smithi113 ceph-mon[36126]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T10:36:10.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:10 smithi113 ceph-mon[36126]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T10:36:10.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:10 smithi113 ceph-mon[36126]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T10:36:10.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:10 smithi060 ceph-mon[31539]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:10.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:10 smithi060 ceph-mon[31539]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T10:36:10.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:10 smithi060 ceph-mon[31539]: Standby manager daemon y started 2024-02-26T10:36:10.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:10 smithi060 ceph-mon[31539]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T10:36:10.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:10 smithi060 ceph-mon[31539]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T10:36:10.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:10 smithi060 ceph-mon[31539]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T10:36:10.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:10 smithi100 ceph-mon[36051]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:10.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:10 smithi100 ceph-mon[36051]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T10:36:10.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:10 smithi100 ceph-mon[36051]: Standby manager daemon y started 2024-02-26T10:36:10.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:10 smithi100 ceph-mon[36051]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T10:36:10.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:10 smithi100 ceph-mon[36051]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T10:36:10.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:10 smithi100 ceph-mon[36051]: from='mgr.? 172.21.15.100:0/1939516315' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T10:36:11.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:11 smithi113 ceph-mon[36126]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-02-26T10:36:11.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:11 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3417411312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f23d95e5-b12d-4215-b22a-879bb7807b94"}]: dispatch 2024-02-26T10:36:11.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:11 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T10:36:11.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:11 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3417411312' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f23d95e5-b12d-4215-b22a-879bb7807b94"}]': finished 2024-02-26T10:36:11.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:11 smithi113 ceph-mon[36126]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T10:36:11.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:11 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:11.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:11 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3543119694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:36:11.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:11 smithi060 ceph-mon[31539]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-02-26T10:36:11.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:11 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3417411312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f23d95e5-b12d-4215-b22a-879bb7807b94"}]: dispatch 2024-02-26T10:36:11.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T10:36:11.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:11 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3417411312' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f23d95e5-b12d-4215-b22a-879bb7807b94"}]': finished 2024-02-26T10:36:11.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:11 smithi060 ceph-mon[31539]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T10:36:11.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:11.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:11 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3543119694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:36:11.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:11 smithi100 ceph-mon[36051]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-02-26T10:36:11.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:11 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3417411312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f23d95e5-b12d-4215-b22a-879bb7807b94"}]: dispatch 2024-02-26T10:36:11.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:11 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T10:36:11.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:11 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3417411312' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f23d95e5-b12d-4215-b22a-879bb7807b94"}]': finished 2024-02-26T10:36:11.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:11 smithi100 ceph-mon[36051]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T10:36:11.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:11 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:11.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:11 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3543119694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:36:12.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:12 smithi113 ceph-mon[36126]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:12.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:12 smithi060 ceph-mon[31539]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:12.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:12 smithi100 ceph-mon[36051]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:14.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:14 smithi113 ceph-mon[36126]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:14.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:14 smithi060 ceph-mon[31539]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:14.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:14 smithi100 ceph-mon[36051]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:16.720 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:16 smithi060 ceph-mon[31539]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:16.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:16 smithi113 ceph-mon[36126]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:16.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:16 smithi100 ceph-mon[36051]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:18.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:18 smithi060 ceph-mon[31539]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:18.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:18 smithi113 ceph-mon[36126]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:18.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:18 smithi100 ceph-mon[36051]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:20.703 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:20 smithi060 ceph-mon[31539]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:20.704 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T10:36:20.704 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:20.704 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:20 smithi060 ceph-mon[31539]: Deploying daemon osd.0 on smithi060 2024-02-26T10:36:20.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:20 smithi113 ceph-mon[36126]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:20.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T10:36:20.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:20.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:20 smithi113 ceph-mon[36126]: Deploying daemon osd.0 on smithi060 2024-02-26T10:36:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:20 smithi100 ceph-mon[36051]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T10:36:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:20 smithi100 ceph-mon[36051]: Deploying daemon osd.0 on smithi060 2024-02-26T10:36:22.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:22 smithi060 ceph-mon[31539]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:22.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:22 smithi113 ceph-mon[36126]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:22.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:22 smithi100 ceph-mon[36051]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:24.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:24 smithi060 ceph-mon[31539]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:24.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:24 smithi113 ceph-mon[36126]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:24 smithi100 ceph-mon[36051]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:26.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:26 smithi060 ceph-mon[31539]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:26.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:26 smithi113 ceph-mon[36126]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:26.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:26 smithi100 ceph-mon[36051]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:28.687 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:28 smithi100 ceph-mon[36051]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:28.688 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:28.688 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:28.688 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:28.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:28 smithi060 ceph-mon[31539]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:28.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:28.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:28.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:28.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:28 smithi113 ceph-mon[36126]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:28.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:28.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:28.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:30.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:30 smithi113 ceph-mon[36126]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:30.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:30 smithi060 ceph-mon[31539]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:30.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:30 smithi100 ceph-mon[36051]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:31 smithi060 ceph-mon[31539]: from='osd.0 [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T10:36:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:31 smithi060 ceph-mon[31539]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T10:36:31.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:31.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:31.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:31.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:31.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:31.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:31 smithi100 ceph-mon[36051]: from='osd.0 [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T10:36:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:31 smithi100 ceph-mon[36051]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T10:36:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.954+0000 7fe813e00700 1 -- 172.21.15.60:0/2488168997 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe81c062190 con 0x7fe800038e80 2024-02-26T10:36:31.959 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 0 on host 'smithi060' 2024-02-26T10:36:31.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe800038e80 msgr2=0x7fe80003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:31.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe800038e80 0x7fe80003b340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00b2e0 tx=0x7fe80c005fb0 comp rx=0 tx=0).stop 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c105040 msgr2=0x7fe81c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c105040 0x7fe81c1983c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe80800e9a0 tx=0x7fe80800ecb0 comp rx=0 tx=0).stop 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 shutdown_connections 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe800038e80 0x7fe80003b340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe81c107970 0x7fe81c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c102710 0x7fe81c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 --2- 172.21.15.60:0/2488168997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe81c105040 0x7fe81c1983c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 >> 172.21.15.60:0/2488168997 conn(0x7fe81c0fc120 msgr2=0x7fe81c103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.958+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 shutdown_connections 2024-02-26T10:36:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:36:31.959+0000 7fe820e00700 1 -- 172.21.15.60:0/2488168997 wait complete. 2024-02-26T10:36:32.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:32.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:32.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:32.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:32.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:32.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:31 smithi113 ceph-mon[36126]: from='osd.0 [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T10:36:32.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:31 smithi113 ceph-mon[36126]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T10:36:32.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:32 smithi060 ceph-mon[31539]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:32.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:32 smithi060 ceph-mon[31539]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T10:36:32.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:32 smithi060 ceph-mon[31539]: from='osd.0 [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:36:32.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:32 smithi060 ceph-mon[31539]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T10:36:32.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:32 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:32.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:32 smithi060 ceph-mon[31539]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:36:32.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:32 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:32.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:32 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:32.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:32 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:32.724 DEBUG:teuthology.orchestra.run.smithi060:osd.0> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.0.service 2024-02-26T10:36:32.728 INFO:tasks.cephadm:Deploying osd.1 on smithi100 with /dev/vg_nvme/lv_4... 2024-02-26T10:36:32.729 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T10:36:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:36:33.089 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:32 smithi100 ceph-mon[36051]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:33.089 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:32 smithi100 ceph-mon[36051]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T10:36:33.089 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:32 smithi100 ceph-mon[36051]: from='osd.0 [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:36:33.089 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:32 smithi100 ceph-mon[36051]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T10:36:33.090 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:32 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:33.090 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:32 smithi100 ceph-mon[36051]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:36:33.090 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:32 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:33.090 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:32 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:33.090 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:32 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:33.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:32 smithi113 ceph-mon[36126]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:33.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:32 smithi113 ceph-mon[36126]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T10:36:33.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:32 smithi113 ceph-mon[36126]: from='osd.0 [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:36:33.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:32 smithi113 ceph-mon[36126]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T10:36:33.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:32 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:33.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:32 smithi113 ceph-mon[36126]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:36:33.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:32 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:33.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:32 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:33.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:32 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:33.999 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:33 smithi100 ceph-mon[36051]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:36:34.000 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:33 smithi100 ceph-mon[36051]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T10:36:34.000 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:33 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:34.000 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:33 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:34.031 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:33 smithi060 ceph-mon[31539]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:36:34.031 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:33 smithi060 ceph-mon[31539]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T10:36:34.031 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:33 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:34.031 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:33 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:34.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:33 smithi113 ceph-mon[36126]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:36:34.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:33 smithi113 ceph-mon[36126]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T10:36:34.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:33 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:34.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:33 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:34.673 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T10:36:34.674 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T10:36:34.674 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-02-26T10:36:34.674 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-02-26T10:36:34.674 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123502 s, 849 MB/s 2024-02-26T10:36:34.674 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-02-26T10:36:34.983 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:34 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:36:34.983 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:34 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:36:34.983 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:34 smithi100 ceph-mon[36051]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:34.983 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:34 smithi100 ceph-mon[36051]: osd.0 [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] boot 2024-02-26T10:36:34.984 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:34 smithi100 ceph-mon[36051]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T10:36:34.984 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:35.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:34 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:36:35.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:34 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:36:35.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:34 smithi113 ceph-mon[36126]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:35.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:34 smithi113 ceph-mon[36126]: osd.0 [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] boot 2024-02-26T10:36:35.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:34 smithi113 ceph-mon[36126]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T10:36:35.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:35.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:34 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:36:35.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:34 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:36:35.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:34 smithi060 ceph-mon[31539]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T10:36:35.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:34 smithi060 ceph-mon[31539]: osd.0 [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] boot 2024-02-26T10:36:35.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:34 smithi060 ceph-mon[31539]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T10:36:35.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T10:36:35.272 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-02-26T10:36:35.506 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:36:35.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:35 smithi100 ceph-mon[36051]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T10:36:35.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:35 smithi100 ceph-mon[36051]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:36.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:35 smithi113 ceph-mon[36126]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T10:36:36.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:35 smithi113 ceph-mon[36126]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:36.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:35 smithi060 ceph-mon[31539]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T10:36:36.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:35 smithi060 ceph-mon[31539]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.984+0000 7f197364c700 1 -- 172.21.15.100:0/172052720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c100c50 msgr2=0x7f196c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.984+0000 7f197364c700 1 --2- 172.21.15.100:0/172052720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c100c50 0x7f196c101070 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f195c0099c0 tx=0x7f195c009cd0 comp rx=0 tx=0).stop 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.984+0000 7f197364c700 1 -- 172.21.15.100:0/172052720 shutdown_connections 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.984+0000 7f197364c700 1 --2- 172.21.15.100:0/172052720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f196c1027f0 0x7f196c1090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.984+0000 7f197364c700 1 --2- 172.21.15.100:0/172052720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c100c50 0x7f196c101070 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.984+0000 7f197364c700 1 --2- 172.21.15.100:0/172052720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f196c101e50 0x7f196c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.984+0000 7f197364c700 1 -- 172.21.15.100:0/172052720 >> 172.21.15.100:0/172052720 conn(0x7f196c0fc1f0 msgr2=0x7f196c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.984+0000 7f197364c700 1 -- 172.21.15.100:0/172052720 shutdown_connections 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.984+0000 7f197364c700 1 -- 172.21.15.100:0/172052720 wait complete. 2024-02-26T10:36:36.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f197364c700 1 Processor -- start 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f197364c700 1 -- start start 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f197364c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f196c100c50 0x7f196c104b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f197364c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f196c101e50 0x7f196c103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f197364c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c1027f0 0x7f196c103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f197364c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c10c910 con 0x7f196c100c50 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f197364c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c10c790 con 0x7f196c1027f0 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f197364c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c10ca90 con 0x7f196c101e50 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f1971be9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c1027f0 0x7f196c103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f1971be9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c1027f0 0x7f196c103770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35088/0 (socket says 172.21.15.100:35088) 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f1971be9700 1 -- 172.21.15.100:0/3047110070 learned_addr learned my addr 172.21.15.100:0/3047110070 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f1970be7700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f196c101e50 0x7f196c103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f1971be9700 1 -- 172.21.15.100:0/3047110070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f196c101e50 msgr2=0x7f196c103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:36.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f1971be9700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f196c101e50 0x7f196c103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.985+0000 7f1971be9700 1 -- 172.21.15.100:0/3047110070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f196c100c50 msgr2=0x7f196c104b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.986+0000 7f1971be9700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f196c100c50 0x7f196c104b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.986+0000 7f1971be9700 1 -- 172.21.15.100:0/3047110070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c104040 con 0x7f196c1027f0 2024-02-26T10:36:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.986+0000 7f1971be9700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c1027f0 0x7f196c103770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f196400b970 tx=0x7f196400bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:36:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.986+0000 7f19627fc700 1 -- 172.21.15.100:0/3047110070 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964019070 con 0x7f196c1027f0 2024-02-26T10:36:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.986+0000 7f19627fc700 1 -- 172.21.15.100:0/3047110070 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1964014440 con 0x7f196c1027f0 2024-02-26T10:36:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.986+0000 7f19627fc700 1 -- 172.21.15.100:0/3047110070 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19640075d0 con 0x7f196c1027f0 2024-02-26T10:36:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.986+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c1a22f0 con 0x7f196c1027f0 2024-02-26T10:36:36.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.986+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c1a28d0 con 0x7f196c1027f0 2024-02-26T10:36:36.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.987+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c068a10 con 0x7f196c1027f0 2024-02-26T10:36:36.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.988+0000 7f19627fc700 1 -- 172.21.15.100:0/3047110070 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7f196401c070 con 0x7f196c1027f0 2024-02-26T10:36:36.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.988+0000 7f19627fc700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f19580a1e00 0x7f19580a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:36:36.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.989+0000 7f19627fc700 1 -- 172.21.15.100:0/3047110070 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f19640cbbe0 con 0x7f196c1027f0 2024-02-26T10:36:36.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.989+0000 7f19713e8700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f19580a1e00 0x7f19580a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:36:36.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.990+0000 7f19713e8700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f19580a1e00 0x7f19580a42c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f195c009fd0 tx=0x7f195c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:36:36.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:36.992+0000 7f19627fc700 1 -- 172.21.15.100:0/3047110070 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f196409aa80 con 0x7f196c1027f0 2024-02-26T10:36:37.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:37.195+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f196c06a110 con 0x7f19580a1e00 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi060 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi060 to 17095M 2024-02-26T10:36:38.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:38.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:38.093 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:36:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:36:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:36:38.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:38.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi060 2024-02-26T10:36:38.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:36:38.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi060 to 17095M 2024-02-26T10:36:38.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:38.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:38.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi060 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi060 to 17095M 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:36:38.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:40.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:39 smithi113 ceph-mon[36126]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:40.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:39 smithi113 ceph-mon[36126]: from='client.? 172.21.15.100:0/267592755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63269103-ac90-4d99-af6e-e73cd5b8846d"}]: dispatch 2024-02-26T10:36:40.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:39 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63269103-ac90-4d99-af6e-e73cd5b8846d"}]: dispatch 2024-02-26T10:36:40.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:39 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63269103-ac90-4d99-af6e-e73cd5b8846d"}]': finished 2024-02-26T10:36:40.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:39 smithi113 ceph-mon[36126]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T10:36:40.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:39 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:40.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:39 smithi060 ceph-mon[31539]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:40.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:39 smithi060 ceph-mon[31539]: from='client.? 172.21.15.100:0/267592755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63269103-ac90-4d99-af6e-e73cd5b8846d"}]: dispatch 2024-02-26T10:36:40.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:39 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63269103-ac90-4d99-af6e-e73cd5b8846d"}]: dispatch 2024-02-26T10:36:40.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:39 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63269103-ac90-4d99-af6e-e73cd5b8846d"}]': finished 2024-02-26T10:36:40.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:39 smithi060 ceph-mon[31539]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T10:36:40.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:39 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:40.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:39 smithi100 ceph-mon[36051]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:40.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:39 smithi100 ceph-mon[36051]: from='client.? 172.21.15.100:0/267592755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63269103-ac90-4d99-af6e-e73cd5b8846d"}]: dispatch 2024-02-26T10:36:40.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:39 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63269103-ac90-4d99-af6e-e73cd5b8846d"}]: dispatch 2024-02-26T10:36:40.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:39 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63269103-ac90-4d99-af6e-e73cd5b8846d"}]': finished 2024-02-26T10:36:40.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:39 smithi100 ceph-mon[36051]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T10:36:40.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:39 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:41.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:40 smithi060 ceph-mon[31539]: from='client.? 172.21.15.100:0/1188175400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:36:41.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:40 smithi100 ceph-mon[36051]: from='client.? 172.21.15.100:0/1188175400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:36:41.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:40 smithi113 ceph-mon[36126]: from='client.? 172.21.15.100:0/1188175400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:36:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:41 smithi060 ceph-mon[31539]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:42.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:41 smithi100 ceph-mon[36051]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:42.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:41 smithi113 ceph-mon[36126]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:44.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:43 smithi060 ceph-mon[31539]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:44.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:43 smithi100 ceph-mon[36051]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:44.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:43 smithi113 ceph-mon[36126]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:46.098 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:45 smithi100 ceph-mon[36051]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:46.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:45 smithi060 ceph-mon[31539]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:46.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:45 smithi113 ceph-mon[36126]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:48.038 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:47 smithi100 ceph-mon[36051]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:48.038 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:47 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T10:36:48.038 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:47 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:48.038 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:47 smithi100 ceph-mon[36051]: Deploying daemon osd.1 on smithi100 2024-02-26T10:36:48.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:47 smithi060 ceph-mon[31539]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:48.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T10:36:48.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:48.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:47 smithi060 ceph-mon[31539]: Deploying daemon osd.1 on smithi100 2024-02-26T10:36:48.143 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:47 smithi113 ceph-mon[36126]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:48.143 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T10:36:48.143 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:36:48.143 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:47 smithi113 ceph-mon[36126]: Deploying daemon osd.1 on smithi100 2024-02-26T10:36:50.111 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:49 smithi100 ceph-mon[36051]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:50.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:49 smithi060 ceph-mon[31539]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:50.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:49 smithi113 ceph-mon[36126]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:52.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:51 smithi060 ceph-mon[31539]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:52.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:51 smithi100 ceph-mon[36051]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:52.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:51 smithi113 ceph-mon[36126]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:53.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:53.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:53.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:52 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:53.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:52 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:53.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:52 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:53.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:52 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:36:53.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:52 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:53.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:52 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:54.276 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:54 smithi100 ceph-mon[36051]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:54.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:54 smithi113 ceph-mon[36126]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:54.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:54 smithi060 ceph-mon[31539]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:55.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.439+0000 7f19627fc700 1 -- 172.21.15.100:0/3047110070 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f196c06a110 con 0x7f19580a1e00 2024-02-26T10:36:55.445 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 1 on host 'smithi100' 2024-02-26T10:36:55.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f19580a1e00 msgr2=0x7f19580a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f19580a1e00 0x7f19580a42c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f195c009fd0 tx=0x7f195c000bc0 comp rx=0 tx=0).stop 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c1027f0 msgr2=0x7f196c103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c1027f0 0x7f196c103770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f196400b970 tx=0x7f196400bd30 comp rx=0 tx=0).stop 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 shutdown_connections 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f19580a1e00 0x7f19580a42c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c1027f0 0x7f196c103770 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f196c100c50 0x7f196c104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 --2- 172.21.15.100:0/3047110070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f196c101e50 0x7f196c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 >> 172.21.15.100:0/3047110070 conn(0x7f196c0fc1f0 msgr2=0x7f196c069a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 shutdown_connections 2024-02-26T10:36:55.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:36:55.442+0000 7f197364c700 1 -- 172.21.15.100:0/3047110070 wait complete. 2024-02-26T10:36:56.063 DEBUG:teuthology.orchestra.run.smithi100:osd.1> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.1.service 2024-02-26T10:36:56.066 INFO:tasks.cephadm:Deploying osd.2 on smithi113 with /dev/vg_nvme/lv_4... 2024-02-26T10:36:56.066 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T10:36:56.308 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:36:56.386 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:36:56 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1[40652]: 2024-02-26T10:36:56.104+0000 7f82692ae7c0 -1 osd.1 0 log_to_monitors true 2024-02-26T10:36:56.749 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:56 smithi113 ceph-mon[36126]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:56.749 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.749 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.749 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:56 smithi113 ceph-mon[36126]: from='osd.1 [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T10:36:56.749 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:56 smithi113 ceph-mon[36126]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T10:36:56.749 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.750 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:56 smithi100 ceph-mon[36051]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:56.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:56 smithi100 ceph-mon[36051]: from='osd.1 [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T10:36:56.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:56 smithi100 ceph-mon[36051]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T10:36:56.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.779 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:56 smithi060 ceph-mon[31539]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:56.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:56 smithi060 ceph-mon[31539]: from='osd.1 [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T10:36:56.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:56 smithi060 ceph-mon[31539]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T10:36:56.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:56.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:36:57.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:57 smithi060 ceph-mon[31539]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T10:36:57.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:57 smithi060 ceph-mon[31539]: from='osd.1 [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:36:57.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:57 smithi060 ceph-mon[31539]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T10:36:57.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:57.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:57 smithi060 ceph-mon[31539]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:36:57.657 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:57 smithi113 ceph-mon[36126]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T10:36:57.657 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:57 smithi113 ceph-mon[36126]: from='osd.1 [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:36:57.657 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:57 smithi113 ceph-mon[36126]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T10:36:57.657 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:57 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:57.657 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:57 smithi113 ceph-mon[36126]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:36:57.859 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:57 smithi100 ceph-mon[36051]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T10:36:57.859 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:57 smithi100 ceph-mon[36051]: from='osd.1 [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:36:57.859 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:57 smithi100 ceph-mon[36051]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T10:36:57.859 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:57 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:57.859 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:57 smithi100 ceph-mon[36051]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:36:58.135 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:36:57 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1[40652]: 2024-02-26T10:36:57.854+0000 7f825e913700 -1 osd.1 0 waiting for initial osdmap 2024-02-26T10:36:58.135 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:36:57 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1[40652]: 2024-02-26T10:36:57.857+0000 7f82566fe700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:36:58.225 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T10:36:58.225 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T10:36:58.225 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-02-26T10:36:58.225 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-02-26T10:36:58.225 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0100947 s, 1.0 GB/s 2024-02-26T10:36:58.225 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-02-26T10:36:58.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:58 smithi113 ceph-mon[36126]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:58.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:58 smithi113 ceph-mon[36126]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:36:58.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:58 smithi113 ceph-mon[36126]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T10:36:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:58 smithi113 ceph-mon[36126]: from='osd.1 ' entity='osd.1' 2024-02-26T10:36:58.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:58 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:58 smithi060 ceph-mon[31539]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:58 smithi060 ceph-mon[31539]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:36:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:58 smithi060 ceph-mon[31539]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T10:36:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:58 smithi060 ceph-mon[31539]: from='osd.1 ' entity='osd.1' 2024-02-26T10:36:58.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:58 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:58 smithi100 ceph-mon[36051]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:36:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:58 smithi100 ceph-mon[36051]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:36:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:58 smithi100 ceph-mon[36051]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T10:36:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:58 smithi100 ceph-mon[36051]: from='osd.1 ' entity='osd.1' 2024-02-26T10:36:58.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:58 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:36:58.903 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi113:vg_nvme/lv_4 2024-02-26T10:36:59.133 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:37:00.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:37:00.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:37:00.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:37:00.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: osd.1 [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] boot 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi100 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi100 to 17080M 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:00.136 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:36:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: osd.1 [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] boot 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi100 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi100 to 17080M 2024-02-26T10:37:00.174 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.175 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:00.175 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:00.175 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:36:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:37:00.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:37:00.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-26T10:37:00.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: osd.1 [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] boot 2024-02-26T10:37:00.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T10:37:00.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T10:37:00.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi100 2024-02-26T10:37:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:37:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi100 to 17080M 2024-02-26T10:37:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:36:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.658+0000 7ff4fd010700 1 -- 172.21.15.113:0/3583786001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f8101e50 msgr2=0x7ff4f81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.658+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3583786001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f8101e50 0x7ff4f81022b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009990 tx=0x7ff4ec009ca0 comp rx=0 tx=0).stop 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.658+0000 7ff4fd010700 1 -- 172.21.15.113:0/3583786001 shutdown_connections 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.658+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3583786001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f81027f0 0x7ff4f81090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.658+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3583786001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4f8100c50 0x7ff4f8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.658+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3583786001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f8101e50 0x7ff4f81022b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.658+0000 7ff4fd010700 1 -- 172.21.15.113:0/3583786001 >> 172.21.15.113:0/3583786001 conn(0x7ff4f80fc1f0 msgr2=0x7ff4f80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.658+0000 7ff4fd010700 1 -- 172.21.15.113:0/3583786001 shutdown_connections 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.658+0000 7ff4fd010700 1 -- 172.21.15.113:0/3583786001 wait complete. 2024-02-26T10:37:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4fd010700 1 Processor -- start 2024-02-26T10:37:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4fd010700 1 -- start start 2024-02-26T10:37:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4fd010700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f8100c50 0x7ff4f81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4fd010700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4f8101e50 0x7ff4f8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4fd010700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f81027f0 0x7ff4f819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4fd010700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f810c900 con 0x7ff4f8100c50 2024-02-26T10:37:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4fd010700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f810c780 con 0x7ff4f8101e50 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4fd010700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f810ca80 con 0x7ff4f81027f0 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4f759e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f81027f0 0x7ff4f819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4f759e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f81027f0 0x7ff4f819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48032/0 (socket says 172.21.15.113:48032) 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4f759e700 1 -- 172.21.15.113:0/3499485332 learned_addr learned my addr 172.21.15.113:0/3499485332 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4f659c700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4f8101e50 0x7ff4f8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4f759e700 1 -- 172.21.15.113:0/3499485332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4f8101e50 msgr2=0x7ff4f8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4f759e700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4f8101e50 0x7ff4f8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4f759e700 1 -- 172.21.15.113:0/3499485332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f8100c50 msgr2=0x7ff4f81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4f759e700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f8100c50 0x7ff4f81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.659+0000 7ff4f759e700 1 -- 172.21.15.113:0/3499485332 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f819fce0 con 0x7ff4f81027f0 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.660+0000 7ff4f759e700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f81027f0 0x7ff4f819f640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000b9a0 tx=0x7ff4f000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.660+0000 7ff4dffff700 1 -- 172.21.15.113:0/3499485332 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0019070 con 0x7ff4f81027f0 2024-02-26T10:37:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.660+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f819ffd0 con 0x7ff4f81027f0 2024-02-26T10:37:00.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.660+0000 7ff4dffff700 1 -- 172.21.15.113:0/3499485332 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4f000ce10 con 0x7ff4f81027f0 2024-02-26T10:37:00.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.660+0000 7ff4dffff700 1 -- 172.21.15.113:0/3499485332 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00073d0 con 0x7ff4f81027f0 2024-02-26T10:37:00.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.660+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f8103070 con 0x7ff4f81027f0 2024-02-26T10:37:00.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.661+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f8068a10 con 0x7ff4f81027f0 2024-02-26T10:37:00.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.662+0000 7ff4dffff700 1 -- 172.21.15.113:0/3499485332 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7ff4f001c070 con 0x7ff4f81027f0 2024-02-26T10:37:00.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.663+0000 7ff4dffff700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4e00a1d80 0x7ff4e00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:00.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.663+0000 7ff4dffff700 1 -- 172.21.15.113:0/3499485332 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7ff4f00cbd60 con 0x7ff4f81027f0 2024-02-26T10:37:00.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.663+0000 7ff4f6d9d700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4e00a1d80 0x7ff4e00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:00.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.664+0000 7ff4f6d9d700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4e00a1d80 0x7ff4e00a4240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8101cb0 tx=0x7ff4e8009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:37:00.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.667+0000 7ff4dffff700 1 -- 172.21.15.113:0/3499485332 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4f009a9c0 con 0x7ff4f81027f0 2024-02-26T10:37:00.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:00.871+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4f80fe410 con 0x7ff4e00a1d80 2024-02-26T10:37:01.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:00 smithi100 ceph-mon[36051]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T10:37:01.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:01.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:01.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:01.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:00 smithi113 ceph-mon[36126]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T10:37:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:01.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:01.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:01.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:01.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:00 smithi060 ceph-mon[31539]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T10:37:01.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:01.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:01.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:01.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:02.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:01 smithi113 ceph-mon[36126]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:02.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:01 smithi113 ceph-mon[36126]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:37:02.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:37:02.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:37:02.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:01 smithi100 ceph-mon[36051]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:01 smithi100 ceph-mon[36051]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:37:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:37:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:37:02.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:02.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:01 smithi060 ceph-mon[31539]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:02.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:01 smithi060 ceph-mon[31539]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:37:02.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:37:02.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:37:02.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:04.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:03 smithi113 ceph-mon[36126]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:04.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:03 smithi113 ceph-mon[36126]: from='client.? 172.21.15.113:0/1635924882' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "857e383d-a190-46b0-a523-79126cf32a69"}]: dispatch 2024-02-26T10:37:04.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:03 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "857e383d-a190-46b0-a523-79126cf32a69"}]: dispatch 2024-02-26T10:37:04.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:03 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "857e383d-a190-46b0-a523-79126cf32a69"}]': finished 2024-02-26T10:37:04.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:03 smithi113 ceph-mon[36126]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T10:37:04.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:04.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:03 smithi060 ceph-mon[31539]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:04.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:03 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/1635924882' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "857e383d-a190-46b0-a523-79126cf32a69"}]: dispatch 2024-02-26T10:37:04.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:03 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "857e383d-a190-46b0-a523-79126cf32a69"}]: dispatch 2024-02-26T10:37:04.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:03 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "857e383d-a190-46b0-a523-79126cf32a69"}]': finished 2024-02-26T10:37:04.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:03 smithi060 ceph-mon[31539]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T10:37:04.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:04.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:03 smithi100 ceph-mon[36051]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:04.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:03 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/1635924882' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "857e383d-a190-46b0-a523-79126cf32a69"}]: dispatch 2024-02-26T10:37:04.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:03 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "857e383d-a190-46b0-a523-79126cf32a69"}]: dispatch 2024-02-26T10:37:04.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:03 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "857e383d-a190-46b0-a523-79126cf32a69"}]': finished 2024-02-26T10:37:04.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:03 smithi100 ceph-mon[36051]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T10:37:04.386 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:05.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:04 smithi113 ceph-mon[36126]: from='client.? 172.21.15.113:0/1472129423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:37:05.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:04 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/1472129423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:37:05.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:04 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/1472129423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:37:06.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:05 smithi113 ceph-mon[36126]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:06.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:05 smithi060 ceph-mon[31539]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:06.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:05 smithi100 ceph-mon[36051]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:08.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:07 smithi113 ceph-mon[36126]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:08.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:07 smithi060 ceph-mon[31539]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:08.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:07 smithi100 ceph-mon[36051]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:10.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:09 smithi113 ceph-mon[36126]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:10.290 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:09 smithi100 ceph-mon[36051]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:10.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:09 smithi060 ceph-mon[31539]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:11.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:10 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T10:37:11.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:10 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:11.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:10 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T10:37:11.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:10 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:11.387 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:10 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T10:37:11.387 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:10 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:12.279 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:11 smithi113 ceph-mon[36126]: Deploying daemon osd.2 on smithi113 2024-02-26T10:37:12.279 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:11 smithi113 ceph-mon[36126]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:11 smithi060 ceph-mon[31539]: Deploying daemon osd.2 on smithi113 2024-02-26T10:37:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:11 smithi060 ceph-mon[31539]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:12.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:11 smithi100 ceph-mon[36051]: Deploying daemon osd.2 on smithi113 2024-02-26T10:37:12.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:11 smithi100 ceph-mon[36051]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:14.233 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:13 smithi113 ceph-mon[36126]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:14.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:13 smithi060 ceph-mon[31539]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:14.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:13 smithi100 ceph-mon[36051]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:16.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:15 smithi113 ceph-mon[36126]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:16.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:15 smithi060 ceph-mon[31539]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:16.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:15 smithi100 ceph-mon[36051]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:17.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:17 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:17.592 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:17 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:17.592 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:17 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:17.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:17 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:17.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:17 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:17.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:17 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:17.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:17 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:17.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:17 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:17.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:17 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:18.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:18 smithi113 ceph-mon[36126]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:18.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:18 smithi060 ceph-mon[31539]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:18.635 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:18 smithi100 ceph-mon[36051]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:19.621 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 2 on host 'smithi113' 2024-02-26T10:37:19.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.616+0000 7ff4dffff700 1 -- 172.21.15.113:0/3499485332 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff4f80fe410 con 0x7ff4e00a1d80 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4e00a1d80 msgr2=0x7ff4e00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4e00a1d80 0x7ff4e00a4240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8101cb0 tx=0x7ff4e8009470 comp rx=0 tx=0).stop 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f81027f0 msgr2=0x7ff4f819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f81027f0 0x7ff4f819f640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000b9a0 tx=0x7ff4f000bcb0 comp rx=0 tx=0).stop 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 shutdown_connections 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4e00a1d80 0x7ff4e00a4240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4f81027f0 0x7ff4f819f640 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f8100c50 0x7ff4f81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 --2- 172.21.15.113:0/3499485332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4f8101e50 0x7ff4f8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:19.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.618+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 >> 172.21.15.113:0/3499485332 conn(0x7ff4f80fc1f0 msgr2=0x7ff4f80fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:37:19.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.619+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 shutdown_connections 2024-02-26T10:37:19.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:37:19.619+0000 7ff4fd010700 1 -- 172.21.15.113:0/3499485332 wait complete. 2024-02-26T10:37:20.218 DEBUG:teuthology.orchestra.run.smithi113:osd.2> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.2.service 2024-02-26T10:37:20.220 INFO:tasks.cephadm:Deploying osd.3 on smithi060 with /dev/vg_nvme/lv_3... 2024-02-26T10:37:20.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T10:37:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:37:20.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:20 smithi113 ceph-mon[36126]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:20.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:20 smithi113 ceph-mon[36126]: from='osd.2 [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T10:37:20.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:20 smithi113 ceph-mon[36126]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T10:37:20.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.848 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:20 smithi060 ceph-mon[31539]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:20.848 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.848 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.848 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:20 smithi060 ceph-mon[31539]: from='osd.2 [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T10:37:20.848 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:20 smithi060 ceph-mon[31539]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T10:37:20.848 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.849 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:20 smithi100 ceph-mon[36051]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:20 smithi100 ceph-mon[36051]: from='osd.2 [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T10:37:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:20 smithi100 ceph-mon[36051]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T10:37:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:20.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:21.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:21 smithi060 ceph-mon[31539]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T10:37:21.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:21 smithi060 ceph-mon[31539]: from='osd.2 [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:37:21.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:21 smithi060 ceph-mon[31539]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T10:37:21.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:21.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:21 smithi060 ceph-mon[31539]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:37:21.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:21 smithi100 ceph-mon[36051]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T10:37:21.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:21 smithi100 ceph-mon[36051]: from='osd.2 [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:37:21.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:21 smithi100 ceph-mon[36051]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T10:37:21.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:21.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:21 smithi100 ceph-mon[36051]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:37:21.947 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:21 smithi113 ceph-mon[36126]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T10:37:21.948 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:21 smithi113 ceph-mon[36126]: from='osd.2 [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:37:21.948 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:21 smithi113 ceph-mon[36126]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T10:37:21.948 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:21.948 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:21 smithi113 ceph-mon[36126]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:37:22.267 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2[41272]: 2024-02-26T10:37:22.028+0000 7f4c6c7c2700 -1 osd.2 0 waiting for initial osdmap 2024-02-26T10:37:22.267 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2[41272]: 2024-02-26T10:37:22.031+0000 7f4c645ad700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:37:22.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:22 smithi060 ceph-mon[31539]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:22.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:22 smithi060 ceph-mon[31539]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:37:22.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:22 smithi060 ceph-mon[31539]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T10:37:22.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:22.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:22.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:22 smithi060 ceph-mon[31539]: from='osd.2 ' entity='osd.2' 2024-02-26T10:37:22.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:22 smithi100 ceph-mon[36051]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:22.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:22 smithi100 ceph-mon[36051]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:37:22.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:22 smithi100 ceph-mon[36051]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T10:37:22.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:22.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:22.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:22 smithi100 ceph-mon[36051]: from='osd.2 ' entity='osd.2' 2024-02-26T10:37:23.033 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:22 smithi113 ceph-mon[36126]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:23.033 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:22 smithi113 ceph-mon[36126]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:37:23.033 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:22 smithi113 ceph-mon[36126]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T10:37:23.033 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:23.033 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:23.033 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:22 smithi113 ceph-mon[36126]: from='osd.2 ' entity='osd.2' 2024-02-26T10:37:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T10:37:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T10:37:23.534 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-02-26T10:37:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-02-26T10:37:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.040369 s, 260 MB/s 2024-02-26T10:37:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-02-26T10:37:23.805 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:23 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:37:23.805 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:23 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:37:23.806 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:23 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:23.806 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:23 smithi113 ceph-mon[36126]: osd.2 [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] boot 2024-02-26T10:37:23.806 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:23 smithi113 ceph-mon[36126]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T10:37:23.806 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:23 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:23 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:37:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:23 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:37:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:23 smithi060 ceph-mon[31539]: osd.2 [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] boot 2024-02-26T10:37:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:23 smithi060 ceph-mon[31539]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T10:37:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:23.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:23 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:37:23.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:23 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:37:23.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:23 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:23.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:23 smithi100 ceph-mon[36051]: osd.2 [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] boot 2024-02-26T10:37:23.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:23 smithi100 ceph-mon[36051]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T10:37:23.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:23 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T10:37:24.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_3 2024-02-26T10:37:24.759 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:37:24.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:24.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:24.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:25.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:25.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:25 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi113 2024-02-26T10:37:25.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:25 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi113 to 17080M 2024-02-26T10:37:25.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:37:25.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T10:37:25.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:25 smithi100 ceph-mon[36051]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T10:37:25.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:37:25.967 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:25 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi113 2024-02-26T10:37:25.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:25 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi113 to 17080M 2024-02-26T10:37:25.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:37:25.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T10:37:25.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:25 smithi060 ceph-mon[31539]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T10:37:25.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:37:26.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:25 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi113 2024-02-26T10:37:26.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:25 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi113 to 17080M 2024-02-26T10:37:26.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:37:26.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T10:37:26.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:25 smithi113 ceph-mon[36126]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T10:37:26.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:37:26.923 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:26 smithi060 ceph-mon[31539]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:26.923 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:26 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T10:37:26.923 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:26 smithi060 ceph-mon[31539]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T10:37:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.969+0000 7faaa2800700 1 -- 172.21.15.60:0/797096447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076d30 msgr2=0x7faa9c078b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.969+0000 7faaa2800700 1 --2- 172.21.15.60:0/797096447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076d30 0x7faa9c078b90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa94009f90 tx=0x7faa9400b330 comp rx=0 tx=0).stop 2024-02-26T10:37:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.969+0000 7faaa2800700 1 -- 172.21.15.60:0/797096447 shutdown_connections 2024-02-26T10:37:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.969+0000 7faaa2800700 1 --2- 172.21.15.60:0/797096447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076d30 0x7faa9c078b90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.969+0000 7faaa2800700 1 --2- 172.21.15.60:0/797096447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c077de0 0x7faa9c078200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.969+0000 7faaa2800700 1 --2- 172.21.15.60:0/797096447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa9c076370 0x7faa9c0767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.969+0000 7faaa2800700 1 -- 172.21.15.60:0/797096447 >> 172.21.15.60:0/797096447 conn(0x7faa9c0ff4d0 msgr2=0x7faa9c101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:37:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.969+0000 7faaa2800700 1 -- 172.21.15.60:0/797096447 shutdown_connections 2024-02-26T10:37:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.969+0000 7faaa2800700 1 -- 172.21.15.60:0/797096447 wait complete. 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.971+0000 7faaa2800700 1 Processor -- start 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.971+0000 7faaa2800700 1 -- start start 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.971+0000 7faaa2800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076370 0x7faa9c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.971+0000 7faaa2800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c076d30 0x7faa9c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.971+0000 7faaa2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa9c077de0 0x7faa9c1a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.971+0000 7faaa2800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c114060 con 0x7faa9c077de0 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.971+0000 7faaa2800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c113ee0 con 0x7faa9c076370 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.971+0000 7faaa2800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c1141e0 con 0x7faa9c076d30 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa9c077de0 0x7faa9c1a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9b400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076370 0x7faa9c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9aa00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c076d30 0x7faa9c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9b400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076370 0x7faa9c19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37274/0 (socket says 172.21.15.60:37274) 2024-02-26T10:37:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9b400700 1 -- 172.21.15.60:0/166379982 learned_addr learned my addr 172.21.15.60:0/166379982 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:37:26.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9b400700 1 -- 172.21.15.60:0/166379982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c076d30 msgr2=0x7faa9c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:26.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9b400700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c076d30 0x7faa9c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9b400700 1 -- 172.21.15.60:0/166379982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa9c077de0 msgr2=0x7faa9c1a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9b400700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa9c077de0 0x7faa9c1a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.972+0000 7faa9b400700 1 -- 172.21.15.60:0/166379982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c1a6220 con 0x7faa9c076370 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.973+0000 7faa9b400700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076370 0x7faa9c19e5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00e9c0 tx=0x7faa8c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.973+0000 7faa93e00700 1 -- 172.21.15.60:0/166379982 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00cbd0 con 0x7faa9c076370 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.973+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9c1a6510 con 0x7faa9c076370 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.973+0000 7faa93e00700 1 -- 172.21.15.60:0/166379982 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa8c00ee60 con 0x7faa9c076370 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.973+0000 7faa93e00700 1 -- 172.21.15.60:0/166379982 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c005680 con 0x7faa9c076370 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.973+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c1a6a50 con 0x7faa9c076370 2024-02-26T10:37:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.974+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9c068a10 con 0x7faa9c076370 2024-02-26T10:37:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.976+0000 7faa93e00700 1 -- 172.21.15.60:0/166379982 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7faa8c004750 con 0x7faa9c076370 2024-02-26T10:37:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.978+0000 7faa93e00700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa800a1e00 0x7faa800a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.978+0000 7faa93e00700 1 -- 172.21.15.60:0/166379982 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7faa8c0ccb90 con 0x7faa9c076370 2024-02-26T10:37:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.978+0000 7faa9aa00700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa800a1e00 0x7faa800a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.979+0000 7faa9aa00700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa800a1e00 0x7faa800a42c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa88009750 tx=0x7faa88009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:37:26.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:26.982+0000 7faa93e00700 1 -- 172.21.15.60:0/166379982 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa8c09b3d0 con 0x7faa9c076370 2024-02-26T10:37:27.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:26 smithi113 ceph-mon[36126]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:27.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:26 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T10:37:27.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:26 smithi113 ceph-mon[36126]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T10:37:27.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:26 smithi100 ceph-mon[36051]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:27.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:26 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T10:37:27.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:26 smithi100 ceph-mon[36051]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T10:37:27.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:27.216+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7faa9c062190 con 0x7faa800a1e00 2024-02-26T10:37:28.025 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:37:28.025 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:37:28.025 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:27 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:28.036 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:27 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:37:28.036 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:27 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:37:28.036 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:27 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:28.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:27 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:37:28.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:27 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:37:28.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:27 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:28.373 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:37:28 smithi060 sudo[51171]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T10:37:28.373 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:37:28 smithi060 sudo[51171]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:28.373 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:37:28 smithi060 sudo[51171]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:28.373 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:37:28 smithi060 sudo[51171]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:28.647 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:37:28 smithi060 sudo[51218]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T10:37:28.647 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:37:28 smithi060 sudo[51218]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:28.647 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:37:28 smithi060 sudo[51218]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:28.648 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:37:28 smithi060 sudo[51218]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:28.914 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:28 smithi100 ceph-mon[36051]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:28.915 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:28 smithi100 ceph-mon[36051]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:37:28.915 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:28 smithi100 ceph-mon[36051]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T10:37:28.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:28 smithi060 ceph-mon[31539]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:28.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:28 smithi060 ceph-mon[31539]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:37:28.952 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:28 smithi060 ceph-mon[31539]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T10:37:29.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:28 smithi113 ceph-mon[36126]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:29.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:28 smithi113 ceph-mon[36126]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:37:29.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:28 smithi113 ceph-mon[36126]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T10:37:29.168 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:37:28 smithi100 sudo[44475]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T10:37:29.168 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:37:28 smithi100 sudo[44475]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:29.168 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:37:28 smithi100 sudo[44475]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:29.168 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:37:28 smithi100 sudo[44475]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:29.635 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:37:29 smithi100 sudo[44478]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T10:37:29.635 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:37:29 smithi100 sudo[44478]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:29.635 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:37:29 smithi100 sudo[44478]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:29.635 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:37:29 smithi100 sudo[44478]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:29.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:29.665+0000 7faa93e00700 1 -- 172.21.15.60:0/166379982 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7faa8c097340 con 0x7faa9c076370 2024-02-26T10:37:29.709 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:29 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:29.709 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:29 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:29.709 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:29 smithi113 sudo[44917]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T10:37:29.709 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:29 smithi113 sudo[44917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:29.710 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:29 smithi113 sudo[44917]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:29.710 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:29 smithi113 sudo[44917]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:29.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:29 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:29.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:29 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:30.091 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:29 smithi113 sudo[44920]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T10:37:30.091 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:29 smithi113 sudo[44920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:30.092 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:29 smithi113 sudo[44920]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:30.092 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:37:29 smithi113 sudo[44920]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:30.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:29 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:30.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:29 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:30.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 sudo[51255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T10:37:30.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 sudo[51255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:30.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 sudo[51255]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:30.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 sudo[51255]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:31.163 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:30 smithi113 ceph-mon[36126]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:31.163 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:30 smithi113 ceph-mon[36126]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-26T10:37:31.164 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:30 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:31.164 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:30 smithi113 ceph-mon[36126]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T10:37:31.164 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 sudo[44923]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T10:37:31.164 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 sudo[44923]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:31.164 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 sudo[44923]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:31.164 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 sudo[44923]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 sudo[51283]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T10:37:31.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 sudo[51283]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:31.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 sudo[51283]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:31.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 sudo[51283]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:31.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 ceph-mon[31539]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:31.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 ceph-mon[31539]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-26T10:37:31.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:31.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:30 smithi060 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T10:37:31.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:30 smithi100 ceph-mon[36051]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:31.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:30 smithi100 ceph-mon[36051]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-26T10:37:31.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:30 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:31.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:30 smithi100 ceph-mon[36051]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T10:37:31.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 sudo[44926]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T10:37:31.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 sudo[44926]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:31.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 sudo[44926]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:31.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 sudo[44926]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 sudo[44482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T10:37:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 sudo[44482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 sudo[44482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 sudo[44482]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 sudo[44485]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T10:37:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 sudo[44485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T10:37:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 sudo[44485]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-26T10:37:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 sudo[44485]: pam_unix(sudo:session): session closed for user root 2024-02-26T10:37:32.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1558209339' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e"}]: dispatch 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T10:37:32.342 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1558209339' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e"}]': finished 2024-02-26T10:37:32.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T10:37:32.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: osdmap e22: 4 total, 3 up, 4 in 2024-02-26T10:37:32.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:32.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:37:32.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:37:32.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:37:32.343 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:31 smithi113 ceph-mon[36126]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:37:32.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1558209339' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e"}]: dispatch 2024-02-26T10:37:32.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T10:37:32.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1558209339' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e"}]': finished 2024-02-26T10:37:32.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T10:37:32.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: osdmap e22: 4 total, 3 up, 4 in 2024-02-26T10:37:32.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:32.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:37:32.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:37:32.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:37:32.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:31 smithi060 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1558209339' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e"}]: dispatch 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1558209339' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e"}]': finished 2024-02-26T10:37:32.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T10:37:32.386 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: osdmap e22: 4 total, 3 up, 4 in 2024-02-26T10:37:32.386 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:32.386 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T10:37:32.386 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T10:37:32.386 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T10:37:32.386 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:31 smithi100 ceph-mon[36051]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T10:37:33.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:32 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1532648169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:37:33.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:32 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1532648169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:37:33.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:32 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1532648169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:37:34.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:33 smithi113 ceph-mon[36126]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:34.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:33 smithi060 ceph-mon[31539]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:34.385 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:33 smithi100 ceph-mon[36051]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:36.195 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:35 smithi060 ceph-mon[31539]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-02-26T10:37:36.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:35 smithi113 ceph-mon[36126]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-02-26T10:37:36.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:35 smithi100 ceph-mon[36051]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-02-26T10:37:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:37 smithi060 ceph-mon[31539]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-26T10:37:38.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:37 smithi113 ceph-mon[36126]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-26T10:37:38.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:37 smithi100 ceph-mon[36051]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-26T10:37:40.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:39 smithi060 ceph-mon[31539]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-26T10:37:40.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:39 smithi113 ceph-mon[36126]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-26T10:37:40.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:39 smithi100 ceph-mon[36051]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-26T10:37:41.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T10:37:41.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:41.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T10:37:41.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:41.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T10:37:41.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:42.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:41 smithi060 ceph-mon[31539]: Deploying daemon osd.3 on smithi060 2024-02-26T10:37:42.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:41 smithi060 ceph-mon[31539]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-26T10:37:42.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:41 smithi113 ceph-mon[36126]: Deploying daemon osd.3 on smithi060 2024-02-26T10:37:42.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:41 smithi113 ceph-mon[36126]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-26T10:37:42.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:41 smithi100 ceph-mon[36051]: Deploying daemon osd.3 on smithi060 2024-02-26T10:37:42.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:41 smithi100 ceph-mon[36051]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-26T10:37:44.335 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:43 smithi060 ceph-mon[31539]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-02-26T10:37:44.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:43 smithi113 ceph-mon[36126]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-02-26T10:37:44.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:43 smithi100 ceph-mon[36051]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-02-26T10:37:46.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:45 smithi060 ceph-mon[31539]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-26T10:37:46.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:45 smithi113 ceph-mon[36126]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-26T10:37:46.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:45 smithi100 ceph-mon[36051]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-26T10:37:48.240 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:47 smithi060 ceph-mon[31539]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:48.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:47 smithi113 ceph-mon[36126]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:48.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:47 smithi100 ceph-mon[36051]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:50.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:50 smithi060 ceph-mon[31539]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:50.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:50.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:50.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:50.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:50 smithi113 ceph-mon[36126]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:50.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:50.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:50.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:50.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:50 smithi100 ceph-mon[36051]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:50.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:50.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:50.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:52.474 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:52 smithi060 ceph-mon[31539]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:52.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:52 smithi113 ceph-mon[36126]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:52.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:52 smithi100 ceph-mon[36051]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:53.769 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 3 on host 'smithi060' 2024-02-26T10:37:53.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.764+0000 7faa93e00700 1 -- 172.21.15.60:0/166379982 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faa9c062190 con 0x7faa800a1e00 2024-02-26T10:37:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.768+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa800a1e00 msgr2=0x7faa800a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.768+0000 7faaa2800700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa800a1e00 0x7faa800a42c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa88009750 tx=0x7faa88009310 comp rx=0 tx=0).stop 2024-02-26T10:37:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.768+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076370 msgr2=0x7faa9c19e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.768+0000 7faaa2800700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076370 0x7faa9c19e5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00e9c0 tx=0x7faa8c00ecd0 comp rx=0 tx=0).stop 2024-02-26T10:37:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.769+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 shutdown_connections 2024-02-26T10:37:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.769+0000 7faaa2800700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa800a1e00 0x7faa800a42c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.769+0000 7faaa2800700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa9c077de0 0x7faa9c1a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.769+0000 7faaa2800700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9c076370 0x7faa9c19e5c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.769+0000 7faaa2800700 1 --2- 172.21.15.60:0/166379982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c076d30 0x7faa9c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.769+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 >> 172.21.15.60:0/166379982 conn(0x7faa9c0ff4d0 msgr2=0x7faa9c100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:37:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.769+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 shutdown_connections 2024-02-26T10:37:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:37:53.770+0000 7faaa2800700 1 -- 172.21.15.60:0/166379982 wait complete. 2024-02-26T10:37:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='osd.3 [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T10:37:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T10:37:54.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:54.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='osd.3 [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:54.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:37:54.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:37:54.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:54.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='osd.3 [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T10:37:54.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T10:37:54.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:37:54.135 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:37:54.690 DEBUG:teuthology.orchestra.run.smithi060:osd.3> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.3.service 2024-02-26T10:37:54.696 INFO:tasks.cephadm:Deploying osd.4 on smithi100 with /dev/vg_nvme/lv_3... 2024-02-26T10:37:54.696 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T10:37:54.941 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:37:55.063 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:54 smithi100 ceph-mon[36051]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T10:37:55.063 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:54 smithi100 ceph-mon[36051]: from='osd.3 [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:37:55.064 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:54 smithi100 ceph-mon[36051]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T10:37:55.064 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:55.064 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:54 smithi100 ceph-mon[36051]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:37:55.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:54 smithi113 ceph-mon[36126]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T10:37:55.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:54 smithi113 ceph-mon[36126]: from='osd.3 [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:37:55.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:54 smithi113 ceph-mon[36126]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T10:37:55.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:55.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:54 smithi113 ceph-mon[36126]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:37:55.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:54 smithi060 ceph-mon[31539]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T10:37:55.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:54 smithi060 ceph-mon[31539]: from='osd.3 [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:37:55.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:54 smithi060 ceph-mon[31539]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T10:37:55.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:55.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:54 smithi060 ceph-mon[31539]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:37:55.803 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:37:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3[53325]: 2024-02-26T10:37:55.447+0000 7f902a400700 -1 osd.3 0 waiting for initial osdmap 2024-02-26T10:37:55.803 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:37:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3[53325]: 2024-02-26T10:37:55.469+0000 7f9020200700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:37:56.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:55 smithi100 ceph-mon[36051]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:56.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:55 smithi100 ceph-mon[36051]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:37:56.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:55 smithi100 ceph-mon[36051]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T10:37:56.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:56.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:56.054 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:55 smithi100 ceph-mon[36051]: from='osd.3 ' entity='osd.3' 2024-02-26T10:37:56.090 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:55 smithi113 ceph-mon[36126]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:56.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:55 smithi113 ceph-mon[36126]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:37:56.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:55 smithi113 ceph-mon[36126]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T10:37:56.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:56.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:56.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:55 smithi113 ceph-mon[36126]: from='osd.3 ' entity='osd.3' 2024-02-26T10:37:56.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:55 smithi060 ceph-mon[31539]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:56.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:55 smithi060 ceph-mon[31539]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:37:56.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:55 smithi060 ceph-mon[31539]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T10:37:56.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:56.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:56.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:55 smithi060 ceph-mon[31539]: from='osd.3 ' entity='osd.3' 2024-02-26T10:37:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T10:37:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T10:37:56.959 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-02-26T10:37:56.959 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-02-26T10:37:56.959 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112648 s, 931 MB/s 2024-02-26T10:37:56.960 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-02-26T10:37:57.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:56 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:37:57.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:56 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:37:57.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:56 smithi113 ceph-mon[36126]: osd.3 [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] boot 2024-02-26T10:37:57.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:56 smithi113 ceph-mon[36126]: osdmap e25: 4 total, 4 up, 4 in 2024-02-26T10:37:57.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:57.092 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:56 smithi113 ceph-mon[36126]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T10:37:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:56 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:37:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:56 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:37:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:56 smithi060 ceph-mon[31539]: osd.3 [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] boot 2024-02-26T10:37:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:56 smithi060 ceph-mon[31539]: osdmap e25: 4 total, 4 up, 4 in 2024-02-26T10:37:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:56 smithi060 ceph-mon[31539]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T10:37:57.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:56 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:37:57.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:56 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:37:57.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:56 smithi100 ceph-mon[36051]: osd.3 [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] boot 2024-02-26T10:37:57.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:56 smithi100 ceph-mon[36051]: osdmap e25: 4 total, 4 up, 4 in 2024-02-26T10:37:57.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T10:37:57.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:56 smithi100 ceph-mon[36051]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T10:37:57.601 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-02-26T10:37:57.842 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:37:58.153 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:57 smithi060 ceph-mon[31539]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:58.153 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:57 smithi060 ceph-mon[31539]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T10:37:58.212 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:57 smithi100 ceph-mon[36051]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:58.212 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:57 smithi100 ceph-mon[36051]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T10:37:58.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:57 smithi113 ceph-mon[36126]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T10:37:58.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:57 smithi113 ceph-mon[36126]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T10:37:59.091 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:58 smithi113 ceph-mon[36126]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T10:37:59.254 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:58 smithi100 ceph-mon[36051]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T10:37:59.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:58 smithi060 ceph-mon[31539]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T10:37:59.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.491+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1352694198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc101e50 msgr2=0x7f0dbc1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:59.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.491+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1352694198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc101e50 0x7f0dbc1022b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0da80099c0 tx=0x7f0da8009cd0 comp rx=0 tx=0).stop 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.491+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1352694198 shutdown_connections 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.491+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1352694198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dbc1027f0 0x7f0dbc1090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.491+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1352694198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dbc100c50 0x7f0dbc101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.491+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1352694198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc101e50 0x7f0dbc1022b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.491+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1352694198 >> 172.21.15.100:0/1352694198 conn(0x7f0dbc0fc1f0 msgr2=0x7f0dbc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.491+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1352694198 shutdown_connections 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.491+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1352694198 wait complete. 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc2d68700 1 Processor -- start 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc2d68700 1 -- start start 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc2d68700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dbc100c50 0x7f0dbc06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc2d68700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dbc101e50 0x7f0dbc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:59.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc2d68700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc1027f0 0x7f0dbc19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc2d68700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc10c8b0 con 0x7f0dbc100c50 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc2d68700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc10c730 con 0x7f0dbc1027f0 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc2d68700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc10ca30 con 0x7f0dbc101e50 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc1305700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc1027f0 0x7f0dbc19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc1305700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc1027f0 0x7f0dbc19f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48310/0 (socket says 172.21.15.100:48310) 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc1305700 1 -- 172.21.15.100:0/1573391615 learned_addr learned my addr 172.21.15.100:0/1573391615 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dbbfff700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dbc101e50 0x7f0dbc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc0b04700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dbc100c50 0x7f0dbc06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc1305700 1 -- 172.21.15.100:0/1573391615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dbc101e50 msgr2=0x7f0dbc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc1305700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dbc101e50 0x7f0dbc19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc1305700 1 -- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dbc100c50 msgr2=0x7f0dbc06a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc1305700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dbc100c50 0x7f0dbc06a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc1305700 1 -- 172.21.15.100:0/1573391615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc19fc40 con 0x7f0dbc1027f0 2024-02-26T10:37:59.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.492+0000 7f0dc0b04700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dbc100c50 0x7f0dbc06a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:37:59.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.493+0000 7f0dc1305700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc1027f0 0x7f0dbc19f370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0db400b970 tx=0x7f0db400bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:37:59.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.493+0000 7f0db9ffb700 1 -- 172.21.15.100:0/1573391615 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4019070 con 0x7f0dbc1027f0 2024-02-26T10:37:59.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.493+0000 7f0db9ffb700 1 -- 172.21.15.100:0/1573391615 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0db4014440 con 0x7f0dbc1027f0 2024-02-26T10:37:59.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.493+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc19ff30 con 0x7f0dbc1027f0 2024-02-26T10:37:59.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.493+0000 7f0db9ffb700 1 -- 172.21.15.100:0/1573391615 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db40075d0 con 0x7f0dbc1027f0 2024-02-26T10:37:59.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.493+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc1a04c0 con 0x7f0dbc1027f0 2024-02-26T10:37:59.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.494+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc068a10 con 0x7f0dbc1027f0 2024-02-26T10:37:59.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.495+0000 7f0db9ffb700 1 -- 172.21.15.100:0/1573391615 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0db401c070 con 0x7f0dbc1027f0 2024-02-26T10:37:59.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.495+0000 7f0db9ffb700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dac0a1e10 0x7f0dac0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:37:59.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.495+0000 7f0db9ffb700 1 -- 172.21.15.100:0/1573391615 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f0db40cbf50 con 0x7f0dbc1027f0 2024-02-26T10:37:59.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.496+0000 7f0dc0b04700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dac0a1e10 0x7f0dac0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:37:59.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.497+0000 7f0dc0b04700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dac0a1e10 0x7f0dac0a42d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0db0005fd0 tx=0x7f0db0005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:37:59.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.500+0000 7f0db9ffb700 1 -- 172.21.15.100:0/1573391615 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0db409a5a0 con 0x7f0dbc1027f0 2024-02-26T10:37:59.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:37:59.701+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dbc062190 con 0x7f0dac0a1e10 2024-02-26T10:38:00.314 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:59 smithi100 ceph-mon[36051]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:00.314 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:59 smithi100 ceph-mon[36051]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T10:38:00.314 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:38:00.314 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:38:00.314 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:37:59 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:00.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:59 smithi113 ceph-mon[36126]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:00.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:59 smithi113 ceph-mon[36126]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T10:38:00.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:38:00.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:38:00.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:37:59 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:00.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:59 smithi060 ceph-mon[31539]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:00.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:59 smithi060 ceph-mon[31539]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T10:38:00.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:38:00.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:38:00.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:37:59 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:00 smithi060 ceph-mon[31539]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:38:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.299 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:00 smithi100 ceph-mon[36051]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:38:01.299 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.299 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.299 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:01.299 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:01.299 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.299 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:01.299 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:01.300 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:00 smithi113 ceph-mon[36126]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:38:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:01.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:02.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:01 smithi113 ceph-mon[36126]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-02-26T10:38:02.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:01 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi060 2024-02-26T10:38:02.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:01 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi060 to 8547M 2024-02-26T10:38:02.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:01 smithi060 ceph-mon[31539]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-02-26T10:38:02.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:01 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi060 2024-02-26T10:38:02.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:01 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi060 to 8547M 2024-02-26T10:38:02.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:01 smithi100 ceph-mon[36051]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-02-26T10:38:02.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:01 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi060 2024-02-26T10:38:02.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:01 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi060 to 8547M 2024-02-26T10:38:03.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:02 smithi100 ceph-mon[36051]: from='client.? 172.21.15.100:0/2664991488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3719710-9869-4caf-bfcf-cbe453306583"}]: dispatch 2024-02-26T10:38:03.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:02 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3719710-9869-4caf-bfcf-cbe453306583"}]: dispatch 2024-02-26T10:38:03.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:02 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3719710-9869-4caf-bfcf-cbe453306583"}]': finished 2024-02-26T10:38:03.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:02 smithi100 ceph-mon[36051]: osdmap e30: 5 total, 4 up, 5 in 2024-02-26T10:38:03.134 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:02 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:03.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:02 smithi113 ceph-mon[36126]: from='client.? 172.21.15.100:0/2664991488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3719710-9869-4caf-bfcf-cbe453306583"}]: dispatch 2024-02-26T10:38:03.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:02 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3719710-9869-4caf-bfcf-cbe453306583"}]: dispatch 2024-02-26T10:38:03.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:02 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3719710-9869-4caf-bfcf-cbe453306583"}]': finished 2024-02-26T10:38:03.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:02 smithi113 ceph-mon[36126]: osdmap e30: 5 total, 4 up, 5 in 2024-02-26T10:38:03.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:02 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:03.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:02 smithi060 ceph-mon[31539]: from='client.? 172.21.15.100:0/2664991488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3719710-9869-4caf-bfcf-cbe453306583"}]: dispatch 2024-02-26T10:38:03.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:02 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3719710-9869-4caf-bfcf-cbe453306583"}]: dispatch 2024-02-26T10:38:03.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:02 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3719710-9869-4caf-bfcf-cbe453306583"}]': finished 2024-02-26T10:38:03.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:02 smithi060 ceph-mon[31539]: osdmap e30: 5 total, 4 up, 5 in 2024-02-26T10:38:03.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:02 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:04.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:03 smithi113 ceph-mon[36126]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-02-26T10:38:04.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:03 smithi113 ceph-mon[36126]: from='client.? 172.21.15.100:0/1136784323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:38:04.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:03 smithi060 ceph-mon[31539]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-02-26T10:38:04.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:03 smithi060 ceph-mon[31539]: from='client.? 172.21.15.100:0/1136784323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:38:04.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:03 smithi100 ceph-mon[36051]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-02-26T10:38:04.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:03 smithi100 ceph-mon[36051]: from='client.? 172.21.15.100:0/1136784323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:38:06.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:05 smithi113 ceph-mon[36126]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-26T10:38:06.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:05 smithi060 ceph-mon[31539]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-26T10:38:06.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:05 smithi100 ceph-mon[36051]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-26T10:38:08.288 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:07 smithi100 ceph-mon[36051]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T10:38:08.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:07 smithi113 ceph-mon[36126]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T10:38:08.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:07 smithi060 ceph-mon[31539]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T10:38:10.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:09 smithi113 ceph-mon[36126]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-26T10:38:10.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:09 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T10:38:10.341 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:09 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:10.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:09 smithi100 ceph-mon[36051]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-26T10:38:10.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:09 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T10:38:10.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:09 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:10.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:09 smithi060 ceph-mon[31539]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-26T10:38:10.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:09 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T10:38:10.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:09 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:11.318 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:10 smithi100 ceph-mon[36051]: Deploying daemon osd.4 on smithi100 2024-02-26T10:38:11.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:10 smithi113 ceph-mon[36126]: Deploying daemon osd.4 on smithi100 2024-02-26T10:38:11.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:10 smithi060 ceph-mon[31539]: Deploying daemon osd.4 on smithi100 2024-02-26T10:38:12.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:11 smithi113 ceph-mon[36126]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:12.349 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:11 smithi100 ceph-mon[36051]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:12.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:11 smithi060 ceph-mon[31539]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:14.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:13 smithi113 ceph-mon[36126]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:14.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:13 smithi060 ceph-mon[31539]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:14.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:13 smithi100 ceph-mon[36051]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:16.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:16 smithi113 ceph-mon[36126]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:16.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:16.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:16.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:16.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:16 smithi060 ceph-mon[31539]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:16.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:16.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:16.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:16.633 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:16 smithi100 ceph-mon[36051]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:16.633 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:16.633 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:16.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:18.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:18 smithi113 ceph-mon[36126]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:18.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:18 smithi060 ceph-mon[31539]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:18.633 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:18 smithi100 ceph-mon[36051]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:18.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.993+0000 7f0db9ffb700 1 -- 172.21.15.100:0/1573391615 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0dbc062190 con 0x7f0dac0a1e10 2024-02-26T10:38:18.996 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 4 on host 'smithi100' 2024-02-26T10:38:18.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dac0a1e10 msgr2=0x7f0dac0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dac0a1e10 0x7f0dac0a42d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0db0005fd0 tx=0x7f0db0005f00 comp rx=0 tx=0).stop 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc1027f0 msgr2=0x7f0dbc19f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc1027f0 0x7f0dbc19f370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0db400b970 tx=0x7f0db400bd30 comp rx=0 tx=0).stop 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 shutdown_connections 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0dac0a1e10 0x7f0dac0a42d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dbc1027f0 0x7f0dbc19f370 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dbc100c50 0x7f0dbc06a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 --2- 172.21.15.100:0/1573391615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dbc101e50 0x7f0dbc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 >> 172.21.15.100:0/1573391615 conn(0x7f0dbc0fc1f0 msgr2=0x7f0dbc105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 shutdown_connections 2024-02-26T10:38:18.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:38:18.995+0000 7f0dc2d68700 1 -- 172.21.15.100:0/1573391615 wait complete. 2024-02-26T10:38:19.612 DEBUG:teuthology.orchestra.run.smithi100:osd.4> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.4.service 2024-02-26T10:38:19.615 INFO:tasks.cephadm:Deploying osd.5 on smithi113 with /dev/vg_nvme/lv_3... 2024-02-26T10:38:19.615 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T10:38:19.848 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:38:20.241 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:19 smithi113 ceph-mon[36126]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:20.241 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.241 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.241 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:19 smithi113 ceph-mon[36126]: from='osd.4 [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T10:38:20.241 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:19 smithi113 ceph-mon[36126]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T10:38:20.241 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.241 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:19 smithi060 ceph-mon[31539]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:20.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:19 smithi060 ceph-mon[31539]: from='osd.4 [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T10:38:20.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:19 smithi060 ceph-mon[31539]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T10:38:20.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:19 smithi100 ceph-mon[36051]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:20.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:19 smithi100 ceph-mon[36051]: from='osd.4 [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T10:38:20.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:19 smithi100 ceph-mon[36051]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T10:38:20.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:20.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:21.315 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:21 smithi113 ceph-mon[36126]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T10:38:21.316 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:21 smithi113 ceph-mon[36126]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T10:38:21.316 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:21.316 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:21 smithi113 ceph-mon[36126]: from='osd.4 [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:38:21.316 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:21 smithi113 ceph-mon[36126]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:38:21.331 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:20 smithi100 ceph-mon[36051]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T10:38:21.331 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:20 smithi100 ceph-mon[36051]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T10:38:21.331 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:21.331 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:20 smithi100 ceph-mon[36051]: from='osd.4 [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:38:21.331 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:20 smithi100 ceph-mon[36051]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:38:21.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:20 smithi060 ceph-mon[31539]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T10:38:21.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:20 smithi060 ceph-mon[31539]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T10:38:21.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:21.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:20 smithi060 ceph-mon[31539]: from='osd.4 [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:38:21.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:20 smithi060 ceph-mon[31539]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:38:21.634 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:38:21 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4[47215]: 2024-02-26T10:38:21.437+0000 7f58e2631700 -1 osd.4 0 waiting for initial osdmap 2024-02-26T10:38:21.634 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:38:21 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4[47215]: 2024-02-26T10:38:21.442+0000 7f58da41c700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:21.856 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T10:38:21.856 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T10:38:21.856 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-02-26T10:38:21.856 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-02-26T10:38:21.857 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00931284 s, 1.1 GB/s 2024-02-26T10:38:21.857 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-02-26T10:38:22.193 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:22 smithi113 ceph-mon[36126]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:22.193 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:22 smithi113 ceph-mon[36126]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:38:22.193 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:22 smithi113 ceph-mon[36126]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T10:38:22.193 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:22.193 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:22.193 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:22 smithi113 ceph-mon[36126]: from='osd.4 ' entity='osd.4' 2024-02-26T10:38:22.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:21 smithi060 ceph-mon[31539]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:22.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:21 smithi060 ceph-mon[31539]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:38:22.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:22 smithi060 ceph-mon[31539]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T10:38:22.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:22.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:22.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:22 smithi060 ceph-mon[31539]: from='osd.4 ' entity='osd.4' 2024-02-26T10:38:22.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:21 smithi100 ceph-mon[36051]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:22.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:21 smithi100 ceph-mon[36051]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:38:22.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:21 smithi100 ceph-mon[36051]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T10:38:22.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:22.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:22.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:22 smithi100 ceph-mon[36051]: from='osd.4 ' entity='osd.4' 2024-02-26T10:38:22.529 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi113:vg_nvme/lv_3 2024-02-26T10:38:22.764 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:38:23.171 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:23 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:38:23.171 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:23 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:38:23.171 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:23 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:23.171 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:23 smithi113 ceph-mon[36126]: osd.4 [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] boot 2024-02-26T10:38:23.171 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:23 smithi113 ceph-mon[36126]: osdmap e33: 5 total, 5 up, 5 in 2024-02-26T10:38:23.171 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:23 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:23.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:23 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:38:23.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:23 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:38:23.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:23.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:23 smithi060 ceph-mon[31539]: osd.4 [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] boot 2024-02-26T10:38:23.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:23 smithi060 ceph-mon[31539]: osdmap e33: 5 total, 5 up, 5 in 2024-02-26T10:38:23.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:23.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:23 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:38:23.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:23 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:38:23.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:23 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:23.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:23 smithi100 ceph-mon[36051]: osd.4 [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] boot 2024-02-26T10:38:23.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:23 smithi100 ceph-mon[36051]: osdmap e33: 5 total, 5 up, 5 in 2024-02-26T10:38:23.384 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:23 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T10:38:24.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.467+0000 7eff77d1a700 1 -- 172.21.15.113:0/2794213556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 msgr2=0x7eff70109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:24.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.467+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2794213556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 0x7eff70109eb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff68009990 tx=0x7eff68009ca0 comp rx=0 tx=0).stop 2024-02-26T10:38:24.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.467+0000 7eff77d1a700 1 -- 172.21.15.113:0/2794213556 shutdown_connections 2024-02-26T10:38:24.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.467+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2794213556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 0x7eff70109eb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:24.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.467+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2794213556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff701027e0 0x7eff70104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:24.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.467+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2794213556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff70105110 0x7eff70107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:24.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.467+0000 7eff77d1a700 1 -- 172.21.15.113:0/2794213556 >> 172.21.15.113:0/2794213556 conn(0x7eff700fc1f0 msgr2=0x7eff700fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:24.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.467+0000 7eff77d1a700 1 -- 172.21.15.113:0/2794213556 shutdown_connections 2024-02-26T10:38:24.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.467+0000 7eff77d1a700 1 -- 172.21.15.113:0/2794213556 wait complete. 2024-02-26T10:38:24.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff77d1a700 1 Processor -- start 2024-02-26T10:38:24.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff77d1a700 1 -- start start 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff77d1a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff701027e0 0x7eff70198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff77d1a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff70105110 0x7eff70198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff77d1a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 0x7eff7019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff77d1a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7010d9a0 con 0x7eff701027e0 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff77d1a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7010d820 con 0x7eff70105110 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff77d1a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7010db20 con 0x7eff70107a40 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff762b7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 0x7eff7019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff752b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff70105110 0x7eff70198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff762b7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 0x7eff7019f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44970/0 (socket says 172.21.15.113:44970) 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff762b7700 1 -- 172.21.15.113:0/2043225412 learned_addr learned my addr 172.21.15.113:0/2043225412 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:38:24.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff752b5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff70105110 0x7eff70198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.113:46996/0 (socket says 172.21.15.113:46996) 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff75ab6700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff701027e0 0x7eff70198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff762b7700 1 -- 172.21.15.113:0/2043225412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff70105110 msgr2=0x7eff70198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff762b7700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff70105110 0x7eff70198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff762b7700 1 -- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff701027e0 msgr2=0x7eff70198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff762b7700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff701027e0 0x7eff70198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff762b7700 1 -- 172.21.15.113:0/2043225412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7019fc70 con 0x7eff70107a40 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.468+0000 7eff762b7700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 0x7eff7019f5d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7eff680059d0 tx=0x7eff6800bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.469+0000 7eff66ffd700 1 -- 172.21.15.113:0/2043225412 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68026070 con 0x7eff70107a40 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.469+0000 7eff66ffd700 1 -- 172.21.15.113:0/2043225412 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff6800f460 con 0x7eff70107a40 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.469+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7019ff00 con 0x7eff70107a40 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.469+0000 7eff66ffd700 1 -- 172.21.15.113:0/2043225412 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68021670 con 0x7eff70107a40 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.469+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff701a02a0 con 0x7eff70107a40 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.470+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff70068a10 con 0x7eff70107a40 2024-02-26T10:38:24.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.471+0000 7eff66ffd700 1 -- 172.21.15.113:0/2043225412 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7eff68029070 con 0x7eff70107a40 2024-02-26T10:38:24.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.471+0000 7eff66ffd700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff5c0a1e10 0x7eff5c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:24.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.472+0000 7eff66ffd700 1 -- 172.21.15.113:0/2043225412 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7eff680ce1b0 con 0x7eff70107a40 2024-02-26T10:38:24.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.472+0000 7eff75ab6700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff5c0a1e10 0x7eff5c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:24.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.472+0000 7eff75ab6700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff5c0a1e10 0x7eff5c0a42d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7eff6c007dc0 tx=0x7eff6c0072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:24.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.475+0000 7eff66ffd700 1 -- 172.21.15.113:0/2043225412 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff68030080 con 0x7eff70107a40 2024-02-26T10:38:24.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:24.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T10:38:24.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:24.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:24.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:24.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:24.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.633 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:24.633 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T10:38:24.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:24.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:24.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:24.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:24.634 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:24.680+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7eff70062190 con 0x7eff5c0a1e10 2024-02-26T10:38:24.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T10:38:24.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T10:38:24.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:24.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:24.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:24.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:24.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:24.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:25.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi100 2024-02-26T10:38:25.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi100 to 8540M 2024-02-26T10:38:25.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:25.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:25.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:38:25.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:25.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:38:25.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:25.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:38:25.591 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:25 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:25.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi100 2024-02-26T10:38:25.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi100 to 8540M 2024-02-26T10:38:25.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:25.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:25.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:38:25.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:25.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:38:25.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:25.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:38:25.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:25 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:25.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi100 2024-02-26T10:38:25.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi100 to 8540M 2024-02-26T10:38:25.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:25.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:25.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:38:25.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:25.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:38:25.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:25.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:38:25.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:25 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:26.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:26 smithi113 ceph-mon[36126]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:26.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:26 smithi060 ceph-mon[31539]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:26.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:26 smithi100 ceph-mon[36051]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:28.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:28 smithi113 ceph-mon[36126]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:28.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:28 smithi113 ceph-mon[36126]: from='client.? 172.21.15.113:0/1874522715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecd0dba3-6cb1-4e97-9b63-c83d0485c759"}]: dispatch 2024-02-26T10:38:28.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:28 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecd0dba3-6cb1-4e97-9b63-c83d0485c759"}]: dispatch 2024-02-26T10:38:28.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:28 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecd0dba3-6cb1-4e97-9b63-c83d0485c759"}]': finished 2024-02-26T10:38:28.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:28 smithi113 ceph-mon[36126]: osdmap e35: 6 total, 5 up, 6 in 2024-02-26T10:38:28.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:28.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:28 smithi113 ceph-mon[36126]: from='client.? 172.21.15.113:0/2898564541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:38:28.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:28 smithi060 ceph-mon[31539]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:28.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:28 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/1874522715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecd0dba3-6cb1-4e97-9b63-c83d0485c759"}]: dispatch 2024-02-26T10:38:28.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:28 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecd0dba3-6cb1-4e97-9b63-c83d0485c759"}]: dispatch 2024-02-26T10:38:28.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:28 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecd0dba3-6cb1-4e97-9b63-c83d0485c759"}]': finished 2024-02-26T10:38:28.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:28 smithi060 ceph-mon[31539]: osdmap e35: 6 total, 5 up, 6 in 2024-02-26T10:38:28.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:28.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:28 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/2898564541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:38:28.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:28 smithi100 ceph-mon[36051]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:28.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:28 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/1874522715' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecd0dba3-6cb1-4e97-9b63-c83d0485c759"}]: dispatch 2024-02-26T10:38:28.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:28 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecd0dba3-6cb1-4e97-9b63-c83d0485c759"}]: dispatch 2024-02-26T10:38:28.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:28 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecd0dba3-6cb1-4e97-9b63-c83d0485c759"}]': finished 2024-02-26T10:38:28.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:28 smithi100 ceph-mon[36051]: osdmap e35: 6 total, 5 up, 6 in 2024-02-26T10:38:28.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:28.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:28 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/2898564541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:38:30.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:30 smithi113 ceph-mon[36126]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:30.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:30 smithi060 ceph-mon[31539]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:30.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:30 smithi100 ceph-mon[36051]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:32.812 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:32 smithi113 ceph-mon[36126]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:32.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:32 smithi060 ceph-mon[31539]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:32.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:32 smithi100 ceph-mon[36051]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:34.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:34 smithi113 ceph-mon[36126]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:34.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:34 smithi060 ceph-mon[31539]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:34.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:34 smithi100 ceph-mon[36051]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:35.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:35 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T10:38:35.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:35 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:35.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:35 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T10:38:35.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:35 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:35.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:35 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T10:38:35.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:35 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:36.805 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:36 smithi113 ceph-mon[36126]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:36.805 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:36 smithi113 ceph-mon[36126]: Deploying daemon osd.5 on smithi113 2024-02-26T10:38:36.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:36 smithi060 ceph-mon[31539]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:36.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:36 smithi060 ceph-mon[31539]: Deploying daemon osd.5 on smithi113 2024-02-26T10:38:36.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:36 smithi100 ceph-mon[36051]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:36.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:36 smithi100 ceph-mon[36051]: Deploying daemon osd.5 on smithi113 2024-02-26T10:38:38.590 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:38 smithi113 ceph-mon[36126]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:38.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:38 smithi060 ceph-mon[31539]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:38.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:38 smithi100 ceph-mon[36051]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:40.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:40 smithi113 ceph-mon[36126]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:40.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:40 smithi060 ceph-mon[31539]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:40.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:40 smithi100 ceph-mon[36051]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:42.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:41 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:42.090 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:41 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:42.090 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:41 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:42.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:41 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:42.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:41 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:42.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:41 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:42.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:41 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:42.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:41 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:42.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:41 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:43.073 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:42 smithi113 ceph-mon[36126]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:43.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:42 smithi060 ceph-mon[31539]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:43.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:42 smithi100 ceph-mon[36051]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:44.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.620+0000 7eff66ffd700 1 -- 172.21.15.113:0/2043225412 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7eff70062190 con 0x7eff5c0a1e10 2024-02-26T10:38:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 5 on host 'smithi113' 2024-02-26T10:38:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff5c0a1e10 msgr2=0x7eff5c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff5c0a1e10 0x7eff5c0a42d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7eff6c007dc0 tx=0x7eff6c0072f0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 msgr2=0x7eff7019f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 0x7eff7019f5d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7eff680059d0 tx=0x7eff6800bbb0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 shutdown_connections 2024-02-26T10:38:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff5c0a1e10 0x7eff5c0a42d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff70107a40 0x7eff7019f5d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff701027e0 0x7eff70198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 --2- 172.21.15.113:0/2043225412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff70105110 0x7eff70198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 >> 172.21.15.113:0/2043225412 conn(0x7eff700fc1f0 msgr2=0x7eff70103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 shutdown_connections 2024-02-26T10:38:44.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:38:44.622+0000 7eff77d1a700 1 -- 172.21.15.113:0/2043225412 wait complete. 2024-02-26T10:38:44.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='osd.5 [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:44.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:45.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:45.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:45.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:45.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='osd.5 [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T10:38:45.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T10:38:45.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='osd.5 [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:45.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:38:45.285 DEBUG:teuthology.orchestra.run.smithi113:osd.5> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.5.service 2024-02-26T10:38:45.287 INFO:tasks.cephadm:Deploying osd.6 on smithi060 with /dev/vg_nvme/lv_2... 2024-02-26T10:38:45.288 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T10:38:45.770 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:38:46.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:45 smithi113 ceph-mon[36126]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T10:38:46.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:45 smithi113 ceph-mon[36126]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T10:38:46.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:46.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:45 smithi113 ceph-mon[36126]: from='osd.5 [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:38:46.090 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:45 smithi113 ceph-mon[36126]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:38:46.090 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:45 smithi113 ceph-mon[36126]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:46.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:45 smithi060 ceph-mon[31539]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T10:38:46.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:45 smithi060 ceph-mon[31539]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T10:38:46.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:46.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:45 smithi060 ceph-mon[31539]: from='osd.5 [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:38:46.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:45 smithi060 ceph-mon[31539]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:38:46.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:45 smithi060 ceph-mon[31539]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:46.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:45 smithi100 ceph-mon[36051]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T10:38:46.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:45 smithi100 ceph-mon[36051]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T10:38:46.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:46.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:45 smithi100 ceph-mon[36051]: from='osd.5 [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:38:46.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:45 smithi100 ceph-mon[36051]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:38:46.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:45 smithi100 ceph-mon[36051]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:46.509 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:38:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:38:46.188+0000 7f5317fb8700 -1 osd.5 0 waiting for initial osdmap 2024-02-26T10:38:46.509 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:38:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:38:46.194+0000 7f530fda3700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:47.071 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:46 smithi113 ceph-mon[36126]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:38:47.071 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:46 smithi113 ceph-mon[36126]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T10:38:47.071 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.071 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.071 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:46 smithi113 ceph-mon[36126]: osdmap e38: 6 total, 5 up, 6 in 2024-02-26T10:38:47.071 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.071 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:46 smithi113 ceph-mon[36126]: from='osd.5 ' entity='osd.5' 2024-02-26T10:38:47.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:46 smithi060 ceph-mon[31539]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:38:47.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:46 smithi060 ceph-mon[31539]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T10:38:47.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:46 smithi060 ceph-mon[31539]: osdmap e38: 6 total, 5 up, 6 in 2024-02-26T10:38:47.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:46 smithi060 ceph-mon[31539]: from='osd.5 ' entity='osd.5' 2024-02-26T10:38:47.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:46 smithi100 ceph-mon[36051]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:38:47.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:46 smithi100 ceph-mon[36051]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T10:38:47.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:46 smithi100 ceph-mon[36051]: osdmap e38: 6 total, 5 up, 6 in 2024-02-26T10:38:47.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:46 smithi100 ceph-mon[36051]: from='osd.5 ' entity='osd.5' 2024-02-26T10:38:47.982 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:47 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:38:47.982 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:47 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:38:47.982 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:47 smithi113 ceph-mon[36126]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:47.982 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:47.982 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:47 smithi113 ceph-mon[36126]: osd.5 [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] boot 2024-02-26T10:38:47.982 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:47 smithi113 ceph-mon[36126]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T10:38:47.982 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:48.002 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:47 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:38:48.002 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:47 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:38:48.002 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:47 smithi060 ceph-mon[31539]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:48.002 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:48.002 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:47 smithi060 ceph-mon[31539]: osd.5 [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] boot 2024-02-26T10:38:48.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:47 smithi060 ceph-mon[31539]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T10:38:48.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:48.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:47 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:38:48.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:47 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:38:48.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:47 smithi100 ceph-mon[36051]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T10:38:48.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:47 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:48.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:47 smithi100 ceph-mon[36051]: osd.5 [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] boot 2024-02-26T10:38:48.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:47 smithi100 ceph-mon[36051]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T10:38:48.133 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:47 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T10:38:49.091 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T10:38:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T10:38:49.092 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-02-26T10:38:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-02-26T10:38:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0379398 s, 276 MB/s 2024-02-26T10:38:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-02-26T10:38:49.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi113 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi113 to 8540M 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:49.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi113 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi113 to 8540M 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:49.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:49.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi113 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi113 to 8540M 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:38:49.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:38:50.131 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_2 2024-02-26T10:38:50.228 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:49 smithi060 ceph-mon[31539]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:38:50.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:49 smithi060 ceph-mon[31539]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T10:38:50.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:49 smithi113 ceph-mon[36126]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:38:50.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:49 smithi113 ceph-mon[36126]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T10:38:50.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:49 smithi100 ceph-mon[36051]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:38:50.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:49 smithi100 ceph-mon[36051]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T10:38:50.607 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:38:51.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:51 smithi060 ceph-mon[31539]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T10:38:51.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:51 smithi113 ceph-mon[36126]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T10:38:51.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:51 smithi100 ceph-mon[36051]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T10:38:52.654 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:52 smithi060 ceph-mon[31539]: pgmap v159: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-02-26T10:38:52.654 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:52 smithi060 ceph-mon[31539]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-02-26T10:38:52.654 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:52 smithi060 ceph-mon[31539]: osdmap e43: 6 total, 6 up, 6 in 2024-02-26T10:38:52.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:52 smithi113 ceph-mon[36126]: pgmap v159: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-02-26T10:38:52.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:52 smithi113 ceph-mon[36126]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-02-26T10:38:52.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:52 smithi113 ceph-mon[36126]: osdmap e43: 6 total, 6 up, 6 in 2024-02-26T10:38:52.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:52 smithi100 ceph-mon[36051]: pgmap v159: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-02-26T10:38:52.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:52 smithi100 ceph-mon[36051]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-02-26T10:38:52.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:52 smithi100 ceph-mon[36051]: osdmap e43: 6 total, 6 up, 6 in 2024-02-26T10:38:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.968+0000 7ff0c0800700 1 -- 172.21.15.60:0/258099911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0b80747b0 msgr2=0x7ff0b8074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.968+0000 7ff0c0800700 1 --2- 172.21.15.60:0/258099911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0b80747b0 0x7ff0b8074c30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009990 tx=0x7ff0a8009ca0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.968+0000 7ff0c0800700 1 -- 172.21.15.60:0/258099911 shutdown_connections 2024-02-26T10:38:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.968+0000 7ff0c0800700 1 --2- 172.21.15.60:0/258099911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b8109c30 0x7ff0b810c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.968+0000 7ff0c0800700 1 --2- 172.21.15.60:0/258099911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0b8075d80 0x7ff0b80741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.968+0000 7ff0c0800700 1 --2- 172.21.15.60:0/258099911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0b80747b0 0x7ff0b8074c30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.968+0000 7ff0c0800700 1 -- 172.21.15.60:0/258099911 >> 172.21.15.60:0/258099911 conn(0x7ff0b80fd2c0 msgr2=0x7ff0b80ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.968+0000 7ff0c0800700 1 -- 172.21.15.60:0/258099911 shutdown_connections 2024-02-26T10:38:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.969+0000 7ff0c0800700 1 -- 172.21.15.60:0/258099911 wait complete. 2024-02-26T10:38:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.970+0000 7ff0c0800700 1 Processor -- start 2024-02-26T10:38:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.970+0000 7ff0c0800700 1 -- start start 2024-02-26T10:38:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.970+0000 7ff0c0800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0b80747b0 0x7ff0b819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.970+0000 7ff0c0800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b8075d80 0x7ff0b819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.970+0000 7ff0c0800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0b8109c30 0x7ff0b81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.970+0000 7ff0c0800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b810fb40 con 0x7ff0b8109c30 2024-02-26T10:38:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.970+0000 7ff0c0800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b810f9c0 con 0x7ff0b8075d80 2024-02-26T10:38:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.970+0000 7ff0c0800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b810fcc0 con 0x7ff0b80747b0 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0be800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0b8109c30 0x7ff0b81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bde00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0b80747b0 0x7ff0b819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bd400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b8075d80 0x7ff0b819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0be800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0b8109c30 0x7ff0b81a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34258/0 (socket says 172.21.15.60:34258) 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bd400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b8075d80 0x7ff0b819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45900/0 (socket says 172.21.15.60:45900) 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bde00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0b80747b0 0x7ff0b819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:42648/0 (socket says 172.21.15.60:42648) 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0be800700 1 -- 172.21.15.60:0/3980599704 learned_addr learned my addr 172.21.15.60:0/3980599704 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bde00700 1 -- 172.21.15.60:0/3980599704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b8075d80 msgr2=0x7ff0b819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bde00700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b8075d80 0x7ff0b819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bde00700 1 -- 172.21.15.60:0/3980599704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0b8109c30 msgr2=0x7ff0b81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bde00700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0b8109c30 0x7ff0b81a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bde00700 1 -- 172.21.15.60:0/3980599704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b81a1de0 con 0x7ff0b80747b0 2024-02-26T10:38:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bd400700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b8075d80 0x7ff0b819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.971+0000 7ff0bde00700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0b80747b0 0x7ff0b819a180 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00e9f0 tx=0x7ff0ac00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.972+0000 7ff0b2a00700 1 -- 172.21.15.60:0/3980599704 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac00c9c0 con 0x7ff0b80747b0 2024-02-26T10:38:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.972+0000 7ff0b2a00700 1 -- 172.21.15.60:0/3980599704 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0ac010430 con 0x7ff0b80747b0 2024-02-26T10:38:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.972+0000 7ff0b2a00700 1 -- 172.21.15.60:0/3980599704 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac018680 con 0x7ff0b80747b0 2024-02-26T10:38:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.972+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b81a20d0 con 0x7ff0b80747b0 2024-02-26T10:38:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.972+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b81a2610 con 0x7ff0b80747b0 2024-02-26T10:38:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.973+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b8068a10 con 0x7ff0b80747b0 2024-02-26T10:38:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.975+0000 7ff0b2a00700 1 -- 172.21.15.60:0/3980599704 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff0ac010950 con 0x7ff0b80747b0 2024-02-26T10:38:52.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.976+0000 7ff0b2a00700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff0a00a1e10 0x7ff0a00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.976+0000 7ff0b2a00700 1 -- 172.21.15.60:0/3980599704 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4533+0+0 (secure 0 0 0) 0x7ff0ac0cd050 con 0x7ff0b80747b0 2024-02-26T10:38:52.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.976+0000 7ff0bd400700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff0a00a1e10 0x7ff0a00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.977+0000 7ff0bd400700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff0a00a1e10 0x7ff0a00a42d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800b7b0 tx=0x7ff0a8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:52.980+0000 7ff0b2a00700 1 -- 172.21.15.60:0/3980599704 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0ac09b2a0 con 0x7ff0b80747b0 2024-02-26T10:38:53.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:38:53.214+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0b8062190 con 0x7ff0a00a1e10 2024-02-26T10:38:53.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:38:53.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:38:53.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:53 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:53.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:38:53.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:38:53.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:53 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:53.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:38:53.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:38:53.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:53 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:38:54.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:54 smithi060 ceph-mon[31539]: pgmap v161: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-02-26T10:38:54.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:54 smithi060 ceph-mon[31539]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:38:54.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:54 smithi113 ceph-mon[36126]: pgmap v161: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-02-26T10:38:54.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:54 smithi113 ceph-mon[36126]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:38:54.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:54 smithi100 ceph-mon[36051]: pgmap v161: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-02-26T10:38:54.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:54 smithi100 ceph-mon[36051]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:38:55.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:55 smithi060 ceph-mon[31539]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-02-26T10:38:55.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:55 smithi060 ceph-mon[31539]: Cluster is now healthy 2024-02-26T10:38:55.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:55 smithi113 ceph-mon[36126]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-02-26T10:38:55.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:55 smithi113 ceph-mon[36126]: Cluster is now healthy 2024-02-26T10:38:55.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:55 smithi100 ceph-mon[36051]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-02-26T10:38:55.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:55 smithi100 ceph-mon[36051]: Cluster is now healthy 2024-02-26T10:38:56.701 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:56 smithi060 ceph-mon[31539]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T10:38:56.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:56 smithi113 ceph-mon[36126]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T10:38:56.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:56 smithi100 ceph-mon[36051]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T10:38:57.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:57 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/155257718' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0"}]: dispatch 2024-02-26T10:38:57.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:57 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0"}]: dispatch 2024-02-26T10:38:57.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:57 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0"}]': finished 2024-02-26T10:38:57.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:57 smithi113 ceph-mon[36126]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T10:38:57.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:57 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:38:57.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:57 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/155257718' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0"}]: dispatch 2024-02-26T10:38:57.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:57 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0"}]: dispatch 2024-02-26T10:38:57.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:57 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0"}]': finished 2024-02-26T10:38:57.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:57 smithi060 ceph-mon[31539]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T10:38:57.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:38:57.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:57 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/155257718' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0"}]: dispatch 2024-02-26T10:38:57.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:57 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0"}]: dispatch 2024-02-26T10:38:57.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:57 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0"}]': finished 2024-02-26T10:38:57.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:57 smithi100 ceph-mon[36051]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T10:38:57.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:57 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:38:58.784 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:58 smithi100 ceph-mon[36051]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-02-26T10:38:58.784 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:38:58 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1146071956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:38:58.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:58 smithi113 ceph-mon[36126]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-02-26T10:38:58.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:38:58 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1146071956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:38:58.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:58 smithi060 ceph-mon[31539]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-02-26T10:38:58.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:38:58 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1146071956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:39:00.589 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:00 smithi113 ceph-mon[36126]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T10:39:00.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:00 smithi060 ceph-mon[31539]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T10:39:00.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:00 smithi100 ceph-mon[36051]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T10:39:02.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:02 smithi060 ceph-mon[31539]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-26T10:39:02.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:02 smithi113 ceph-mon[36126]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-26T10:39:02.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:02 smithi100 ceph-mon[36051]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-26T10:39:04.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:04 smithi060 ceph-mon[31539]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T10:39:04.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:04 smithi113 ceph-mon[36126]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T10:39:04.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:04 smithi100 ceph-mon[36051]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T10:39:06.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:06 smithi113 ceph-mon[36126]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:06.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:06 smithi060 ceph-mon[31539]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:06.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:06 smithi100 ceph-mon[36051]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:07.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:07 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T10:39:07.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:07 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:07.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:07 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T10:39:07.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:07 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:07.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:07 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T10:39:07.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:07 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:08.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:08 smithi060 ceph-mon[31539]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:08.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:08 smithi060 ceph-mon[31539]: Deploying daemon osd.6 on smithi060 2024-02-26T10:39:08.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:08 smithi113 ceph-mon[36126]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:08.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:08 smithi113 ceph-mon[36126]: Deploying daemon osd.6 on smithi060 2024-02-26T10:39:08.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:08 smithi100 ceph-mon[36051]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:08.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:08 smithi100 ceph-mon[36051]: Deploying daemon osd.6 on smithi060 2024-02-26T10:39:10.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:10 smithi060 ceph-mon[31539]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:10.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:10 smithi113 ceph-mon[36126]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:10.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:10 smithi100 ceph-mon[36051]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:12.611 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:12 smithi060 ceph-mon[31539]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:12.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:12 smithi113 ceph-mon[36126]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:12.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:12 smithi100 ceph-mon[36051]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:14.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:14 smithi113 ceph-mon[36126]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:14.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:14 smithi060 ceph-mon[31539]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:14.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:14 smithi100 ceph-mon[36051]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:16 smithi113 ceph-mon[36126]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:16.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:16.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:16.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:16 smithi060 ceph-mon[31539]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:16.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:16 smithi100 ceph-mon[36051]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:16.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:16.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:16.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:18.689 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:18 smithi060 ceph-mon[31539]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:18.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:18 smithi113 ceph-mon[36126]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:18.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:18 smithi100 ceph-mon[36051]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:20.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:20 smithi060 ceph-mon[31539]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:20.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:20 smithi060 ceph-mon[31539]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T10:39:20.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:20.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:20.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:20 smithi113 ceph-mon[36126]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:20.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:20 smithi113 ceph-mon[36126]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T10:39:20.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:20.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:20.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:20 smithi100 ceph-mon[36051]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:20.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:20 smithi100 ceph-mon[36051]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T10:39:20.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:20.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.260 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 6 on host 'smithi060' 2024-02-26T10:39:21.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.257+0000 7ff0b2a00700 1 -- 172.21.15.60:0/3980599704 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff0b8062190 con 0x7ff0a00a1e10 2024-02-26T10:39:21.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.261+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff0a00a1e10 msgr2=0x7ff0a00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.261+0000 7ff0c0800700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff0a00a1e10 0x7ff0a00a42d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800b7b0 tx=0x7ff0a8005fd0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0b80747b0 msgr2=0x7ff0b819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0b80747b0 0x7ff0b819a180 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00e9f0 tx=0x7ff0ac00ed00 comp rx=0 tx=0).stop 2024-02-26T10:39:21.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 shutdown_connections 2024-02-26T10:39:21.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff0a00a1e10 0x7ff0a00a42d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0b8109c30 0x7ff0b81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0b80747b0 0x7ff0b819a180 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 --2- 172.21.15.60:0/3980599704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b8075d80 0x7ff0b819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 >> 172.21.15.60:0/3980599704 conn(0x7ff0b80fd2c0 msgr2=0x7ff0b8105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:21.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 shutdown_connections 2024-02-26T10:39:21.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:39:21.262+0000 7ff0c0800700 1 -- 172.21.15.60:0/3980599704 wait complete. 2024-02-26T10:39:21.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T10:39:21.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T10:39:21.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:39:21.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:21.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:21.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:21.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:21.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:39:21.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:21 smithi113 ceph-mon[36126]: osdmap e46: 7 total, 6 up, 7 in 2024-02-26T10:39:21.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:39:21.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:21 smithi060 ceph-mon[31539]: osdmap e46: 7 total, 6 up, 7 in 2024-02-26T10:39:21.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T10:39:21.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T10:39:21.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:39:21.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:21.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:21.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:21.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:21.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:21.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:39:21.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:21 smithi100 ceph-mon[36051]: osdmap e46: 7 total, 6 up, 7 in 2024-02-26T10:39:22.208 DEBUG:teuthology.orchestra.run.smithi060:osd.6> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.6.service 2024-02-26T10:39:22.213 INFO:tasks.cephadm:Deploying osd.7 on smithi100 with /dev/vg_nvme/lv_2... 2024-02-26T10:39:22.214 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T10:39:22.444 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:39:22.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:22 smithi060 ceph-mon[31539]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:22.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:22.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:22.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:22 smithi060 ceph-mon[31539]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' 2024-02-26T10:39:22.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:22.824 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:22 smithi100 ceph-mon[36051]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:22.825 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:22.825 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:22.825 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:22 smithi100 ceph-mon[36051]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' 2024-02-26T10:39:22.825 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:22.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:22 smithi113 ceph-mon[36126]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:22.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:22.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:22.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:22 smithi113 ceph-mon[36126]: from='osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688]' entity='osd.6' 2024-02-26T10:39:22.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:23.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:23 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:39:23.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:23 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:39:23.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:23 smithi113 ceph-mon[36126]: osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] boot 2024-02-26T10:39:23.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:23 smithi113 ceph-mon[36126]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T10:39:23.839 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:23 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:23.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:23 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:39:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:23 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:39:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:23 smithi060 ceph-mon[31539]: osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] boot 2024-02-26T10:39:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:23 smithi060 ceph-mon[31539]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T10:39:23.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:23.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:23 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:39:23.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:23 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:39:23.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:23 smithi100 ceph-mon[36051]: osd.6 [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] boot 2024-02-26T10:39:23.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:23 smithi100 ceph-mon[36051]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T10:39:23.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:23 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T10:39:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T10:39:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T10:39:24.450 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-02-26T10:39:24.450 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-02-26T10:39:24.451 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121009 s, 867 MB/s 2024-02-26T10:39:24.451 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-02-26T10:39:24.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:24 smithi060 ceph-mon[31539]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:24.818 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:24 smithi100 ceph-mon[36051]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:24.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:24 smithi113 ceph-mon[36126]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-02-26T10:39:25.106 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-02-26T10:39:25.346 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:39:25.616 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:25 smithi100 ceph-mon[36051]: osdmap e48: 7 total, 7 up, 7 in 2024-02-26T10:39:25.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:25 smithi113 ceph-mon[36126]: osdmap e48: 7 total, 7 up, 7 in 2024-02-26T10:39:25.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:25 smithi060 ceph-mon[31539]: osdmap e48: 7 total, 7 up, 7 in 2024-02-26T10:39:26.794 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:26 smithi100 ceph-mon[36051]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:26.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:26 smithi113 ceph-mon[36126]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:26.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:26 smithi060 ceph-mon[31539]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:26.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 -- 172.21.15.100:0/860720783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c201038f0 msgr2=0x7f6c2010a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 --2- 172.21.15.100:0/860720783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c201038f0 0x7f6c2010a1a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800b0d0 tx=0x7f6c1800b3e0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 -- 172.21.15.100:0/860720783 shutdown_connections 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 --2- 172.21.15.100:0/860720783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c201038f0 0x7f6c2010a1a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 --2- 172.21.15.100:0/860720783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c20101d50 0x7f6c20102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 --2- 172.21.15.100:0/860720783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c20102f50 0x7f6c201033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 -- 172.21.15.100:0/860720783 >> 172.21.15.100:0/860720783 conn(0x7f6c200fd330 msgr2=0x7f6c200ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 -- 172.21.15.100:0/860720783 shutdown_connections 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 -- 172.21.15.100:0/860720783 wait complete. 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 Processor -- start 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.975+0000 7f6c25773700 1 -- start start 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c25773700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c20101d50 0x7f6c20075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c25773700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c20102f50 0x7f6c20074140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c25773700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c201038f0 0x7f6c20074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c25773700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2010da30 con 0x7f6c201038f0 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c25773700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2010d8b0 con 0x7f6c20102f50 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c25773700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2010dbb0 con 0x7f6c20101d50 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c20102f50 0x7f6c20074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c20102f50 0x7f6c20074140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33896/0 (socket says 172.21.15.100:33896) 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1e7fc700 1 -- 172.21.15.100:0/482317206 learned_addr learned my addr 172.21.15.100:0/482317206 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1effd700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c20101d50 0x7f6c20075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1e7fc700 1 -- 172.21.15.100:0/482317206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c20101d50 msgr2=0x7f6c20075af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1e7fc700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c20101d50 0x7f6c20075af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1e7fc700 1 -- 172.21.15.100:0/482317206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c201038f0 msgr2=0x7f6c20074680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1e7fc700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c201038f0 0x7f6c20074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1e7fc700 1 -- 172.21.15.100:0/482317206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c20074f30 con 0x7f6c20102f50 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c1e7fc700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c20102f50 0x7f6c20074140 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b530 tx=0x7f6c1400b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c07fff700 1 -- 172.21.15.100:0/482317206 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14015020 con 0x7f6c20102f50 2024-02-26T10:39:26.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c07fff700 1 -- 172.21.15.100:0/482317206 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c14011d10 con 0x7f6c20102f50 2024-02-26T10:39:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c201a21e0 con 0x7f6c20102f50 2024-02-26T10:39:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c07fff700 1 -- 172.21.15.100:0/482317206 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14010e50 con 0x7f6c20102f50 2024-02-26T10:39:26.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.976+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c201a2720 con 0x7f6c20102f50 2024-02-26T10:39:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.977+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c20068a10 con 0x7f6c20102f50 2024-02-26T10:39:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.978+0000 7f6c07fff700 1 -- 172.21.15.100:0/482317206 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6c14018020 con 0x7f6c20102f50 2024-02-26T10:39:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.979+0000 7f6c07fff700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c080a1fb0 0x7f6c080a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.979+0000 7f6c07fff700 1 -- 172.21.15.100:0/482317206 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f6c140ccf10 con 0x7f6c20102f50 2024-02-26T10:39:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.979+0000 7f6c1effd700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c080a1fb0 0x7f6c080a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.980+0000 7f6c1effd700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c080a1fb0 0x7f6c080a4470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006fd0 tx=0x7f6c10008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:26.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:26.982+0000 7f6c07fff700 1 -- 172.21.15.100:0/482317206 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c1401f020 con 0x7f6c20102f50 2024-02-26T10:39:27.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:27.185+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c20062190 con 0x7f6c080a1fb0 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi060 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi060 to 5698M 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:28.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:39:28.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:39:28.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:39:28.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:28 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:28.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi060 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi060 to 5698M 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:39:28.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:39:28.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:39:28.340 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:28 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:28.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi060 2024-02-26T10:39:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi060 to 5698M 2024-02-26T10:39:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:28.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:28.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:28.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:39:28.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:39:28.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:39:28.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:28 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:30.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:30 smithi113 ceph-mon[36126]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:30.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:30 smithi113 ceph-mon[36126]: from='client.? 172.21.15.100:0/2550403274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07a3c8c-b460-46fd-8b14-7e042a98495d"}]: dispatch 2024-02-26T10:39:30.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:30 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07a3c8c-b460-46fd-8b14-7e042a98495d"}]: dispatch 2024-02-26T10:39:30.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:30 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c07a3c8c-b460-46fd-8b14-7e042a98495d"}]': finished 2024-02-26T10:39:30.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:30 smithi113 ceph-mon[36126]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T10:39:30.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:30 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:30.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:30 smithi060 ceph-mon[31539]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:30.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:30 smithi060 ceph-mon[31539]: from='client.? 172.21.15.100:0/2550403274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07a3c8c-b460-46fd-8b14-7e042a98495d"}]: dispatch 2024-02-26T10:39:30.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:30 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07a3c8c-b460-46fd-8b14-7e042a98495d"}]: dispatch 2024-02-26T10:39:30.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:30 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c07a3c8c-b460-46fd-8b14-7e042a98495d"}]': finished 2024-02-26T10:39:30.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:30 smithi060 ceph-mon[31539]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T10:39:30.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:30 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:30.382 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:30 smithi100 ceph-mon[36051]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:30.382 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:30 smithi100 ceph-mon[36051]: from='client.? 172.21.15.100:0/2550403274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07a3c8c-b460-46fd-8b14-7e042a98495d"}]: dispatch 2024-02-26T10:39:30.382 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:30 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07a3c8c-b460-46fd-8b14-7e042a98495d"}]: dispatch 2024-02-26T10:39:30.382 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:30 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c07a3c8c-b460-46fd-8b14-7e042a98495d"}]': finished 2024-02-26T10:39:30.382 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:30 smithi100 ceph-mon[36051]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T10:39:30.382 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:30 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:31.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:31 smithi113 ceph-mon[36126]: from='client.? 172.21.15.100:0/13647379' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:39:31.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:31 smithi060 ceph-mon[31539]: from='client.? 172.21.15.100:0/13647379' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:39:31.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:31 smithi100 ceph-mon[36051]: from='client.? 172.21.15.100:0/13647379' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:39:32.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:32 smithi113 ceph-mon[36126]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:32.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:32 smithi060 ceph-mon[31539]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:32.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:32 smithi100 ceph-mon[36051]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:34.172 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:34 smithi100 ceph-mon[36051]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:34.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:34 smithi113 ceph-mon[36126]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:34.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:34 smithi060 ceph-mon[31539]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:36.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:36 smithi113 ceph-mon[36126]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:36.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:36 smithi060 ceph-mon[31539]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:36.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:36 smithi100 ceph-mon[36051]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:38.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:38 smithi113 ceph-mon[36126]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:38.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T10:39:38.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:38.339 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:38 smithi113 ceph-mon[36126]: Deploying daemon osd.7 on smithi100 2024-02-26T10:39:38.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:38 smithi060 ceph-mon[31539]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:38.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T10:39:38.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:38.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:38 smithi060 ceph-mon[31539]: Deploying daemon osd.7 on smithi100 2024-02-26T10:39:38.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:38 smithi100 ceph-mon[36051]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:38.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T10:39:38.382 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:38.382 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:38 smithi100 ceph-mon[36051]: Deploying daemon osd.7 on smithi100 2024-02-26T10:39:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:40 smithi100 ceph-mon[36051]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:40.338 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:40 smithi113 ceph-mon[36126]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:40.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:40 smithi060 ceph-mon[31539]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:42.132 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:42 smithi100 ceph-mon[36051]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:42.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:42 smithi060 ceph-mon[31539]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:42.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:42 smithi113 ceph-mon[36126]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:43.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:43.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:43.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:43.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:43 smithi100 ceph-mon[36051]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:44.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:44.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:44.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:44.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:43 smithi113 ceph-mon[36126]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:44.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:44.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:44.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:44.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:43 smithi060 ceph-mon[31539]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:46.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.522+0000 7f6c07fff700 1 -- 172.21.15.100:0/482317206 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6c20062190 con 0x7f6c080a1fb0 2024-02-26T10:39:46.522 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 7 on host 'smithi100' 2024-02-26T10:39:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c080a1fb0 msgr2=0x7f6c080a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c080a1fb0 0x7f6c080a4470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006fd0 tx=0x7f6c10008040 comp rx=0 tx=0).stop 2024-02-26T10:39:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c20102f50 msgr2=0x7f6c20074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c20102f50 0x7f6c20074140 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b530 tx=0x7f6c1400b8f0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 shutdown_connections 2024-02-26T10:39:46.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c080a1fb0 0x7f6c080a4470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c201038f0 0x7f6c20074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c20101d50 0x7f6c20075af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 --2- 172.21.15.100:0/482317206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c20102f50 0x7f6c20074140 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 >> 172.21.15.100:0/482317206 conn(0x7f6c200fd330 msgr2=0x7f6c20106320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:46.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 shutdown_connections 2024-02-26T10:39:46.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:39:46.524+0000 7f6c25773700 1 -- 172.21.15.100:0/482317206 wait complete. 2024-02-26T10:39:46.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='osd.7 [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T10:39:46.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:46 smithi100 ceph-mon[36051]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T10:39:47.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:47.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:47.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:47.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:47.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='osd.7 [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T10:39:47.089 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:46 smithi113 ceph-mon[36126]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T10:39:47.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:47.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:47.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:47.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:39:47.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:47.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='osd.7 [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T10:39:47.121 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:46 smithi060 ceph-mon[31539]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T10:39:47.223 DEBUG:teuthology.orchestra.run.smithi100:osd.7> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.7.service 2024-02-26T10:39:47.225 INFO:tasks.cephadm:Deploying osd.8 on smithi113 with /dev/vg_nvme/lv_2... 2024-02-26T10:39:47.225 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T10:39:47.470 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:39:48.614 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:48 smithi113 ceph-mon[36126]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:48.614 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:48 smithi113 ceph-mon[36126]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T10:39:48.614 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:48 smithi113 ceph-mon[36126]: from='osd.7 [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:39:48.614 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:48 smithi113 ceph-mon[36126]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T10:39:48.614 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:48.614 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:48 smithi113 ceph-mon[36126]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:39:48.758 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:48 smithi100 ceph-mon[36051]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:48.759 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:48 smithi100 ceph-mon[36051]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T10:39:48.759 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:48 smithi100 ceph-mon[36051]: from='osd.7 [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:39:48.759 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:48 smithi100 ceph-mon[36051]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T10:39:48.759 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:48.759 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:48 smithi100 ceph-mon[36051]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:39:48.759 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:39:48 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:39:48.522+0000 7faaa33b7700 -1 osd.7 0 waiting for initial osdmap 2024-02-26T10:39:48.759 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:39:48 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:39:48.532+0000 7faa9f1aa700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:39:48.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:48 smithi060 ceph-mon[31539]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:48.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:48 smithi060 ceph-mon[31539]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T10:39:48.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:48 smithi060 ceph-mon[31539]: from='osd.7 [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:39:48.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:48 smithi060 ceph-mon[31539]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T10:39:48.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:48.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:48 smithi060 ceph-mon[31539]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:39:49.487 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T10:39:49.488 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T10:39:49.488 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-02-26T10:39:49.488 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-02-26T10:39:49.488 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010256 s, 1.0 GB/s 2024-02-26T10:39:49.488 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-02-26T10:39:49.782 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:49 smithi113 ceph-mon[36126]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:39:49.783 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:49 smithi113 ceph-mon[36126]: osdmap e51: 8 total, 7 up, 8 in 2024-02-26T10:39:49.783 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:49 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:49.783 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:49 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:49.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:49 smithi060 ceph-mon[31539]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:39:49.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:49 smithi060 ceph-mon[31539]: osdmap e51: 8 total, 7 up, 8 in 2024-02-26T10:39:49.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:49 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:49.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:49 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:49.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:49 smithi100 ceph-mon[36051]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:39:49.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:49 smithi100 ceph-mon[36051]: osdmap e51: 8 total, 7 up, 8 in 2024-02-26T10:39:49.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:49.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:50.139 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi113:vg_nvme/lv_2 2024-02-26T10:39:50.385 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:39:50.770 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:39:50.770 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:39:50.770 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: osd.7 [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] boot 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.771 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:50 smithi113 ceph-mon[36126]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T10:39:50.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:39:50.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:39:50.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:50.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: osd.7 [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] boot 2024-02-26T10:39:50.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T10:39:50.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:50.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:50.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:50.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:50.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:50.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:50.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:50 smithi060 ceph-mon[31539]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T10:39:50.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: osd.7 [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] boot 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:39:50.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:39:50.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:50 smithi100 ceph-mon[36051]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T10:39:51.827 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:51 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi100 2024-02-26T10:39:51.827 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:51 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi100 to 5693M 2024-02-26T10:39:51.827 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:51 smithi113 ceph-mon[36126]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T10:39:51.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:51 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi100 2024-02-26T10:39:51.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:51 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi100 to 5693M 2024-02-26T10:39:51.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:51 smithi060 ceph-mon[31539]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T10:39:51.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:51 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi100 2024-02-26T10:39:51.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:51 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi100 to 5693M 2024-02-26T10:39:51.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:51 smithi100 ceph-mon[36051]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T10:39:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.035+0000 7f304e534700 1 -- 172.21.15.113:0/861799433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30481000a0 msgr2=0x7f30481004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.035+0000 7f304e534700 1 --2- 172.21.15.113:0/861799433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30481000a0 0x7f30481004c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3038009990 tx=0x7f3038009ca0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.035+0000 7f304e534700 1 -- 172.21.15.113:0/861799433 shutdown_connections 2024-02-26T10:39:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.035+0000 7f304e534700 1 --2- 172.21.15.113:0/861799433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3048103ac0 0x7f304810a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.035+0000 7f304e534700 1 --2- 172.21.15.113:0/861799433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30481000a0 0x7f30481004c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.035+0000 7f304e534700 1 --2- 172.21.15.113:0/861799433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3048103120 0x7f3048103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.035+0000 7f304e534700 1 -- 172.21.15.113:0/861799433 >> 172.21.15.113:0/861799433 conn(0x7f30480fb9e0 msgr2=0x7f30480fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.035+0000 7f304e534700 1 -- 172.21.15.113:0/861799433 shutdown_connections 2024-02-26T10:39:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.035+0000 7f304e534700 1 -- 172.21.15.113:0/861799433 wait complete. 2024-02-26T10:39:52.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.036+0000 7f304e534700 1 Processor -- start 2024-02-26T10:39:52.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.036+0000 7f304e534700 1 -- start start 2024-02-26T10:39:52.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f304e534700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30481000a0 0x7f3048102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f304e534700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3048103120 0x7f30481010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f304e534700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3048103ac0 0x7f3048101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f304e534700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304810da20 con 0x7f3048103ac0 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f304e534700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304810d8a0 con 0x7f30481000a0 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f304e534700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304810dba0 con 0x7f3048103120 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30477fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3048103120 0x7f30481010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30477fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3048103120 0x7f30481010d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33736/0 (socket says 172.21.15.113:33736) 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30477fe700 1 -- 172.21.15.113:0/1515758430 learned_addr learned my addr 172.21.15.113:0/1515758430 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f3047fff700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30481000a0 0x7f3048102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30477fe700 1 -- 172.21.15.113:0/1515758430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30481000a0 msgr2=0x7f3048102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30477fe700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30481000a0 0x7f3048102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30477fe700 1 -- 172.21.15.113:0/1515758430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3048103ac0 msgr2=0x7f3048101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30477fe700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3048103ac0 0x7f3048101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30477fe700 1 -- 172.21.15.113:0/1515758430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048101ec0 con 0x7f3048103120 2024-02-26T10:39:52.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30477fe700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3048103120 0x7f30481010d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f303c00b530 tx=0x7f303c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:52.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30457fa700 1 -- 172.21.15.113:0/1515758430 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c015020 con 0x7f3048103120 2024-02-26T10:39:52.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30457fa700 1 -- 172.21.15.113:0/1515758430 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f303c011d10 con 0x7f3048103120 2024-02-26T10:39:52.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30481a2190 con 0x7f3048103120 2024-02-26T10:39:52.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f30457fa700 1 -- 172.21.15.113:0/1515758430 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c010e50 con 0x7f3048103120 2024-02-26T10:39:52.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.037+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30481a2720 con 0x7f3048103120 2024-02-26T10:39:52.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.038+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3048068a10 con 0x7f3048103120 2024-02-26T10:39:52.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.039+0000 7f30457fa700 1 -- 172.21.15.113:0/1515758430 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f303c018020 con 0x7f3048103120 2024-02-26T10:39:52.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.040+0000 7f30457fa700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f30300a1fb0 0x7f30300a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.040+0000 7f30457fa700 1 -- 172.21.15.113:0/1515758430 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f303c0ccb00 con 0x7f3048103120 2024-02-26T10:39:52.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.040+0000 7f3047fff700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f30300a1fb0 0x7f30300a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.041+0000 7f3047fff700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f30300a1fb0 0x7f30300a4470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30380039c0 tx=0x7f3038005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:52.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.044+0000 7f30457fa700 1 -- 172.21.15.113:0/1515758430 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f303c01f020 con 0x7f3048103120 2024-02-26T10:39:52.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:39:52.244+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3048062190 con 0x7f30300a1fb0 2024-02-26T10:39:52.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:52 smithi113 ceph-mon[36126]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:52.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:52 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:39:52.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:52 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:39:52.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:52 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:52.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:52 smithi060 ceph-mon[31539]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:52.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:39:52.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:39:52.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:52 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:52.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:52 smithi100 ceph-mon[36051]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:52.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:52 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:39:52.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:52 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:39:52.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:52 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:39:53.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:53 smithi113 ceph-mon[36126]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:39:53.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:53 smithi113 ceph-mon[36126]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T10:39:53.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:53 smithi060 ceph-mon[31539]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:39:53.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:53 smithi060 ceph-mon[31539]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T10:39:53.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:53 smithi100 ceph-mon[36051]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:39:53.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:53 smithi100 ceph-mon[36051]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T10:39:54.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:54 smithi113 ceph-mon[36126]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:54.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:54 smithi113 ceph-mon[36126]: osdmap e56: 8 total, 8 up, 8 in 2024-02-26T10:39:54.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:54 smithi060 ceph-mon[31539]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:54.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:54 smithi060 ceph-mon[31539]: osdmap e56: 8 total, 8 up, 8 in 2024-02-26T10:39:54.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:54 smithi100 ceph-mon[36051]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:54.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:54 smithi100 ceph-mon[36051]: osdmap e56: 8 total, 8 up, 8 in 2024-02-26T10:39:55.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:55 smithi113 ceph-mon[36126]: from='client.? 172.21.15.113:0/3897606316' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26d9b49-c7af-40f2-96b1-25018e1e9ddf"}]: dispatch 2024-02-26T10:39:55.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:55 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26d9b49-c7af-40f2-96b1-25018e1e9ddf"}]: dispatch 2024-02-26T10:39:55.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:55 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26d9b49-c7af-40f2-96b1-25018e1e9ddf"}]': finished 2024-02-26T10:39:55.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:55 smithi113 ceph-mon[36126]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T10:39:55.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:39:55.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:55 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/3897606316' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26d9b49-c7af-40f2-96b1-25018e1e9ddf"}]: dispatch 2024-02-26T10:39:55.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:55 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26d9b49-c7af-40f2-96b1-25018e1e9ddf"}]: dispatch 2024-02-26T10:39:55.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:55 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26d9b49-c7af-40f2-96b1-25018e1e9ddf"}]': finished 2024-02-26T10:39:55.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:55 smithi060 ceph-mon[31539]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T10:39:55.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:39:55.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:55 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/3897606316' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26d9b49-c7af-40f2-96b1-25018e1e9ddf"}]: dispatch 2024-02-26T10:39:55.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:55 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26d9b49-c7af-40f2-96b1-25018e1e9ddf"}]: dispatch 2024-02-26T10:39:55.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:55 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26d9b49-c7af-40f2-96b1-25018e1e9ddf"}]': finished 2024-02-26T10:39:55.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:55 smithi100 ceph-mon[36051]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T10:39:55.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:39:56.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:56 smithi113 ceph-mon[36126]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:56.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:56 smithi113 ceph-mon[36126]: from='client.? 172.21.15.113:0/3362250925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:39:56.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:56 smithi060 ceph-mon[31539]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:56.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:56 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/3362250925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:39:56.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:56 smithi100 ceph-mon[36051]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:56.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:56 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/3362250925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:39:58.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:39:58 smithi060 ceph-mon[31539]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:58.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:39:58 smithi100 ceph-mon[36051]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:39:59.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:39:58 smithi113 ceph-mon[36126]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:00.820 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:00 smithi113 ceph-mon[36126]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:00.820 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:00 smithi113 ceph-mon[36126]: overall HEALTH_OK 2024-02-26T10:40:00.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:00 smithi060 ceph-mon[31539]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:00.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:00 smithi060 ceph-mon[31539]: overall HEALTH_OK 2024-02-26T10:40:00.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:00 smithi100 ceph-mon[36051]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:00.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:00 smithi100 ceph-mon[36051]: overall HEALTH_OK 2024-02-26T10:40:02.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:02 smithi060 ceph-mon[31539]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:02.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:02 smithi100 ceph-mon[36051]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:03.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:02 smithi113 ceph-mon[36126]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:03.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T10:40:03.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:03.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:03 smithi060 ceph-mon[31539]: Deploying daemon osd.8 on smithi113 2024-02-26T10:40:03.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T10:40:03.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:03.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:03 smithi100 ceph-mon[36051]: Deploying daemon osd.8 on smithi113 2024-02-26T10:40:04.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T10:40:04.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:04.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:03 smithi113 ceph-mon[36126]: Deploying daemon osd.8 on smithi113 2024-02-26T10:40:04.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:04 smithi113 ceph-mon[36126]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:04.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:04 smithi060 ceph-mon[31539]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:04.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:04 smithi100 ceph-mon[36051]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:06.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:06 smithi060 ceph-mon[31539]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:06.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:06 smithi100 ceph-mon[36051]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:07.037 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:06 smithi113 ceph-mon[36126]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:08.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:08 smithi100 ceph-mon[36051]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:08.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:08 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:08.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:08 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:08.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:08 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:09.042 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:08 smithi113 ceph-mon[36126]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:09.042 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:08 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:09.042 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:08 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:09.042 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:08 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:09.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:08 smithi060 ceph-mon[31539]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:09.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:09.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:09.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:08 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:10.818 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:10 smithi113 ceph-mon[36126]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:10.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:10 smithi100 ceph-mon[36051]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:11.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:10 smithi060 ceph-mon[31539]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:12.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.383+0000 7f30457fa700 1 -- 172.21.15.113:0/1515758430 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3048062190 con 0x7f30300a1fb0 2024-02-26T10:40:12.385 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 8 on host 'smithi113' 2024-02-26T10:40:12.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f30300a1fb0 msgr2=0x7f30300a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:12.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f30300a1fb0 0x7f30300a4470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30380039c0 tx=0x7f3038005fb0 comp rx=0 tx=0).stop 2024-02-26T10:40:12.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3048103120 msgr2=0x7f30481010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:12.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3048103120 0x7f30481010d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f303c00b530 tx=0x7f303c00b8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:12.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 shutdown_connections 2024-02-26T10:40:12.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f30300a1fb0 0x7f30300a4470 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3048103ac0 0x7f3048101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30481000a0 0x7f3048102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 --2- 172.21.15.113:0/1515758430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3048103120 0x7f30481010d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 >> 172.21.15.113:0/1515758430 conn(0x7f30480fb9e0 msgr2=0x7f3048106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 shutdown_connections 2024-02-26T10:40:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:40:12.385+0000 7f304e534700 1 -- 172.21.15.113:0/1515758430 wait complete. 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='osd.8 [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.588 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:12 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:12.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:12.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:12.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:12.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='osd.8 [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T10:40:12.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T10:40:12.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:12 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:12.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:12.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:12.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:12.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='osd.8 [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T10:40:12.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T10:40:12.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:12.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:12 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:13.031 DEBUG:teuthology.orchestra.run.smithi113:osd.8> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.8.service 2024-02-26T10:40:13.034 INFO:tasks.cephadm:Deploying osd.9 on smithi060 with /dev/vg_nvme/lv_1... 2024-02-26T10:40:13.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T10:40:13.503 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:40:13.921 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:13 smithi113 ceph-mon[36126]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T10:40:13.921 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:13 smithi113 ceph-mon[36126]: from='osd.8 [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:40:13.922 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:13 smithi113 ceph-mon[36126]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T10:40:13.922 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:13 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:40:13.922 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:13 smithi113 ceph-mon[36126]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:40:13.953 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:13 smithi060 ceph-mon[31539]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T10:40:13.953 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:13 smithi060 ceph-mon[31539]: from='osd.8 [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:40:13.953 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:13 smithi060 ceph-mon[31539]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T10:40:13.953 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:13 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:40:13.953 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:13 smithi060 ceph-mon[31539]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:40:14.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:13 smithi100 ceph-mon[36051]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T10:40:14.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:13 smithi100 ceph-mon[36051]: from='osd.8 [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:40:14.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:13 smithi100 ceph-mon[36051]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T10:40:14.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:13 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:40:14.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:13 smithi100 ceph-mon[36051]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:40:14.263 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:40:14.027+0000 7fd9ce94d700 -1 osd.8 0 waiting for initial osdmap 2024-02-26T10:40:14.263 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:40:14.037+0000 7fd9c6738700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:40:14.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-mon[36126]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:14.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-mon[36126]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:40:14.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-mon[36126]: osdmap e59: 9 total, 8 up, 9 in 2024-02-26T10:40:14.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:40:14.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-mon[36126]: from='osd.8 ' entity='osd.8' 2024-02-26T10:40:14.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-mon[36126]: osd.8 [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] boot 2024-02-26T10:40:14.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-mon[36126]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T10:40:14.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:40:14.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:14 smithi060 ceph-mon[31539]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:14.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:14 smithi060 ceph-mon[31539]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:40:14.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:14 smithi060 ceph-mon[31539]: osdmap e59: 9 total, 8 up, 9 in 2024-02-26T10:40:14.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:40:14.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:14 smithi060 ceph-mon[31539]: from='osd.8 ' entity='osd.8' 2024-02-26T10:40:14.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:14 smithi060 ceph-mon[31539]: osd.8 [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] boot 2024-02-26T10:40:14.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:14 smithi060 ceph-mon[31539]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T10:40:14.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:40:15.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:14 smithi100 ceph-mon[36051]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-02-26T10:40:15.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:14 smithi100 ceph-mon[36051]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:40:15.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:14 smithi100 ceph-mon[36051]: osdmap e59: 9 total, 8 up, 9 in 2024-02-26T10:40:15.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:40:15.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:14 smithi100 ceph-mon[36051]: from='osd.8 ' entity='osd.8' 2024-02-26T10:40:15.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:14 smithi100 ceph-mon[36051]: osd.8 [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] boot 2024-02-26T10:40:15.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:14 smithi100 ceph-mon[36051]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T10:40:15.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T10:40:15.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:15 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:40:15.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:15 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:40:15.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:15 smithi113 ceph-mon[36126]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T10:40:15.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:15 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:40:15.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:15 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:40:15.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:15 smithi060 ceph-mon[31539]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T10:40:16.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:15 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:40:16.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:15 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:40:16.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:15 smithi100 ceph-mon[36051]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T10:40:16.469 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T10:40:16.469 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T10:40:16.469 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-02-26T10:40:16.469 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-02-26T10:40:16.469 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036607 s, 286 MB/s 2024-02-26T10:40:16.469 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-02-26T10:40:16.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:16.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:16.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:16 smithi113 ceph-mon[36126]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T10:40:16.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:16.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:16.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:16.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:16 smithi060 ceph-mon[31539]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T10:40:17.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:17.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:17.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:17.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:17.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:17.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:17.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:17.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:17.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:17.132 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:17.132 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:16 smithi100 ceph-mon[36051]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T10:40:17.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_1 2024-02-26T10:40:17.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:17 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi113 2024-02-26T10:40:17.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:17 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi113 to 5693M 2024-02-26T10:40:17.899 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:40:18.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:17 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi113 2024-02-26T10:40:18.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:17 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi113 to 5693M 2024-02-26T10:40:18.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:17 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi113 2024-02-26T10:40:18.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:17 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi113 to 5693M 2024-02-26T10:40:18.989 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:18 smithi113 ceph-mon[36126]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:18.989 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:18 smithi113 ceph-mon[36126]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T10:40:19.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:18 smithi060 ceph-mon[31539]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:19.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:18 smithi060 ceph-mon[31539]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T10:40:19.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:18 smithi100 ceph-mon[36051]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:19.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:18 smithi100 ceph-mon[36051]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T10:40:19.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:19 smithi060 ceph-mon[31539]: osdmap e64: 9 total, 9 up, 9 in 2024-02-26T10:40:20.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:19 smithi113 ceph-mon[36126]: osdmap e64: 9 total, 9 up, 9 in 2024-02-26T10:40:20.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:19 smithi100 ceph-mon[36051]: osdmap e64: 9 total, 9 up, 9 in 2024-02-26T10:40:20.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.130+0000 7fca0c000700 1 -- 172.21.15.60:0/932642810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 msgr2=0x7fca04109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.130+0000 7fca0c000700 1 --2- 172.21.15.60:0/932642810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 0x7fca04109a20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009990 tx=0x7fc9f0009ca0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.131+0000 7fca0c000700 1 -- 172.21.15.60:0/932642810 shutdown_connections 2024-02-26T10:40:20.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.131+0000 7fca0c000700 1 --2- 172.21.15.60:0/932642810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04109f60 0x7fca04110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.131+0000 7fca0c000700 1 --2- 172.21.15.60:0/932642810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca041083c0 0x7fca041087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.131+0000 7fca0c000700 1 --2- 172.21.15.60:0/932642810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 0x7fca04109a20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.131+0000 7fca0c000700 1 -- 172.21.15.60:0/932642810 >> 172.21.15.60:0/932642810 conn(0x7fca04077390 msgr2=0x7fca040797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:20.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.131+0000 7fca0c000700 1 -- 172.21.15.60:0/932642810 shutdown_connections 2024-02-26T10:40:20.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.131+0000 7fca0c000700 1 -- 172.21.15.60:0/932642810 wait complete. 2024-02-26T10:40:20.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.132+0000 7fca0c000700 1 Processor -- start 2024-02-26T10:40:20.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.132+0000 7fca0c000700 1 -- start start 2024-02-26T10:40:20.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.132+0000 7fca0c000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca041083c0 0x7fca0419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.132+0000 7fca0c000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 0x7fca0419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.132+0000 7fca0c000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04109f60 0x7fca041a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.132+0000 7fca0c000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04113fe0 con 0x7fca041083c0 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.132+0000 7fca0c000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04113e60 con 0x7fca04109f60 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.132+0000 7fca0c000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04114160 con 0x7fca041095c0 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca08c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 0x7fca0419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca08c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 0x7fca0419eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:43018/0 (socket says 172.21.15.60:43018) 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca08c00700 1 -- 172.21.15.60:0/782664693 learned_addr learned my addr 172.21.15.60:0/782664693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca0a000700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04109f60 0x7fca041a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca09600700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca041083c0 0x7fca0419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca08c00700 1 -- 172.21.15.60:0/782664693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04109f60 msgr2=0x7fca041a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca08c00700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04109f60 0x7fca041a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca08c00700 1 -- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca041083c0 msgr2=0x7fca0419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca08c00700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca041083c0 0x7fca0419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca08c00700 1 -- 172.21.15.60:0/782664693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca041a6250 con 0x7fca041095c0 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca09600700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca041083c0 0x7fca0419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:20.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.133+0000 7fca08c00700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 0x7fca0419eb30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00057c0 tx=0x7fc9f00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:20.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.134+0000 7fca02000700 1 -- 172.21.15.60:0/782664693 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0027070 con 0x7fca041095c0 2024-02-26T10:40:20.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.134+0000 7fca02000700 1 -- 172.21.15.60:0/782664693 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9f0004150 con 0x7fca041095c0 2024-02-26T10:40:20.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.134+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca041a64e0 con 0x7fca041095c0 2024-02-26T10:40:20.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.134+0000 7fca02000700 1 -- 172.21.15.60:0/782664693 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000f460 con 0x7fca041095c0 2024-02-26T10:40:20.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.134+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca041a6990 con 0x7fca041095c0 2024-02-26T10:40:20.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.135+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca04068a10 con 0x7fca041095c0 2024-02-26T10:40:20.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.137+0000 7fca02000700 1 -- 172.21.15.60:0/782664693 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc9f002a070 con 0x7fca041095c0 2024-02-26T10:40:20.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.139+0000 7fca02000700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9ec0a1e90 0x7fc9ec0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.139+0000 7fca02000700 1 -- 172.21.15.60:0/782664693 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fc9f00ce290 con 0x7fca041095c0 2024-02-26T10:40:20.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.139+0000 7fca09600700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9ec0a1e90 0x7fc9ec0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.139+0000 7fca09600700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9ec0a1e90 0x7fc9ec0a4350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fca04109420 tx=0x7fc9f800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:20.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.142+0000 7fca02000700 1 -- 172.21.15.60:0/782664693 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9f005dd20 con 0x7fca041095c0 2024-02-26T10:40:20.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:20.377+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fca04062190 con 0x7fc9ec0a1e90 2024-02-26T10:40:21.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:20 smithi113 ceph-mon[36126]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:21.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:40:21.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:40:21.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:21.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:20 smithi060 ceph-mon[31539]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:21.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:40:21.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:40:21.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:21.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:20 smithi100 ceph-mon[36051]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:21.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:40:21.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:40:21.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:22.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:21 smithi113 ceph-mon[36126]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:40:22.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:21 smithi060 ceph-mon[31539]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:40:22.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:21 smithi100 ceph-mon[36051]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:40:23.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:22 smithi060 ceph-mon[31539]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-02-26T10:40:23.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:22 smithi113 ceph-mon[36126]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-02-26T10:40:23.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:22 smithi100 ceph-mon[36051]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-02-26T10:40:25.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:24 smithi113 ceph-mon[36126]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-02-26T10:40:25.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:24 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/833313434' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2beba404-c823-40fc-9102-990ad3810ce5"}]: dispatch 2024-02-26T10:40:25.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:24 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2beba404-c823-40fc-9102-990ad3810ce5"}]: dispatch 2024-02-26T10:40:25.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:24 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2beba404-c823-40fc-9102-990ad3810ce5"}]': finished 2024-02-26T10:40:25.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:24 smithi113 ceph-mon[36126]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T10:40:25.088 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:24 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:25.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:24 smithi060 ceph-mon[31539]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-02-26T10:40:25.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:24 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/833313434' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2beba404-c823-40fc-9102-990ad3810ce5"}]: dispatch 2024-02-26T10:40:25.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:24 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2beba404-c823-40fc-9102-990ad3810ce5"}]: dispatch 2024-02-26T10:40:25.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:24 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2beba404-c823-40fc-9102-990ad3810ce5"}]': finished 2024-02-26T10:40:25.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:24 smithi060 ceph-mon[31539]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T10:40:25.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:24 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:25.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:24 smithi100 ceph-mon[36051]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-02-26T10:40:25.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:24 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/833313434' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2beba404-c823-40fc-9102-990ad3810ce5"}]: dispatch 2024-02-26T10:40:25.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:24 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2beba404-c823-40fc-9102-990ad3810ce5"}]: dispatch 2024-02-26T10:40:25.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:24 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2beba404-c823-40fc-9102-990ad3810ce5"}]': finished 2024-02-26T10:40:25.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:24 smithi100 ceph-mon[36051]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T10:40:25.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:24 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:26.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:25 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3489290795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:40:26.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:25 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3489290795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:40:26.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:25 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3489290795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:40:26.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:26 smithi060 ceph-mon[31539]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-02-26T10:40:27.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:26 smithi113 ceph-mon[36126]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-02-26T10:40:27.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:26 smithi100 ceph-mon[36051]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-02-26T10:40:27.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:27 smithi060 ceph-mon[31539]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-26T10:40:28.032 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:27 smithi100 ceph-mon[36051]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-26T10:40:28.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:27 smithi113 ceph-mon[36126]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-26T10:40:30.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:29 smithi113 ceph-mon[36126]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T10:40:30.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:29 smithi060 ceph-mon[31539]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T10:40:30.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:29 smithi100 ceph-mon[36051]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-26T10:40:32.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:31 smithi060 ceph-mon[31539]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:32.337 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:31 smithi113 ceph-mon[36126]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:32.383 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:31 smithi100 ceph-mon[36051]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:34.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:33 smithi060 ceph-mon[31539]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:34.337 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:33 smithi113 ceph-mon[36126]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:34.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:33 smithi100 ceph-mon[36051]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:35.327 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T10:40:35.327 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:35.327 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:34 smithi060 ceph-mon[31539]: Deploying daemon osd.9 on smithi060 2024-02-26T10:40:35.337 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T10:40:35.337 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:35.337 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:34 smithi113 ceph-mon[36126]: Deploying daemon osd.9 on smithi060 2024-02-26T10:40:35.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T10:40:35.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:35.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:34 smithi100 ceph-mon[36051]: Deploying daemon osd.9 on smithi060 2024-02-26T10:40:36.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:35 smithi060 ceph-mon[31539]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:36.337 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:35 smithi113 ceph-mon[36126]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:36.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:35 smithi100 ceph-mon[36051]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:38.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:37 smithi060 ceph-mon[31539]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:38.337 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:37 smithi113 ceph-mon[36126]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:38.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:37 smithi100 ceph-mon[36051]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:40.177 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:39 smithi060 ceph-mon[31539]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:40.337 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:39 smithi113 ceph-mon[36126]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:40.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:39 smithi100 ceph-mon[36051]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:42.199 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:41 smithi060 ceph-mon[31539]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:42.337 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:41 smithi113 ceph-mon[36126]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:42.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:41 smithi100 ceph-mon[36051]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:44.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:44 smithi060 ceph-mon[31539]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:44.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:44.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:44.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:44.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:44 smithi113 ceph-mon[36126]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:44.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:44.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:44.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:44.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:44 smithi100 ceph-mon[36051]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:44.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:44.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:44.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:46.451 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:46 smithi060 ceph-mon[31539]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:46.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:46 smithi113 ceph-mon[36126]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:46.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:46 smithi100 ceph-mon[36051]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:47.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:47 smithi060 ceph-mon[31539]: from='osd.9 [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T10:40:47.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:47 smithi060 ceph-mon[31539]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T10:40:47.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:47.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:47.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:47 smithi113 ceph-mon[36126]: from='osd.9 [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T10:40:47.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:47 smithi113 ceph-mon[36126]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T10:40:47.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:47.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:47.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:47 smithi100 ceph-mon[36051]: from='osd.9 [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T10:40:47.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:47 smithi100 ceph-mon[36051]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T10:40:47.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:47 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:47.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:47 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:48.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:48 smithi060 ceph-mon[31539]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:48.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:48 smithi060 ceph-mon[31539]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T10:40:48.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:48 smithi060 ceph-mon[31539]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T10:40:48.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:48.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:48 smithi060 ceph-mon[31539]: from='osd.9 [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:40:48.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:48 smithi060 ceph-mon[31539]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:40:48.393 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 9 on host 'smithi060' 2024-02-26T10:40:48.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.392+0000 7fca02000700 1 -- 172.21.15.60:0/782664693 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fca04062190 con 0x7fc9ec0a1e90 2024-02-26T10:40:48.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.396+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9ec0a1e90 msgr2=0x7fc9ec0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:48.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.396+0000 7fca0c000700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9ec0a1e90 0x7fc9ec0a4350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fca04109420 tx=0x7fc9f800b670 comp rx=0 tx=0).stop 2024-02-26T10:40:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.396+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 msgr2=0x7fca0419eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.396+0000 7fca0c000700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 0x7fca0419eb30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00057c0 tx=0x7fc9f00057f0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.396+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 shutdown_connections 2024-02-26T10:40:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.396+0000 7fca0c000700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9ec0a1e90 0x7fc9ec0a4350 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.397+0000 7fca0c000700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04109f60 0x7fca041a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.397+0000 7fca0c000700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca041083c0 0x7fca0419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.397+0000 7fca0c000700 1 --2- 172.21.15.60:0/782664693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca041095c0 0x7fca0419eb30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.397+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 >> 172.21.15.60:0/782664693 conn(0x7fca04077390 msgr2=0x7fca0410c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.397+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 shutdown_connections 2024-02-26T10:40:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:40:48.397+0000 7fca0c000700 1 -- 172.21.15.60:0/782664693 wait complete. 2024-02-26T10:40:48.488 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:48 smithi113 ceph-mon[36126]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:48.488 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:48 smithi113 ceph-mon[36126]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T10:40:48.488 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:48 smithi113 ceph-mon[36126]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T10:40:48.488 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:48.488 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:48 smithi113 ceph-mon[36126]: from='osd.9 [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:40:48.489 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:48 smithi113 ceph-mon[36126]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:40:48.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:48 smithi100 ceph-mon[36051]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:48.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:48 smithi100 ceph-mon[36051]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T10:40:48.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:48 smithi100 ceph-mon[36051]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T10:40:48.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:48.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:48 smithi100 ceph-mon[36051]: from='osd.9 [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:40:48.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:48 smithi100 ceph-mon[36051]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T10:40:49.240 DEBUG:teuthology.orchestra.run.smithi060:osd.9> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.9.service 2024-02-26T10:40:49.244 INFO:tasks.cephadm:Deploying osd.10 on smithi100 with /dev/vg_nvme/lv_1... 2024-02-26T10:40:49.244 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T10:40:49.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:49 smithi060 ceph-mon[31539]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:40:49.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:49 smithi060 ceph-mon[31539]: osdmap e67: 10 total, 9 up, 10 in 2024-02-26T10:40:49.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:49 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:49.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:49 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:49.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:49 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:49.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:49 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:49.488 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:40:49.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:49 smithi113 ceph-mon[36126]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:40:49.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:49 smithi113 ceph-mon[36126]: osdmap e67: 10 total, 9 up, 10 in 2024-02-26T10:40:49.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:49 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:49.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:49 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:49.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:49 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:49.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:49 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:49.610 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:49 smithi100 ceph-mon[36051]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T10:40:49.610 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:49 smithi100 ceph-mon[36051]: osdmap e67: 10 total, 9 up, 10 in 2024-02-26T10:40:49.610 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:49.610 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:49.610 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:49.610 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:49 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:50.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:50 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:40:50.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:50 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:40:50.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:50 smithi100 ceph-mon[36051]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:50.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:50.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:50 smithi100 ceph-mon[36051]: osd.9 [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] boot 2024-02-26T10:40:50.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:50 smithi100 ceph-mon[36051]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T10:40:50.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:50.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:50 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:40:50.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:50 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:40:50.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:50 smithi113 ceph-mon[36126]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:50.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:50.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:50 smithi113 ceph-mon[36126]: osd.9 [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] boot 2024-02-26T10:40:50.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:50 smithi113 ceph-mon[36126]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T10:40:50.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:50.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:50 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:40:50.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:50 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:40:50.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:50 smithi060 ceph-mon[31539]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-02-26T10:40:50.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:50.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:50 smithi060 ceph-mon[31539]: osd.9 [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] boot 2024-02-26T10:40:50.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:50 smithi060 ceph-mon[31539]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T10:40:50.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T10:40:51.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:51 smithi060 ceph-mon[31539]: osdmap e69: 10 total, 10 up, 10 in 2024-02-26T10:40:51.542 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:51 smithi100 ceph-mon[36051]: osdmap e69: 10 total, 10 up, 10 in 2024-02-26T10:40:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T10:40:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T10:40:51.542 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-02-26T10:40:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-02-26T10:40:51.542 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125098 s, 838 MB/s 2024-02-26T10:40:51.542 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-02-26T10:40:51.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:51 smithi113 ceph-mon[36126]: osdmap e69: 10 total, 10 up, 10 in 2024-02-26T10:40:52.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-02-26T10:40:52.469 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:40:52.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:52 smithi113 ceph-mon[36126]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:52.590 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:52 smithi100 ceph-mon[36051]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:52.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:52 smithi060 ceph-mon[31539]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 -- 172.21.15.100:0/641885335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 msgr2=0x7f1ef810a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 --2- 172.21.15.100:0/641885335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 0x7f1ef810a1b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80099c0 tx=0x7f1ee8009cd0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 -- 172.21.15.100:0/641885335 shutdown_connections 2024-02-26T10:40:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 --2- 172.21.15.100:0/641885335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 0x7f1ef810a1b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 --2- 172.21.15.100:0/641885335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef8101d60 0x7f1ef8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 --2- 172.21.15.100:0/641885335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef8102f60 0x7f1ef81033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 -- 172.21.15.100:0/641885335 >> 172.21.15.100:0/641885335 conn(0x7f1ef80fd2c0 msgr2=0x7f1ef80ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 -- 172.21.15.100:0/641885335 shutdown_connections 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 -- 172.21.15.100:0/641885335 wait complete. 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.225+0000 7f1eff809700 1 Processor -- start 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1eff809700 1 -- start start 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1eff809700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef8101d60 0x7f1ef8075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1eff809700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef8102f60 0x7f1ef8074140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1eff809700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 0x7f1ef8074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1eff809700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef810da40 con 0x7f1ef8102f60 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1eff809700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef810d8c0 con 0x7f1ef8103900 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1eff809700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef810dbc0 con 0x7f1ef8101d60 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efdda6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 0x7f1ef8074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efdda6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 0x7f1ef8074680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52606/0 (socket says 172.21.15.100:52606) 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efdda6700 1 -- 172.21.15.100:0/66275666 learned_addr learned my addr 172.21.15.100:0/66275666 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efd5a5700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef8101d60 0x7f1ef8075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efdda6700 1 -- 172.21.15.100:0/66275666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef8101d60 msgr2=0x7f1ef8075af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efdda6700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef8101d60 0x7f1ef8075af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efdda6700 1 -- 172.21.15.100:0/66275666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef8102f60 msgr2=0x7f1ef8074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efdda6700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef8102f60 0x7f1ef8074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efdda6700 1 -- 172.21.15.100:0/66275666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8074f30 con 0x7f1ef8103900 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1efdda6700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 0x7f1ef8074680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009990 tx=0x7f1ee80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.226+0000 7f1ef67fc700 1 -- 172.21.15.100:0/66275666 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8027070 con 0x7f1ef8103900 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.227+0000 7f1ef67fc700 1 -- 172.21.15.100:0/66275666 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8022470 con 0x7f1ef8103900 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.227+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef81a2180 con 0x7f1ef8103900 2024-02-26T10:40:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.227+0000 7f1ef67fc700 1 -- 172.21.15.100:0/66275666 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800f6c0 con 0x7f1ef8103900 2024-02-26T10:40:54.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.227+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef81a2600 con 0x7f1ef8103900 2024-02-26T10:40:54.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.228+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef802d7f0 con 0x7f1ef8103900 2024-02-26T10:40:54.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.229+0000 7f1ef67fc700 1 -- 172.21.15.100:0/66275666 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1ee802a070 con 0x7f1ef8103900 2024-02-26T10:40:54.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.229+0000 7f1ef67fc700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ee00a1e10 0x7f1ee00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.229+0000 7f1ef67fc700 1 -- 172.21.15.100:0/66275666 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1ee80cea80 con 0x7f1ef8103900 2024-02-26T10:40:54.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.230+0000 7f1efd5a5700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ee00a1e10 0x7f1ee00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:54.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.231+0000 7f1efd5a5700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ee00a1e10 0x7f1ee00a42d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8102dc0 tx=0x7f1ef00094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:54.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.233+0000 7f1ef67fc700 1 -- 172.21.15.100:0/66275666 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ee8013070 con 0x7f1ef8103900 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:54.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:54 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:40:54.438+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ef8062190 con 0x7f1ee00a1e10 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:54.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:54 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:54.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:40:54.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:54.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:54.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:54.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:54 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:55.349 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi060 2024-02-26T10:40:55.349 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi060 to 4273M 2024-02-26T10:40:55.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:40:55.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:40:55.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:40:55.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:55.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:55.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:55.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:55.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:55 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi060 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi060 to 4273M 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:55.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:55 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:55.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi060 2024-02-26T10:40:55.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi060 to 4273M 2024-02-26T10:40:55.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:40:55.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:40:55.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:40:55.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:55.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:40:55.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:40:55.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:40:55.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:55 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:40:56.716 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:56 smithi100 ceph-mon[36051]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:56.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:56 smithi060 ceph-mon[31539]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:57.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:56 smithi113 ceph-mon[36126]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:57.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:57 smithi060 ceph-mon[31539]: from='client.? 172.21.15.100:0/284281913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4"}]: dispatch 2024-02-26T10:40:57.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:57 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4"}]: dispatch 2024-02-26T10:40:57.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:57 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4"}]': finished 2024-02-26T10:40:57.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:57 smithi060 ceph-mon[31539]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T10:40:57.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:40:57.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:57 smithi100 ceph-mon[36051]: from='client.? 172.21.15.100:0/284281913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4"}]: dispatch 2024-02-26T10:40:57.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:57 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4"}]: dispatch 2024-02-26T10:40:57.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:57 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4"}]': finished 2024-02-26T10:40:57.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:57 smithi100 ceph-mon[36051]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T10:40:57.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:57 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:40:58.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:57 smithi113 ceph-mon[36126]: from='client.? 172.21.15.100:0/284281913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4"}]: dispatch 2024-02-26T10:40:58.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:57 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4"}]: dispatch 2024-02-26T10:40:58.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:57 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4"}]': finished 2024-02-26T10:40:58.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:57 smithi113 ceph-mon[36126]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T10:40:58.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:57 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:40:58.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:58 smithi060 ceph-mon[31539]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:58.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:40:58 smithi060 ceph-mon[31539]: from='client.? 172.21.15.100:0/820810628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:40:58.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:58 smithi100 ceph-mon[36051]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:58.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:40:58 smithi100 ceph-mon[36051]: from='client.? 172.21.15.100:0/820810628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:40:59.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:58 smithi113 ceph-mon[36126]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:40:59.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:40:58 smithi113 ceph-mon[36126]: from='client.? 172.21.15.100:0/820810628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:41:00.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:00 smithi100 ceph-mon[36051]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:01.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:00 smithi113 ceph-mon[36126]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:01.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:00 smithi060 ceph-mon[31539]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:03.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:02 smithi113 ceph-mon[36126]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:03.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:02 smithi060 ceph-mon[31539]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:03.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:02 smithi100 ceph-mon[36051]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:04.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:04 smithi100 ceph-mon[36051]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:05.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:04 smithi113 ceph-mon[36126]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:05.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:04 smithi060 ceph-mon[31539]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:05.835 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T10:41:05.835 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:05 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:06.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T10:41:06.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:05 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:06.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T10:41:06.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:05 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:06.830 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:06 smithi100 ceph-mon[36051]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:06.830 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:06 smithi100 ceph-mon[36051]: Deploying daemon osd.10 on smithi100 2024-02-26T10:41:07.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:06 smithi113 ceph-mon[36126]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:07.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:06 smithi113 ceph-mon[36126]: Deploying daemon osd.10 on smithi100 2024-02-26T10:41:07.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:06 smithi060 ceph-mon[31539]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:07.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:06 smithi060 ceph-mon[31539]: Deploying daemon osd.10 on smithi100 2024-02-26T10:41:08.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:08 smithi060 ceph-mon[31539]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:08.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:08 smithi100 ceph-mon[36051]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:09.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:08 smithi113 ceph-mon[36126]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:10.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:10 smithi100 ceph-mon[36051]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:10.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:10 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:10.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:10 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:10.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:10 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:11.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:10 smithi113 ceph-mon[36126]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:11.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:10 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:11.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:10 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:11.087 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:10 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:11.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:10 smithi060 ceph-mon[31539]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:11.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:10 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:11.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:10 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:11.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:10 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:12.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:12 smithi100 ceph-mon[36051]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:13.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:12 smithi113 ceph-mon[36126]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:13.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:12 smithi060 ceph-mon[31539]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:14.130 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 10 on host 'smithi100' 2024-02-26T10:41:14.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.130+0000 7f1ef67fc700 1 -- 172.21.15.100:0/66275666 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1ef8062190 con 0x7f1ee00a1e10 2024-02-26T10:41:14.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ee00a1e10 msgr2=0x7f1ee00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ee00a1e10 0x7f1ee00a42d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8102dc0 tx=0x7f1ef00094f0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 msgr2=0x7f1ef8074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 0x7f1ef8074680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009990 tx=0x7f1ee80047b0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 shutdown_connections 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ee00a1e10 0x7f1ee00a42d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef8103900 0x7f1ef8074680 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef8101d60 0x7f1ef8075af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 --2- 172.21.15.100:0/66275666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef8102f60 0x7f1ef8074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 >> 172.21.15.100:0/66275666 conn(0x7f1ef80fd2c0 msgr2=0x7f1ef81062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 shutdown_connections 2024-02-26T10:41:14.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-26T10:41:14.132+0000 7f1eff809700 1 -- 172.21.15.100:0/66275666 wait complete. 2024-02-26T10:41:14.763 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:14 smithi100 ceph-mon[36051]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:14.763 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:14.763 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:14.763 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:14 smithi100 ceph-mon[36051]: from='osd.10 [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T10:41:14.763 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:14 smithi100 ceph-mon[36051]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T10:41:14.789 DEBUG:teuthology.orchestra.run.smithi100:osd.10> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.10.service 2024-02-26T10:41:14.791 INFO:tasks.cephadm:Deploying osd.11 on smithi113 with /dev/vg_nvme/lv_1... 2024-02-26T10:41:14.791 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T10:41:15.038 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:41:15.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:14 smithi113 ceph-mon[36126]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:15.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:15.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:15.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:14 smithi113 ceph-mon[36126]: from='osd.10 [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T10:41:15.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:14 smithi113 ceph-mon[36126]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T10:41:15.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:14 smithi060 ceph-mon[31539]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:15.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:15.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:15.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:14 smithi060 ceph-mon[31539]: from='osd.10 [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T10:41:15.119 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:14 smithi060 ceph-mon[31539]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T10:41:16.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:15 smithi100 ceph-mon[36051]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:16.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:15 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:16.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:15 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:16.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:15 smithi100 ceph-mon[36051]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T10:41:16.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:15 smithi100 ceph-mon[36051]: from='osd.10 [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:41:16.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:15 smithi100 ceph-mon[36051]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T10:41:16.128 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:15 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:16.128 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:15 smithi100 ceph-mon[36051]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:41:16.304 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:15 smithi113 ceph-mon[36126]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:16.305 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:15 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:16.305 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:15 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:16.305 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:15 smithi113 ceph-mon[36126]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T10:41:16.305 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:15 smithi113 ceph-mon[36126]: from='osd.10 [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:41:16.305 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:15 smithi113 ceph-mon[36126]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T10:41:16.305 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:15 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:16.305 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:15 smithi113 ceph-mon[36126]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:41:16.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:15 smithi060 ceph-mon[31539]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:16.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:15 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:16.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:15 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:16.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:15 smithi060 ceph-mon[31539]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T10:41:16.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:15 smithi060 ceph-mon[31539]: from='osd.10 [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:41:16.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:15 smithi060 ceph-mon[31539]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T10:41:16.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:15 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:16.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:15 smithi060 ceph-mon[31539]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-26T10:41:16.380 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:41:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:41:16.127+0000 7fb7222d8700 -1 osd.10 0 waiting for initial osdmap 2024-02-26T10:41:16.380 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:41:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:41:16.142+0000 7fb71e0cb700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:41:17.209 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T10:41:17.209 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T10:41:17.210 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-02-26T10:41:17.210 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-02-26T10:41:17.210 INFO:teuthology.orchestra.run.smithi113.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0152315 s, 688 MB/s 2024-02-26T10:41:17.210 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-02-26T10:41:17.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:17 smithi100 ceph-mon[36051]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:41:17.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:17 smithi100 ceph-mon[36051]: osdmap e72: 11 total, 10 up, 11 in 2024-02-26T10:41:17.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:17 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:17.391 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:17 smithi113 ceph-mon[36126]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:41:17.391 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:17 smithi113 ceph-mon[36126]: osdmap e72: 11 total, 10 up, 11 in 2024-02-26T10:41:17.391 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:17 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:17.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:17 smithi060 ceph-mon[31539]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-26T10:41:17.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:17 smithi060 ceph-mon[31539]: osdmap e72: 11 total, 10 up, 11 in 2024-02-26T10:41:17.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:17 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:18.082 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi113:vg_nvme/lv_1 2024-02-26T10:41:18.324 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:41:18.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:18 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:41:18.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:18 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:41:18.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:18 smithi113 ceph-mon[36126]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:18.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:18 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:18.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:18 smithi113 ceph-mon[36126]: osd.10 [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] boot 2024-02-26T10:41:18.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:18 smithi113 ceph-mon[36126]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T10:41:18.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:18 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:18.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:18 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:41:18.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:18 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:41:18.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:18 smithi100 ceph-mon[36051]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:18.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:18 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:18.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:18 smithi100 ceph-mon[36051]: osd.10 [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] boot 2024-02-26T10:41:18.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:18 smithi100 ceph-mon[36051]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T10:41:18.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:18 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:18.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:18 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:41:18.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:18 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:41:18.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:18 smithi060 ceph-mon[31539]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-02-26T10:41:18.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:18 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:18.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:18 smithi060 ceph-mon[31539]: osd.10 [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] boot 2024-02-26T10:41:18.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:18 smithi060 ceph-mon[31539]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T10:41:18.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:18 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T10:41:19.480 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T10:41:19.480 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi100 2024-02-26T10:41:19.480 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.480 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.480 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi100 to 4270M 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:19.481 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:19.482 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:19 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T10:41:19.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi100 2024-02-26T10:41:19.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi100 to 4270M 2024-02-26T10:41:19.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:19.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:19.620 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:19.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:19.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:19.621 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:19 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi100 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi100 to 4270M 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:19.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:19.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:19.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:19.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:19.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:19.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:19 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:20.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 -- 172.21.15.113:0/1504361353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca2010e330 msgr2=0x7fca20110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 --2- 172.21.15.113:0/1504361353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca2010e330 0x7fca20110720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca1800b0a0 tx=0x7fca1800b3b0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 -- 172.21.15.113:0/1504361353 shutdown_connections 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 --2- 172.21.15.113:0/1504361353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca2010e330 0x7fca20110720 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 --2- 172.21.15.113:0/1504361353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca20075d80 0x7fca200741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 --2- 172.21.15.113:0/1504361353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca200747b0 0x7fca20074c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 -- 172.21.15.113:0/1504361353 >> 172.21.15.113:0/1504361353 conn(0x7fca200fd330 msgr2=0x7fca200ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 -- 172.21.15.113:0/1504361353 shutdown_connections 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 -- 172.21.15.113:0/1504361353 wait complete. 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.029+0000 7fca280ea700 1 Processor -- start 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca280ea700 1 -- start start 2024-02-26T10:41:20.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca280ea700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca200747b0 0x7fca2019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca280ea700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca20075d80 0x7fca2019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca280ea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2010e330 0x7fca201a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca280ea700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20114050 con 0x7fca200747b0 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca280ea700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20113ed0 con 0x7fca2010e330 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca280ea700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca201141d0 con 0x7fca20075d80 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca25685700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca20075d80 0x7fca2019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca25685700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca20075d80 0x7fca2019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50814/0 (socket says 172.21.15.113:50814) 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca25685700 1 -- 172.21.15.113:0/3215301177 learned_addr learned my addr 172.21.15.113:0/3215301177 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca26687700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2010e330 0x7fca201a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca25685700 1 -- 172.21.15.113:0/3215301177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2010e330 msgr2=0x7fca201a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca25685700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2010e330 0x7fca201a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca25685700 1 -- 172.21.15.113:0/3215301177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca200747b0 msgr2=0x7fca2019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca25685700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca200747b0 0x7fca2019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca25685700 1 -- 172.21.15.113:0/3215301177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca201a6270 con 0x7fca20075d80 2024-02-26T10:41:20.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca26687700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2010e330 0x7fca201a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:20.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca25685700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca20075d80 0x7fca2019eb50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fca1000b530 tx=0x7fca1000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:20.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca16ffd700 1 -- 172.21.15.113:0/3215301177 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10015020 con 0x7fca20075d80 2024-02-26T10:41:20.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca201a6560 con 0x7fca20075d80 2024-02-26T10:41:20.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca201a6b70 con 0x7fca20075d80 2024-02-26T10:41:20.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca16ffd700 1 -- 172.21.15.113:0/3215301177 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca10011d10 con 0x7fca20075d80 2024-02-26T10:41:20.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.030+0000 7fca16ffd700 1 -- 172.21.15.113:0/3215301177 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1000f2e0 con 0x7fca20075d80 2024-02-26T10:41:20.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.031+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca20068a10 con 0x7fca20075d80 2024-02-26T10:41:20.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.032+0000 7fca16ffd700 1 -- 172.21.15.113:0/3215301177 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fca10018020 con 0x7fca20075d80 2024-02-26T10:41:20.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.033+0000 7fca16ffd700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca0c0a1e10 0x7fca0c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.033+0000 7fca16ffd700 1 -- 172.21.15.113:0/3215301177 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fca100ccfa0 con 0x7fca20075d80 2024-02-26T10:41:20.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.034+0000 7fca25e86700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca0c0a1e10 0x7fca0c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.034+0000 7fca25e86700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca0c0a1e10 0x7fca0c0a42d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca20076160 tx=0x7fca1c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:20.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.037+0000 7fca16ffd700 1 -- 172.21.15.113:0/3215301177 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca1001f020 con 0x7fca20075d80 2024-02-26T10:41:20.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:20.240+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fca20062190 con 0x7fca0c0a1e10 2024-02-26T10:41:20.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:20 smithi113 ceph-mon[36126]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:20.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:20 smithi113 ceph-mon[36126]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T10:41:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:20 smithi060 ceph-mon[31539]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:20 smithi060 ceph-mon[31539]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T10:41:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:20 smithi100 ceph-mon[36051]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:20 smithi100 ceph-mon[36051]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T10:41:21.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:21 smithi113 ceph-mon[36126]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T10:41:21.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:21 smithi113 ceph-mon[36126]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:41:21.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:41:21.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:41:21.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:21 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:21.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:21 smithi060 ceph-mon[31539]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T10:41:21.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:21 smithi060 ceph-mon[31539]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:41:21.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:41:21.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:41:21.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:21 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:21.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:21 smithi100 ceph-mon[36051]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T10:41:21.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:21 smithi100 ceph-mon[36051]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:41:21.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T10:41:21.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T10:41:21.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:21 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:22.445 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:22 smithi113 ceph-mon[36126]: pgmap v268: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T10:41:22.445 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:22 smithi113 ceph-mon[36126]: osdmap e77: 11 total, 11 up, 11 in 2024-02-26T10:41:22.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:22 smithi060 ceph-mon[31539]: pgmap v268: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T10:41:22.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:22 smithi060 ceph-mon[31539]: osdmap e77: 11 total, 11 up, 11 in 2024-02-26T10:41:22.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:22 smithi100 ceph-mon[36051]: pgmap v268: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T10:41:22.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:22 smithi100 ceph-mon[36051]: osdmap e77: 11 total, 11 up, 11 in 2024-02-26T10:41:23.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:23 smithi113 ceph-mon[36126]: from='client.? 172.21.15.113:0/3992851331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ecbb968-c217-40ea-b64a-91e5de0c7327"}]: dispatch 2024-02-26T10:41:23.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:23 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ecbb968-c217-40ea-b64a-91e5de0c7327"}]: dispatch 2024-02-26T10:41:23.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:23 smithi113 ceph-mon[36126]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ecbb968-c217-40ea-b64a-91e5de0c7327"}]': finished 2024-02-26T10:41:23.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:23 smithi113 ceph-mon[36126]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T10:41:23.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:23 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:23.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:23 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/3992851331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ecbb968-c217-40ea-b64a-91e5de0c7327"}]: dispatch 2024-02-26T10:41:23.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:23 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ecbb968-c217-40ea-b64a-91e5de0c7327"}]: dispatch 2024-02-26T10:41:23.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:23 smithi060 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ecbb968-c217-40ea-b64a-91e5de0c7327"}]': finished 2024-02-26T10:41:23.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:23 smithi060 ceph-mon[31539]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T10:41:23.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:23.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:23 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/3992851331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ecbb968-c217-40ea-b64a-91e5de0c7327"}]: dispatch 2024-02-26T10:41:23.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:23 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ecbb968-c217-40ea-b64a-91e5de0c7327"}]: dispatch 2024-02-26T10:41:23.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:23 smithi100 ceph-mon[36051]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ecbb968-c217-40ea-b64a-91e5de0c7327"}]': finished 2024-02-26T10:41:23.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:23 smithi100 ceph-mon[36051]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T10:41:23.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:23 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:24.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:24 smithi113 ceph-mon[36126]: pgmap v270: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-26T10:41:24.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:24 smithi113 ceph-mon[36126]: from='client.? 172.21.15.113:0/3679871286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:41:24.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:24 smithi060 ceph-mon[31539]: pgmap v270: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-26T10:41:24.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:24 smithi060 ceph-mon[31539]: from='client.? 172.21.15.113:0/3679871286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:41:24.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:24 smithi100 ceph-mon[36051]: pgmap v270: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-26T10:41:24.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:24 smithi100 ceph-mon[36051]: from='client.? 172.21.15.113:0/3679871286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T10:41:26.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:26 smithi113 ceph-mon[36126]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-02-26T10:41:26.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:26 smithi060 ceph-mon[31539]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-02-26T10:41:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:26 smithi100 ceph-mon[36051]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-02-26T10:41:28.254 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:28 smithi113 ceph-mon[36126]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:28.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:28 smithi060 ceph-mon[31539]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:28.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:28 smithi100 ceph-mon[36051]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:30.492 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:30 smithi113 ceph-mon[36126]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:30.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:30 smithi060 ceph-mon[31539]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:30.621 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:30 smithi100 ceph-mon[36051]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:31.453 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T10:41:31.453 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:31 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:31.453 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:31 smithi113 ceph-mon[36126]: Deploying daemon osd.11 on smithi113 2024-02-26T10:41:31.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T10:41:31.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:31 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:31.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:31 smithi060 ceph-mon[31539]: Deploying daemon osd.11 on smithi113 2024-02-26T10:41:31.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T10:41:31.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:31 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:31.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:31 smithi100 ceph-mon[36051]: Deploying daemon osd.11 on smithi113 2024-02-26T10:41:32.460 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:32 smithi113 ceph-mon[36126]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:32.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:32 smithi060 ceph-mon[31539]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:32.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:32 smithi100 ceph-mon[36051]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:34.468 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:34 smithi113 ceph-mon[36126]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:34.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:34 smithi060 ceph-mon[31539]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:34.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:34 smithi100 ceph-mon[36051]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:36.279 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:36 smithi113 ceph-mon[36126]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:36.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:36 smithi060 ceph-mon[31539]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:36.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:36 smithi100 ceph-mon[36051]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:37.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:37.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:37.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:37.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:37.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:37.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:37.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:37.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:37.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:38.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:38 smithi113 ceph-mon[36126]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:38.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:38 smithi060 ceph-mon[31539]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:38.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:38 smithi100 ceph-mon[36051]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:40.378 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 11 on host 'smithi113' 2024-02-26T10:41:40.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.378+0000 7fca16ffd700 1 -- 172.21.15.113:0/3215301177 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fca20062190 con 0x7fca0c0a1e10 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca0c0a1e10 msgr2=0x7fca0c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca0c0a1e10 0x7fca0c0a42d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca20076160 tx=0x7fca1c006d20 comp rx=0 tx=0).stop 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca20075d80 msgr2=0x7fca2019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca20075d80 0x7fca2019eb50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fca1000b530 tx=0x7fca1000b8f0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 shutdown_connections 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca0c0a1e10 0x7fca0c0a42d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2010e330 0x7fca201a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca200747b0 0x7fca2019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 --2- 172.21.15.113:0/3215301177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca20075d80 0x7fca2019eb50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 >> 172.21.15.113:0/3215301177 conn(0x7fca200fd330 msgr2=0x7fca201016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 shutdown_connections 2024-02-26T10:41:40.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:40.381+0000 7fca280ea700 1 -- 172.21.15.113:0/3215301177 wait complete. 2024-02-26T10:41:40.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='osd.11 [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:40 smithi113 ceph-mon[36126]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T10:41:40.691 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:40.691 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.691 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.691 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:40.691 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:40.691 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.691 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.691 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:40.691 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='osd.11 [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T10:41:40.692 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.692 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:40 smithi100 ceph-mon[36051]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T10:41:40.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:40.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:40.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:40.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:40.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='osd.11 [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T10:41:40.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:40.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:40 smithi060 ceph-mon[31539]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T10:41:41.321 DEBUG:teuthology.orchestra.run.smithi113:osd.11> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.11.service 2024-02-26T10:41:41.323 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-26T10:41:41.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd stat -f json 2024-02-26T10:41:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:41:42.547 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:42 smithi113 ceph-mon[36126]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:42.548 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:42 smithi113 ceph-mon[36126]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T10:41:42.548 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:42 smithi113 ceph-mon[36126]: from='osd.11 [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:41:42.548 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:42 smithi113 ceph-mon[36126]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T10:41:42.548 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:42 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:42.548 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:42 smithi113 ceph-mon[36126]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:41:42.548 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:41:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:41:42.341+0000 7fcda53f0700 -1 osd.11 0 waiting for initial osdmap 2024-02-26T10:41:42.548 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:41:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:41:42.356+0000 7fcda11e3700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:41:42.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:42 smithi060 ceph-mon[31539]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:42.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:42 smithi060 ceph-mon[31539]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T10:41:42.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:42 smithi060 ceph-mon[31539]: from='osd.11 [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:41:42.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:42 smithi060 ceph-mon[31539]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T10:41:42.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:42 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:42.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:42 smithi060 ceph-mon[31539]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:41:42.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:42 smithi100 ceph-mon[36051]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:42.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:42 smithi100 ceph-mon[36051]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T10:41:42.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:42 smithi100 ceph-mon[36051]: from='osd.11 [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:41:42.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:42 smithi100 ceph-mon[36051]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T10:41:42.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:42.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:42 smithi100 ceph-mon[36051]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-02-26T10:41:43.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:43 smithi060 ceph-mon[31539]: purged_snaps scrub starts 2024-02-26T10:41:43.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:43 smithi060 ceph-mon[31539]: purged_snaps scrub ok 2024-02-26T10:41:43.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:43 smithi060 ceph-mon[31539]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:41:43.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:43 smithi060 ceph-mon[31539]: osdmap e80: 12 total, 11 up, 12 in 2024-02-26T10:41:43.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:43.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:43.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:43.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:43 smithi100 ceph-mon[36051]: purged_snaps scrub starts 2024-02-26T10:41:43.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:43 smithi100 ceph-mon[36051]: purged_snaps scrub ok 2024-02-26T10:41:43.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:43 smithi100 ceph-mon[36051]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:41:43.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:43 smithi100 ceph-mon[36051]: osdmap e80: 12 total, 11 up, 12 in 2024-02-26T10:41:43.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:43.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:43.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:43.681 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:43 smithi113 ceph-mon[36126]: purged_snaps scrub starts 2024-02-26T10:41:43.681 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:43 smithi113 ceph-mon[36126]: purged_snaps scrub ok 2024-02-26T10:41:43.681 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:43 smithi113 ceph-mon[36126]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-02-26T10:41:43.681 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:43 smithi113 ceph-mon[36126]: osdmap e80: 12 total, 11 up, 12 in 2024-02-26T10:41:43.681 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:43.681 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:43.682 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:44.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.177+0000 7fe30b200700 1 -- 172.21.15.60:0/1844790340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3040747b0 msgr2=0x7fe304074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.177+0000 7fe30b200700 1 --2- 172.21.15.60:0/1844790340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3040747b0 0x7fe304074c30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009990 tx=0x7fe2ec009ca0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.178+0000 7fe30b200700 1 -- 172.21.15.60:0/1844790340 shutdown_connections 2024-02-26T10:41:44.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.178+0000 7fe30b200700 1 --2- 172.21.15.60:0/1844790340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe304109c60 0x7fe30410c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.178+0000 7fe30b200700 1 --2- 172.21.15.60:0/1844790340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe304075d80 0x7fe3040741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.178+0000 7fe30b200700 1 --2- 172.21.15.60:0/1844790340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3040747b0 0x7fe304074c30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.178+0000 7fe30b200700 1 -- 172.21.15.60:0/1844790340 >> 172.21.15.60:0/1844790340 conn(0x7fe3040fd2f0 msgr2=0x7fe3040ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.178+0000 7fe30b200700 1 -- 172.21.15.60:0/1844790340 shutdown_connections 2024-02-26T10:41:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.178+0000 7fe30b200700 1 -- 172.21.15.60:0/1844790340 wait complete. 2024-02-26T10:41:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.179+0000 7fe30b200700 1 Processor -- start 2024-02-26T10:41:44.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.179+0000 7fe30b200700 1 -- start start 2024-02-26T10:41:44.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.179+0000 7fe30b200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3040747b0 0x7fe30419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.179+0000 7fe30b200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe304075d80 0x7fe30419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.179+0000 7fe30b200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe304109c60 0x7fe3041a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.179+0000 7fe30b200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30410fbc0 con 0x7fe304075d80 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.179+0000 7fe30b200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30410fa40 con 0x7fe3040747b0 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.179+0000 7fe30b200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30410fd40 con 0x7fe304109c60 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe303e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3040747b0 0x7fe30419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe309200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe304109c60 0x7fe3041a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe2fbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe304075d80 0x7fe30419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe303e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3040747b0 0x7fe30419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:55164/0 (socket says 172.21.15.60:55164) 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe303e00700 1 -- 172.21.15.60:0/1416287776 learned_addr learned my addr 172.21.15.60:0/1416287776 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe309200700 1 -- 172.21.15.60:0/1416287776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3040747b0 msgr2=0x7fe30419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe309200700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3040747b0 0x7fe30419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe309200700 1 -- 172.21.15.60:0/1416287776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe304075d80 msgr2=0x7fe30419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe309200700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe304075d80 0x7fe30419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe309200700 1 -- 172.21.15.60:0/1416287776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3041a1e30 con 0x7fe304109c60 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.180+0000 7fe309200700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe304109c60 0x7fe3041a1790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009f60 tx=0x7fe2fc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.181+0000 7fe301600700 1 -- 172.21.15.60:0/1416287776 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0092e0 con 0x7fe304109c60 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.181+0000 7fe301600700 1 -- 172.21.15.60:0/1416287776 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2fc004bb0 con 0x7fe304109c60 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.181+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3041a20c0 con 0x7fe304109c60 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.181+0000 7fe301600700 1 -- 172.21.15.60:0/1416287776 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc004280 con 0x7fe304109c60 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.181+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3041a2650 con 0x7fe304109c60 2024-02-26T10:41:44.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.182+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe304068a10 con 0x7fe304109c60 2024-02-26T10:41:44.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.184+0000 7fe301600700 1 -- 172.21.15.60:0/1416287776 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe2fc01b030 con 0x7fe304109c60 2024-02-26T10:41:44.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.185+0000 7fe301600700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe2e80a1e90 0x7fe2e80a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.185+0000 7fe301600700 1 -- 172.21.15.60:0/1416287776 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2fc0cd370 con 0x7fe304109c60 2024-02-26T10:41:44.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.185+0000 7fe303e00700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe2e80a1e90 0x7fe2e80a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.186+0000 7fe303e00700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe2e80a1e90 0x7fe2e80a4350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4005950 tx=0x7fe2f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:44.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.189+0000 7fe301600700 1 -- 172.21.15.60:0/1416287776 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe2fc022020 con 0x7fe304109c60 2024-02-26T10:41:44.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.419+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fe304062960 con 0x7fe304109c60 2024-02-26T10:41:44.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.420+0000 7fe301600700 1 -- 172.21.15.60:0/1416287776 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7fe2fc010040 con 0x7fe304109c60 2024-02-26T10:41:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:41:44.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe2e80a1e90 msgr2=0x7fe2e80a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe2e80a1e90 0x7fe2e80a4350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4005950 tx=0x7fe2f4009260 comp rx=0 tx=0).stop 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe304109c60 msgr2=0x7fe3041a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe304109c60 0x7fe3041a1790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009f60 tx=0x7fe2fc009f90 comp rx=0 tx=0).stop 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 shutdown_connections 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe2e80a1e90 0x7fe2e80a4350 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe304109c60 0x7fe3041a1790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3040747b0 0x7fe30419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 --2- 172.21.15.60:0/1416287776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe304075d80 0x7fe30419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 >> 172.21.15.60:0/1416287776 conn(0x7fe3040fd2f0 msgr2=0x7fe304105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 shutdown_connections 2024-02-26T10:41:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:44.423+0000 7fe30b200700 1 -- 172.21.15.60:0/1416287776 wait complete. 2024-02-26T10:41:44.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:44 smithi113 ceph-mon[36126]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:44.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:44 smithi113 ceph-mon[36126]: osd.11 [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] boot 2024-02-26T10:41:44.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:44 smithi113 ceph-mon[36126]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T10:41:44.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:44.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:44 smithi060 ceph-mon[31539]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:44.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:44 smithi060 ceph-mon[31539]: osd.11 [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] boot 2024-02-26T10:41:44.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:44 smithi060 ceph-mon[31539]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T10:41:44.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:44.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:44 smithi100 ceph-mon[36051]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-02-26T10:41:44.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:44 smithi100 ceph-mon[36051]: osd.11 [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] boot 2024-02-26T10:41:44.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:44 smithi100 ceph-mon[36051]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T10:41:44.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T10:41:45.305 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1708944103,"num_in_osds":12,"osd_in_since":1708944083,"num_remapped_pgs":0} 2024-02-26T10:41:45.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T10:41:45.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T10:41:45.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: Detected new or changed devices on smithi113 2024-02-26T10:41:45.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1416287776' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T10:41:45.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: Adjusting osd_memory_target on smithi113 to 4270M 2024-02-26T10:41:45.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:45.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:45.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T10:41:45.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: Detected new or changed devices on smithi113 2024-02-26T10:41:45.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1416287776' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T10:41:45.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: Adjusting osd_memory_target on smithi113 to 4270M 2024-02-26T10:41:45.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:45.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:45.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.751 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:41:45.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T10:41:45.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: Detected new or changed devices on smithi113 2024-02-26T10:41:45.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1416287776' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T10:41:45.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T10:41:45.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: Adjusting osd_memory_target on smithi113 to 4270M 2024-02-26T10:41:45.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:45.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:45.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:45.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:46.723 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:46 smithi060 ceph-mon[31539]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:46.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:46 smithi113 ceph-mon[36126]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:46.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:46 smithi100 ceph-mon[36051]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.959+0000 7f818c800700 1 -- 172.21.15.60:0/3962259821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184069a00 msgr2=0x7f8184069e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.959+0000 7f818c800700 1 --2- 172.21.15.60:0/3962259821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184069a00 0x7f8184069e60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f81740099c0 tx=0x7f8174009cd0 comp rx=0 tx=0).stop 2024-02-26T10:41:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.959+0000 7f818c800700 1 -- 172.21.15.60:0/3962259821 shutdown_connections 2024-02-26T10:41:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.959+0000 7f818c800700 1 --2- 172.21.15.60:0/3962259821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818406a3a0 0x7f818410ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.959+0000 7f818c800700 1 --2- 172.21.15.60:0/3962259821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8184107da0 0x7f8184108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.959+0000 7f818c800700 1 --2- 172.21.15.60:0/3962259821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184069a00 0x7f8184069e60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.959+0000 7f818c800700 1 -- 172.21.15.60:0/3962259821 >> 172.21.15.60:0/3962259821 conn(0x7f8184077b80 msgr2=0x7f8184077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.959+0000 7f818c800700 1 -- 172.21.15.60:0/3962259821 shutdown_connections 2024-02-26T10:41:47.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.959+0000 7f818c800700 1 -- 172.21.15.60:0/3962259821 wait complete. 2024-02-26T10:41:47.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.960+0000 7f818c800700 1 Processor -- start 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.960+0000 7f818c800700 1 -- start start 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818c800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8184069a00 0x7f81841a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818c800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818406a3a0 0x7f81841a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818c800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184107da0 0x7f818419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818c800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184112000 con 0x7f8184107da0 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818c800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184111e80 con 0x7f818406a3a0 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818c800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184112180 con 0x7f8184069a00 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818a800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184107da0 0x7f818419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818a800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184107da0 0x7f818419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55570/0 (socket says 172.21.15.60:55570) 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818a800700 1 -- 172.21.15.60:0/2268790925 learned_addr learned my addr 172.21.15.60:0/2268790925 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f8189e00700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8184069a00 0x7f81841a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f8189400700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818406a3a0 0x7f81841a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818a800700 1 -- 172.21.15.60:0/2268790925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8184069a00 msgr2=0x7f81841a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818a800700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8184069a00 0x7f81841a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818a800700 1 -- 172.21.15.60:0/2268790925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818406a3a0 msgr2=0x7f81841a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.961+0000 7f818a800700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818406a3a0 0x7f81841a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.962+0000 7f818a800700 1 -- 172.21.15.60:0/2268790925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818419ada0 con 0x7f8184107da0 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.962+0000 7f8189e00700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8184069a00 0x7f81841a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.962+0000 7f818a800700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184107da0 0x7f818419a640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8180007ee0 tx=0x7f818000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.962+0000 7f817ea00700 1 -- 172.21.15.60:0/2268790925 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180004bb0 con 0x7f8184107da0 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.962+0000 7f817ea00700 1 -- 172.21.15.60:0/2268790925 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8180004d50 con 0x7f8184107da0 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.962+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818419b090 con 0x7f8184107da0 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.962+0000 7f817ea00700 1 -- 172.21.15.60:0/2268790925 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180005960 con 0x7f8184107da0 2024-02-26T10:41:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.962+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81841a7300 con 0x7f8184107da0 2024-02-26T10:41:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.963+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8184068a10 con 0x7f8184107da0 2024-02-26T10:41:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.965+0000 7f817ea00700 1 -- 172.21.15.60:0/2268790925 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f818001a020 con 0x7f8184107da0 2024-02-26T10:41:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.967+0000 7f817ea00700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f816c0a1e90 0x7f816c0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.967+0000 7f817ea00700 1 -- 172.21.15.60:0/2268790925 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81800ccff0 con 0x7f8184107da0 2024-02-26T10:41:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.967+0000 7f8189e00700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f816c0a1e90 0x7f816c0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.968+0000 7f8189e00700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f816c0a1e90 0x7f816c0a4350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8178005950 tx=0x7f817800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:47.972+0000 7f817ea00700 1 -- 172.21.15.60:0/2268790925 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8180010070 con 0x7f8184107da0 2024-02-26T10:41:48.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.196+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81841a7530 con 0x7f8184107da0 2024-02-26T10:41:48.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.198+0000 7f817ea00700 1 -- 172.21.15.60:0/2268790925 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18465 (secure 0 0 0) 0x7f818009a860 con 0x7f8184107da0 2024-02-26T10:41:48.197 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:41:48.197 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:41:44.354883+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f816c0a1e90 msgr2=0x7f816c0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f816c0a1e90 0x7f816c0a4350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8178005950 tx=0x7f817800b410 comp rx=0 tx=0).stop 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184107da0 msgr2=0x7f818419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184107da0 0x7f818419a640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8180007ee0 tx=0x7f818000d3b0 comp rx=0 tx=0).stop 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 shutdown_connections 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f816c0a1e90 0x7f816c0a4350 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8184107da0 0x7f818419a640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8184069a00 0x7f81841a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 --2- 172.21.15.60:0/2268790925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f818406a3a0 0x7f81841a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 >> 172.21.15.60:0/2268790925 conn(0x7f8184077b80 msgr2=0x7f81840ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:48.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.201+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 shutdown_connections 2024-02-26T10:41:48.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:48.202+0000 7f818c800700 1 -- 172.21.15.60:0/2268790925 wait complete. 2024-02-26T10:41:48.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:48 smithi113 ceph-mon[36126]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:48.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:48 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2268790925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:41:48.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:48 smithi060 ceph-mon[31539]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:48.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:48 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2268790925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:41:48.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:48 smithi100 ceph-mon[36051]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:48.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:48 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2268790925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:41:49.100 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:41:49.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-26T10:41:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:41:50.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:50 smithi060 ceph-mon[31539]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:50.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:50 smithi113 ceph-mon[36126]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:50.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:50 smithi100 ceph-mon[36051]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:51.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.919+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3876643593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc109160 msgr2=0x7ff9dc110d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:51.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.919+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3876643593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc109160 0x7ff9dc110d30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400b0d0 tx=0x7ff9d400b3e0 comp rx=0 tx=0).stop 2024-02-26T10:41:51.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.919+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3876643593 shutdown_connections 2024-02-26T10:41:51.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.919+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3876643593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc109160 0x7ff9dc110d30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:51.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.919+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3876643593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc104080 0x7ff9dc104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:51.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.919+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3876643593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9dc104a30 0x7ff9dc108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:51.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.919+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3876643593 >> 172.21.15.60:0/3876643593 conn(0x7ff9dc0ff8f0 msgr2=0x7ff9dc101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:51.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.919+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3876643593 shutdown_connections 2024-02-26T10:41:51.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.919+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3876643593 wait complete. 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.920+0000 7ff9e0c00700 1 Processor -- start 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.920+0000 7ff9e0c00700 1 -- start start 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9e0c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc104080 0x7ff9dc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9e0c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc104a30 0x7ff9dc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9e0c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9dc109160 0x7ff9dc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9e0c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc1141c0 con 0x7ff9dc104080 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9e0c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc114040 con 0x7ff9dc104a30 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9e0c00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc114340 con 0x7ff9dc109160 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9db400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc104080 0x7ff9dc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9db400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc104080 0x7ff9dc19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55590/0 (socket says 172.21.15.60:55590) 2024-02-26T10:41:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9db400700 1 -- 172.21.15.60:0/3843988060 learned_addr learned my addr 172.21.15.60:0/3843988060 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:41:51.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9dbe00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9dc109160 0x7ff9dc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:51.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9daa00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc104a30 0x7ff9dc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:51.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9dbe00700 1 -- 172.21.15.60:0/3843988060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc104a30 msgr2=0x7ff9dc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:51.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.921+0000 7ff9dbe00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc104a30 0x7ff9dc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:51.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.922+0000 7ff9dbe00700 1 -- 172.21.15.60:0/3843988060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc104080 msgr2=0x7ff9dc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:51.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.922+0000 7ff9dbe00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc104080 0x7ff9dc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:51.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.922+0000 7ff9dbe00700 1 -- 172.21.15.60:0/3843988060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc19f5d0 con 0x7ff9dc109160 2024-02-26T10:41:51.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.922+0000 7ff9dbe00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9dc109160 0x7ff9dc19ee90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400d010 tx=0x7ff9d4009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:51.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.922+0000 7ff9d3e00700 1 -- 172.21.15.60:0/3843988060 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4003a00 con 0x7ff9dc109160 2024-02-26T10:41:51.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.922+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc19f860 con 0x7ff9dc109160 2024-02-26T10:41:51.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.922+0000 7ff9d3e00700 1 -- 172.21.15.60:0/3843988060 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4022070 con 0x7ff9dc109160 2024-02-26T10:41:51.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.922+0000 7ff9d3e00700 1 -- 172.21.15.60:0/3843988060 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d401b6c0 con 0x7ff9dc109160 2024-02-26T10:41:51.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.922+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc1a7140 con 0x7ff9dc109160 2024-02-26T10:41:51.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.923+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9dc068a10 con 0x7ff9dc109160 2024-02-26T10:41:51.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.925+0000 7ff9d3e00700 1 -- 172.21.15.60:0/3843988060 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff9d4003ba0 con 0x7ff9dc109160 2024-02-26T10:41:51.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.927+0000 7ff9d3e00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff9c00a1e10 0x7ff9c00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:51.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.927+0000 7ff9d3e00700 1 -- 172.21.15.60:0/3843988060 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff9d400f020 con 0x7ff9dc109160 2024-02-26T10:41:51.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.927+0000 7ff9db400700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff9c00a1e10 0x7ff9c00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:51.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.928+0000 7ff9db400700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff9c00a1e10 0x7ff9c00a42d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009d20 tx=0x7ff9cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:51.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:51.929+0000 7ff9d3e00700 1 -- 172.21.15.60:0/3843988060 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9d4011390 con 0x7ff9dc109160 2024-02-26T10:41:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.155+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9dc19b920 con 0x7ff9dc109160 2024-02-26T10:41:52.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.156+0000 7ff9d3e00700 1 -- 172.21.15.60:0/3843988060 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9d409bf10 con 0x7ff9dc109160 2024-02-26T10:41:52.155 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:41:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.159+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff9c00a1e10 msgr2=0x7ff9c00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.159+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff9c00a1e10 0x7ff9c00a42d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009d20 tx=0x7ff9cc009260 comp rx=0 tx=0).stop 2024-02-26T10:41:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.159+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9dc109160 msgr2=0x7ff9dc19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.159+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9dc109160 0x7ff9dc19ee90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400d010 tx=0x7ff9d4009810 comp rx=0 tx=0).stop 2024-02-26T10:41:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.160+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 shutdown_connections 2024-02-26T10:41:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.160+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff9c00a1e10 0x7ff9c00a42d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.160+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9dc109160 0x7ff9dc19ee90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.160+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc104080 0x7ff9dc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.160+0000 7ff9e0c00700 1 --2- 172.21.15.60:0/3843988060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9dc104a30 0x7ff9dc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.160+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 >> 172.21.15.60:0/3843988060 conn(0x7ff9dc0ff8f0 msgr2=0x7ff9dc100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:52.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.160+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 shutdown_connections 2024-02-26T10:41:52.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:52.160+0000 7ff9e0c00700 1 -- 172.21.15.60:0/3843988060 wait complete. 2024-02-26T10:41:52.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:52 smithi113 ceph-mon[36126]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:52.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:52 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3843988060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:41:52.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:52 smithi060 ceph-mon[31539]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:52.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:52 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3843988060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:41:52.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:52 smithi100 ceph-mon[36051]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:52.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:52 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3843988060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.a on smithi060 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.d on smithi060 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.g on smithi060 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.j on smithi060 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.b on smithi100 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.e on smithi100 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.h on smithi100 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.k on smithi100 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.c on smithi113 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.f on smithi113 2024-02-26T10:41:52.958 INFO:tasks.cephadm:Adding mds.i on smithi113 2024-02-26T10:41:52.959 INFO:tasks.cephadm:Adding mds.l on smithi113 2024-02-26T10:41:52.959 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph orch apply mds all '12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l' 2024-02-26T10:41:53.209 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:41:54.680 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:54 smithi113 ceph-mon[36126]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:54.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:54 smithi060 ceph-mon[31539]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:54.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:54 smithi100 ceph-mon[36051]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:54.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 -- 172.21.15.113:0/3883211649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24103920 msgr2=0x7fed2410a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 --2- 172.21.15.113:0/3883211649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24103920 0x7fed2410a1d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00b0a0 tx=0x7fed1c00b3b0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 -- 172.21.15.113:0/3883211649 shutdown_connections 2024-02-26T10:41:54.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 --2- 172.21.15.113:0/3883211649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24103920 0x7fed2410a1d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 --2- 172.21.15.113:0/3883211649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24101d80 0x7fed241021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 --2- 172.21.15.113:0/3883211649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24102f80 0x7fed241033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 -- 172.21.15.113:0/3883211649 >> 172.21.15.113:0/3883211649 conn(0x7fed240fd360 msgr2=0x7fed240ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:54.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 -- 172.21.15.113:0/3883211649 shutdown_connections 2024-02-26T10:41:54.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 -- 172.21.15.113:0/3883211649 wait complete. 2024-02-26T10:41:54.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 Processor -- start 2024-02-26T10:41:54.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 -- start start 2024-02-26T10:41:54.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24101d80 0x7fed24197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24102f80 0x7fed241984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.896+0000 7fed2ac74700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24103920 0x7fed2419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed2ac74700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2410d9e0 con 0x7fed24103920 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed2ac74700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2410d860 con 0x7fed24102f80 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed2ac74700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2410db60 con 0x7fed24101d80 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed28a10700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24101d80 0x7fed24197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed28a10700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24101d80 0x7fed24197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35436/0 (socket says 172.21.15.113:35436) 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed28a10700 1 -- 172.21.15.113:0/368787353 learned_addr learned my addr 172.21.15.113:0/368787353 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed23fff700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24102f80 0x7fed241984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed28a10700 1 -- 172.21.15.113:0/368787353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24102f80 msgr2=0x7fed241984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed28a10700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24102f80 0x7fed241984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed28a10700 1 -- 172.21.15.113:0/368787353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24103920 msgr2=0x7fed2419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed28a10700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24103920 0x7fed2419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed28a10700 1 -- 172.21.15.113:0/368787353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2419fc00 con 0x7fed24101d80 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed28a10700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24101d80 0x7fed24197fa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fed1800d9b0 tx=0x7fed1800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:54.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed21ffb700 1 -- 172.21.15.113:0/368787353 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18017070 con 0x7fed24101d80 2024-02-26T10:41:54.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed21ffb700 1 -- 172.21.15.113:0/368787353 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed18009e10 con 0x7fed24101d80 2024-02-26T10:41:54.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2419fef0 con 0x7fed24101d80 2024-02-26T10:41:54.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed21ffb700 1 -- 172.21.15.113:0/368787353 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800f620 con 0x7fed24101d80 2024-02-26T10:41:54.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.897+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed241a0360 con 0x7fed24101d80 2024-02-26T10:41:54.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.898+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed24068a10 con 0x7fed24101d80 2024-02-26T10:41:54.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.899+0000 7fed21ffb700 1 -- 172.21.15.113:0/368787353 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fed18009930 con 0x7fed24101d80 2024-02-26T10:41:54.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.900+0000 7fed21ffb700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fed140a1e10 0x7fed140a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.900+0000 7fed21ffb700 1 -- 172.21.15.113:0/368787353 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fed180cd4e0 con 0x7fed24101d80 2024-02-26T10:41:54.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.901+0000 7fed23fff700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fed140a1e10 0x7fed140a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.901+0000 7fed23fff700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fed140a1e10 0x7fed140a42d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fed10009a10 tx=0x7fed10009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:54.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:54.903+0000 7fed21ffb700 1 -- 172.21.15.113:0/368787353 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed1809ad10 con 0x7fed24101d80 2024-02-26T10:41:55.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.107+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fed24062450 con 0x7fed140a1e10 2024-02-26T10:41:55.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.124+0000 7fed21ffb700 1 -- 172.21.15.113:0/368787353 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fed24062450 con 0x7fed140a1e10 2024-02-26T10:41:55.123 INFO:teuthology.orchestra.run.smithi113.stdout:Scheduled mds.all update... 2024-02-26T10:41:55.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fed140a1e10 msgr2=0x7fed140a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fed140a1e10 0x7fed140a42d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fed10009a10 tx=0x7fed10009190 comp rx=0 tx=0).stop 2024-02-26T10:41:55.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24101d80 msgr2=0x7fed24197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24101d80 0x7fed24197fa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fed1800d9b0 tx=0x7fed1800dcc0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 shutdown_connections 2024-02-26T10:41:55.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fed140a1e10 0x7fed140a42d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24103920 0x7fed2419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed24101d80 0x7fed24197fa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 --2- 172.21.15.113:0/368787353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24102f80 0x7fed241984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 >> 172.21.15.113:0/368787353 conn(0x7fed240fd360 msgr2=0x7fed24106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:55.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 shutdown_connections 2024-02-26T10:41:55.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-02-26T10:41:55.127+0000 7fed2ac74700 1 -- 172.21.15.113:0/368787353 wait complete. 2024-02-26T10:41:55.822 DEBUG:teuthology.orchestra.run.smithi060:mds.a> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.a.service 2024-02-26T10:41:55.828 DEBUG:teuthology.orchestra.run.smithi060:mds.d> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.d.service 2024-02-26T10:41:55.874 DEBUG:teuthology.orchestra.run.smithi060:mds.g> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.g.service 2024-02-26T10:41:55.918 DEBUG:teuthology.orchestra.run.smithi060:mds.j> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.j.service 2024-02-26T10:41:55.965 DEBUG:teuthology.orchestra.run.smithi100:mds.b> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.b.service 2024-02-26T10:41:55.967 DEBUG:teuthology.orchestra.run.smithi100:mds.e> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.e.service 2024-02-26T10:41:56.010 DEBUG:teuthology.orchestra.run.smithi100:mds.h> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.h.service 2024-02-26T10:41:56.053 DEBUG:teuthology.orchestra.run.smithi100:mds.k> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.k.service 2024-02-26T10:41:56.096 DEBUG:teuthology.orchestra.run.smithi113:mds.c> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.c.service 2024-02-26T10:41:56.099 DEBUG:teuthology.orchestra.run.smithi113:mds.f> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.f.service 2024-02-26T10:41:56.143 DEBUG:teuthology.orchestra.run.smithi113:mds.i> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.i.service 2024-02-26T10:41:56.187 DEBUG:teuthology.orchestra.run.smithi113:mds.l> sudo journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.l.service 2024-02-26T10:41:56.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:56 smithi060 ceph-mon[31539]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:56.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:56 smithi060 ceph-mon[31539]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:41:56.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:56 smithi060 ceph-mon[31539]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l;count:12 2024-02-26T10:41:56.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:56.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:56 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:56.230 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-26T10:41:56.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:56.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:56 smithi100 ceph-mon[36051]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:56.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:56 smithi100 ceph-mon[36051]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:41:56.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:56 smithi100 ceph-mon[36051]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l;count:12 2024-02-26T10:41:56.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:56.380 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:56 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.465+0000 7f5720c00640 1 -- 172.21.15.60:0/2913502063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571c101e40 msgr2=0x7f571c104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.465+0000 7f5720c00640 1 --2- 172.21.15.60:0/2913502063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571c101e40 0x7f571c104230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f570c009900 tx=0x7f570c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:41:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.465+0000 7f5720c00640 1 -- 172.21.15.60:0/2913502063 shutdown_connections 2024-02-26T10:41:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.465+0000 7f5720c00640 1 --2- 172.21.15.60:0/2913502063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571c1070a0 0x7f571c109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.465+0000 7f5720c00640 1 --2- 172.21.15.60:0/2913502063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c104770 0x7f571c106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.465+0000 7f5720c00640 1 --2- 172.21.15.60:0/2913502063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571c101e40 0x7f571c104230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.465+0000 7f5720c00640 1 -- 172.21.15.60:0/2913502063 >> 172.21.15.60:0/2913502063 conn(0x7f571c0fb790 msgr2=0x7f571c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.465+0000 7f5720c00640 1 -- 172.21.15.60:0/2913502063 shutdown_connections 2024-02-26T10:41:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.465+0000 7f5720c00640 1 -- 172.21.15.60:0/2913502063 wait complete. 2024-02-26T10:41:56.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.466+0000 7f5720c00640 1 Processor -- start 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5720c00640 1 -- start start 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5720c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571c101e40 0x7f571c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5720c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571c104770 0x7f571c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5720c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c1070a0 0x7f571c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5720c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c10cf80 con 0x7f571c104770 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5720c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c10ce00 con 0x7f571c101e40 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5720c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c10d100 con 0x7f571c1070a0 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5719400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571c104770 0x7f571c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5719e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571c101e40 0x7f571c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f571a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c1070a0 0x7f571c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5719400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571c104770 0x7f571c199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51388/0 (socket says 172.21.15.60:51388) 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.467+0000 7f5719400640 1 -- 172.21.15.60:0/1540498370 learned_addr learned my addr 172.21.15.60:0/1540498370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.468+0000 7f5719e00640 1 -- 172.21.15.60:0/1540498370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c1070a0 msgr2=0x7f571c1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.468+0000 7f5719e00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c1070a0 0x7f571c1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.468+0000 7f5719e00640 1 -- 172.21.15.60:0/1540498370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571c104770 msgr2=0x7f571c199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.468+0000 7f5719e00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571c104770 0x7f571c199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.468+0000 7f5719e00640 1 -- 172.21.15.60:0/1540498370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c1a1370 con 0x7f571c101e40 2024-02-26T10:41:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.468+0000 7f571a800640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c1070a0 0x7f571c1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:41:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.468+0000 7f5719e00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571c101e40 0x7f571c199750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f570c036040 tx=0x7f570c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.468+0000 7f5712a00640 1 -- 172.21.15.60:0/1540498370 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c043020 con 0x7f571c101e40 2024-02-26T10:41:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.468+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c1a1600 con 0x7f571c101e40 2024-02-26T10:41:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.469+0000 7f5712a00640 1 -- 172.21.15.60:0/1540498370 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f570c0054e0 con 0x7f571c101e40 2024-02-26T10:41:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.469+0000 7f5712a00640 1 -- 172.21.15.60:0/1540498370 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c003c30 con 0x7f571c101e40 2024-02-26T10:41:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.469+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c1a1b70 con 0x7f571c101e40 2024-02-26T10:41:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.470+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c108b90 con 0x7f571c101e40 2024-02-26T10:41:56.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.473+0000 7f5712a00640 1 -- 172.21.15.60:0/1540498370 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f570c046020 con 0x7f571c101e40 2024-02-26T10:41:56.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.474+0000 7f5712a00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57000a1d80 0x7f57000a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.474+0000 7f5712a00640 1 -- 172.21.15.60:0/1540498370 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f570c0ef910 con 0x7f571c101e40 2024-02-26T10:41:56.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.474+0000 7f5719400640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57000a1d80 0x7f57000a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.475+0000 7f5719400640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57000a1d80 0x7f57000a4240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f571c19ac60 tx=0x7f5708005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.478+0000 7f5712a00640 1 -- 172.21.15.60:0/1540498370 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f570c04d020 con 0x7f571c101e40 2024-02-26T10:41:56.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:56 smithi113 ceph-mon[36126]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:56.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:56 smithi113 ceph-mon[36126]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:41:56.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:56 smithi113 ceph-mon[36126]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l;count:12 2024-02-26T10:41:56.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:56.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:56 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:41:56.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.684+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f571c061e60 con 0x7f571c101e40 2024-02-26T10:41:56.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.686+0000 7f5712a00640 1 -- 172.21.15.60:0/1540498370 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18465 (secure 0 0 0) 0x7f570c0bd1c0 con 0x7f571c101e40 2024-02-26T10:41:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:41:56.685 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:41:44.354883+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:56.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57000a1d80 msgr2=0x7f57000a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57000a1d80 0x7f57000a4240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f571c19ac60 tx=0x7f5708005f90 comp rx=0 tx=0).stop 2024-02-26T10:41:56.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571c101e40 msgr2=0x7f571c199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571c101e40 0x7f571c199750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f570c036040 tx=0x7f570c02c650 comp rx=0 tx=0).stop 2024-02-26T10:41:56.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 shutdown_connections 2024-02-26T10:41:56.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57000a1d80 0x7f57000a4240 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f571c1070a0 0x7f571c1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571c104770 0x7f571c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 --2- 172.21.15.60:0/1540498370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f571c101e40 0x7f571c199750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 >> 172.21.15.60:0/1540498370 conn(0x7f571c0fb790 msgr2=0x7f571c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:56.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 shutdown_connections 2024-02-26T10:41:56.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.689+0000 7f5720c00640 1 -- 172.21.15.60:0/1540498370 wait complete. 2024-02-26T10:41:56.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:41:56.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:41:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.918+0000 7f180f200640 1 -- 172.21.15.60:0/2600740918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101bf0 msgr2=0x7f1810108480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.918+0000 7f180f200640 1 --2- 172.21.15.60:0/2600740918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101bf0 0x7f1810108480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1808009ca0 tx=0x7f180802c0f0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.918+0000 7f180f200640 1 -- 172.21.15.60:0/2600740918 shutdown_connections 2024-02-26T10:41:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.918+0000 7f180f200640 1 --2- 172.21.15.60:0/2600740918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1810101230 0x7f18101016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.918+0000 7f180f200640 1 --2- 172.21.15.60:0/2600740918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101bf0 0x7f1810108480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.918+0000 7f180f200640 1 --2- 172.21.15.60:0/2600740918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1810100040 0x7f1810100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.918+0000 7f180f200640 1 -- 172.21.15.60:0/2600740918 >> 172.21.15.60:0/2600740918 conn(0x7f18100fb5d0 msgr2=0x7f18100fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.918+0000 7f180f200640 1 -- 172.21.15.60:0/2600740918 shutdown_connections 2024-02-26T10:41:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.918+0000 7f180f200640 1 -- 172.21.15.60:0/2600740918 wait complete. 2024-02-26T10:41:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.919+0000 7f180f200640 1 Processor -- start 2024-02-26T10:41:56.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180f200640 1 -- start start 2024-02-26T10:41:56.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1810100040 0x7f18101973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101230 0x7f1810197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1810101bf0 0x7f181019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181010bc90 con 0x7f1810101bf0 2024-02-26T10:41:56.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181010bb10 con 0x7f1810100040 2024-02-26T10:41:56.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181010be10 con 0x7f1810101230 2024-02-26T10:41:56.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101230 0x7f1810197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101230 0x7f1810197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:33392/0 (socket says 172.21.15.60:33392) 2024-02-26T10:41:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180d400640 1 -- 172.21.15.60:0/577190364 learned_addr learned my addr 172.21.15.60:0/577190364 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:41:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180de00640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1810100040 0x7f18101973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.920+0000 7f180e800640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1810101bf0 0x7f181019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f180d400640 1 -- 172.21.15.60:0/577190364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1810100040 msgr2=0x7f18101973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f180d400640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1810100040 0x7f18101973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f180d400640 1 -- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1810101bf0 msgr2=0x7f181019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f180d400640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1810101bf0 0x7f181019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f180d400640 1 -- 172.21.15.60:0/577190364 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181019efe0 con 0x7f1810101230 2024-02-26T10:41:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f180d400640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101230 0x7f1810197900 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00e9f0 tx=0x7f17fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f1806a00640 1 -- 172.21.15.60:0/577190364 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00cd80 con 0x7f1810101230 2024-02-26T10:41:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181019f2d0 con 0x7f1810101230 2024-02-26T10:41:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f1806a00640 1 -- 172.21.15.60:0/577190364 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17fc010910 con 0x7f1810101230 2024-02-26T10:41:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f1806a00640 1 -- 172.21.15.60:0/577190364 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0188f0 con 0x7f1810101230 2024-02-26T10:41:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.921+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181019f870 con 0x7f1810101230 2024-02-26T10:41:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.923+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181019fad0 con 0x7f1810101230 2024-02-26T10:41:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.924+0000 7f1806a00640 1 -- 172.21.15.60:0/577190364 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f17fc010ac0 con 0x7f1810101230 2024-02-26T10:41:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.926+0000 7f1806a00640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17f40a1d80 0x7f17f40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.926+0000 7f1806a00640 1 -- 172.21.15.60:0/577190364 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f17fc0cde00 con 0x7f1810101230 2024-02-26T10:41:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.926+0000 7f180de00640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17f40a1d80 0x7f17f40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.927+0000 7f180de00640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17f40a1d80 0x7f17f40a4240 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1810101090 tx=0x7f1800005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:56.933+0000 7f1806a00640 1 -- 172.21.15.60:0/577190364 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17fc09b630 con 0x7f1810101230 2024-02-26T10:41:57.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.136+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f181019fca0 con 0x7f1810101230 2024-02-26T10:41:57.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.136+0000 7f1806a00640 1 -- 172.21.15.60:0/577190364 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17fc09af30 con 0x7f1810101230 2024-02-26T10:41:57.135 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:41:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.139+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17f40a1d80 msgr2=0x7f17f40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.139+0000 7f180f200640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17f40a1d80 0x7f17f40a4240 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1810101090 tx=0x7f1800005f90 comp rx=0 tx=0).stop 2024-02-26T10:41:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.139+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101230 msgr2=0x7f1810197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.139+0000 7f180f200640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101230 0x7f1810197900 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00e9f0 tx=0x7f17fc00eeb0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.140+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 shutdown_connections 2024-02-26T10:41:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.140+0000 7f180f200640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17f40a1d80 0x7f17f40a4240 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.140+0000 7f180f200640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1810101bf0 0x7f181019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.140+0000 7f180f200640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1810101230 0x7f1810197900 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.140+0000 7f180f200640 1 --2- 172.21.15.60:0/577190364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1810100040 0x7f18101973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.140+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 >> 172.21.15.60:0/577190364 conn(0x7f18100fb5d0 msgr2=0x7f18100fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:57.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.140+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 shutdown_connections 2024-02-26T10:41:57.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.140+0000 7f180f200640 1 -- 172.21.15.60:0/577190364 wait complete. 2024-02-26T10:41:57.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:57.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:57.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:57.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:57.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:41:57.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:41:57.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:57 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:57.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:57 smithi060 ceph-mon[31539]: Deploying daemon mds.l on smithi113 2024-02-26T10:41:57.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:57 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1540498370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:41:57.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:57 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:57.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:57 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:57.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:57 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:57.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:57 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:41:57.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:57 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:41:57.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:57 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:57.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:57 smithi113 ceph-mon[36126]: Deploying daemon mds.l on smithi113 2024-02-26T10:41:57.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:57 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1540498370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:41:57.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.359+0000 7f0351000640 1 -- 172.21.15.60:0/1999917039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f034c101bd0 msgr2=0x7f034c103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.359+0000 7f0351000640 1 --2- 172.21.15.60:0/1999917039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f034c101bd0 0x7f034c103fc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f033c009900 tx=0x7f033c02bc40 comp rx=0 tx=0).stop 2024-02-26T10:41:57.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.359+0000 7f0351000640 1 -- 172.21.15.60:0/1999917039 shutdown_connections 2024-02-26T10:41:57.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.359+0000 7f0351000640 1 --2- 172.21.15.60:0/1999917039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c104500 0x7f034c1068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.359+0000 7f0351000640 1 --2- 172.21.15.60:0/1999917039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f034c106e30 0x7f034c109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.359+0000 7f0351000640 1 --2- 172.21.15.60:0/1999917039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f034c101bd0 0x7f034c103fc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.359+0000 7f0351000640 1 -- 172.21.15.60:0/1999917039 >> 172.21.15.60:0/1999917039 conn(0x7f034c0fb540 msgr2=0x7f034c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:57.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.359+0000 7f0351000640 1 -- 172.21.15.60:0/1999917039 shutdown_connections 2024-02-26T10:41:57.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.359+0000 7f0351000640 1 -- 172.21.15.60:0/1999917039 wait complete. 2024-02-26T10:41:57.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.360+0000 7f0351000640 1 Processor -- start 2024-02-26T10:41:57.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f0351000640 1 -- start start 2024-02-26T10:41:57.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f0351000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f034c101bd0 0x7f034c101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f0351000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f034c104500 0x7f034c0ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f0351000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c106e30 0x7f034c1000c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f0351000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c10cdb0 con 0x7f034c104500 2024-02-26T10:41:57.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f0351000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c10cc30 con 0x7f034c106e30 2024-02-26T10:41:57.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f0351000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c10cf30 con 0x7f034c101bd0 2024-02-26T10:41:57.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f034b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f034c101bd0 0x7f034c101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f034b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f034c101bd0 0x7f034c101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:33416/0 (socket says 172.21.15.60:33416) 2024-02-26T10:41:57.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f034b400640 1 -- 172.21.15.60:0/2899351746 learned_addr learned my addr 172.21.15.60:0/2899351746 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:41:57.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f0343e00640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f034c104500 0x7f034c0ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.361+0000 7f034be00640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c106e30 0x7f034c1000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f034b400640 1 -- 172.21.15.60:0/2899351746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c106e30 msgr2=0x7f034c1000c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f034b400640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c106e30 0x7f034c1000c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f034b400640 1 -- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f034c104500 msgr2=0x7f034c0ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f034b400640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f034c104500 0x7f034c0ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f034b400640 1 -- 172.21.15.60:0/2899351746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c100950 con 0x7f034c101bd0 2024-02-26T10:41:57.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f034b400640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f034c101bd0 0x7f034c101540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f033c02c8f0 tx=0x7f033c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:57.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f0348c00640 1 -- 172.21.15.60:0/2899351746 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c041070 con 0x7f034c101bd0 2024-02-26T10:41:57.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f0348c00640 1 -- 172.21.15.60:0/2899351746 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f033c0044d0 con 0x7f034c101bd0 2024-02-26T10:41:57.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c197000 con 0x7f034c101bd0 2024-02-26T10:41:57.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.362+0000 7f0348c00640 1 -- 172.21.15.60:0/2899351746 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c034440 con 0x7f034c101bd0 2024-02-26T10:41:57.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.363+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c1975a0 con 0x7f034c101bd0 2024-02-26T10:41:57.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.364+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c0fd030 con 0x7f034c101bd0 2024-02-26T10:41:57.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.365+0000 7f0348c00640 1 -- 172.21.15.60:0/2899351746 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f033c044070 con 0x7f034c101bd0 2024-02-26T10:41:57.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.367+0000 7f0348c00640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03300a1e00 0x7f03300a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.367+0000 7f0348c00640 1 -- 172.21.15.60:0/2899351746 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f033c0ef4a0 con 0x7f034c101bd0 2024-02-26T10:41:57.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.367+0000 7f0343e00640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03300a1e00 0x7f03300a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.367+0000 7f0343e00640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03300a1e00 0x7f03300a42c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f034c1012b0 tx=0x7f0334016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:57.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.371+0000 7f0348c00640 1 -- 172.21.15.60:0/2899351746 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f033c0bc680 con 0x7f034c101bd0 2024-02-26T10:41:57.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:57 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:57.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:57 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:41:57.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:57 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:41:57.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:57 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:41:57.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:57 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:41:57.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:57 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:41:57.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:57 smithi100 ceph-mon[36051]: Deploying daemon mds.l on smithi113 2024-02-26T10:41:57.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:57 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1540498370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:41:57.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.610+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f034c02ce30 con 0x7f034c101bd0 2024-02-26T10:41:57.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.611+0000 7f0348c00640 1 -- 172.21.15.60:0/2899351746 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f033c0c0720 con 0x7f034c101bd0 2024-02-26T10:41:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:41:57.609 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-26T10:41:57.609 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 1 2024-02-26T10:41:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03300a1e00 msgr2=0x7f03300a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03300a1e00 0x7f03300a42c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f034c1012b0 tx=0x7f0334016040 comp rx=0 tx=0).stop 2024-02-26T10:41:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f034c101bd0 msgr2=0x7f034c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f034c101bd0 0x7f034c101540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f033c02c8f0 tx=0x7f033c005d50 comp rx=0 tx=0).stop 2024-02-26T10:41:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 shutdown_connections 2024-02-26T10:41:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f03300a1e00 0x7f03300a42c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f034c106e30 0x7f034c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f034c104500 0x7f034c0ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 --2- 172.21.15.60:0/2899351746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f034c101bd0 0x7f034c101540 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.613+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 >> 172.21.15.60:0/2899351746 conn(0x7f034c0fb540 msgr2=0x7f034c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.614+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 shutdown_connections 2024-02-26T10:41:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:57.614+0000 7f0351000640 1 -- 172.21.15.60:0/2899351746 wait complete. 2024-02-26T10:41:58.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:58 smithi113 ceph-mon[36126]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:58.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:58 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/577190364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:41:58.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:41:58 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2899351746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:41:58.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:58 smithi060 ceph-mon[31539]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:58.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:58 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/577190364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:41:58.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:41:58 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2899351746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:41:58.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:58 smithi100 ceph-mon[36051]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:41:58.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:58 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/577190364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:41:58.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:41:58 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2899351746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:41:58.778 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:41:58 smithi113 systemd[1]: Starting Ceph mds.l for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:41:59.045 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:41:58 smithi113 podman[66026]: 2024-02-26 10:41:58.683561635 +0000 UTC m=+0.017194304 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:41:59.045 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:41:58 smithi113 podman[66026]: 2024-02-26 10:41:58.811524268 +0000 UTC m=+0.145156916 container create 2110a133da3d24a4f6c7563bd9dceccff00d82eee098862e3bbbf70a1923b055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-l, ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-02-26T10:41:59.045 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:41:59 smithi113 podman[66026]: 2024-02-26 10:41:59.047640661 +0000 UTC m=+0.381273315 container init 2110a133da3d24a4f6c7563bd9dceccff00d82eee098862e3bbbf70a1923b055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-26T10:41:59.336 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:41:59 smithi113 podman[66026]: 2024-02-26 10:41:59.050367422 +0000 UTC m=+0.384000077 container start 2110a133da3d24a4f6c7563bd9dceccff00d82eee098862e3bbbf70a1923b055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:41:59.336 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:41:59 smithi113 ceph-mds[66044]: -- 172.21.15.113:0/352460800 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563b84b85a40 con 0x563b84b59000 2024-02-26T10:41:59.336 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:41:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-l[66040]: starting mds.l at 2024-02-26T10:41:59.336 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:41:59 smithi113 bash[66026]: 2110a133da3d24a4f6c7563bd9dceccff00d82eee098862e3bbbf70a1923b055 2024-02-26T10:41:59.336 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:41:59 smithi113 systemd[1]: Started Ceph mds.l for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:41:59.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.859+0000 7f85d8000640 1 -- 172.21.15.60:0/2269835577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85d0076e60 msgr2=0x7f85d00772c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.859+0000 7f85d8000640 1 --2- 172.21.15.60:0/2269835577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85d0076e60 0x7f85d00772c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009900 tx=0x7f85c002bc30 comp rx=0 tx=0).stop 2024-02-26T10:41:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.859+0000 7f85d8000640 1 -- 172.21.15.60:0/2269835577 shutdown_connections 2024-02-26T10:41:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.859+0000 7f85d8000640 1 --2- 172.21.15.60:0/2269835577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d0077800 0x7f85d00780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.859+0000 7f85d8000640 1 --2- 172.21.15.60:0/2269835577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85d0076e60 0x7f85d00772c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.859+0000 7f85d8000640 1 --2- 172.21.15.60:0/2269835577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d0075c20 0x7f85d0076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.859+0000 7f85d8000640 1 -- 172.21.15.60:0/2269835577 >> 172.21.15.60:0/2269835577 conn(0x7f85d00fea10 msgr2=0x7f85d0100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.859+0000 7f85d8000640 1 -- 172.21.15.60:0/2269835577 shutdown_connections 2024-02-26T10:41:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.859+0000 7f85d8000640 1 -- 172.21.15.60:0/2269835577 wait complete. 2024-02-26T10:41:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.860+0000 7f85d8000640 1 Processor -- start 2024-02-26T10:41:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.860+0000 7f85d8000640 1 -- start start 2024-02-26T10:41:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.860+0000 7f85d8000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85d0075c20 0x7f85d019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.860+0000 7f85d8000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d0076e60 0x7f85d019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.860+0000 7f85d8000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d0077800 0x7f85d01a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.860+0000 7f85d8000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d01134f0 con 0x7f85d0076e60 2024-02-26T10:41:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.860+0000 7f85d8000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0113370 con 0x7f85d0077800 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.860+0000 7f85d8000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0113670 con 0x7f85d0075c20 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d0076e60 0x7f85d019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85d0075c20 0x7f85d019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d0077800 0x7f85d01a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d0076e60 0x7f85d019e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51436/0 (socket says 172.21.15.60:51436) 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d4c00640 1 -- 172.21.15.60:0/380481733 learned_addr learned my addr 172.21.15.60:0/380481733 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d0077800 0x7f85d01a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53240/0 (socket says 172.21.15.60:53240) 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85d0075c20 0x7f85d019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:33436/0 (socket says 172.21.15.60:33436) 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d6000640 1 -- 172.21.15.60:0/380481733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85d0075c20 msgr2=0x7f85d019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d6000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85d0075c20 0x7f85d019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d6000640 1 -- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d0076e60 msgr2=0x7f85d019e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d6000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d0076e60 0x7f85d019e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d6000640 1 -- 172.21.15.60:0/380481733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d01a57b0 con 0x7f85d0077800 2024-02-26T10:41:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d4c00640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d0076e60 0x7f85d019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:41:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85d6000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d0077800 0x7f85d01a5140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009fd0 tx=0x7f85c800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.861+0000 7f85ce000640 1 -- 172.21.15.60:0/380481733 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c800ce20 con 0x7f85d0077800 2024-02-26T10:41:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.862+0000 7f85ce000640 1 -- 172.21.15.60:0/380481733 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85c8005020 con 0x7f85d0077800 2024-02-26T10:41:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.862+0000 7f85ce000640 1 -- 172.21.15.60:0/380481733 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8010720 con 0x7f85d0077800 2024-02-26T10:41:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.862+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d01a5aa0 con 0x7f85d0077800 2024-02-26T10:41:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.862+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d01a5fc0 con 0x7f85d0077800 2024-02-26T10:41:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.863+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d010fff0 con 0x7f85d0077800 2024-02-26T10:41:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.865+0000 7f85ce000640 1 -- 172.21.15.60:0/380481733 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f85c8005540 con 0x7f85d0077800 2024-02-26T10:41:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.866+0000 7f85ce000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f85b80a1e00 0x7f85b80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.866+0000 7f85ce000640 1 -- 172.21.15.60:0/380481733 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f85c80cdf40 con 0x7f85d0077800 2024-02-26T10:41:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.866+0000 7f85d5600640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f85b80a1e00 0x7f85b80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.867+0000 7f85d5600640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f85b80a1e00 0x7f85b80a42c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f85c4005d00 tx=0x7f85c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:59.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:41:59.870+0000 7f85ce000640 1 -- 172.21.15.60:0/380481733 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85c809b770 con 0x7f85d0077800 2024-02-26T10:42:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.114+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85d0061e60 con 0x7f85d0077800 2024-02-26T10:42:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.115+0000 7f85ce000640 1 -- 172.21.15.60:0/380481733 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f85c809b070 con 0x7f85d0077800 2024-02-26T10:42:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:00.113 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-26T10:42:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 2 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f85b80a1e00 msgr2=0x7f85b80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f85b80a1e00 0x7f85b80a42c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f85c4005d00 tx=0x7f85c400b410 comp rx=0 tx=0).stop 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d0077800 msgr2=0x7f85d01a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d0077800 0x7f85d01a5140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009fd0 tx=0x7f85c800ef10 comp rx=0 tx=0).stop 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 shutdown_connections 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f85b80a1e00 0x7f85b80a42c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d0077800 0x7f85d01a5140 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d0076e60 0x7f85d019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 --2- 172.21.15.60:0/380481733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85d0075c20 0x7f85d019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.117+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 >> 172.21.15.60:0/380481733 conn(0x7f85d00fea10 msgr2=0x7f85d0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.118+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 shutdown_connections 2024-02-26T10:42:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:00.118+0000 7f85d8000640 1 -- 172.21.15.60:0/380481733 wait complete. 2024-02-26T10:42:00.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:00.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.113:6832/1265971356,v1:172.21.15.113:6833/1265971356] up:boot 2024-02-26T10:42:00.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: fsmap 1 up:standby 2024-02-26T10:42:00.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T10:42:00.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T10:42:00.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:00.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:00.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:00.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:00.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:00.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:00.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:00 smithi100 ceph-mon[36051]: Deploying daemon mds.k on smithi100 2024-02-26T10:42:00.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:00.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.113:6832/1265971356,v1:172.21.15.113:6833/1265971356] up:boot 2024-02-26T10:42:00.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: fsmap 1 up:standby 2024-02-26T10:42:00.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T10:42:00.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T10:42:00.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:00.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:00.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:00.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:00.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:00.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:00.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:00 smithi060 ceph-mon[31539]: Deploying daemon mds.k on smithi100 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.113:6832/1265971356,v1:172.21.15.113:6833/1265971356] up:boot 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: fsmap 1 up:standby 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:00.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:00.587 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:00 smithi113 ceph-mon[36126]: Deploying daemon mds.k on smithi100 2024-02-26T10:42:01.337 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:01 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/380481733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:01.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:01 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/380481733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:01.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:01 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/380481733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:01.629 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:42:01 smithi100 systemd[1]: Starting Ceph mds.k for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:42:01.992 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:42:01 smithi100 podman[65679]: 2024-02-26 10:42:01.626805492 +0000 UTC m=+0.017739327 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:42:01.992 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:42:01 smithi100 podman[65679]: 2024-02-26 10:42:01.741095862 +0000 UTC m=+0.132029696 container create 25c2b7bad73b39e9d72ca0a28682ed8f53a68124acdaedbdc719ccbffdc3aa81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222) 2024-02-26T10:42:01.992 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:42:01 smithi100 podman[65679]: 2024-02-26 10:42:01.968928876 +0000 UTC m=+0.359862721 container init 25c2b7bad73b39e9d72ca0a28682ed8f53a68124acdaedbdc719ccbffdc3aa81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-26T10:42:01.992 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:42:01 smithi100 podman[65679]: 2024-02-26 10:42:01.971627448 +0000 UTC m=+0.362561284 container start 25c2b7bad73b39e9d72ca0a28682ed8f53a68124acdaedbdc719ccbffdc3aa81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-26T10:42:02.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:02.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:02 smithi060 ceph-mon[31539]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.377+0000 7fbbf6200640 1 -- 172.21.15.60:0/88513234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf01001a0 msgr2=0x7fbbf01005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.377+0000 7fbbf6200640 1 --2- 172.21.15.60:0/88513234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf01001a0 0x7fbbf01005c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009900 tx=0x7fbbe002bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.378+0000 7fbbf6200640 1 -- 172.21.15.60:0/88513234 shutdown_connections 2024-02-26T10:42:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.378+0000 7fbbf6200640 1 --2- 172.21.15.60:0/88513234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf0101d50 0x7fbbf01085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.378+0000 7fbbf6200640 1 --2- 172.21.15.60:0/88513234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbf0101390 0x7fbbf0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.378+0000 7fbbf6200640 1 --2- 172.21.15.60:0/88513234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf01001a0 0x7fbbf01005c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.378+0000 7fbbf6200640 1 -- 172.21.15.60:0/88513234 >> 172.21.15.60:0/88513234 conn(0x7fbbf00fb710 msgr2=0x7fbbf00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.378+0000 7fbbf6200640 1 -- 172.21.15.60:0/88513234 shutdown_connections 2024-02-26T10:42:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.378+0000 7fbbf6200640 1 -- 172.21.15.60:0/88513234 wait complete. 2024-02-26T10:42:02.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.379+0000 7fbbf6200640 1 Processor -- start 2024-02-26T10:42:02.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.379+0000 7fbbf6200640 1 -- start start 2024-02-26T10:42:02.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.379+0000 7fbbf6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf01001a0 0x7fbbf01040e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.379+0000 7fbbf6200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf0101390 0x7fbbf0102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.379+0000 7fbbf6200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbf0101d50 0x7fbbf0102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.379+0000 7fbbf6200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf010be80 con 0x7fbbf0101390 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.379+0000 7fbbf6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf010bd00 con 0x7fbbf01001a0 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.379+0000 7fbbf6200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf010c000 con 0x7fbbf0101d50 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf0101390 0x7fbbf0102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf01001a0 0x7fbbf01040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbeea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf0101390 0x7fbbf0102770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51452/0 (socket says 172.21.15.60:51452) 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbeea00640 1 -- 172.21.15.60:0/4165021021 learned_addr learned my addr 172.21.15.60:0/4165021021 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbefe00640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbf0101d50 0x7fbbf0102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbef400640 1 -- 172.21.15.60:0/4165021021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbf0101d50 msgr2=0x7fbbf0102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbef400640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbf0101d50 0x7fbbf0102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbef400640 1 -- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf0101390 msgr2=0x7fbbf0102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbef400640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf0101390 0x7fbbf0102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbef400640 1 -- 172.21.15.60:0/4165021021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf0103570 con 0x7fbbf01001a0 2024-02-26T10:42:02.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.380+0000 7fbbeea00640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf0101390 0x7fbbf0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:02.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.381+0000 7fbbef400640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf01001a0 0x7fbbf01040e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0036040 tx=0x7fbbe0004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:02.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.381+0000 7fbbe7e00640 1 -- 172.21.15.60:0/4165021021 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0043020 con 0x7fbbf01001a0 2024-02-26T10:42:02.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.381+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf01a1720 con 0x7fbbf01001a0 2024-02-26T10:42:02.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.381+0000 7fbbe7e00640 1 -- 172.21.15.60:0/4165021021 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbe00054e0 con 0x7fbbf01001a0 2024-02-26T10:42:02.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.381+0000 7fbbe7e00640 1 -- 172.21.15.60:0/4165021021 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0003be0 con 0x7fbbf01001a0 2024-02-26T10:42:02.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.381+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf01a1c90 con 0x7fbbf01001a0 2024-02-26T10:42:02.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:02 smithi100 ceph-mon[36051]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:02.382 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:42:01 smithi100 ceph-mds[65698]: -- 172.21.15.100:0/2221486072 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557d21b6fa40 con 0x557d21b42c00 2024-02-26T10:42:02.382 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:42:01 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-k[65694]: starting mds.k at 2024-02-26T10:42:02.382 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:42:02 smithi100 bash[65679]: 25c2b7bad73b39e9d72ca0a28682ed8f53a68124acdaedbdc719ccbffdc3aa81 2024-02-26T10:42:02.382 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:42:02 smithi100 systemd[1]: Started Ceph mds.k for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:02.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.382+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf00677c0 con 0x7fbbf01001a0 2024-02-26T10:42:02.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.384+0000 7fbbe7e00640 1 -- 172.21.15.60:0/4165021021 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbbe0046020 con 0x7fbbf01001a0 2024-02-26T10:42:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.385+0000 7fbbe7e00640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbbd40a1d80 0x7fbbd40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.385+0000 7fbbe7e00640 1 -- 172.21.15.60:0/4165021021 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbbe00ef8a0 con 0x7fbbf01001a0 2024-02-26T10:42:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.385+0000 7fbbeea00640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbbd40a1d80 0x7fbbd40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.386+0000 7fbbeea00640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbbd40a1d80 0x7fbbd40a4240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0103e50 tx=0x7fbbdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:02.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.389+0000 7fbbe7e00640 1 -- 172.21.15.60:0/4165021021 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbe004d020 con 0x7fbbf01001a0 2024-02-26T10:42:02.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:02 smithi113 ceph-mon[36126]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:02.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.635+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbf01a1f80 con 0x7fbbf01001a0 2024-02-26T10:42:02.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.636+0000 7fbbe7e00640 1 -- 172.21.15.60:0/4165021021 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fbbe00bd0d0 con 0x7fbbf01001a0 2024-02-26T10:42:02.634 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:02.634 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-26T10:42:02.635 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbbd40a1d80 msgr2=0x7fbbd40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbbd40a1d80 0x7fbbd40a4240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0103e50 tx=0x7fbbdc005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf01001a0 msgr2=0x7fbbf01040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf01001a0 0x7fbbf01040e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0036040 tx=0x7fbbe0004970 comp rx=0 tx=0).stop 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 shutdown_connections 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbbd40a1d80 0x7fbbd40a4240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbf0101d50 0x7fbbf0102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbf0101390 0x7fbbf0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 --2- 172.21.15.60:0/4165021021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbf01001a0 0x7fbbf01040e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 >> 172.21.15.60:0/4165021021 conn(0x7fbbf00fb710 msgr2=0x7fbbf00fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:02.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 shutdown_connections 2024-02-26T10:42:02.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:02.639+0000 7fbbf6200640 1 -- 172.21.15.60:0/4165021021 wait complete. 2024-02-26T10:42:03.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6832/1828064682,v1:172.21.15.100:6833/1828064682] up:boot 2024-02-26T10:42:03.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: fsmap 2 up:standby 2024-02-26T10:42:03.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T10:42:03.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:03.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:03.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:03.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:03.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:03.373 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:03.373 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: Deploying daemon mds.j on smithi060 2024-02-26T10:42:03.373 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:03 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/4165021021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:03.378 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6832/1828064682,v1:172.21.15.100:6833/1828064682] up:boot 2024-02-26T10:42:03.378 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: fsmap 2 up:standby 2024-02-26T10:42:03.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T10:42:03.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:03.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:03.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:03.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:03.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:03.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:03.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: Deploying daemon mds.j on smithi060 2024-02-26T10:42:03.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:03 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4165021021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6832/1828064682,v1:172.21.15.100:6833/1828064682] up:boot 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: fsmap 2 up:standby 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: Deploying daemon mds.j on smithi060 2024-02-26T10:42:03.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:03 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/4165021021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:04.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:04 smithi060 ceph-mon[31539]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:04.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:04 smithi113 ceph-mon[36126]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:04.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:04 smithi100 ceph-mon[36051]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:04.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:04.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:04.998+0000 7fdf19c00640 1 -- 172.21.15.60:0/367979769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14104e10 msgr2=0x7fdf14105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:04.998+0000 7fdf19c00640 1 --2- 172.21.15.60:0/367979769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14104e10 0x7fdf14105290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009900 tx=0x7fdf0c02bc80 comp rx=0 tx=0).stop 2024-02-26T10:42:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.000+0000 7fdf19c00640 1 -- 172.21.15.60:0/367979769 shutdown_connections 2024-02-26T10:42:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.000+0000 7fdf19c00640 1 --2- 172.21.15.60:0/367979769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14104e10 0x7fdf14105290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.000+0000 7fdf19c00640 1 --2- 172.21.15.60:0/367979769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14102460 0x7fdf141048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.000+0000 7fdf19c00640 1 --2- 172.21.15.60:0/367979769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf140ffaf0 0x7fdf14101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.000+0000 7fdf19c00640 1 -- 172.21.15.60:0/367979769 >> 172.21.15.60:0/367979769 conn(0x7fdf140fb6f0 msgr2=0x7fdf140fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.000+0000 7fdf19c00640 1 -- 172.21.15.60:0/367979769 shutdown_connections 2024-02-26T10:42:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.000+0000 7fdf19c00640 1 -- 172.21.15.60:0/367979769 wait complete. 2024-02-26T10:42:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.001+0000 7fdf19c00640 1 Processor -- start 2024-02-26T10:42:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.002+0000 7fdf19c00640 1 -- start start 2024-02-26T10:42:05.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.003+0000 7fdf19c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf140ffaf0 0x7fdf1419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.003+0000 7fdf19c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14102460 0x7fdf1419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.003+0000 7fdf19c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14104e10 0x7fdf141a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.003+0000 7fdf12000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14102460 0x7fdf1419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.003+0000 7fdf12000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14102460 0x7fdf1419dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:60274/0 (socket says 172.21.15.60:60274) 2024-02-26T10:42:05.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.004+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf14113520 con 0x7fdf14104e10 2024-02-26T10:42:05.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.004+0000 7fdf12000640 1 -- 172.21.15.60:0/2207020704 learned_addr learned my addr 172.21.15.60:0/2207020704 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:05.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.004+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf141133a0 con 0x7fdf140ffaf0 2024-02-26T10:42:05.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.004+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf141136a0 con 0x7fdf14102460 2024-02-26T10:42:05.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.004+0000 7fdf12a00640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf140ffaf0 0x7fdf1419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.004+0000 7fdf13400640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14104e10 0x7fdf141a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.005+0000 7fdf13400640 1 -- 172.21.15.60:0/2207020704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14102460 msgr2=0x7fdf1419dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.005+0000 7fdf13400640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14102460 0x7fdf1419dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.005+0000 7fdf13400640 1 -- 172.21.15.60:0/2207020704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf140ffaf0 msgr2=0x7fdf1419da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.005+0000 7fdf13400640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf140ffaf0 0x7fdf1419da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.005+0000 7fdf13400640 1 -- 172.21.15.60:0/2207020704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf141a56b0 con 0x7fdf14104e10 2024-02-26T10:42:05.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.005+0000 7fdf13400640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14104e10 0x7fdf141a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c02c1c0 tx=0x7fdf0c02eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:05.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.006+0000 7fdf0b400640 1 -- 172.21.15.60:0/2207020704 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c041020 con 0x7fdf14104e10 2024-02-26T10:42:05.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.006+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf141a5940 con 0x7fdf14104e10 2024-02-26T10:42:05.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.007+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf141a5e50 con 0x7fdf14104e10 2024-02-26T10:42:05.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.007+0000 7fdf0b400640 1 -- 172.21.15.60:0/2207020704 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c02c620 con 0x7fdf14104e10 2024-02-26T10:42:05.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.007+0000 7fdf0b400640 1 -- 172.21.15.60:0/2207020704 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c004fe0 con 0x7fdf14104e10 2024-02-26T10:42:05.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.009+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef00052c0 con 0x7fdf14104e10 2024-02-26T10:42:05.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.013+0000 7fdf0b400640 1 -- 172.21.15.60:0/2207020704 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fdf0c044020 con 0x7fdf14104e10 2024-02-26T10:42:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.014+0000 7fdf0b400640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdef80a1e00 0x7fdef80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.014+0000 7fdf0b400640 1 -- 172.21.15.60:0/2207020704 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdf0c0ef090 con 0x7fdf14104e10 2024-02-26T10:42:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.014+0000 7fdf12a00640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdef80a1e00 0x7fdef80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.015+0000 7fdf12a00640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdef80a1e00 0x7fdef80a42c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdefc005d00 tx=0x7fdefc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:05.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.019+0000 7fdf0b400640 1 -- 172.21.15.60:0/2207020704 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf0c04b020 con 0x7fdf14104e10 2024-02-26T10:42:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.259+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdef0005880 con 0x7fdf14104e10 2024-02-26T10:42:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.260+0000 7fdf0b400640 1 -- 172.21.15.60:0/2207020704 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fdf0c0bc940 con 0x7fdf14104e10 2024-02-26T10:42:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:05.258 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-26T10:42:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdef80a1e00 msgr2=0x7fdef80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdef80a1e00 0x7fdef80a42c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdefc005d00 tx=0x7fdefc00b410 comp rx=0 tx=0).stop 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14104e10 msgr2=0x7fdf141a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14104e10 0x7fdf141a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c02c1c0 tx=0x7fdf0c02eba0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 shutdown_connections 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdef80a1e00 0x7fdef80a42c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14104e10 0x7fdf141a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14102460 0x7fdf1419dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 --2- 172.21.15.60:0/2207020704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf140ffaf0 0x7fdf1419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.262+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 >> 172.21.15.60:0/2207020704 conn(0x7fdf140fb6f0 msgr2=0x7fdf14100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.263+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 shutdown_connections 2024-02-26T10:42:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:05.263+0000 7fdf19c00640 1 -- 172.21.15.60:0/2207020704 wait complete. 2024-02-26T10:42:05.869 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:42:05 smithi060 podman[71897]: 2024-02-26 10:42:05.616214446 +0000 UTC m=+0.159392635 container create d938fc1db71a026c61c412bea42b03867debf04cd9f7639a393364e9c98898f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-26T10:42:06.210 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:42:05 smithi060 podman[71897]: 2024-02-26 10:42:05.895319419 +0000 UTC m=+0.438497615 container init d938fc1db71a026c61c412bea42b03867debf04cd9f7639a393364e9c98898f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2) 2024-02-26T10:42:06.210 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:42:05 smithi060 podman[71897]: 2024-02-26 10:42:05.906224616 +0000 UTC m=+0.449402787 container start d938fc1db71a026c61c412bea42b03867debf04cd9f7639a393364e9c98898f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-26T10:42:06.210 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:42:05 smithi060 ceph-mds[71916]: -- 172.21.15.60:0/2495258795 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a072cdda40 con 0x55a072cb0800 2024-02-26T10:42:06.210 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:42:05 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j[71912]: starting mds.j at 2024-02-26T10:42:06.210 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:42:06 smithi060 bash[71897]: d938fc1db71a026c61c412bea42b03867debf04cd9f7639a393364e9c98898f8 2024-02-26T10:42:06.210 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:42:06 smithi060 systemd[1]: Started Ceph mds.j for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:06.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:06 smithi113 ceph-mon[36126]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:06.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:06 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2207020704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:06.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:06 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:06.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:06 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:06.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:06 smithi060 ceph-mon[31539]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:06.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:06 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2207020704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:06.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:06 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:06.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:06 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:06.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:06 smithi100 ceph-mon[36051]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:06.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:06 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2207020704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:06.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:06 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:06.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:06 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:07.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:07.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.502+0000 7fdf1c400640 1 -- 172.21.15.60:0/1118840769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14104690 msgr2=0x7fdf14104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:07.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.502+0000 7fdf1c400640 1 --2- 172.21.15.60:0/1118840769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14104690 0x7fdf14104b10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009900 tx=0x7fdf0002bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:07.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.503+0000 7fdf1c400640 1 -- 172.21.15.60:0/1118840769 shutdown_connections 2024-02-26T10:42:07.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.503+0000 7fdf1c400640 1 --2- 172.21.15.60:0/1118840769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf14105050 0x7fdf1410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:07.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.503+0000 7fdf1c400640 1 --2- 172.21.15.60:0/1118840769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf14104690 0x7fdf14104b10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:07.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.503+0000 7fdf1c400640 1 --2- 172.21.15.60:0/1118840769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf141034a0 0x7fdf141038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:07.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.503+0000 7fdf1c400640 1 -- 172.21.15.60:0/1118840769 >> 172.21.15.60:0/1118840769 conn(0x7fdf140fea10 msgr2=0x7fdf14100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:07.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.503+0000 7fdf1c400640 1 -- 172.21.15.60:0/1118840769 shutdown_connections 2024-02-26T10:42:07.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.503+0000 7fdf1c400640 1 -- 172.21.15.60:0/1118840769 wait complete. 2024-02-26T10:42:07.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.504+0000 7fdf1c400640 1 Processor -- start 2024-02-26T10:42:07.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.504+0000 7fdf1c400640 1 -- start start 2024-02-26T10:42:07.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.504+0000 7fdf1c400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141034a0 0x7fdf14199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:07.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.504+0000 7fdf1c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf14104690 0x7fdf14199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:07.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.504+0000 7fdf1c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14105050 0x7fdf141a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.504+0000 7fdf1c400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1410f180 con 0x7fdf141034a0 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.504+0000 7fdf1c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1410f000 con 0x7fdf14104690 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.504+0000 7fdf1c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1410f300 con 0x7fdf14105050 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf19a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141034a0 0x7fdf14199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf19a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141034a0 0x7fdf14199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46730/0 (socket says 172.21.15.60:46730) 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf19a00640 1 -- 172.21.15.60:0/454304060 learned_addr learned my addr 172.21.15.60:0/454304060 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf19000640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf14104690 0x7fdf14199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf1a400640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14105050 0x7fdf141a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf19a00640 1 -- 172.21.15.60:0/454304060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14105050 msgr2=0x7fdf141a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf19a00640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14105050 0x7fdf141a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf19a00640 1 -- 172.21.15.60:0/454304060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf14104690 msgr2=0x7fdf14199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf19a00640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf14104690 0x7fdf14199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf19a00640 1 -- 172.21.15.60:0/454304060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf141a1390 con 0x7fdf141034a0 2024-02-26T10:42:07.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.505+0000 7fdf1a400640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14105050 0x7fdf141a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:07.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.506+0000 7fdf19a00640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141034a0 0x7fdf14199770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800e9c0 tx=0x7fdf0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:07.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.506+0000 7fdf12000640 1 -- 172.21.15.60:0/454304060 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800cd80 con 0x7fdf141034a0 2024-02-26T10:42:07.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.506+0000 7fdf12000640 1 -- 172.21.15.60:0/454304060 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf080035f0 con 0x7fdf141034a0 2024-02-26T10:42:07.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.506+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf141a1680 con 0x7fdf141034a0 2024-02-26T10:42:07.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.506+0000 7fdf12000640 1 -- 172.21.15.60:0/454304060 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08010690 con 0x7fdf141034a0 2024-02-26T10:42:07.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.506+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf141a1ba0 con 0x7fdf141034a0 2024-02-26T10:42:07.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.507+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1410a760 con 0x7fdf141034a0 2024-02-26T10:42:07.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.509+0000 7fdf12000640 1 -- 172.21.15.60:0/454304060 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fdf080037a0 con 0x7fdf141034a0 2024-02-26T10:42:07.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.511+0000 7fdf12000640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdefc0a1e00 0x7fdefc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:07.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.511+0000 7fdf12000640 1 -- 172.21.15.60:0/454304060 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdf080cdcc0 con 0x7fdf141034a0 2024-02-26T10:42:07.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.511+0000 7fdf19000640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdefc0a1e00 0x7fdefc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:07.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.512+0000 7fdf19000640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdefc0a1e00 0x7fdefc0a42c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf0002c240 tx=0x7fdf0002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:07.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.516+0000 7fdf12000640 1 -- 172.21.15.60:0/454304060 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf080d4080 con 0x7fdf141034a0 2024-02-26T10:42:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:07 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.60:6834/2576791717,v1:172.21.15.60:6835/2576791717] up:boot 2024-02-26T10:42:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:07 smithi060 ceph-mon[31539]: fsmap 3 up:standby 2024-02-26T10:42:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:07 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T10:42:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:07 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:07 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:07 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:07 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:07 smithi060 ceph-mon[31539]: Deploying daemon mds.d on smithi060 2024-02-26T10:42:07.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:07 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.60:6834/2576791717,v1:172.21.15.60:6835/2576791717] up:boot 2024-02-26T10:42:07.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:07 smithi113 ceph-mon[36126]: fsmap 3 up:standby 2024-02-26T10:42:07.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:07 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T10:42:07.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:07 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:07.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:07 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:07.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:07 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:07.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:07 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:07.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:07 smithi113 ceph-mon[36126]: Deploying daemon mds.d on smithi060 2024-02-26T10:42:07.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:07 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.60:6834/2576791717,v1:172.21.15.60:6835/2576791717] up:boot 2024-02-26T10:42:07.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:07 smithi100 ceph-mon[36051]: fsmap 3 up:standby 2024-02-26T10:42:07.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:07 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T10:42:07.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:07 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:07.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:07 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:07.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:07 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:07.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:07 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:07.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:07 smithi100 ceph-mon[36051]: Deploying daemon mds.d on smithi060 2024-02-26T10:42:07.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.766+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf1402ce30 con 0x7fdf141034a0 2024-02-26T10:42:07.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.767+0000 7fdf12000640 1 -- 172.21.15.60:0/454304060 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7fdf0809b4f0 con 0x7fdf141034a0 2024-02-26T10:42:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2024-02-26T10:42:07.765 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 4 2024-02-26T10:42:07.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdefc0a1e00 msgr2=0x7fdefc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:07.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdefc0a1e00 0x7fdefc0a42c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf0002c240 tx=0x7fdf0002df90 comp rx=0 tx=0).stop 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141034a0 msgr2=0x7fdf14199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141034a0 0x7fdf14199770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800e9c0 tx=0x7fdf0800ee80 comp rx=0 tx=0).stop 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 shutdown_connections 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdefc0a1e00 0x7fdefc0a42c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf14104690 0x7fdf14199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf14105050 0x7fdf141a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 --2- 172.21.15.60:0/454304060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141034a0 0x7fdf14199770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.769+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 >> 172.21.15.60:0/454304060 conn(0x7fdf140fea10 msgr2=0x7fdf14107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.770+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 shutdown_connections 2024-02-26T10:42:07.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:07.770+0000 7fdf1c400640 1 -- 172.21.15.60:0/454304060 wait complete. 2024-02-26T10:42:08.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:08 smithi113 ceph-mon[36126]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:08.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:08 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/454304060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:08.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:08 smithi060 ceph-mon[31539]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:08.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:08 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/454304060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:08.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:08 smithi100 ceph-mon[36051]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:08.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:08 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/454304060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:09.790 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:09.814 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:42:09 smithi060 systemd[1]: Starting Ceph mds.d for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:42:10.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.021+0000 7f94ebe00640 1 -- 172.21.15.60:0/4102753742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102c50 msgr2=0x7f94ec1094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.021+0000 7f94ebe00640 1 --2- 172.21.15.60:0/4102753742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102c50 0x7f94ec1094e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009ca0 tx=0x7f94e402c0f0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.021+0000 7f94ebe00640 1 -- 172.21.15.60:0/4102753742 shutdown_connections 2024-02-26T10:42:10.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.021+0000 7f94ebe00640 1 --2- 172.21.15.60:0/4102753742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102c50 0x7f94ec1094e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.021+0000 7f94ebe00640 1 --2- 172.21.15.60:0/4102753742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ec102290 0x7f94ec102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.021+0000 7f94ebe00640 1 --2- 172.21.15.60:0/4102753742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94ec1010a0 0x7f94ec1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.021+0000 7f94ebe00640 1 -- 172.21.15.60:0/4102753742 >> 172.21.15.60:0/4102753742 conn(0x7f94ec0fc610 msgr2=0x7f94ec0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:10.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.022+0000 7f94ebe00640 1 -- 172.21.15.60:0/4102753742 shutdown_connections 2024-02-26T10:42:10.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.022+0000 7f94ebe00640 1 -- 172.21.15.60:0/4102753742 wait complete. 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94ebe00640 1 Processor -- start 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94ebe00640 1 -- start start 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94ec1010a0 0x7f94ec199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94ebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102290 0x7f94ec199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ec102c50 0x7f94ec1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94ebe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec10cd30 con 0x7f94ec102290 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94ebe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec10cbb0 con 0x7f94ec1010a0 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94ebe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec10ceb0 con 0x7f94ec102c50 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94e3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102290 0x7f94ec199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94e3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102290 0x7f94ec199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46750/0 (socket says 172.21.15.60:46750) 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94e3e00640 1 -- 172.21.15.60:0/1764267290 learned_addr learned my addr 172.21.15.60:0/1764267290 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94eaa00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94ec1010a0 0x7f94ec199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.023+0000 7f94eb400640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ec102c50 0x7f94ec1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94e3e00640 1 -- 172.21.15.60:0/1764267290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ec102c50 msgr2=0x7f94ec1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94e3e00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ec102c50 0x7f94ec1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94e3e00640 1 -- 172.21.15.60:0/1764267290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94ec1010a0 msgr2=0x7f94ec199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94e3e00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94ec1010a0 0x7f94ec199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94e3e00640 1 -- 172.21.15.60:0/1764267290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec1a1190 con 0x7f94ec102290 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94e3e00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102290 0x7f94ec199ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f94d400e9f0 tx=0x7f94d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94e3400640 1 -- 172.21.15.60:0/1764267290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400cd80 con 0x7f94ec102290 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94e3400640 1 -- 172.21.15.60:0/1764267290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94d4005500 con 0x7f94ec102290 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec1a1480 con 0x7f94ec102290 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.024+0000 7f94e3400640 1 -- 172.21.15.60:0/1764267290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4010690 con 0x7f94ec102290 2024-02-26T10:42:10.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.025+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec1a19d0 con 0x7f94ec102290 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.026+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec1a1c90 con 0x7f94ec102290 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.028+0000 7f94e3400640 1 -- 172.21.15.60:0/1764267290 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f94d40046c0 con 0x7f94ec102290 2024-02-26T10:42:10.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.029+0000 7f94e3400640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94d00a1e00 0x7f94d00a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.029+0000 7f94e3400640 1 -- 172.21.15.60:0/1764267290 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f94d40cdd40 con 0x7f94ec102290 2024-02-26T10:42:10.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.029+0000 7f94eaa00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94d00a1e00 0x7f94d00a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.030+0000 7f94eaa00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94d00a1e00 0x7f94d00a42c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0095f0 tx=0x7f94dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.033+0000 7f94e3400640 1 -- 172.21.15.60:0/1764267290 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94d409b570 con 0x7f94ec102290 2024-02-26T10:42:10.118 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:42:09 smithi060 podman[72242]: 2024-02-26 10:42:09.724289691 +0000 UTC m=+0.036237584 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:42:10.118 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:42:09 smithi060 podman[72242]: 2024-02-26 10:42:09.849546247 +0000 UTC m=+0.161494098 container create 15451d5bb6800e0bb5ab2b18e9584c8c9090b63b5678e202fab0fda18c62e576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-26T10:42:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.284+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94ec1a1e60 con 0x7f94ec102290 2024-02-26T10:42:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.284+0000 7f94e3400640 1 -- 172.21.15.60:0/1764267290 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f94d409ae70 con 0x7f94ec102290 2024-02-26T10:42:10.283 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:10.283 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2024-02-26T10:42:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 5 2024-02-26T10:42:10.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94d00a1e00 msgr2=0x7f94d00a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94d00a1e00 0x7f94d00a42c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0095f0 tx=0x7f94dc005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:10.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102290 msgr2=0x7f94ec199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102290 0x7f94ec199ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f94d400e9f0 tx=0x7f94d400edf0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 shutdown_connections 2024-02-26T10:42:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94d00a1e00 0x7f94d00a42c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ec102c50 0x7f94ec1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ec102290 0x7f94ec199ab0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 --2- 172.21.15.60:0/1764267290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94ec1010a0 0x7f94ec199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.287+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 >> 172.21.15.60:0/1764267290 conn(0x7f94ec0fc610 msgr2=0x7f94ec073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.288+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 shutdown_connections 2024-02-26T10:42:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:10.288+0000 7f94ebe00640 1 -- 172.21.15.60:0/1764267290 wait complete. 2024-02-26T10:42:10.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:10 smithi113 ceph-mon[36126]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:10.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:10 smithi060 ceph-mon[31539]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:10.618 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:42:10 smithi060 podman[72242]: 2024-02-26 10:42:10.145461685 +0000 UTC m=+0.457409581 container init 15451d5bb6800e0bb5ab2b18e9584c8c9090b63b5678e202fab0fda18c62e576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-02-26T10:42:10.618 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:42:10 smithi060 podman[72242]: 2024-02-26 10:42:10.154451907 +0000 UTC m=+0.466399759 container start 15451d5bb6800e0bb5ab2b18e9584c8c9090b63b5678e202fab0fda18c62e576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:42:10.618 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:42:10 smithi060 ceph-mds[72300]: -- 172.21.15.60:0/1450461982 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5587d6329a40 con 0x5587d62fc800 2024-02-26T10:42:10.618 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:42:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d[72277]: starting mds.d at 2024-02-26T10:42:10.618 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:42:10 smithi060 bash[72242]: 15451d5bb6800e0bb5ab2b18e9584c8c9090b63b5678e202fab0fda18c62e576 2024-02-26T10:42:10.618 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:42:10 smithi060 systemd[1]: Started Ceph mds.d for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:10.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:10 smithi100 ceph-mon[36051]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:11.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.60:6836/2267710624,v1:172.21.15.60:6837/2267710624] up:boot 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: fsmap 4 up:standby 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1764267290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:11.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:11 smithi100 ceph-mon[36051]: Deploying daemon mds.e on smithi100 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.60:6836/2267710624,v1:172.21.15.60:6837/2267710624] up:boot 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: fsmap 4 up:standby 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1764267290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:11.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:11 smithi113 ceph-mon[36126]: Deploying daemon mds.e on smithi100 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.60:6836/2267710624,v1:172.21.15.60:6837/2267710624] up:boot 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: fsmap 4 up:standby 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1764267290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:11.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:11 smithi060 ceph-mon[31539]: Deploying daemon mds.e on smithi100 2024-02-26T10:42:12.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:12.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.541+0000 7f181b000640 1 -- 172.21.15.60:0/2602451153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18141088b0 msgr2=0x7f1814108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:12.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.541+0000 7f181b000640 1 --2- 172.21.15.60:0/2602451153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18141088b0 0x7f1814108d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1804009900 tx=0x7f180402bc50 comp rx=0 tx=0).stop 2024-02-26T10:42:12.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.541+0000 7f181b000640 1 -- 172.21.15.60:0/2602451153 shutdown_connections 2024-02-26T10:42:12.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.541+0000 7f181b000640 1 --2- 172.21.15.60:0/2602451153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18141088b0 0x7f1814108d30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.541+0000 7f181b000640 1 --2- 172.21.15.60:0/2602451153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1814109270 0x7f181410fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.541+0000 7f181b000640 1 --2- 172.21.15.60:0/2602451153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18141076c0 0x7f1814107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.541+0000 7f181b000640 1 -- 172.21.15.60:0/2602451153 >> 172.21.15.60:0/2602451153 conn(0x7f1814076670 msgr2=0x7f1814078a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:12.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.541+0000 7f181b000640 1 -- 172.21.15.60:0/2602451153 shutdown_connections 2024-02-26T10:42:12.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.541+0000 7f181b000640 1 -- 172.21.15.60:0/2602451153 wait complete. 2024-02-26T10:42:12.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.542+0000 7f181b000640 1 Processor -- start 2024-02-26T10:42:12.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.542+0000 7f181b000640 1 -- start start 2024-02-26T10:42:12.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.542+0000 7f181b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18141076c0 0x7f181419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:12.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.542+0000 7f181b000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18141088b0 0x7f181419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:12.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f181b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1814109270 0x7f18141a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:12.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f181b000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1814113370 con 0x7f18141088b0 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f181b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141131f0 con 0x7f1814109270 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f181b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141134f0 con 0x7f18141076c0 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f1819200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18141088b0 0x7f181419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f1819200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18141088b0 0x7f181419dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46786/0 (socket says 172.21.15.60:46786) 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f1819200640 1 -- 172.21.15.60:0/3634798076 learned_addr learned my addr 172.21.15.60:0/3634798076 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f1819c00640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18141076c0 0x7f181419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f181a600640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1814109270 0x7f18141a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f1819200640 1 -- 172.21.15.60:0/3634798076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18141076c0 msgr2=0x7f181419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f1819200640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18141076c0 0x7f181419d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f1819200640 1 -- 172.21.15.60:0/3634798076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1814109270 msgr2=0x7f18141a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f1819200640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1814109270 0x7f18141a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.543+0000 7f1819200640 1 -- 172.21.15.60:0/3634798076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18141a55a0 con 0x7f18141088b0 2024-02-26T10:42:12.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.544+0000 7f1819200640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18141088b0 0x7f181419dec0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1804005ab0 tx=0x7f180402c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:12.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.544+0000 7f1812000640 1 -- 172.21.15.60:0/3634798076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804041070 con 0x7f18141088b0 2024-02-26T10:42:12.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.544+0000 7f1812000640 1 -- 172.21.15.60:0/3634798076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1804004450 con 0x7f18141088b0 2024-02-26T10:42:12.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.544+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18141a5830 con 0x7f18141088b0 2024-02-26T10:42:12.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.544+0000 7f1812000640 1 -- 172.21.15.60:0/3634798076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804034440 con 0x7f18141088b0 2024-02-26T10:42:12.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.544+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18141a5d80 con 0x7f18141088b0 2024-02-26T10:42:12.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.545+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181410e980 con 0x7f18141088b0 2024-02-26T10:42:12.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.547+0000 7f1812000640 1 -- 172.21.15.60:0/3634798076 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1804044070 con 0x7f18141088b0 2024-02-26T10:42:12.545 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:12 smithi100 ceph-mon[36051]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:12.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.549+0000 7f1812000640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17fc0a1d80 0x7f17fc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:12.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.549+0000 7f1812000640 1 -- 172.21.15.60:0/3634798076 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18040ef610 con 0x7f18141088b0 2024-02-26T10:42:12.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.549+0000 7f1819c00640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17fc0a1d80 0x7f17fc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:12.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.550+0000 7f1819c00640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17fc0a1d80 0x7f17fc0a4240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1814108710 tx=0x7f1808005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:12.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.553+0000 7f1812000640 1 -- 172.21.15.60:0/3634798076 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18040bce40 con 0x7f18141088b0 2024-02-26T10:42:12.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.790+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f181402ce30 con 0x7f18141088b0 2024-02-26T10:42:12.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.791+0000 7f1812000640 1 -- 172.21.15.60:0/3634798076 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f18040bc740 con 0x7f18141088b0 2024-02-26T10:42:12.789 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:12.789 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2024-02-26T10:42:12.790 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 5 2024-02-26T10:42:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17fc0a1d80 msgr2=0x7f17fc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17fc0a1d80 0x7f17fc0a4240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1814108710 tx=0x7f1808005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18141088b0 msgr2=0x7f181419dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18141088b0 0x7f181419dec0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1804005ab0 tx=0x7f180402c6e0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 shutdown_connections 2024-02-26T10:42:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f17fc0a1d80 0x7f17fc0a4240 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1814109270 0x7f18141a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18141088b0 0x7f181419dec0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 --2- 172.21.15.60:0/3634798076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18141076c0 0x7f181419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:12.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 >> 172.21.15.60:0/3634798076 conn(0x7f1814076670 msgr2=0x7f181410bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:12.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 shutdown_connections 2024-02-26T10:42:12.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:12.794+0000 7f181b000640 1 -- 172.21.15.60:0/3634798076 wait complete. 2024-02-26T10:42:12.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:12 smithi060 ceph-mon[31539]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:12.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:12 smithi113 ceph-mon[36126]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:12.879 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:42:12 smithi100 systemd[1]: Starting Ceph mds.e for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:42:13.221 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:42:12 smithi100 podman[65995]: 2024-02-26 10:42:12.837544874 +0000 UTC m=+0.017349513 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:42:13.221 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:42:12 smithi100 podman[65995]: 2024-02-26 10:42:12.95469223 +0000 UTC m=+0.134496854 container create e36bf7a5fde2888107c2c3daee0ea14c045769d794bd715db3731aab02576fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-26T10:42:13.221 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:42:13 smithi100 podman[65995]: 2024-02-26 10:42:13.224127397 +0000 UTC m=+0.403932032 container init e36bf7a5fde2888107c2c3daee0ea14c045769d794bd715db3731aab02576fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T10:42:13.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:13 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3634798076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:13.629 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:42:13 smithi100 podman[65995]: 2024-02-26 10:42:13.226737812 +0000 UTC m=+0.406542430 container start e36bf7a5fde2888107c2c3daee0ea14c045769d794bd715db3731aab02576fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:42:13.629 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:42:13 smithi100 ceph-mds[66014]: -- 172.21.15.100:0/1878423970 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563233271a40 con 0x563233245000 2024-02-26T10:42:13.629 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:42:13 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e[66010]: starting mds.e at 2024-02-26T10:42:13.629 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:42:13 smithi100 bash[65995]: e36bf7a5fde2888107c2c3daee0ea14c045769d794bd715db3731aab02576fe4 2024-02-26T10:42:13.629 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:42:13 smithi100 systemd[1]: Started Ceph mds.e for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:13.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:13 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3634798076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:13.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:13 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3634798076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6834/1201544557,v1:172.21.15.100:6835/1201544557] up:boot 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: fsmap 5 up:standby 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:14.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:14 smithi100 ceph-mon[36051]: Deploying daemon mds.f on smithi113 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6834/1201544557,v1:172.21.15.100:6835/1201544557] up:boot 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: fsmap 5 up:standby 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:14.776 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:14 smithi113 ceph-mon[36126]: Deploying daemon mds.f on smithi113 2024-02-26T10:42:14.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:14.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:14.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:14.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:14.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6834/1201544557,v1:172.21.15.100:6835/1201544557] up:boot 2024-02-26T10:42:14.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: fsmap 5 up:standby 2024-02-26T10:42:14.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T10:42:14.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:14.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:14.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:14.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:14.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:14 smithi060 ceph-mon[31539]: Deploying daemon mds.f on smithi113 2024-02-26T10:42:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.044+0000 7ff388c00640 1 -- 172.21.15.60:0/31016761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384108a90 msgr2=0x7ff384108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.044+0000 7ff388c00640 1 --2- 172.21.15.60:0/31016761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384108a90 0x7ff384108f10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009900 tx=0x7ff36c02bc40 comp rx=0 tx=0).stop 2024-02-26T10:42:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.045+0000 7ff388c00640 1 -- 172.21.15.60:0/31016761 shutdown_connections 2024-02-26T10:42:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.045+0000 7ff388c00640 1 --2- 172.21.15.60:0/31016761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff384108a90 0x7ff384108f10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.045+0000 7ff388c00640 1 --2- 172.21.15.60:0/31016761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff384109450 0x7ff38410fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.045+0000 7ff388c00640 1 --2- 172.21.15.60:0/31016761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3841078a0 0x7ff384107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.045+0000 7ff388c00640 1 -- 172.21.15.60:0/31016761 >> 172.21.15.60:0/31016761 conn(0x7ff384076870 msgr2=0x7ff384078c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.045+0000 7ff388c00640 1 -- 172.21.15.60:0/31016761 shutdown_connections 2024-02-26T10:42:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.045+0000 7ff388c00640 1 -- 172.21.15.60:0/31016761 wait complete. 2024-02-26T10:42:15.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.046+0000 7ff388c00640 1 Processor -- start 2024-02-26T10:42:15.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.046+0000 7ff388c00640 1 -- start start 2024-02-26T10:42:15.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff388c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3841078a0 0x7ff38419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff388c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff384108a90 0x7ff38419e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff388c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff384109450 0x7ff3841a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff388c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff384113550 con 0x7ff384108a90 2024-02-26T10:42:15.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff388c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3841133d0 con 0x7ff3841078a0 2024-02-26T10:42:15.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff388c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3841136d0 con 0x7ff384109450 2024-02-26T10:42:15.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff382800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff384109450 0x7ff3841a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff379400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff384108a90 0x7ff38419e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff382800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff384109450 0x7ff3841a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:35206/0 (socket says 172.21.15.60:35206) 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff382800640 1 -- 172.21.15.60:0/1907831876 learned_addr learned my addr 172.21.15.60:0/1907831876 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff381e00640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3841078a0 0x7ff38419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff382800640 1 -- 172.21.15.60:0/1907831876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3841078a0 msgr2=0x7ff38419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff382800640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3841078a0 0x7ff38419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff382800640 1 -- 172.21.15.60:0/1907831876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff384108a90 msgr2=0x7ff38419e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff382800640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff384108a90 0x7ff38419e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.047+0000 7ff382800640 1 -- 172.21.15.60:0/1907831876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3841a5780 con 0x7ff384109450 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.048+0000 7ff382800640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff384109450 0x7ff3841a5110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00eaa0 tx=0x7ff37c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.048+0000 7ff37b400640 1 -- 172.21.15.60:0/1907831876 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c00cdb0 con 0x7ff384109450 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.048+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3841a5a70 con 0x7ff384109450 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.048+0000 7ff37b400640 1 -- 172.21.15.60:0/1907831876 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff37c005020 con 0x7ff384109450 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.048+0000 7ff37b400640 1 -- 172.21.15.60:0/1907831876 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c0116b0 con 0x7ff384109450 2024-02-26T10:42:15.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.048+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3841a5f90 con 0x7ff384109450 2024-02-26T10:42:15.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.049+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38410eb60 con 0x7ff384109450 2024-02-26T10:42:15.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.051+0000 7ff37b400640 1 -- 172.21.15.60:0/1907831876 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff37c005560 con 0x7ff384109450 2024-02-26T10:42:15.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.053+0000 7ff37b400640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff3680a1d80 0x7ff3680a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.053+0000 7ff37b400640 1 -- 172.21.15.60:0/1907831876 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff37c0cdb80 con 0x7ff384109450 2024-02-26T10:42:15.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.053+0000 7ff381e00640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff3680a1d80 0x7ff3680a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.054+0000 7ff381e00640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff3680a1d80 0x7ff3680a4240 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff3841088f0 tx=0x7ff374005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.057+0000 7ff37b400640 1 -- 172.21.15.60:0/1907831876 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff37c016030 con 0x7ff384109450 2024-02-26T10:42:15.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.296+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff384061e60 con 0x7ff384109450 2024-02-26T10:42:15.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.297+0000 7ff37b400640 1 -- 172.21.15.60:0/1907831876 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7ff37c09ad60 con 0x7ff384109450 2024-02-26T10:42:15.295 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:15.295 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-26T10:42:15.295 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2024-02-26T10:42:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff3680a1d80 msgr2=0x7ff3680a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff3680a1d80 0x7ff3680a4240 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff3841088f0 tx=0x7ff374005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff384109450 msgr2=0x7ff3841a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff384109450 0x7ff3841a5110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00eaa0 tx=0x7ff37c00ef60 comp rx=0 tx=0).stop 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 shutdown_connections 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff3680a1d80 0x7ff3680a4240 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff384109450 0x7ff3841a5110 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff384108a90 0x7ff38419e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 --2- 172.21.15.60:0/1907831876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3841078a0 0x7ff38419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.299+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 >> 172.21.15.60:0/1907831876 conn(0x7ff384076870 msgr2=0x7ff38410bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.300+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 shutdown_connections 2024-02-26T10:42:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:15.300+0000 7ff388c00640 1 -- 172.21.15.60:0/1907831876 wait complete. 2024-02-26T10:42:15.474 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:15 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1907831876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:15.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:15 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1907831876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:15.836 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:42:15 smithi113 systemd[1]: Starting Ceph mds.f for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:42:15.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:15 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1907831876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:16.132 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:42:15 smithi113 podman[66331]: 2024-02-26 10:42:15.77836634 +0000 UTC m=+0.017597305 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:42:16.132 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:42:15 smithi113 podman[66331]: 2024-02-26 10:42:15.899243694 +0000 UTC m=+0.138474692 container create 8d5f20c4882e2349bb607baf781b6714aaccf5ec2748ee15800480745c79d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:42:16.414 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:42:16 smithi113 podman[66331]: 2024-02-26 10:42:16.135169074 +0000 UTC m=+0.374400037 container init 8d5f20c4882e2349bb607baf781b6714aaccf5ec2748ee15800480745c79d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:42:16.415 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:42:16 smithi113 podman[66331]: 2024-02-26 10:42:16.138075008 +0000 UTC m=+0.377305961 container start 8d5f20c4882e2349bb607baf781b6714aaccf5ec2748ee15800480745c79d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True) 2024-02-26T10:42:16.415 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:42:16 smithi113 ceph-mds[66350]: -- 172.21.15.113:0/1438078578 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dda88d1a40 con 0x55dda88a4c00 2024-02-26T10:42:16.415 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:42:16 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f[66346]: starting mds.f at 2024-02-26T10:42:16.415 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:42:16 smithi113 bash[66331]: 8d5f20c4882e2349bb607baf781b6714aaccf5ec2748ee15800480745c79d74c 2024-02-26T10:42:16.415 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:42:16 smithi113 systemd[1]: Started Ceph mds.f for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:16.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:16 smithi113 ceph-mon[36126]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:16.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:16.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:16.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:16.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:16.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:16.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:16 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:16.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:16 smithi060 ceph-mon[31539]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:16.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:16.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:16.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:16.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:16.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:16.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:16 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:16.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:16 smithi100 ceph-mon[36051]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:16.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:16.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:16.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:16.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:16.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:16.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:16 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:17.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:17.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.553+0000 7fb891800640 1 -- 172.21.15.60:0/2350578096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 msgr2=0x7fb88c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.553+0000 7fb891800640 1 --2- 172.21.15.60:0/2350578096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 0x7fb88c1014e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009900 tx=0x7fb87c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:17.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.553+0000 7fb891800640 1 -- 172.21.15.60:0/2350578096 shutdown_connections 2024-02-26T10:42:17.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.553+0000 7fb891800640 1 --2- 172.21.15.60:0/2350578096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb88c102c70 0x7fb88c109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.553+0000 7fb891800640 1 --2- 172.21.15.60:0/2350578096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb88c1022b0 0x7fb88c102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.553+0000 7fb891800640 1 --2- 172.21.15.60:0/2350578096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 0x7fb88c1014e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.553+0000 7fb891800640 1 -- 172.21.15.60:0/2350578096 >> 172.21.15.60:0/2350578096 conn(0x7fb88c0fc630 msgr2=0x7fb88c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:17.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.553+0000 7fb891800640 1 -- 172.21.15.60:0/2350578096 shutdown_connections 2024-02-26T10:42:17.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.553+0000 7fb891800640 1 -- 172.21.15.60:0/2350578096 wait complete. 2024-02-26T10:42:17.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.554+0000 7fb891800640 1 Processor -- start 2024-02-26T10:42:17.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.554+0000 7fb891800640 1 -- start start 2024-02-26T10:42:17.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb891800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 0x7fb88c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb891800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb88c1022b0 0x7fb88c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb891800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb88c102c70 0x7fb88c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb891800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c10cda0 con 0x7fb88c1010c0 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb891800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c10cc20 con 0x7fb88c102c70 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb891800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c10cf20 con 0x7fb88c1022b0 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 0x7fb88c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 0x7fb88c1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41604/0 (socket says 172.21.15.60:41604) 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88be00640 1 -- 172.21.15.60:0/512363639 learned_addr learned my addr 172.21.15.60:0/512363639 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88b400640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb88c1022b0 0x7fb88c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb890e00640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb88c102c70 0x7fb88c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88be00640 1 -- 172.21.15.60:0/512363639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb88c1022b0 msgr2=0x7fb88c199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88be00640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb88c1022b0 0x7fb88c199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88be00640 1 -- 172.21.15.60:0/512363639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb88c102c70 msgr2=0x7fb88c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88be00640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb88c102c70 0x7fb88c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88be00640 1 -- 172.21.15.60:0/512363639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c1a1270 con 0x7fb88c1010c0 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb88b400640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb88c1022b0 0x7fb88c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.555+0000 7fb890e00640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb88c102c70 0x7fb88c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.556+0000 7fb88be00640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 0x7fb88c1996a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb87c000c00 tx=0x7fb87c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.556+0000 7fb888c00640 1 -- 172.21.15.60:0/512363639 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c043020 con 0x7fb88c1010c0 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.556+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c1a1500 con 0x7fb88c1010c0 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.556+0000 7fb888c00640 1 -- 172.21.15.60:0/512363639 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb87c0054e0 con 0x7fb88c1010c0 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.556+0000 7fb888c00640 1 -- 172.21.15.60:0/512363639 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c003ce0 con 0x7fb88c1010c0 2024-02-26T10:42:17.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.556+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c1a1a20 con 0x7fb88c1010c0 2024-02-26T10:42:17.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.557+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c1a1c20 con 0x7fb88c1010c0 2024-02-26T10:42:17.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.559+0000 7fb888c00640 1 -- 172.21.15.60:0/512363639 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb87c046020 con 0x7fb88c1010c0 2024-02-26T10:42:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.560+0000 7fb888c00640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb86c0a1d80 0x7fb86c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.561+0000 7fb888c00640 1 -- 172.21.15.60:0/512363639 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb87c0efa40 con 0x7fb88c1010c0 2024-02-26T10:42:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.561+0000 7fb88b400640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb86c0a1d80 0x7fb86c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.561+0000 7fb88b400640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb86c0a1d80 0x7fb86c0a4240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb88c19ab60 tx=0x7fb874009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:17.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.565+0000 7fb888c00640 1 -- 172.21.15.60:0/512363639 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb87c04d020 con 0x7fb88c1010c0 2024-02-26T10:42:17.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:17 smithi113 ceph-mon[36126]: Deploying daemon mds.c on smithi113 2024-02-26T10:42:17.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:17 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.113:6834/2109318506,v1:172.21.15.113:6835/2109318506] up:boot 2024-02-26T10:42:17.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:17 smithi113 ceph-mon[36126]: fsmap 6 up:standby 2024-02-26T10:42:17.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:17 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T10:42:17.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.806+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb88c1a1df0 con 0x7fb88c1010c0 2024-02-26T10:42:17.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.807+0000 7fb888c00640 1 -- 172.21.15.60:0/512363639 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7fb87c0bd310 con 0x7fb88c1010c0 2024-02-26T10:42:17.806 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:17.806 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-02-26T10:42:17.807 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 7 2024-02-26T10:42:17.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb86c0a1d80 msgr2=0x7fb86c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb86c0a1d80 0x7fb86c0a4240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb88c19ab60 tx=0x7fb874009500 comp rx=0 tx=0).stop 2024-02-26T10:42:17.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 msgr2=0x7fb88c1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 0x7fb88c1996a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb87c000c00 tx=0x7fb87c02c6c0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 shutdown_connections 2024-02-26T10:42:17.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb86c0a1d80 0x7fb86c0a4240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb88c102c70 0x7fb88c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb88c1022b0 0x7fb88c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 --2- 172.21.15.60:0/512363639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb88c1010c0 0x7fb88c1996a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 >> 172.21.15.60:0/512363639 conn(0x7fb88c0fc630 msgr2=0x7fb88c073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:17.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 shutdown_connections 2024-02-26T10:42:17.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:17.810+0000 7fb891800640 1 -- 172.21.15.60:0/512363639 wait complete. 2024-02-26T10:42:17.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:17 smithi060 ceph-mon[31539]: Deploying daemon mds.c on smithi113 2024-02-26T10:42:17.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:17 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.113:6834/2109318506,v1:172.21.15.113:6835/2109318506] up:boot 2024-02-26T10:42:17.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:17 smithi060 ceph-mon[31539]: fsmap 6 up:standby 2024-02-26T10:42:17.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:17 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T10:42:17.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:17 smithi100 ceph-mon[36051]: Deploying daemon mds.c on smithi113 2024-02-26T10:42:17.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:17 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.113:6834/2109318506,v1:172.21.15.113:6835/2109318506] up:boot 2024-02-26T10:42:17.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:17 smithi100 ceph-mon[36051]: fsmap 6 up:standby 2024-02-26T10:42:17.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:17 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T10:42:18.674 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:18 smithi113 ceph-mon[36126]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:18.675 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:18 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/512363639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:18.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:18 smithi060 ceph-mon[31539]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:18.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:18 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/512363639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:18.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:18 smithi100 ceph-mon[36051]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:18.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:18 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/512363639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:19.293 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:42:19 smithi113 systemd[1]: Started Ceph mds.c for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:19.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:20.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.069+0000 7f5590e00640 1 -- 172.21.15.60:0/2351712174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558c1044a0 msgr2=0x7f558c104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.069+0000 7f5590e00640 1 --2- 172.21.15.60:0/2351712174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558c1044a0 0x7f558c104920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5574009900 tx=0x7f557402bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:20.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.069+0000 7f5590e00640 1 -- 172.21.15.60:0/2351712174 shutdown_connections 2024-02-26T10:42:20.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.069+0000 7f5590e00640 1 --2- 172.21.15.60:0/2351712174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558c104e60 0x7f558c10b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.069+0000 7f5590e00640 1 --2- 172.21.15.60:0/2351712174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558c1044a0 0x7f558c104920 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.069+0000 7f5590e00640 1 --2- 172.21.15.60:0/2351712174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c1032b0 0x7f558c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.069+0000 7f5590e00640 1 -- 172.21.15.60:0/2351712174 >> 172.21.15.60:0/2351712174 conn(0x7f558c0fe840 msgr2=0x7f558c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:20.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.069+0000 7f5590e00640 1 -- 172.21.15.60:0/2351712174 shutdown_connections 2024-02-26T10:42:20.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.069+0000 7f5590e00640 1 -- 172.21.15.60:0/2351712174 wait complete. 2024-02-26T10:42:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f5590e00640 1 Processor -- start 2024-02-26T10:42:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f5590e00640 1 -- start start 2024-02-26T10:42:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f5590e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558c1032b0 0x7f558c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f5590e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558c1044a0 0x7f558c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f5590e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c104e60 0x7f558c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f5590e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c10ef40 con 0x7f558c1032b0 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f5590e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c10edc0 con 0x7f558c1044a0 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f5590e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c10f0c0 con 0x7f558c104e60 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f558be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c104e60 0x7f558c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.071+0000 7f558b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558c1032b0 0x7f558c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f558be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c104e60 0x7f558c1a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:35246/0 (socket says 172.21.15.60:35246) 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f558be00640 1 -- 172.21.15.60:0/3604414037 learned_addr learned my addr 172.21.15.60:0/3604414037 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f5582a00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558c1044a0 0x7f558c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f558b400640 1 -- 172.21.15.60:0/3604414037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c104e60 msgr2=0x7f558c1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f558b400640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c104e60 0x7f558c1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f558b400640 1 -- 172.21.15.60:0/3604414037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558c1044a0 msgr2=0x7f558c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f558b400640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558c1044a0 0x7f558c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f558b400640 1 -- 172.21.15.60:0/3604414037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c1a1190 con 0x7f558c1032b0 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f558be00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c104e60 0x7f558c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.072+0000 7f558b400640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558c1032b0 0x7f558c199570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e9c0 tx=0x7f557c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.073+0000 7f5588c00640 1 -- 172.21.15.60:0/3604414037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00ccc0 con 0x7f558c1032b0 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.073+0000 7f5588c00640 1 -- 172.21.15.60:0/3604414037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f557c0035f0 con 0x7f558c1032b0 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.073+0000 7f5588c00640 1 -- 172.21.15.60:0/3604414037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00ce60 con 0x7f558c1032b0 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.073+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558c1a1480 con 0x7f558c1032b0 2024-02-26T10:42:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.073+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558c1a1950 con 0x7f558c1032b0 2024-02-26T10:42:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.074+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558c10a650 con 0x7f558c1032b0 2024-02-26T10:42:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.076+0000 7f5588c00640 1 -- 172.21.15.60:0/3604414037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f557c0037a0 con 0x7f558c1032b0 2024-02-26T10:42:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.077+0000 7f5588c00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f556c0a1d80 0x7f556c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.078+0000 7f5588c00640 1 -- 172.21.15.60:0/3604414037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f557c0cdd10 con 0x7f558c1032b0 2024-02-26T10:42:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.078+0000 7f5582a00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f556c0a1d80 0x7f556c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.079+0000 7f5582a00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f556c0a1d80 0x7f556c0a4240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f557402c7b0 tx=0x7f5574005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.082+0000 7f5588c00640 1 -- 172.21.15.60:0/3604414037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f557c0d3020 con 0x7f558c1032b0 2024-02-26T10:42:20.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.325+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f558c02ce30 con 0x7f558c1032b0 2024-02-26T10:42:20.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.326+0000 7f5588c00640 1 -- 172.21.15.60:0/3604414037 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6086 (secure 0 0 0) 0x7f557c09b540 con 0x7f558c1032b0 2024-02-26T10:42:20.325 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:20.325 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-26T10:42:20.326 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-02-26T10:42:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f556c0a1d80 msgr2=0x7f556c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f556c0a1d80 0x7f556c0a4240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f557402c7b0 tx=0x7f5574005fd0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558c1032b0 msgr2=0x7f558c199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558c1032b0 0x7f558c199570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e9c0 tx=0x7f557c00ee80 comp rx=0 tx=0).stop 2024-02-26T10:42:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 shutdown_connections 2024-02-26T10:42:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f556c0a1d80 0x7f556c0a4240 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558c1044a0 0x7f558c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c104e60 0x7f558c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 --2- 172.21.15.60:0/3604414037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558c1032b0 0x7f558c199570 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.329+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 >> 172.21.15.60:0/3604414037 conn(0x7f558c0fe840 msgr2=0x7f558c107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:20.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.330+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 shutdown_connections 2024-02-26T10:42:20.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:20.330+0000 7f5590e00640 1 -- 172.21.15.60:0/3604414037 wait complete. 2024-02-26T10:42:20.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:20.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:20.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:20.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:20.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:20.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:20.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:20.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: Deploying daemon mds.h on smithi100 2024-02-26T10:42:20.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.113:6836/3968092815,v1:172.21.15.113:6837/3968092815] up:boot 2024-02-26T10:42:20.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: fsmap 7 up:standby 2024-02-26T10:42:20.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T10:42:20.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:20 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3604414037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:20.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: Deploying daemon mds.h on smithi100 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.113:6836/3968092815,v1:172.21.15.113:6837/3968092815] up:boot 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: fsmap 7 up:standby 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T10:42:20.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:20 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3604414037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: Deploying daemon mds.h on smithi100 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.113:6836/3968092815,v1:172.21.15.113:6837/3968092815] up:boot 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: fsmap 7 up:standby 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T10:42:20.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:20 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3604414037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:21.629 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:42:21 smithi100 systemd[1]: Starting Ceph mds.h for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:42:22.018 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:42:21 smithi100 podman[66307]: 2024-02-26 10:42:21.63863299 +0000 UTC m=+0.017576072 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:42:22.018 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:42:21 smithi100 podman[66307]: 2024-02-26 10:42:21.759801813 +0000 UTC m=+0.138744864 container create bc785a3a01e38a0b314ef42f0ae1e99a121559c900a32a28e6de52940f5982fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-26T10:42:22.335 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:42:22 smithi100 podman[66307]: 2024-02-26 10:42:22.021280505 +0000 UTC m=+0.400223544 container init bc785a3a01e38a0b314ef42f0ae1e99a121559c900a32a28e6de52940f5982fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-26T10:42:22.336 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:42:22 smithi100 podman[66307]: 2024-02-26 10:42:22.023889945 +0000 UTC m=+0.402832977 container start bc785a3a01e38a0b314ef42f0ae1e99a121559c900a32a28e6de52940f5982fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=) 2024-02-26T10:42:22.336 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:42:22 smithi100 ceph-mds[66328]: -- 172.21.15.100:0/3767451359 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559fc9f0da40 con 0x559fc9ee0800 2024-02-26T10:42:22.336 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:42:22 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: starting mds.h at 2024-02-26T10:42:22.336 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:42:22 smithi100 bash[66307]: bc785a3a01e38a0b314ef42f0ae1e99a121559c900a32a28e6de52940f5982fe 2024-02-26T10:42:22.336 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:42:22 smithi100 systemd[1]: Started Ceph mds.h for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:22.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.577+0000 7f4dc7400640 1 -- 172.21.15.60:0/4163364797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 msgr2=0x7f4dc00736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.577+0000 7f4dc7400640 1 --2- 172.21.15.60:0/4163364797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 0x7f4dc00736f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009900 tx=0x7f4db402bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.578+0000 7f4dc7400640 1 -- 172.21.15.60:0/4163364797 shutdown_connections 2024-02-26T10:42:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.578+0000 7f4dc7400640 1 --2- 172.21.15.60:0/4163364797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc0073cc0 0x7f4dc0074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.578+0000 7f4dc7400640 1 --2- 172.21.15.60:0/4163364797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dc010d850 0x7f4dc010fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.578+0000 7f4dc7400640 1 --2- 172.21.15.60:0/4163364797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 0x7f4dc00736f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.578+0000 7f4dc7400640 1 -- 172.21.15.60:0/4163364797 >> 172.21.15.60:0/4163364797 conn(0x7f4dc00fc810 msgr2=0x7f4dc00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.578+0000 7f4dc7400640 1 -- 172.21.15.60:0/4163364797 shutdown_connections 2024-02-26T10:42:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.578+0000 7f4dc7400640 1 -- 172.21.15.60:0/4163364797 wait complete. 2024-02-26T10:42:22.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.579+0000 7f4dc7400640 1 Processor -- start 2024-02-26T10:42:22.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.579+0000 7f4dc7400640 1 -- start start 2024-02-26T10:42:22.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.579+0000 7f4dc7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dc0073cc0 0x7f4dc019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:22.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dc7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 0x7f4dc019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dc7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc010d850 0x7f4dc01a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dc7400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc0113540 con 0x7f4dc0073cc0 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dc7400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc01133c0 con 0x7f4dc010d850 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dc7400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc01136c0 con 0x7f4dc00752a0 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dbfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 0x7f4dc019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dbfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 0x7f4dc019e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:35264/0 (socket says 172.21.15.60:35264) 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dbfe00640 1 -- 172.21.15.60:0/1209907005 learned_addr learned my addr 172.21.15.60:0/1209907005 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dc4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dc0073cc0 0x7f4dc019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dc5400640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc010d850 0x7f4dc01a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dbfe00640 1 -- 172.21.15.60:0/1209907005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc010d850 msgr2=0x7f4dc01a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dbfe00640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc010d850 0x7f4dc01a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dbfe00640 1 -- 172.21.15.60:0/1209907005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dc0073cc0 msgr2=0x7f4dc019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dbfe00640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dc0073cc0 0x7f4dc019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dbfe00640 1 -- 172.21.15.60:0/1209907005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc01a5790 con 0x7f4dc00752a0 2024-02-26T10:42:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.580+0000 7f4dbfe00640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 0x7f4dc019e0b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00e9f0 tx=0x7f4dac00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.581+0000 7f4dbd600640 1 -- 172.21.15.60:0/1209907005 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00cd80 con 0x7f4dc00752a0 2024-02-26T10:42:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.581+0000 7f4dbd600640 1 -- 172.21.15.60:0/1209907005 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dac005500 con 0x7f4dc00752a0 2024-02-26T10:42:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.581+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc01a5a80 con 0x7f4dc00752a0 2024-02-26T10:42:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.581+0000 7f4dbd600640 1 -- 172.21.15.60:0/1209907005 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac010690 con 0x7f4dc00752a0 2024-02-26T10:42:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.581+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc01a5fd0 con 0x7f4dc00752a0 2024-02-26T10:42:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.582+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc0103a20 con 0x7f4dc00752a0 2024-02-26T10:42:22.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.584+0000 7f4dbd600640 1 -- 172.21.15.60:0/1209907005 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4dac0046c0 con 0x7f4dc00752a0 2024-02-26T10:42:22.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.585+0000 7f4dbd600640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4da80a1e00 0x7f4da80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:22.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.586+0000 7f4dbd600640 1 -- 172.21.15.60:0/1209907005 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4dac0cddf0 con 0x7f4dc00752a0 2024-02-26T10:42:22.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.586+0000 7f4dc4a00640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4da80a1e00 0x7f4da80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:22.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.586+0000 7f4dc4a00640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4da80a1e00 0x7f4da80a42c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4db402c240 tx=0x7f4db4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:22.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.589+0000 7f4dbd600640 1 -- 172.21.15.60:0/1209907005 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4dac09b620 con 0x7f4dc00752a0 2024-02-26T10:42:22.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:22 smithi060 ceph-mon[31539]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:22.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:22.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:22.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:22.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:22.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:22.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:22 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:22.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:22 smithi100 ceph-mon[36051]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:22.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:22.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:22.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:22.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:22.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:22.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:22 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.834+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dc002ce30 con 0x7f4dc00752a0 2024-02-26T10:42:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.835+0000 7f4dbd600640 1 -- 172.21.15.60:0/1209907005 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6880 (secure 0 0 0) 0x7f4dac09af20 con 0x7f4dc00752a0 2024-02-26T10:42:22.834 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:22.834 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-26T10:42:22.835 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 9 2024-02-26T10:42:22.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:22 smithi113 ceph-mon[36126]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:22.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:22.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:22.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:22.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:22.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:22.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:22 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4da80a1e00 msgr2=0x7f4da80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4da80a1e00 0x7f4da80a42c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4db402c240 tx=0x7f4db4000bc0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 msgr2=0x7f4dc019e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 0x7f4dc019e0b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00e9f0 tx=0x7f4dac00edf0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 shutdown_connections 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4da80a1e00 0x7f4da80a42c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dc010d850 0x7f4dc01a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dc00752a0 0x7f4dc019e0b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 --2- 172.21.15.60:0/1209907005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dc0073cc0 0x7f4dc019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 >> 172.21.15.60:0/1209907005 conn(0x7f4dc00fc810 msgr2=0x7f4dc0100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 shutdown_connections 2024-02-26T10:42:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:22.838+0000 7f4dc7400640 1 -- 172.21.15.60:0/1209907005 wait complete. 2024-02-26T10:42:23.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:23 smithi113 ceph-mon[36126]: Deploying daemon mds.i on smithi113 2024-02-26T10:42:23.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:23 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] up:boot 2024-02-26T10:42:23.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:23 smithi113 ceph-mon[36126]: fsmap 8 up:standby 2024-02-26T10:42:23.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:23 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T10:42:23.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:23 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1209907005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:23.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:23 smithi060 ceph-mon[31539]: Deploying daemon mds.i on smithi113 2024-02-26T10:42:23.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:23 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] up:boot 2024-02-26T10:42:23.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:23 smithi060 ceph-mon[31539]: fsmap 8 up:standby 2024-02-26T10:42:23.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:23 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T10:42:23.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:23 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1209907005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:23.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:23 smithi100 ceph-mon[36051]: Deploying daemon mds.i on smithi113 2024-02-26T10:42:23.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:23 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] up:boot 2024-02-26T10:42:23.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:23 smithi100 ceph-mon[36051]: fsmap 8 up:standby 2024-02-26T10:42:23.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:23 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T10:42:23.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:23 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1209907005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:24.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:24 smithi113 ceph-mon[36126]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:24.586 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:42:24 smithi113 systemd[1]: Starting Ceph mds.i for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:42:24.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:24 smithi060 ceph-mon[31539]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:24.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:24 smithi100 ceph-mon[36051]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:24.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:25.003 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:42:24 smithi113 podman[66965]: 2024-02-26 10:42:24.592990452 +0000 UTC m=+0.017595550 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:42:25.003 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:42:24 smithi113 podman[66965]: 2024-02-26 10:42:24.707322725 +0000 UTC m=+0.131927849 container create a587f82898b496712c4e35bf940a91ba4764121bf2de33daa90a66419fc68cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0) 2024-02-26T10:42:25.003 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:42:25 smithi113 podman[66965]: 2024-02-26 10:42:25.006121049 +0000 UTC m=+0.430726134 container init a587f82898b496712c4e35bf940a91ba4764121bf2de33daa90a66419fc68cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-26T10:42:25.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.091+0000 7faff0c00640 1 -- 172.21.15.60:0/1075984585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec077800 msgr2=0x7fafec0780a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.091+0000 7faff0c00640 1 --2- 172.21.15.60:0/1075984585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec077800 0x7fafec0780a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009ca0 tx=0x7fafe402c100 comp rx=0 tx=0).stop 2024-02-26T10:42:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.091+0000 7faff0c00640 1 -- 172.21.15.60:0/1075984585 shutdown_connections 2024-02-26T10:42:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.091+0000 7faff0c00640 1 --2- 172.21.15.60:0/1075984585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafec076e60 0x7fafec0772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.091+0000 7faff0c00640 1 --2- 172.21.15.60:0/1075984585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec077800 0x7fafec0780a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.091+0000 7faff0c00640 1 --2- 172.21.15.60:0/1075984585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafec075c20 0x7fafec076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.091+0000 7faff0c00640 1 -- 172.21.15.60:0/1075984585 >> 172.21.15.60:0/1075984585 conn(0x7fafec0fea10 msgr2=0x7fafec100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.091+0000 7faff0c00640 1 -- 172.21.15.60:0/1075984585 shutdown_connections 2024-02-26T10:42:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.091+0000 7faff0c00640 1 -- 172.21.15.60:0/1075984585 wait complete. 2024-02-26T10:42:25.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.092+0000 7faff0c00640 1 Processor -- start 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.092+0000 7faff0c00640 1 -- start start 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.092+0000 7faff0c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec075c20 0x7fafec19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.092+0000 7faff0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafec076e60 0x7fafec19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.092+0000 7faff0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafec077800 0x7fafec1a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.092+0000 7faff0c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec113510 con 0x7fafec075c20 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.092+0000 7faff0c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec113390 con 0x7fafec076e60 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.092+0000 7faff0c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec113690 con 0x7fafec077800 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafe9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec075c20 0x7fafec19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafe9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec075c20 0x7fafec19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48658/0 (socket says 172.21.15.60:48658) 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafe9e00640 1 -- 172.21.15.60:0/1629936116 learned_addr learned my addr 172.21.15.60:0/1629936116 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafe9400640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafec076e60 0x7fafec19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafea800640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafec077800 0x7fafec1a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafe9e00640 1 -- 172.21.15.60:0/1629936116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafec077800 msgr2=0x7fafec1a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafe9e00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafec077800 0x7fafec1a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafe9e00640 1 -- 172.21.15.60:0/1629936116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafec076e60 msgr2=0x7fafec19e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafe9e00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafec076e60 0x7fafec19e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafe9e00640 1 -- 172.21.15.60:0/1629936116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec1a57c0 con 0x7fafec075c20 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.093+0000 7fafea800640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafec077800 0x7fafec1a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.094+0000 7fafe9e00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec075c20 0x7fafec19dba0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00e9f0 tx=0x7fafdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.094+0000 7fafe2a00640 1 -- 172.21.15.60:0/1629936116 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc00cd80 con 0x7fafec075c20 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.094+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec1a5ab0 con 0x7fafec075c20 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.094+0000 7fafe2a00640 1 -- 172.21.15.60:0/1629936116 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafdc005500 con 0x7fafec075c20 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.094+0000 7fafe2a00640 1 -- 172.21.15.60:0/1629936116 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc010690 con 0x7fafec075c20 2024-02-26T10:42:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.094+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec1a5fb0 con 0x7fafec075c20 2024-02-26T10:42:25.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.096+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec10fdf0 con 0x7fafec075c20 2024-02-26T10:42:25.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.098+0000 7fafe2a00640 1 -- 172.21.15.60:0/1629936116 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fafdc0046c0 con 0x7fafec075c20 2024-02-26T10:42:25.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.099+0000 7fafe2a00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fafd00a1d80 0x7fafd00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.099+0000 7fafe2a00640 1 -- 172.21.15.60:0/1629936116 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fafdc0cdda0 con 0x7fafec075c20 2024-02-26T10:42:25.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.099+0000 7fafe9400640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fafd00a1d80 0x7fafd00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.100+0000 7fafe9400640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fafd00a1d80 0x7fafd00a4240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fafec19f0b0 tx=0x7fafd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:25.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.103+0000 7fafe2a00640 1 -- 172.21.15.60:0/1629936116 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fafdc09b5d0 con 0x7fafec075c20 2024-02-26T10:42:25.335 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:42:25 smithi113 podman[66965]: 2024-02-26 10:42:25.008868615 +0000 UTC m=+0.433473706 container start a587f82898b496712c4e35bf940a91ba4764121bf2de33daa90a66419fc68cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:42:25.335 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:42:25 smithi113 ceph-mds[66984]: -- 172.21.15.113:0/1344729522 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558448f41a40 con 0x558448f14800 2024-02-26T10:42:25.335 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:42:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: starting mds.i at 2024-02-26T10:42:25.335 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:42:25 smithi113 bash[66965]: a587f82898b496712c4e35bf940a91ba4764121bf2de33daa90a66419fc68cda 2024-02-26T10:42:25.335 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:42:25 smithi113 systemd[1]: Started Ceph mds.i for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:25.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.365+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafec02ce30 con 0x7fafec075c20 2024-02-26T10:42:25.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.370+0000 7fafe2a00640 1 -- 172.21.15.60:0/1629936116 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7676 (secure 0 0 0) 0x7fafdc09aed0 con 0x7fafec075c20 2024-02-26T10:42:25.369 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:25.369 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T10:42:25.369 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 10 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fafd00a1d80 msgr2=0x7fafd00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fafd00a1d80 0x7fafd00a4240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fafec19f0b0 tx=0x7fafd800b410 comp rx=0 tx=0).stop 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec075c20 msgr2=0x7fafec19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec075c20 0x7fafec19dba0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00e9f0 tx=0x7fafdc00edf0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 shutdown_connections 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fafd00a1d80 0x7fafd00a4240 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafec076e60 0x7fafec19e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafec077800 0x7fafec1a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 --2- 172.21.15.60:0/1629936116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafec075c20 0x7fafec19dba0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 >> 172.21.15.60:0/1629936116 conn(0x7fafec0fea10 msgr2=0x7fafec100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:25.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.373+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 shutdown_connections 2024-02-26T10:42:25.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:25.374+0000 7faff0c00640 1 -- 172.21.15.60:0/1629936116 wait complete. 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: Deploying daemon mds.g on smithi060 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] up:boot 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: fsmap 9 up:standby 2024-02-26T10:42:26.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1629936116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:26.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:26 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T10:42:26.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:26.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:26.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:26.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:26.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:26.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:26.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:26.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: Deploying daemon mds.g on smithi060 2024-02-26T10:42:26.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] up:boot 2024-02-26T10:42:26.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: fsmap 9 up:standby 2024-02-26T10:42:26.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1629936116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:26.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:26 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: Deploying daemon mds.g on smithi060 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] up:boot 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: fsmap 9 up:standby 2024-02-26T10:42:26.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1629936116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:26.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:26 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T10:42:27.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:27.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.624+0000 7fcd4c200640 1 -- 172.21.15.60:0/3032427581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd44105050 msgr2=0x7fcd4410b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.624+0000 7fcd4c200640 1 --2- 172.21.15.60:0/3032427581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd44105050 0x7fcd4410b8e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009ca0 tx=0x7fcd3c02c0f0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.625+0000 7fcd4c200640 1 -- 172.21.15.60:0/3032427581 shutdown_connections 2024-02-26T10:42:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.625+0000 7fcd4c200640 1 --2- 172.21.15.60:0/3032427581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd44105050 0x7fcd4410b8e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.625+0000 7fcd4c200640 1 --2- 172.21.15.60:0/3032427581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd44104690 0x7fcd44104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.625+0000 7fcd4c200640 1 --2- 172.21.15.60:0/3032427581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd441034a0 0x7fcd441038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.625+0000 7fcd4c200640 1 -- 172.21.15.60:0/3032427581 >> 172.21.15.60:0/3032427581 conn(0x7fcd440fea50 msgr2=0x7fcd44100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.625+0000 7fcd4c200640 1 -- 172.21.15.60:0/3032427581 shutdown_connections 2024-02-26T10:42:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.625+0000 7fcd4c200640 1 -- 172.21.15.60:0/3032427581 wait complete. 2024-02-26T10:42:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.626+0000 7fcd4c200640 1 Processor -- start 2024-02-26T10:42:27.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.626+0000 7fcd4c200640 1 -- start start 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd4c200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd441034a0 0x7fcd44199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd4c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd44104690 0x7fcd44199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd4c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd44105050 0x7fcd441a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd4c200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4410f130 con 0x7fcd441034a0 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd4c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4410efb0 con 0x7fcd44105050 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd4c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4410f2b0 con 0x7fcd44104690 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd49800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd441034a0 0x7fcd44199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd49800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd441034a0 0x7fcd44199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48684/0 (socket says 172.21.15.60:48684) 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd49800640 1 -- 172.21.15.60:0/611871926 learned_addr learned my addr 172.21.15.60:0/611871926 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd4a200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd44105050 0x7fcd441a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.627+0000 7fcd48e00640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd44104690 0x7fcd44199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.628+0000 7fcd4a200640 1 -- 172.21.15.60:0/611871926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd44104690 msgr2=0x7fcd44199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.628+0000 7fcd4a200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd44104690 0x7fcd44199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.628+0000 7fcd4a200640 1 -- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd441034a0 msgr2=0x7fcd44199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.628+0000 7fcd4a200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd441034a0 0x7fcd44199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.628+0000 7fcd4a200640 1 -- 172.21.15.60:0/611871926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd441a1360 con 0x7fcd44105050 2024-02-26T10:42:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.628+0000 7fcd49800640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd441034a0 0x7fcd44199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.628+0000 7fcd48e00640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd44104690 0x7fcd44199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.628+0000 7fcd4a200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd44105050 0x7fcd441a0cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c02c630 tx=0x7fcd3c032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.629+0000 7fcd42000640 1 -- 172.21.15.60:0/611871926 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c041020 con 0x7fcd44105050 2024-02-26T10:42:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.629+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd441a15f0 con 0x7fcd44105050 2024-02-26T10:42:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.629+0000 7fcd42000640 1 -- 172.21.15.60:0/611871926 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd3c034cd0 con 0x7fcd44105050 2024-02-26T10:42:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.629+0000 7fcd42000640 1 -- 172.21.15.60:0/611871926 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c039040 con 0x7fcd44105050 2024-02-26T10:42:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.629+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd441a1b00 con 0x7fcd44105050 2024-02-26T10:42:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.630+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4410a6f0 con 0x7fcd44105050 2024-02-26T10:42:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.633+0000 7fcd42000640 1 -- 172.21.15.60:0/611871926 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fcd3c044020 con 0x7fcd44105050 2024-02-26T10:42:27.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.635+0000 7fcd42000640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd2c0a1d80 0x7fcd2c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.635+0000 7fcd42000640 1 -- 172.21.15.60:0/611871926 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcd3c0ef730 con 0x7fcd44105050 2024-02-26T10:42:27.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.635+0000 7fcd49800640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd2c0a1d80 0x7fcd2c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.636+0000 7fcd49800640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd2c0a1d80 0x7fcd2c0a4240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd380058c0 tx=0x7fcd38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.639+0000 7fcd42000640 1 -- 172.21.15.60:0/611871926 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd3c04b020 con 0x7fcd44105050 2024-02-26T10:42:27.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.918+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd44061e60 con 0x7fcd44105050 2024-02-26T10:42:27.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.919+0000 7fcd42000640 1 -- 172.21.15.60:0/611871926 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7676 (secure 0 0 0) 0x7fcd3c0bcff0 con 0x7fcd44105050 2024-02-26T10:42:27.916 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:27.916 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T10:42:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 10 2024-02-26T10:42:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.921+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd2c0a1d80 msgr2=0x7fcd2c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.921+0000 7fcd4c200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd2c0a1d80 0x7fcd2c0a4240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd380058c0 tx=0x7fcd38005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.921+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd44105050 msgr2=0x7fcd441a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.921+0000 7fcd4c200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd44105050 0x7fcd441a0cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c02c630 tx=0x7fcd3c032f20 comp rx=0 tx=0).stop 2024-02-26T10:42:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.921+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 shutdown_connections 2024-02-26T10:42:27.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.921+0000 7fcd4c200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd2c0a1d80 0x7fcd2c0a4240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.921+0000 7fcd4c200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd44105050 0x7fcd441a0cf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.921+0000 7fcd4c200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd44104690 0x7fcd44199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.922+0000 7fcd4c200640 1 --2- 172.21.15.60:0/611871926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd441034a0 0x7fcd44199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.922+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 >> 172.21.15.60:0/611871926 conn(0x7fcd440fea50 msgr2=0x7fcd441078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:27.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.922+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 shutdown_connections 2024-02-26T10:42:27.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:27.922+0000 7fcd4c200640 1 -- 172.21.15.60:0/611871926 wait complete. 2024-02-26T10:42:28.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:28 smithi060 ceph-mon[31539]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:28.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:28 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/611871926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:28.431 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:28 smithi100 ceph-mon[36051]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:28.431 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:28 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/611871926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:28.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:28 smithi113 ceph-mon[36126]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:28.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:28 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/611871926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:28.617 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:42:28 smithi060 systemd[1]: Starting Ceph mds.g for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:42:28.868 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:42:28 smithi060 podman[72884]: 2024-02-26 10:42:28.526249508 +0000 UTC m=+0.037767812 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:42:28.868 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:42:28 smithi060 podman[72884]: 2024-02-26 10:42:28.655955657 +0000 UTC m=+0.167473988 container create 07cd66bddb7afb2319197e8782400f69303204b3ae9a974329e9111ea347e5ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-02-26T10:42:29.367 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:42:28 smithi060 podman[72884]: 2024-02-26 10:42:28.935259505 +0000 UTC m=+0.446777900 container init 07cd66bddb7afb2319197e8782400f69303204b3ae9a974329e9111ea347e5ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-02-26T10:42:29.367 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:42:28 smithi060 podman[72884]: 2024-02-26 10:42:28.947466736 +0000 UTC m=+0.458985051 container start 07cd66bddb7afb2319197e8782400f69303204b3ae9a974329e9111ea347e5ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-02-26T10:42:29.367 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:42:28 smithi060 ceph-mds[72904]: -- 172.21.15.60:0/1896468330 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55776c427a40 con 0x55776c3fb000 2024-02-26T10:42:29.368 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:42:28 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g[72899]: starting mds.g at 2024-02-26T10:42:29.368 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:42:29 smithi060 bash[72884]: 07cd66bddb7afb2319197e8782400f69303204b3ae9a974329e9111ea347e5ab 2024-02-26T10:42:29.368 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:42:29 smithi060 systemd[1]: Started Ceph mds.g for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:29.941 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:30.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.146+0000 7fe572800640 1 -- 172.21.15.60:0/639371561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 msgr2=0x7fe56c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.146+0000 7fe572800640 1 --2- 172.21.15.60:0/639371561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 0x7fe56c076040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009900 tx=0x7fe55c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:30.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.146+0000 7fe572800640 1 -- 172.21.15.60:0/639371561 shutdown_connections 2024-02-26T10:42:30.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.146+0000 7fe572800640 1 --2- 172.21.15.60:0/639371561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe56c076e60 0x7fe56c0772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.146+0000 7fe572800640 1 --2- 172.21.15.60:0/639371561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe56c077800 0x7fe56c0780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.146+0000 7fe572800640 1 --2- 172.21.15.60:0/639371561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 0x7fe56c076040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.146+0000 7fe572800640 1 -- 172.21.15.60:0/639371561 >> 172.21.15.60:0/639371561 conn(0x7fe56c0fea10 msgr2=0x7fe56c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:30.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.146+0000 7fe572800640 1 -- 172.21.15.60:0/639371561 shutdown_connections 2024-02-26T10:42:30.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.146+0000 7fe572800640 1 -- 172.21.15.60:0/639371561 wait complete. 2024-02-26T10:42:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.147+0000 7fe572800640 1 Processor -- start 2024-02-26T10:42:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.147+0000 7fe572800640 1 -- start start 2024-02-26T10:42:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe572800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 0x7fe56c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe572800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe56c076e60 0x7fe56c19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe572800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe56c077800 0x7fe56c1a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe572800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c113510 con 0x7fe56c077800 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe572800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c113390 con 0x7fe56c076e60 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe572800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c113690 con 0x7fe56c075c20 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe56b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 0x7fe56c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe56b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 0x7fe56c19dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:33602/0 (socket says 172.21.15.60:33602) 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe56b400640 1 -- 172.21.15.60:0/182386592 learned_addr learned my addr 172.21.15.60:0/182386592 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe56be00640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe56c077800 0x7fe56c1a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.148+0000 7fe563e00640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe56c076e60 0x7fe56c19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe56b400640 1 -- 172.21.15.60:0/182386592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe56c076e60 msgr2=0x7fe56c19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe56b400640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe56c076e60 0x7fe56c19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe56b400640 1 -- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe56c077800 msgr2=0x7fe56c1a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe56b400640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe56c077800 0x7fe56c1a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe56b400640 1 -- 172.21.15.60:0/182386592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c1a5850 con 0x7fe56c075c20 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe563e00640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe56c076e60 0x7fe56c19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe56b400640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 0x7fe56c19dc40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005ab0 tx=0x7fe55c02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe568c00640 1 -- 172.21.15.60:0/182386592 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c041070 con 0x7fe56c075c20 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56c1a5ae0 con 0x7fe56c075c20 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe568c00640 1 -- 172.21.15.60:0/182386592 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe55c004450 con 0x7fe56c075c20 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe568c00640 1 -- 172.21.15.60:0/182386592 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c003d70 con 0x7fe56c075c20 2024-02-26T10:42:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.149+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56c1a6020 con 0x7fe56c075c20 2024-02-26T10:42:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.151+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56c10fdf0 con 0x7fe56c075c20 2024-02-26T10:42:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.152+0000 7fe568c00640 1 -- 172.21.15.60:0/182386592 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe55c044070 con 0x7fe56c075c20 2024-02-26T10:42:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.154+0000 7fe568c00640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe5500a1e00 0x7fe5500a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.154+0000 7fe568c00640 1 -- 172.21.15.60:0/182386592 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe55c0ef550 con 0x7fe56c075c20 2024-02-26T10:42:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.154+0000 7fe563e00640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe5500a1e00 0x7fe5500a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.155+0000 7fe563e00640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe5500a1e00 0x7fe5500a42c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe554009fc0 tx=0x7fe554005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.158+0000 7fe568c00640 1 -- 172.21.15.60:0/182386592 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe55c0bcd80 con 0x7fe56c075c20 2024-02-26T10:42:30.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:30.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.60:6838/2551740513,v1:172.21.15.60:6839/2551740513] up:boot 2024-02-26T10:42:30.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: fsmap 10 up:standby 2024-02-26T10:42:30.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T10:42:30.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:30.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:30.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:30.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:30.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:30.281 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:30.281 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:30 smithi060 ceph-mon[31539]: Deploying daemon mds.a on smithi060 2024-02-26T10:42:30.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.409+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe56c02ce30 con 0x7fe56c075c20 2024-02-26T10:42:30.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.410+0000 7fe568c00640 1 -- 172.21.15.60:0/182386592 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7fe55c0bc680 con 0x7fe56c075c20 2024-02-26T10:42:30.408 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:30.408 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T10:42:30.408 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 11 2024-02-26T10:42:30.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe5500a1e00 msgr2=0x7fe5500a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe5500a1e00 0x7fe5500a42c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe554009fc0 tx=0x7fe554005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:30.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 msgr2=0x7fe56c19dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 0x7fe56c19dc40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005ab0 tx=0x7fe55c02c6c0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 shutdown_connections 2024-02-26T10:42:30.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe5500a1e00 0x7fe5500a42c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe56c076e60 0x7fe56c19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe56c077800 0x7fe56c1a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 --2- 172.21.15.60:0/182386592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56c075c20 0x7fe56c19dc40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 >> 172.21.15.60:0/182386592 conn(0x7fe56c0fea10 msgr2=0x7fe56c101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 shutdown_connections 2024-02-26T10:42:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:30.413+0000 7fe572800640 1 -- 172.21.15.60:0/182386592 wait complete. 2024-02-26T10:42:30.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:30.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.60:6838/2551740513,v1:172.21.15.60:6839/2551740513] up:boot 2024-02-26T10:42:30.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: fsmap 10 up:standby 2024-02-26T10:42:30.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T10:42:30.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:30.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:30.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:30.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:30.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:30.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:30.586 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:30 smithi113 ceph-mon[36126]: Deploying daemon mds.a on smithi060 2024-02-26T10:42:30.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.60:6838/2551740513,v1:172.21.15.60:6839/2551740513] up:boot 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: fsmap 10 up:standby 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:30.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:30 smithi100 ceph-mon[36051]: Deploying daemon mds.a on smithi060 2024-02-26T10:42:31.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:31 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/182386592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:31.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:31 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/182386592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:31.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:31 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/182386592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:32.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:32.477 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:32 smithi060 ceph-mon[31539]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:32.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:32 smithi113 ceph-mon[36126]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:32.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:32 smithi100 ceph-mon[36051]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:32.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.652+0000 7fca34e00640 1 -- 172.21.15.60:0/475086192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30076e60 msgr2=0x7fca300772c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.652+0000 7fca34e00640 1 --2- 172.21.15.60:0/475086192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30076e60 0x7fca300772c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009900 tx=0x7fca1c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:32.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.652+0000 7fca34e00640 1 -- 172.21.15.60:0/475086192 shutdown_connections 2024-02-26T10:42:32.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.652+0000 7fca34e00640 1 --2- 172.21.15.60:0/475086192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30076e60 0x7fca300772c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.652+0000 7fca34e00640 1 --2- 172.21.15.60:0/475086192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca30077800 0x7fca300780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.652+0000 7fca34e00640 1 --2- 172.21.15.60:0/475086192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30075c20 0x7fca30076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.652+0000 7fca34e00640 1 -- 172.21.15.60:0/475086192 >> 172.21.15.60:0/475086192 conn(0x7fca300fea10 msgr2=0x7fca30100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:32.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.652+0000 7fca34e00640 1 -- 172.21.15.60:0/475086192 shutdown_connections 2024-02-26T10:42:32.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.652+0000 7fca34e00640 1 -- 172.21.15.60:0/475086192 wait complete. 2024-02-26T10:42:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.653+0000 7fca34e00640 1 Processor -- start 2024-02-26T10:42:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.653+0000 7fca34e00640 1 -- start start 2024-02-26T10:42:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca34e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30075c20 0x7fca3019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca34e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca30076e60 0x7fca3019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca34e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30077800 0x7fca301a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca34e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca30113570 con 0x7fca30076e60 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca34e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca301133f0 con 0x7fca30075c20 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca34e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca301136f0 con 0x7fca30077800 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca2d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca30076e60 0x7fca3019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca2d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca30076e60 0x7fca3019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48756/0 (socket says 172.21.15.60:48756) 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca2d400640 1 -- 172.21.15.60:0/1332552683 learned_addr learned my addr 172.21.15.60:0/1332552683 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca2de00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30075c20 0x7fca3019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.654+0000 7fca2e800640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30077800 0x7fca301a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca2d400640 1 -- 172.21.15.60:0/1332552683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30077800 msgr2=0x7fca301a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca2d400640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30077800 0x7fca301a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca2d400640 1 -- 172.21.15.60:0/1332552683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30075c20 msgr2=0x7fca3019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca2d400640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30075c20 0x7fca3019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca2d400640 1 -- 172.21.15.60:0/1332552683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca301a57c0 con 0x7fca30076e60 2024-02-26T10:42:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca2de00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30075c20 0x7fca3019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca2d400640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca30076e60 0x7fca3019e0e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca26a00640 1 -- 172.21.15.60:0/1332552683 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c043020 con 0x7fca30076e60 2024-02-26T10:42:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca301a5a50 con 0x7fca30076e60 2024-02-26T10:42:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca26a00640 1 -- 172.21.15.60:0/1332552683 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca1c0054e0 con 0x7fca30076e60 2024-02-26T10:42:32.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca301a5fa0 con 0x7fca30076e60 2024-02-26T10:42:32.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.655+0000 7fca26a00640 1 -- 172.21.15.60:0/1332552683 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c003b70 con 0x7fca30076e60 2024-02-26T10:42:32.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.657+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3010ff40 con 0x7fca30076e60 2024-02-26T10:42:32.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.659+0000 7fca26a00640 1 -- 172.21.15.60:0/1332552683 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fca1c046020 con 0x7fca30076e60 2024-02-26T10:42:32.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.660+0000 7fca26a00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca140a1d80 0x7fca140a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.660+0000 7fca26a00640 1 -- 172.21.15.60:0/1332552683 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fca1c0ef9a0 con 0x7fca30076e60 2024-02-26T10:42:32.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.660+0000 7fca2de00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca140a1d80 0x7fca140a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.661+0000 7fca2de00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca140a1d80 0x7fca140a4240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca20009fc0 tx=0x7fca20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:32.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.665+0000 7fca26a00640 1 -- 172.21.15.60:0/1332552683 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca1c04d020 con 0x7fca30076e60 2024-02-26T10:42:32.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.911+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca3002d210 con 0x7fca30076e60 2024-02-26T10:42:32.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.912+0000 7fca26a00640 1 -- 172.21.15.60:0/1332552683 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7fca3002d210 con 0x7fca30076e60 2024-02-26T10:42:32.910 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:32.910 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T10:42:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 11 2024-02-26T10:42:32.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca140a1d80 msgr2=0x7fca140a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca140a1d80 0x7fca140a4240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca20009fc0 tx=0x7fca20005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca30076e60 msgr2=0x7fca3019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca30076e60 0x7fca3019e0e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c02c650 comp rx=0 tx=0).stop 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 shutdown_connections 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca140a1d80 0x7fca140a4240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30077800 0x7fca301a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca30076e60 0x7fca3019e0e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 --2- 172.21.15.60:0/1332552683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca30075c20 0x7fca3019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 >> 172.21.15.60:0/1332552683 conn(0x7fca300fea10 msgr2=0x7fca301000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 shutdown_connections 2024-02-26T10:42:32.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:32.915+0000 7fca34e00640 1 -- 172.21.15.60:0/1332552683 wait complete. 2024-02-26T10:42:33.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:33 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1332552683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:33.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:33 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1332552683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:33.617 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 10:42:33 smithi060 systemd[1]: Started Ceph mds.a for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:33.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:33 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1332552683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:34.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:34.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:34.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:34.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:34.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:34.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:34.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:34.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: Deploying daemon mds.b on smithi100 2024-02-26T10:42:34.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.60:6840/1960047882,v1:172.21.15.60:6841/1960047882] up:boot 2024-02-26T10:42:34.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: fsmap 11 up:standby 2024-02-26T10:42:34.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:34 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T10:42:34.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:34.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:34.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:34.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:34.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:34.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:34.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:34.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: Deploying daemon mds.b on smithi100 2024-02-26T10:42:34.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.60:6840/1960047882,v1:172.21.15.60:6841/1960047882] up:boot 2024-02-26T10:42:34.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: fsmap 11 up:standby 2024-02-26T10:42:34.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:34 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T10:42:34.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:34.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:34.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:34.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:34.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T10:42:34.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T10:42:34.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:34.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: Deploying daemon mds.b on smithi100 2024-02-26T10:42:34.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.60:6840/1960047882,v1:172.21.15.60:6841/1960047882] up:boot 2024-02-26T10:42:34.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: fsmap 11 up:standby 2024-02-26T10:42:34.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:34 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T10:42:34.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:35.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.170+0000 7f751b200640 1 -- 172.21.15.60:0/3847691851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 msgr2=0x7f751c1036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.170+0000 7f751b200640 1 --2- 172.21.15.60:0/3847691851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 0x7f751c1036d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f750c009900 tx=0x7f750c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.171+0000 7f751b200640 1 -- 172.21.15.60:0/3847691851 shutdown_connections 2024-02-26T10:42:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.171+0000 7f751b200640 1 --2- 172.21.15.60:0/3847691851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f751c1044a0 0x7f751c104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.171+0000 7f751b200640 1 --2- 172.21.15.60:0/3847691851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f751c104e60 0x7f751c10b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.171+0000 7f751b200640 1 --2- 172.21.15.60:0/3847691851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 0x7f751c1036d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.171+0000 7f751b200640 1 -- 172.21.15.60:0/3847691851 >> 172.21.15.60:0/3847691851 conn(0x7f751c0fe840 msgr2=0x7f751c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.171+0000 7f751b200640 1 -- 172.21.15.60:0/3847691851 shutdown_connections 2024-02-26T10:42:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.171+0000 7f751b200640 1 -- 172.21.15.60:0/3847691851 wait complete. 2024-02-26T10:42:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f751b200640 1 Processor -- start 2024-02-26T10:42:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f751b200640 1 -- start start 2024-02-26T10:42:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f751b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 0x7f751c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f751b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f751c1044a0 0x7f751c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f751b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f751c104e60 0x7f751c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f751b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c10ef90 con 0x7f751c1032b0 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f751b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c10ee10 con 0x7f751c104e60 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f751b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c10f110 con 0x7f751c1044a0 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f7519e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 0x7f751c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f7519e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 0x7f751c1995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43792/0 (socket says 172.21.15.60:43792) 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.172+0000 7f7519e00640 1 -- 172.21.15.60:0/3778792754 learned_addr learned my addr 172.21.15.60:0/3778792754 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f7519400640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f751c1044a0 0x7f751c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f751a800640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f751c104e60 0x7f751c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f7519e00640 1 -- 172.21.15.60:0/3778792754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f751c1044a0 msgr2=0x7f751c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f7519e00640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f751c1044a0 0x7f751c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f7519e00640 1 -- 172.21.15.60:0/3778792754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f751c104e60 msgr2=0x7f751c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f7519e00640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f751c104e60 0x7f751c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f7519e00640 1 -- 172.21.15.60:0/3778792754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c1a11d0 con 0x7f751c1032b0 2024-02-26T10:42:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f751a800640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f751c104e60 0x7f751c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f7519e00640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 0x7f751c1995b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f750c000c00 tx=0x7f750c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.173+0000 7f7512a00640 1 -- 172.21.15.60:0/3778792754 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c043020 con 0x7f751c1032b0 2024-02-26T10:42:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.174+0000 7f7512a00640 1 -- 172.21.15.60:0/3778792754 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f750c0054e0 con 0x7f751c1032b0 2024-02-26T10:42:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.174+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751c1a1460 con 0x7f751c1032b0 2024-02-26T10:42:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.174+0000 7f7512a00640 1 -- 172.21.15.60:0/3778792754 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c003c40 con 0x7f751c1032b0 2024-02-26T10:42:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.174+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751c1a19b0 con 0x7f751c1032b0 2024-02-26T10:42:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.175+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751c10a570 con 0x7f751c1032b0 2024-02-26T10:42:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.177+0000 7f7512a00640 1 -- 172.21.15.60:0/3778792754 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f750c046020 con 0x7f751c1032b0 2024-02-26T10:42:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.178+0000 7f7512a00640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75000a1d80 0x7f75000a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.178+0000 7f7512a00640 1 -- 172.21.15.60:0/3778792754 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f750c0efa10 con 0x7f751c1032b0 2024-02-26T10:42:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.178+0000 7f7519400640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75000a1d80 0x7f75000a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.179+0000 7f7519400640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75000a1d80 0x7f75000a4240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f751c19aac0 tx=0x7f7508005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:35.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.182+0000 7f7512a00640 1 -- 172.21.15.60:0/3778792754 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f750c04d020 con 0x7f751c1032b0 2024-02-26T10:42:35.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.425+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f751c02d050 con 0x7f751c1032b0 2024-02-26T10:42:35.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.427+0000 7f7512a00640 1 -- 172.21.15.60:0/3778792754 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9260 (secure 0 0 0) 0x7f750c0bd240 con 0x7f751c1032b0 2024-02-26T10:42:35.425 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:35.425 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-26T10:42:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 12 2024-02-26T10:42:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.429+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75000a1d80 msgr2=0x7f75000a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.429+0000 7f751b200640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75000a1d80 0x7f75000a4240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f751c19aac0 tx=0x7f7508005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.429+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 msgr2=0x7f751c1995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.429+0000 7f751b200640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 0x7f751c1995b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f750c000c00 tx=0x7f750c02c650 comp rx=0 tx=0).stop 2024-02-26T10:42:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.430+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 shutdown_connections 2024-02-26T10:42:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.430+0000 7f751b200640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75000a1d80 0x7f75000a4240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.430+0000 7f751b200640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f751c104e60 0x7f751c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.430+0000 7f751b200640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f751c1044a0 0x7f751c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.430+0000 7f751b200640 1 --2- 172.21.15.60:0/3778792754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f751c1032b0 0x7f751c1995b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.430+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 >> 172.21.15.60:0/3778792754 conn(0x7f751c0fe840 msgr2=0x7f751c107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.430+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 shutdown_connections 2024-02-26T10:42:35.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:35.430+0000 7f751b200640 1 -- 172.21.15.60:0/3778792754 wait complete. 2024-02-26T10:42:35.832 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:35 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3778792754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:35.833 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:35 smithi100 systemd[1]: Starting Ceph mds.b for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:42:35.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:35 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3778792754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:35.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:35 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3778792754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:36.090 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:35 smithi100 podman[66629]: 2024-02-26 10:42:35.737795656 +0000 UTC m=+0.017809477 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:42:36.090 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:35 smithi100 podman[66629]: 2024-02-26 10:42:35.857140707 +0000 UTC m=+0.137154557 container create ccbf93bac5b91c86b99630adf11d1bfac608cbfdc1592cc16244131625ce58b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-26T10:42:36.090 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:36 smithi100 podman[66629]: 2024-02-26 10:42:36.093294877 +0000 UTC m=+0.373308701 container init ccbf93bac5b91c86b99630adf11d1bfac608cbfdc1592cc16244131625ce58b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-26T10:42:36.378 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:36 smithi100 podman[66629]: 2024-02-26 10:42:36.096034347 +0000 UTC m=+0.376048156 container start ccbf93bac5b91c86b99630adf11d1bfac608cbfdc1592cc16244131625ce58b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD) 2024-02-26T10:42:36.379 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:36 smithi100 ceph-mds[66649]: -- 172.21.15.100:0/2789409792 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5645cf6bba40 con 0x5645cf68f000 2024-02-26T10:42:36.379 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:36 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b[66645]: starting mds.b at 2024-02-26T10:42:36.379 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:36 smithi100 bash[66629]: ccbf93bac5b91c86b99630adf11d1bfac608cbfdc1592cc16244131625ce58b4 2024-02-26T10:42:36.379 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:36 smithi100 systemd[1]: Started Ceph mds.b for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:42:36.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:36 smithi113 ceph-mon[36126]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:36.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:36 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:36 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:36 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:36 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:36 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:36 smithi060 ceph-mon[31539]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:36.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:36 smithi100 ceph-mon[36051]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:36.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:36 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:36 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:36 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:36 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:36.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:36 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:37.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:37.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.679+0000 7f8329c00640 1 -- 172.21.15.60:0/2694905846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8324105050 msgr2=0x7f832410b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:37.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.679+0000 7f8329c00640 1 --2- 172.21.15.60:0/2694905846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8324105050 0x7f832410b8e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f831c009be0 tx=0x7f831c02bef0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.680+0000 7f8329c00640 1 -- 172.21.15.60:0/2694905846 shutdown_connections 2024-02-26T10:42:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.680+0000 7f8329c00640 1 --2- 172.21.15.60:0/2694905846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8324104690 0x7f8324104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.680+0000 7f8329c00640 1 --2- 172.21.15.60:0/2694905846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8324105050 0x7f832410b8e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.680+0000 7f8329c00640 1 --2- 172.21.15.60:0/2694905846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83241034a0 0x7f83241038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.680+0000 7f8329c00640 1 -- 172.21.15.60:0/2694905846 >> 172.21.15.60:0/2694905846 conn(0x7f83240fea10 msgr2=0x7f8324100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.680+0000 7f8329c00640 1 -- 172.21.15.60:0/2694905846 shutdown_connections 2024-02-26T10:42:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.680+0000 7f8329c00640 1 -- 172.21.15.60:0/2694905846 wait complete. 2024-02-26T10:42:37.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.681+0000 7f8329c00640 1 Processor -- start 2024-02-26T10:42:37.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.681+0000 7f8329c00640 1 -- start start 2024-02-26T10:42:37.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.681+0000 7f8329c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241034a0 0x7f83241997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:37.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.681+0000 7f8329c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8324104690 0x7f8324199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.681+0000 7f8329c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8324105050 0x7f83241a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.681+0000 7f8329c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832410f150 con 0x7f8324105050 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.681+0000 7f8329c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832410efd0 con 0x7f8324104690 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.681+0000 7f8329c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832410f2d0 con 0x7f83241034a0 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241034a0 0x7f83241997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8324104690 0x7f8324199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241034a0 0x7f83241997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:43072/0 (socket says 172.21.15.60:43072) 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322a00640 1 -- 172.21.15.60:0/3785737384 learned_addr learned my addr 172.21.15.60:0/3785737384 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8323400640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8324105050 0x7f83241a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322000640 1 -- 172.21.15.60:0/3785737384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241034a0 msgr2=0x7f83241997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322000640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241034a0 0x7f83241997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322000640 1 -- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8324105050 msgr2=0x7f83241a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322000640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8324105050 0x7f83241a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322000640 1 -- 172.21.15.60:0/3785737384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83241a13d0 con 0x7f8324104690 2024-02-26T10:42:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322a00640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241034a0 0x7f83241997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.682+0000 7f8322000640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8324104690 0x7f8324199cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f831000e9f0 tx=0x7f831000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.683+0000 7f831b400640 1 -- 172.21.15.60:0/3785737384 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831000ccc0 con 0x7f8324104690 2024-02-26T10:42:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.683+0000 7f831b400640 1 -- 172.21.15.60:0/3785737384 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8310010910 con 0x7f8324104690 2024-02-26T10:42:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.683+0000 7f831b400640 1 -- 172.21.15.60:0/3785737384 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310018960 con 0x7f8324104690 2024-02-26T10:42:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.683+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83241a16c0 con 0x7f8324104690 2024-02-26T10:42:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.683+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83241a1bc0 con 0x7f8324104690 2024-02-26T10:42:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.684+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832410a760 con 0x7f8324104690 2024-02-26T10:42:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.686+0000 7f831b400640 1 -- 172.21.15.60:0/3785737384 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8310010ac0 con 0x7f8324104690 2024-02-26T10:42:37.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.687+0000 7f831b400640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83080a1d80 0x7f83080a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:37.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.687+0000 7f831b400640 1 -- 172.21.15.60:0/3785737384 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f83100cdd30 con 0x7f8324104690 2024-02-26T10:42:37.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.688+0000 7f8322a00640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83080a1d80 0x7f83080a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:37.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.688+0000 7f8322a00640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83080a1d80 0x7f83080a4240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8314011490 tx=0x7f8314009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:37.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.691+0000 7f831b400640 1 -- 172.21.15.60:0/3785737384 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f831009af10 con 0x7f8324104690 2024-02-26T10:42:37.693 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:37 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] up:boot 2024-02-26T10:42:37.693 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:37 smithi100 ceph-mon[36051]: fsmap 12 up:standby 2024-02-26T10:42:37.693 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T10:42:37.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:42:37.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:37.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:37 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] up:boot 2024-02-26T10:42:37.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:37 smithi113 ceph-mon[36126]: fsmap 12 up:standby 2024-02-26T10:42:37.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T10:42:37.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:42:37.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:37.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:37 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] up:boot 2024-02-26T10:42:37.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:37 smithi060 ceph-mon[31539]: fsmap 12 up:standby 2024-02-26T10:42:37.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T10:42:37.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:42:37.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:37.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.937+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f832402ce30 con 0x7f8324104690 2024-02-26T10:42:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.938+0000 7f831b400640 1 -- 172.21.15.60:0/3785737384 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10055 (secure 0 0 0) 0x7f831009efb0 con 0x7f8324104690 2024-02-26T10:42:37.936 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:37.937 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-02-26T10:42:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 13 2024-02-26T10:42:37.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83080a1d80 msgr2=0x7f83080a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:37.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83080a1d80 0x7f83080a4240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8314011490 tx=0x7f8314009260 comp rx=0 tx=0).stop 2024-02-26T10:42:37.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8324104690 msgr2=0x7f8324199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:37.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8324104690 0x7f8324199cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f831000e9f0 tx=0x7f831000eeb0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 shutdown_connections 2024-02-26T10:42:37.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83080a1d80 0x7f83080a4240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8324104690 0x7f8324199cf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8324105050 0x7f83241a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 --2- 172.21.15.60:0/3785737384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241034a0 0x7f83241997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.941+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 >> 172.21.15.60:0/3785737384 conn(0x7f83240fea10 msgr2=0x7f8324107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:37.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.942+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 shutdown_connections 2024-02-26T10:42:37.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:37.942+0000 7f8329c00640 1 -- 172.21.15.60:0/3785737384 wait complete. 2024-02-26T10:42:37.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.171+0000 7fb062400640 1 -- 172.21.15.60:0/2397731581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb05c1078a0 msgr2=0x7fb05c107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.171+0000 7fb062400640 1 --2- 172.21.15.60:0/2397731581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb05c1078a0 0x7fb05c107cc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009900 tx=0x7fb04c02bc50 comp rx=0 tx=0).stop 2024-02-26T10:42:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.172+0000 7fb062400640 1 -- 172.21.15.60:0/2397731581 shutdown_connections 2024-02-26T10:42:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.172+0000 7fb062400640 1 --2- 172.21.15.60:0/2397731581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c109450 0x7fb05c10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.172+0000 7fb062400640 1 --2- 172.21.15.60:0/2397731581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c108a90 0x7fb05c108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.172+0000 7fb062400640 1 --2- 172.21.15.60:0/2397731581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb05c1078a0 0x7fb05c107cc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.172+0000 7fb062400640 1 -- 172.21.15.60:0/2397731581 >> 172.21.15.60:0/2397731581 conn(0x7fb05c076870 msgr2=0x7fb05c078c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.172+0000 7fb062400640 1 -- 172.21.15.60:0/2397731581 shutdown_connections 2024-02-26T10:42:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.172+0000 7fb062400640 1 -- 172.21.15.60:0/2397731581 wait complete. 2024-02-26T10:42:38.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.173+0000 7fb062400640 1 Processor -- start 2024-02-26T10:42:38.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.173+0000 7fb062400640 1 -- start start 2024-02-26T10:42:38.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.173+0000 7fb062400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1078a0 0x7fb05c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.173+0000 7fb062400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c108a90 0x7fb05c19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.173+0000 7fb062400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb05c109450 0x7fb05c1a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.173+0000 7fb062400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c113540 con 0x7fb05c108a90 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.173+0000 7fb062400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c1133c0 con 0x7fb05c1078a0 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.173+0000 7fb062400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c1136c0 con 0x7fb05c109450 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1078a0 0x7fb05c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1078a0 0x7fb05c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52838/0 (socket says 172.21.15.60:52838) 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05b400640 1 -- 172.21.15.60:0/1094087419 learned_addr learned my addr 172.21.15.60:0/1094087419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb052a00640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c108a90 0x7fb05c19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05be00640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb05c109450 0x7fb05c1a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05b400640 1 -- 172.21.15.60:0/1094087419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb05c109450 msgr2=0x7fb05c1a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05b400640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb05c109450 0x7fb05c1a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05b400640 1 -- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c108a90 msgr2=0x7fb05c19e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05b400640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c108a90 0x7fb05c19e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05b400640 1 -- 172.21.15.60:0/1094087419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c1a57a0 con 0x7fb05c1078a0 2024-02-26T10:42:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb052a00640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c108a90 0x7fb05c19e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.174+0000 7fb05b400640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1078a0 0x7fb05c19db80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb04c02c260 tx=0x7fb04c02c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.175+0000 7fb058c00640 1 -- 172.21.15.60:0/1094087419 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c041070 con 0x7fb05c1078a0 2024-02-26T10:42:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.175+0000 7fb058c00640 1 -- 172.21.15.60:0/1094087419 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb04c004450 con 0x7fb05c1078a0 2024-02-26T10:42:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.175+0000 7fb058c00640 1 -- 172.21.15.60:0/1094087419 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c034440 con 0x7fb05c1078a0 2024-02-26T10:42:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.175+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05c1a5a30 con 0x7fb05c1078a0 2024-02-26T10:42:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.175+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05c1a5f30 con 0x7fb05c1078a0 2024-02-26T10:42:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.176+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05c10eb60 con 0x7fb05c1078a0 2024-02-26T10:42:38.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.178+0000 7fb058c00640 1 -- 172.21.15.60:0/1094087419 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb04c044070 con 0x7fb05c1078a0 2024-02-26T10:42:38.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.179+0000 7fb058c00640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0400a1e00 0x7fb0400a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:38.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.179+0000 7fb058c00640 1 -- 172.21.15.60:0/1094087419 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb04c0ef540 con 0x7fb05c1078a0 2024-02-26T10:42:38.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.180+0000 7fb052a00640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0400a1e00 0x7fb0400a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:38.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.180+0000 7fb052a00640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0400a1e00 0x7fb0400a42c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb05c19f090 tx=0x7fb044009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:38.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.183+0000 7fb058c00640 1 -- 172.21.15.60:0/1094087419 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb04c0bcd70 con 0x7fb05c1078a0 2024-02-26T10:42:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.439+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb05c061e60 con 0x7fb05c1078a0 2024-02-26T10:42:38.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.440+0000 7fb058c00640 1 -- 172.21.15.60:0/1094087419 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18465 (secure 0 0 0) 0x7fb04c0bc670 con 0x7fb05c1078a0 2024-02-26T10:42:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:38.438 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:41:44.354883+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0400a1e00 msgr2=0x7fb0400a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0400a1e00 0x7fb0400a42c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb05c19f090 tx=0x7fb044009500 comp rx=0 tx=0).stop 2024-02-26T10:42:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1078a0 msgr2=0x7fb05c19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1078a0 0x7fb05c19db80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb04c02c260 tx=0x7fb04c02c670 comp rx=0 tx=0).stop 2024-02-26T10:42:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 shutdown_connections 2024-02-26T10:42:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0400a1e00 0x7fb0400a42c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb05c109450 0x7fb05c1a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c108a90 0x7fb05c19e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 --2- 172.21.15.60:0/1094087419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb05c1078a0 0x7fb05c19db80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 >> 172.21.15.60:0/1094087419 conn(0x7fb05c076870 msgr2=0x7fb05c10bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 shutdown_connections 2024-02-26T10:42:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.443+0000 7fb062400640 1 -- 172.21.15.60:0/1094087419 wait complete. 2024-02-26T10:42:38.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:42:38.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:42:38.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:38 smithi060 ceph-mon[31539]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:38.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:38 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3785737384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:38.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:38 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1094087419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.668+0000 7f9510a00640 1 -- 172.21.15.60:0/1838489293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 msgr2=0x7f950c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.668+0000 7f9510a00640 1 --2- 172.21.15.60:0/1838489293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 0x7f950c076040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009900 tx=0x7f94fc02bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.668+0000 7f9510a00640 1 -- 172.21.15.60:0/1838489293 shutdown_connections 2024-02-26T10:42:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.668+0000 7f9510a00640 1 --2- 172.21.15.60:0/1838489293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950c077800 0x7f950c0780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.668+0000 7f9510a00640 1 --2- 172.21.15.60:0/1838489293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f950c076e60 0x7f950c0772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.668+0000 7f9510a00640 1 --2- 172.21.15.60:0/1838489293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 0x7f950c076040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.668+0000 7f9510a00640 1 -- 172.21.15.60:0/1838489293 >> 172.21.15.60:0/1838489293 conn(0x7f950c0fea10 msgr2=0x7f950c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.668+0000 7f9510a00640 1 -- 172.21.15.60:0/1838489293 shutdown_connections 2024-02-26T10:42:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.668+0000 7f9510a00640 1 -- 172.21.15.60:0/1838489293 wait complete. 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.669+0000 7f9510a00640 1 Processor -- start 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9510a00640 1 -- start start 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9510a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 0x7f950c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9510a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950c076e60 0x7f950c19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9510a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f950c077800 0x7f950c1a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9510a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c113570 con 0x7f950c075c20 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9510a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c1133f0 con 0x7f950c076e60 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9510a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c1136f0 con 0x7f950c077800 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9509e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 0x7f950c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9509e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 0x7f950c19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43854/0 (socket says 172.21.15.60:43854) 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9509e00640 1 -- 172.21.15.60:0/96981294 learned_addr learned my addr 172.21.15.60:0/96981294 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f950a800640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f950c077800 0x7f950c1a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.670+0000 7f9509400640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950c076e60 0x7f950c19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9509400640 1 -- 172.21.15.60:0/96981294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f950c077800 msgr2=0x7f950c1a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9509400640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f950c077800 0x7f950c1a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9509400640 1 -- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 msgr2=0x7f950c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9509400640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 0x7f950c19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9509400640 1 -- 172.21.15.60:0/96981294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c1a57c0 con 0x7f950c076e60 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9509e00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 0x7f950c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f950a800640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f950c077800 0x7f950c1a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9509400640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950c076e60 0x7f950c19e0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94f800e9f0 tx=0x7f94f800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9502a00640 1 -- 172.21.15.60:0/96981294 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f800ccc0 con 0x7f950c076e60 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9502a00640 1 -- 172.21.15.60:0/96981294 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94f8005020 con 0x7f950c076e60 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9502a00640 1 -- 172.21.15.60:0/96981294 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8004a00 con 0x7f950c076e60 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950c1a5ab0 con 0x7f950c076e60 2024-02-26T10:42:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.671+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950c1a5fb0 con 0x7f950c076e60 2024-02-26T10:42:38.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.673+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950c10fe10 con 0x7f950c076e60 2024-02-26T10:42:38.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.674+0000 7f9502a00640 1 -- 172.21.15.60:0/96981294 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f94f80051d0 con 0x7f950c076e60 2024-02-26T10:42:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.676+0000 7f9502a00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94f00a1e00 0x7f94f00a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.676+0000 7f9502a00640 1 -- 172.21.15.60:0/96981294 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f94f80cd810 con 0x7f950c076e60 2024-02-26T10:42:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.676+0000 7f9509e00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94f00a1e00 0x7f94f00a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.677+0000 7f9509e00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94f00a1e00 0x7f94f00a42c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94fc02c8e0 tx=0x7f94fc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.679+0000 7f9502a00640 1 -- 172.21.15.60:0/96981294 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94f809b080 con 0x7f950c076e60 2024-02-26T10:42:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:38 smithi113 ceph-mon[36126]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:38 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3785737384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:38 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1094087419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:38 smithi100 ceph-mon[36051]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:38.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:38 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3785737384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:38.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:38.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:38 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1094087419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:38.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.886+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f950c061e60 con 0x7f950c076e60 2024-02-26T10:42:38.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.887+0000 7f9502a00640 1 -- 172.21.15.60:0/96981294 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94f809a980 con 0x7f950c076e60 2024-02-26T10:42:38.884 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:42:38.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.889+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94f00a1e00 msgr2=0x7f94f00a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.889+0000 7f9510a00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94f00a1e00 0x7f94f00a42c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94fc02c8e0 tx=0x7f94fc02df90 comp rx=0 tx=0).stop 2024-02-26T10:42:38.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.889+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950c076e60 msgr2=0x7f950c19e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:38.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.889+0000 7f9510a00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950c076e60 0x7f950c19e0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94f800e9f0 tx=0x7f94f800edf0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.890+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 shutdown_connections 2024-02-26T10:42:38.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.890+0000 7f9510a00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94f00a1e00 0x7f94f00a42c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.890+0000 7f9510a00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950c076e60 0x7f950c19e0e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.890+0000 7f9510a00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f950c077800 0x7f950c1a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.890+0000 7f9510a00640 1 --2- 172.21.15.60:0/96981294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f950c075c20 0x7f950c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:38.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.890+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 >> 172.21.15.60:0/96981294 conn(0x7f950c0fea10 msgr2=0x7f950c101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:38.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.890+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 shutdown_connections 2024-02-26T10:42:38.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:38.890+0000 7f9510a00640 1 -- 172.21.15.60:0/96981294 wait complete. 2024-02-26T10:42:38.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T10:42:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.113+0000 7f6c45600640 1 -- 172.21.15.60:0/4177969877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c40106f80 msgr2=0x7f6c401093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.113+0000 7f6c45600640 1 --2- 172.21.15.60:0/4177969877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c40106f80 0x7f6c401093d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009be0 tx=0x7f6c3802bef0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.113+0000 7f6c45600640 1 -- 172.21.15.60:0/4177969877 shutdown_connections 2024-02-26T10:42:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.113+0000 7f6c45600640 1 --2- 172.21.15.60:0/4177969877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c40104650 0x7f6c40106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.113+0000 7f6c45600640 1 --2- 172.21.15.60:0/4177969877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c40106f80 0x7f6c401093d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.113+0000 7f6c45600640 1 --2- 172.21.15.60:0/4177969877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c40101d20 0x7f6c40104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.113+0000 7f6c45600640 1 -- 172.21.15.60:0/4177969877 >> 172.21.15.60:0/4177969877 conn(0x7f6c400fb710 msgr2=0x7f6c400fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.113+0000 7f6c45600640 1 -- 172.21.15.60:0/4177969877 shutdown_connections 2024-02-26T10:42:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.113+0000 7f6c45600640 1 -- 172.21.15.60:0/4177969877 wait complete. 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.114+0000 7f6c45600640 1 Processor -- start 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c45600640 1 -- start start 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c45600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c40101d20 0x7f6c401997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c45600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c40104650 0x7f6c40199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c45600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c40106f80 0x7f6c401a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c45600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4010cf80 con 0x7f6c40101d20 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c45600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4010ce00 con 0x7f6c40106f80 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c45600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4010d100 con 0x7f6c40104650 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c3ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c40101d20 0x7f6c401997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c3ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c40101d20 0x7f6c401997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43862/0 (socket says 172.21.15.60:43862) 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c3ea00640 1 -- 172.21.15.60:0/2744516235 learned_addr learned my addr 172.21.15.60:0/2744516235 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c3e000640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c40104650 0x7f6c40199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.115+0000 7f6c3f400640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c40106f80 0x7f6c401a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c3ea00640 1 -- 172.21.15.60:0/2744516235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c40104650 msgr2=0x7f6c40199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c3ea00640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c40104650 0x7f6c40199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c3ea00640 1 -- 172.21.15.60:0/2744516235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c40106f80 msgr2=0x7f6c401a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c3ea00640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c40106f80 0x7f6c401a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c3ea00640 1 -- 172.21.15.60:0/2744516235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c401a13d0 con 0x7f6c40101d20 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c3ea00640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c40101d20 0x7f6c401997b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000e9c0 tx=0x7f6c3000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c37400640 1 -- 172.21.15.60:0/2744516235 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000cde0 con 0x7f6c40101d20 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c401a16c0 con 0x7f6c40101d20 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c37400640 1 -- 172.21.15.60:0/2744516235 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c300035f0 con 0x7f6c40101d20 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.116+0000 7f6c37400640 1 -- 172.21.15.60:0/2744516235 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30010650 con 0x7f6c40101d20 2024-02-26T10:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.117+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c401a1c10 con 0x7f6c40101d20 2024-02-26T10:42:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.118+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c40108a70 con 0x7f6c40101d20 2024-02-26T10:42:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.120+0000 7f6c37400640 1 -- 172.21.15.60:0/2744516235 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6c300037a0 con 0x7f6c40101d20 2024-02-26T10:42:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.122+0000 7f6c37400640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c240a1d80 0x7f6c240a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.122+0000 7f6c37400640 1 -- 172.21.15.60:0/2744516235 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6c300cdc80 con 0x7f6c40101d20 2024-02-26T10:42:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.122+0000 7f6c3e000640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c240a1d80 0x7f6c240a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.123+0000 7f6c3e000640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c240a1d80 0x7f6c240a4240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c4019acc0 tx=0x7f6c2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.125+0000 7f6c37400640 1 -- 172.21.15.60:0/2744516235 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c3009b4b0 con 0x7f6c40101d20 2024-02-26T10:42:39.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.358+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6c40061e60 con 0x7f6c40101d20 2024-02-26T10:42:39.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.359+0000 7f6c37400640 1 -- 172.21.15.60:0/2744516235 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6c3009adb0 con 0x7f6c40101d20 2024-02-26T10:42:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-02-26T10:42:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:42:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-02-26T10:42:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-02-26T10:42:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:42:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-02-26T10:42:39.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c240a1d80 msgr2=0x7f6c240a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c240a1d80 0x7f6c240a4240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c4019acc0 tx=0x7f6c2c005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c40101d20 msgr2=0x7f6c401997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c40101d20 0x7f6c401997b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000e9c0 tx=0x7f6c3000ee80 comp rx=0 tx=0).stop 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 shutdown_connections 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c240a1d80 0x7f6c240a4240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c40106f80 0x7f6c401a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c40104650 0x7f6c40199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 --2- 172.21.15.60:0/2744516235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c40101d20 0x7f6c401997b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 >> 172.21.15.60:0/2744516235 conn(0x7f6c400fb710 msgr2=0x7f6c401013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:39.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 shutdown_connections 2024-02-26T10:42:39.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.362+0000 7f6c45600640 1 -- 172.21.15.60:0/2744516235 wait complete. 2024-02-26T10:42:39.379 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-26T10:42:39.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-26T10:42:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.598+0000 7f0a45600640 1 -- 172.21.15.60:0/1636184360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a40106e30 msgr2=0x7f0a40109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.598+0000 7f0a45600640 1 --2- 172.21.15.60:0/1636184360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a40106e30 0x7f0a40109280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009ca0 tx=0x7f0a3802c0f0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.598+0000 7f0a45600640 1 -- 172.21.15.60:0/1636184360 shutdown_connections 2024-02-26T10:42:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.598+0000 7f0a45600640 1 --2- 172.21.15.60:0/1636184360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a40106e30 0x7f0a40109280 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.598+0000 7f0a45600640 1 --2- 172.21.15.60:0/1636184360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a40104500 0x7f0a401068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.598+0000 7f0a45600640 1 --2- 172.21.15.60:0/1636184360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a40101bd0 0x7f0a40103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.598+0000 7f0a45600640 1 -- 172.21.15.60:0/1636184360 >> 172.21.15.60:0/1636184360 conn(0x7f0a400fb5e0 msgr2=0x7f0a400fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.598+0000 7f0a45600640 1 -- 172.21.15.60:0/1636184360 shutdown_connections 2024-02-26T10:42:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.598+0000 7f0a45600640 1 -- 172.21.15.60:0/1636184360 wait complete. 2024-02-26T10:42:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.599+0000 7f0a45600640 1 Processor -- start 2024-02-26T10:42:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.599+0000 7f0a45600640 1 -- start start 2024-02-26T10:42:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a45600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a40101bd0 0x7f0a40101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a45600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a40104500 0x7f0a400ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a45600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a40106e30 0x7f0a401000c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a45600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4010cde0 con 0x7f0a40106e30 2024-02-26T10:42:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a45600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4010cc60 con 0x7f0a40104500 2024-02-26T10:42:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a45600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4010cf60 con 0x7f0a40101bd0 2024-02-26T10:42:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a3fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a40101bd0 0x7f0a40101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a37e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a40104500 0x7f0a400ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a3fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a40101bd0 0x7f0a40101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:43140/0 (socket says 172.21.15.60:43140) 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a44c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a40106e30 0x7f0a401000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.600+0000 7f0a3fe00640 1 -- 172.21.15.60:0/2666011347 learned_addr learned my addr 172.21.15.60:0/2666011347 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a44c00640 1 -- 172.21.15.60:0/2666011347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a40101bd0 msgr2=0x7f0a40101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a44c00640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a40101bd0 0x7f0a40101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a44c00640 1 -- 172.21.15.60:0/2666011347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a40104500 msgr2=0x7f0a400ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a44c00640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a40104500 0x7f0a400ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a44c00640 1 -- 172.21.15.60:0/2666011347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a40100950 con 0x7f0a40106e30 2024-02-26T10:42:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a3fe00640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a40101bd0 0x7f0a40101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a44c00640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a40106e30 0x7f0a401000c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a3802c630 tx=0x7f0a38032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a3d600640 1 -- 172.21.15.60:0/2666011347 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38041020 con 0x7f0a40106e30 2024-02-26T10:42:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a401a14f0 con 0x7f0a40106e30 2024-02-26T10:42:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.601+0000 7f0a3d600640 1 -- 172.21.15.60:0/2666011347 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a38034cd0 con 0x7f0a40106e30 2024-02-26T10:42:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.602+0000 7f0a3d600640 1 -- 172.21.15.60:0/2666011347 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38039040 con 0x7f0a40106e30 2024-02-26T10:42:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.602+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a401a1a60 con 0x7f0a40106e30 2024-02-26T10:42:39.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.603+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a400fd0d0 con 0x7f0a40106e30 2024-02-26T10:42:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.605+0000 7f0a3d600640 1 -- 172.21.15.60:0/2666011347 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0a38044020 con 0x7f0a40106e30 2024-02-26T10:42:39.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.607+0000 7f0a3d600640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0a240a1d80 0x7f0a240a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.608+0000 7f0a3d600640 1 -- 172.21.15.60:0/2666011347 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0a380ef730 con 0x7f0a40106e30 2024-02-26T10:42:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.608+0000 7f0a3fe00640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0a240a1d80 0x7f0a240a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.608+0000 7f0a3fe00640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0a240a1d80 0x7f0a240a4240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0a30011550 tx=0x7f0a30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.612+0000 7f0a3d600640 1 -- 172.21.15.60:0/2666011347 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a3804b020 con 0x7f0a40106e30 2024-02-26T10:42:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:39.814+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0a40061e60 con 0x7f0a40106e30 2024-02-26T10:42:39.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:39 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/96981294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:42:39.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:39 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2744516235' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T10:42:39.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:39 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/96981294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:42:39.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:39 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2744516235' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T10:42:39.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:39 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/96981294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:42:39.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:39 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2744516235' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T10:42:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.565+0000 7f0a3d600640 1 -- 172.21.15.60:0/2666011347 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0a380bcff0 con 0x7f0a40106e30 2024-02-26T10:42:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-02-26T10:42:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0a240a1d80 msgr2=0x7f0a240a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0a240a1d80 0x7f0a240a4240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0a30011550 tx=0x7f0a30005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a40106e30 msgr2=0x7f0a401000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a40106e30 0x7f0a401000c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a3802c630 tx=0x7f0a38032f20 comp rx=0 tx=0).stop 2024-02-26T10:42:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 shutdown_connections 2024-02-26T10:42:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0a240a1d80 0x7f0a240a4240 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a40104500 0x7f0a400ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a40106e30 0x7f0a401000c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 --2- 172.21.15.60:0/2666011347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a40101bd0 0x7f0a40101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.570+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 >> 172.21.15.60:0/2666011347 conn(0x7f0a400fb5e0 msgr2=0x7f0a40102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.571+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 shutdown_connections 2024-02-26T10:42:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.571+0000 7f0a45600640 1 -- 172.21.15.60:0/2666011347 wait complete. 2024-02-26T10:42:40.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-02-26T10:42:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.798+0000 7fcdace00640 1 -- 172.21.15.60:0/2391523852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 msgr2=0x7fcda8106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.798+0000 7fcdace00640 1 --2- 172.21.15.60:0/2391523852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 0x7fcda8106910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009900 tx=0x7fcd9402bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.798+0000 7fcdace00640 1 -- 172.21.15.60:0/2391523852 shutdown_connections 2024-02-26T10:42:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.798+0000 7fcdace00640 1 --2- 172.21.15.60:0/2391523852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8106e50 0x7fcda81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.798+0000 7fcdace00640 1 --2- 172.21.15.60:0/2391523852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 0x7fcda8106910 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.798+0000 7fcdace00640 1 --2- 172.21.15.60:0/2391523852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8101bf0 0x7fcda8103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.798+0000 7fcdace00640 1 -- 172.21.15.60:0/2391523852 >> 172.21.15.60:0/2391523852 conn(0x7fcda80fb5e0 msgr2=0x7fcda80fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.798+0000 7fcdace00640 1 -- 172.21.15.60:0/2391523852 shutdown_connections 2024-02-26T10:42:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.798+0000 7fcdace00640 1 -- 172.21.15.60:0/2391523852 wait complete. 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.799+0000 7fcdace00640 1 Processor -- start 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.799+0000 7fcdace00640 1 -- start start 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.799+0000 7fcdace00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8101bf0 0x7fcda81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.799+0000 7fcdace00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 0x7fcda8199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.799+0000 7fcdace00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8106e50 0x7fcda81a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.799+0000 7fcdace00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda810ce50 con 0x7fcda8101bf0 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.799+0000 7fcdace00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda810ccd0 con 0x7fcda8106e50 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.799+0000 7fcdace00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda810cfd0 con 0x7fcda8104520 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 0x7fcda8199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 0x7fcda8199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:43168/0 (socket says 172.21.15.60:43168) 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8106e50 0x7fcda81a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda6a00640 1 -- 172.21.15.60:0/3136938879 learned_addr learned my addr 172.21.15.60:0/3136938879 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda7400640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8101bf0 0x7fcda81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda6a00640 1 -- 172.21.15.60:0/3136938879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8106e50 msgr2=0x7fcda81a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda6a00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8106e50 0x7fcda81a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda6a00640 1 -- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8101bf0 msgr2=0x7fcda81996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda6a00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8101bf0 0x7fcda81996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda6a00640 1 -- 172.21.15.60:0/3136938879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda81a12c0 con 0x7fcda8104520 2024-02-26T10:42:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda7e00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8106e50 0x7fcda81a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.800+0000 7fcda6a00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 0x7fcda8199be0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcd94000c00 tx=0x7fcd9402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.801+0000 7fcd9fe00640 1 -- 172.21.15.60:0/3136938879 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd94043020 con 0x7fcda8104520 2024-02-26T10:42:40.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.801+0000 7fcd9fe00640 1 -- 172.21.15.60:0/3136938879 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd940054e0 con 0x7fcda8104520 2024-02-26T10:42:40.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.801+0000 7fcd9fe00640 1 -- 172.21.15.60:0/3136938879 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd94003c40 con 0x7fcda8104520 2024-02-26T10:42:40.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.801+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda81a1550 con 0x7fcda8104520 2024-02-26T10:42:40.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.801+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda81a1af0 con 0x7fcda8104520 2024-02-26T10:42:40.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.802+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda81089a0 con 0x7fcda8104520 2024-02-26T10:42:40.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.805+0000 7fcd9fe00640 1 -- 172.21.15.60:0/3136938879 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fcd94046020 con 0x7fcda8104520 2024-02-26T10:42:40.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.806+0000 7fcd9fe00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd8c0a1e00 0x7fcd8c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.806+0000 7fcd9fe00640 1 -- 172.21.15.60:0/3136938879 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fcd940efa70 con 0x7fcda8104520 2024-02-26T10:42:40.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.806+0000 7fcda7400640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd8c0a1e00 0x7fcd8c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.807+0000 7fcda7400640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd8c0a1e00 0x7fcd8c0a42c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcd980095f0 tx=0x7fcd98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:40.810+0000 7fcd9fe00640 1 -- 172.21.15.60:0/3136938879 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd9404d020 con 0x7fcda8104520 2024-02-26T10:42:40.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:40 smithi113 ceph-mon[36126]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:40.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:40.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:40.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:40.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:42:40.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:40.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:40 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2666011347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T10:42:40.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:42:40.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:40 smithi060 ceph-mon[31539]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:40.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:40.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:40.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:40.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:42:40.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:40.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:40 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2666011347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T10:42:40.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:42:40.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:40 smithi100 ceph-mon[36051]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:40.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:40.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:40.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:40.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:42:40.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:40.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:40 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2666011347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T10:42:40.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:42:41.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.011+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fcda802ce30 con 0x7fcda8104520 2024-02-26T10:42:41.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.595+0000 7fcd9fe00640 1 -- 172.21.15.60:0/3136938879 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) v1 ==== 171+0+0 (secure 0 0 0) 0x7fcd940bd130 con 0x7fcda8104520 2024-02-26T10:42:41.594 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-02-26T10:42:41.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.601+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd8c0a1e00 msgr2=0x7fcd8c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.601+0000 7fcdace00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd8c0a1e00 0x7fcd8c0a42c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcd980095f0 tx=0x7fcd98005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:41.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.601+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 msgr2=0x7fcda8199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.601+0000 7fcdace00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 0x7fcda8199be0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcd94000c00 tx=0x7fcd9402c650 comp rx=0 tx=0).stop 2024-02-26T10:42:41.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.602+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 shutdown_connections 2024-02-26T10:42:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.602+0000 7fcdace00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcd8c0a1e00 0x7fcd8c0a42c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.602+0000 7fcdace00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8106e50 0x7fcda81a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.602+0000 7fcdace00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda8104520 0x7fcda8199be0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.602+0000 7fcdace00640 1 --2- 172.21.15.60:0/3136938879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda8101bf0 0x7fcda81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.602+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 >> 172.21.15.60:0/3136938879 conn(0x7fcda80fb5e0 msgr2=0x7fcda8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.603+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 shutdown_connections 2024-02-26T10:42:41.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.604+0000 7fcdace00640 1 -- 172.21.15.60:0/3136938879 wait complete. 2024-02-26T10:42:41.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-26T10:42:41.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.816+0000 7fd24a400640 1 -- 172.21.15.60:0/920496287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244075c20 msgr2=0x7fd244076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.816+0000 7fd24a400640 1 --2- 172.21.15.60:0/920496287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244075c20 0x7fd244076040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd234009900 tx=0x7fd23402bc40 comp rx=0 tx=0).stop 2024-02-26T10:42:41.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.816+0000 7fd24a400640 1 -- 172.21.15.60:0/920496287 shutdown_connections 2024-02-26T10:42:41.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.816+0000 7fd24a400640 1 --2- 172.21.15.60:0/920496287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd244077800 0x7fd2440780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.816+0000 7fd24a400640 1 --2- 172.21.15.60:0/920496287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244076e60 0x7fd2440772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.816+0000 7fd24a400640 1 --2- 172.21.15.60:0/920496287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244075c20 0x7fd244076040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.816+0000 7fd24a400640 1 -- 172.21.15.60:0/920496287 >> 172.21.15.60:0/920496287 conn(0x7fd2440fea10 msgr2=0x7fd244100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.816+0000 7fd24a400640 1 -- 172.21.15.60:0/920496287 shutdown_connections 2024-02-26T10:42:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.816+0000 7fd24a400640 1 -- 172.21.15.60:0/920496287 wait complete. 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.817+0000 7fd24a400640 1 Processor -- start 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.817+0000 7fd24a400640 1 -- start start 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd24a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd244075c20 0x7fd24419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd24a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244076e60 0x7fd24419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd24a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244077800 0x7fd2441a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd24a400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244113570 con 0x7fd244075c20 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd24a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2441133f0 con 0x7fd244076e60 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd24a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2441136f0 con 0x7fd244077800 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd243400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd244075c20 0x7fd24419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd243400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd244075c20 0x7fd24419dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43914/0 (socket says 172.21.15.60:43914) 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd243e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244077800 0x7fd2441a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd243400640 1 -- 172.21.15.60:0/443043562 learned_addr learned my addr 172.21.15.60:0/443043562 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd242a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244076e60 0x7fd24419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd243e00640 1 -- 172.21.15.60:0/443043562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244076e60 msgr2=0x7fd24419e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd243e00640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244076e60 0x7fd24419e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd243e00640 1 -- 172.21.15.60:0/443043562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd244075c20 msgr2=0x7fd24419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd243e00640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd244075c20 0x7fd24419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.818+0000 7fd243e00640 1 -- 172.21.15.60:0/443043562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2441a57c0 con 0x7fd244077800 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.819+0000 7fd243e00640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244077800 0x7fd2441a5150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e940 tx=0x7fd23c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.819+0000 7fd23be00640 1 -- 172.21.15.60:0/443043562 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c004d20 con 0x7fd244077800 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.819+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2441a5ab0 con 0x7fd244077800 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.819+0000 7fd23be00640 1 -- 172.21.15.60:0/443043562 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd23c005020 con 0x7fd244077800 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.819+0000 7fd23be00640 1 -- 172.21.15.60:0/443043562 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0116b0 con 0x7fd244077800 2024-02-26T10:42:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.819+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2441a6000 con 0x7fd244077800 2024-02-26T10:42:41.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.820+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24410ffb0 con 0x7fd244077800 2024-02-26T10:42:41.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.822+0000 7fd23be00640 1 -- 172.21.15.60:0/443043562 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd23c0054d0 con 0x7fd244077800 2024-02-26T10:42:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.823+0000 7fd23be00640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd2280a1d80 0x7fd2280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.823+0000 7fd23be00640 1 -- 172.21.15.60:0/443043562 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fd23c0cdcd0 con 0x7fd244077800 2024-02-26T10:42:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.823+0000 7fd243400640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd2280a1d80 0x7fd2280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.824+0000 7fd243400640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd2280a1d80 0x7fd2280a4240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd234006010 tx=0x7fd23402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:41.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:41.827+0000 7fd23be00640 1 -- 172.21.15.60:0/443043562 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd23c016030 con 0x7fd244077800 2024-02-26T10:42:41.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:41 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2666011347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T10:42:41.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:41 smithi113 ceph-mon[36126]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T10:42:41.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:41 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2666011347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T10:42:41.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:41 smithi100 ceph-mon[36051]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T10:42:42.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:42.072+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd24402ce30 con 0x7fd244077800 2024-02-26T10:42:42.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:41 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2666011347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T10:42:42.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:41 smithi060 ceph-mon[31539]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T10:42:42.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: pgmap v315: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:42.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3136938879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/443043562' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T10:42:42.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:42 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.066+0000 7fd23be00640 1 -- 172.21.15.60:0/443043562 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd23c09abc0 con 0x7fd244077800 2024-02-26T10:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 2 and data pool 3 2024-02-26T10:42:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.069+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd2280a1d80 msgr2=0x7fd2280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.069+0000 7fd24a400640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd2280a1d80 0x7fd2280a4240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd234006010 tx=0x7fd23402cf90 comp rx=0 tx=0).stop 2024-02-26T10:42:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.069+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244077800 msgr2=0x7fd2441a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.069+0000 7fd24a400640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244077800 0x7fd2441a5150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e940 tx=0x7fd23c00ee00 comp rx=0 tx=0).stop 2024-02-26T10:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.070+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 shutdown_connections 2024-02-26T10:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.070+0000 7fd24a400640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd2280a1d80 0x7fd2280a4240 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.070+0000 7fd24a400640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244076e60 0x7fd24419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.070+0000 7fd24a400640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244077800 0x7fd2441a5150 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.070+0000 7fd24a400640 1 --2- 172.21.15.60:0/443043562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd244075c20 0x7fd24419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.070+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 >> 172.21.15.60:0/443043562 conn(0x7fd2440fea10 msgr2=0x7fd244100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.071+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 shutdown_connections 2024-02-26T10:42:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.072+0000 7fd24a400640 1 -- 172.21.15.60:0/443043562 wait complete. 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: pgmap v315: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3136938879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/443043562' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T10:42:43.085 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T10:42:43.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:43.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T10:42:43.086 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:42 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:43.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: pgmap v315: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:43.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3136938879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-26T10:42:43.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-26T10:42:43.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T10:42:43.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-26T10:42:43.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T10:42:43.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:43.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:43.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/443043562' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T10:42:43.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T10:42:43.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T10:42:43.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:43.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T10:42:43.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:42 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:43.095 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-02-26T10:42:43.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-02-26T10:42:43.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.307+0000 7f98f5600640 1 -- 172.21.15.60:0/2099955388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 msgr2=0x7f98f010fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.307+0000 7f98f5600640 1 --2- 172.21.15.60:0/2099955388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 0x7f98f010fcc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009ca0 tx=0x7f98e802c100 comp rx=0 tx=0).stop 2024-02-26T10:42:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.307+0000 7f98f5600640 1 -- 172.21.15.60:0/2099955388 shutdown_connections 2024-02-26T10:42:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.307+0000 7f98f5600640 1 --2- 172.21.15.60:0/2099955388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 0x7f98f010fcc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.307+0000 7f98f5600640 1 --2- 172.21.15.60:0/2099955388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98f0108a70 0x7f98f0108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.307+0000 7f98f5600640 1 --2- 172.21.15.60:0/2099955388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0107880 0x7f98f0107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.307+0000 7f98f5600640 1 -- 172.21.15.60:0/2099955388 >> 172.21.15.60:0/2099955388 conn(0x7f98f0076820 msgr2=0x7f98f0078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.307+0000 7f98f5600640 1 -- 172.21.15.60:0/2099955388 shutdown_connections 2024-02-26T10:42:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.307+0000 7f98f5600640 1 -- 172.21.15.60:0/2099955388 wait complete. 2024-02-26T10:42:43.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.308+0000 7f98f5600640 1 Processor -- start 2024-02-26T10:42:43.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.308+0000 7f98f5600640 1 -- start start 2024-02-26T10:42:43.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98f5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98f0107880 0x7f98f019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98f5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0108a70 0x7f98f019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98f5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 0x7f98f01a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98f5600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f01134e0 con 0x7f98f0107880 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98f5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0113360 con 0x7f98f0109430 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98f5600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0113660 con 0x7f98f0108a70 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98ef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 0x7f98f01a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98ee000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0108a70 0x7f98f019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98eea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98f0107880 0x7f98f019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98ef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 0x7f98f01a50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52958/0 (socket says 172.21.15.60:52958) 2024-02-26T10:42:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.309+0000 7f98ef400640 1 -- 172.21.15.60:0/500959553 learned_addr learned my addr 172.21.15.60:0/500959553 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:43.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.310+0000 7f98ef400640 1 -- 172.21.15.60:0/500959553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0108a70 msgr2=0x7f98f019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.310+0000 7f98ef400640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0108a70 0x7f98f019e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.310+0000 7f98ef400640 1 -- 172.21.15.60:0/500959553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98f0107880 msgr2=0x7f98f019db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.310+0000 7f98ef400640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98f0107880 0x7f98f019db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.310+0000 7f98ef400640 1 -- 172.21.15.60:0/500959553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f01a5740 con 0x7f98f0109430 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.310+0000 7f98ee000640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0108a70 0x7f98f019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.310+0000 7f98ef400640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 0x7f98f01a50d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98e802c640 tx=0x7f98e8035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.310+0000 7f98e7400640 1 -- 172.21.15.60:0/500959553 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8039040 con 0x7f98f0109430 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.310+0000 7f98e7400640 1 -- 172.21.15.60:0/500959553 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98e8044070 con 0x7f98f0109430 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.311+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f01a59d0 con 0x7f98f0109430 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.311+0000 7f98e7400640 1 -- 172.21.15.60:0/500959553 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8004460 con 0x7f98f0109430 2024-02-26T10:42:43.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.311+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f01a5ec0 con 0x7f98f0109430 2024-02-26T10:42:43.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.312+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f010eba0 con 0x7f98f0109430 2024-02-26T10:42:43.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.315+0000 7f98e7400640 1 -- 172.21.15.60:0/500959553 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f98e8004600 con 0x7f98f0109430 2024-02-26T10:42:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.316+0000 7f98e7400640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98d40a1d80 0x7f98d40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:43.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.316+0000 7f98e7400640 1 -- 172.21.15.60:0/500959553 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f98e80f0340 con 0x7f98f0109430 2024-02-26T10:42:43.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.317+0000 7f98eea00640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98d40a1d80 0x7f98d40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:43.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.317+0000 7f98eea00640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98d40a1d80 0x7f98d40a4240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f98e0005fd0 tx=0x7f98e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.321+0000 7f98e7400640 1 -- 172.21.15.60:0/500959553 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f98e80bd820 con 0x7f98f0109430 2024-02-26T10:42:43.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:43.531+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f98f002d010 con 0x7f98f0109430 2024-02-26T10:42:44.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.176+0000 7f98e7400640 1 -- 172.21.15.60:0/500959553 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v87) v1 ==== 157+0+0 (secure 0 0 0) 0x7f98e80098b0 con 0x7f98f0109430 2024-02-26T10:42:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.180+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98d40a1d80 msgr2=0x7f98d40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.180+0000 7f98f5600640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98d40a1d80 0x7f98d40a4240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f98e0005fd0 tx=0x7f98e000b410 comp rx=0 tx=0).stop 2024-02-26T10:42:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.180+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 msgr2=0x7f98f01a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.180+0000 7f98f5600640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 0x7f98f01a50d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98e802c640 tx=0x7f98e8035d10 comp rx=0 tx=0).stop 2024-02-26T10:42:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.181+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 shutdown_connections 2024-02-26T10:42:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.181+0000 7f98f5600640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98d40a1d80 0x7f98d40a4240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.181+0000 7f98f5600640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98f0109430 0x7f98f01a50d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.181+0000 7f98f5600640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0108a70 0x7f98f019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.181+0000 7f98f5600640 1 --2- 172.21.15.60:0/500959553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98f0107880 0x7f98f019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.181+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 >> 172.21.15.60:0/500959553 conn(0x7f98f0076820 msgr2=0x7f98f010bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:44.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.181+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 shutdown_connections 2024-02-26T10:42:44.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.181+0000 7f98f5600640 1 -- 172.21.15.60:0/500959553 wait complete. 2024-02-26T10:42:44.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T10:42:44.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: pgmap v318: 97 pgs: 8 creating+peering, 13 active+clean, 76 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:44.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T10:42:44.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T10:42:44.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T10:42:44.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:44.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T10:42:44.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:44.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:44.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:42:44.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:44.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/500959553' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-26T10:42:44.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-26T10:42:44.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T10:42:44.336 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:44 smithi113 ceph-mon[36126]: Cluster is now healthy 2024-02-26T10:42:44.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: pgmap v318: 97 pgs: 8 creating+peering, 13 active+clean, 76 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:44.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T10:42:44.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T10:42:44.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/500959553' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T10:42:44.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:44 smithi060 ceph-mon[31539]: Cluster is now healthy 2024-02-26T10:42:44.378 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: pgmap v318: 97 pgs: 8 creating+peering, 13 active+clean, 76 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T10:42:44.378 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T10:42:44.378 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T10:42:44.378 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/500959553' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T10:42:44.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:44 smithi100 ceph-mon[36051]: Cluster is now healthy 2024-02-26T10:42:44.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.443+0000 7f4ba7e00640 1 -- 172.21.15.60:0/3984546321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8101b30 msgr2=0x7f4ba8103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.443+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/3984546321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8101b30 0x7f4ba8103f20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009900 tx=0x7f4b9802bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:44.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.443+0000 7f4ba7e00640 1 -- 172.21.15.60:0/3984546321 shutdown_connections 2024-02-26T10:42:44.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.443+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/3984546321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba8104460 0x7f4ba8106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.443+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/3984546321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ba8106d90 0x7f4ba81091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.443+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/3984546321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8101b30 0x7f4ba8103f20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.443+0000 7f4ba7e00640 1 -- 172.21.15.60:0/3984546321 >> 172.21.15.60:0/3984546321 conn(0x7f4ba80fb540 msgr2=0x7f4ba80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:44.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.443+0000 7f4ba7e00640 1 -- 172.21.15.60:0/3984546321 shutdown_connections 2024-02-26T10:42:44.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.443+0000 7f4ba7e00640 1 -- 172.21.15.60:0/3984546321 wait complete. 2024-02-26T10:42:44.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.444+0000 7f4ba7e00640 1 Processor -- start 2024-02-26T10:42:44.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.444+0000 7f4ba7e00640 1 -- start start 2024-02-26T10:42:44.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.444+0000 7f4ba7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba8101b30 0x7f4ba81995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.444+0000 7f4ba7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ba8104460 0x7f4ba8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.444+0000 7f4ba7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8106d90 0x7f4ba81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.444+0000 7f4ba7e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba810cd90 con 0x7f4ba8106d90 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.444+0000 7f4ba7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba810cc10 con 0x7f4ba8101b30 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.444+0000 7f4ba7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba810cf10 con 0x7f4ba8104460 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba8101b30 0x7f4ba81995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba8101b30 0x7f4ba81995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:43356/0 (socket says 172.21.15.60:43356) 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6a00640 1 -- 172.21.15.60:0/266687560 learned_addr learned my addr 172.21.15.60:0/266687560 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba7400640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8106d90 0x7f4ba81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6000640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ba8104460 0x7f4ba8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6a00640 1 -- 172.21.15.60:0/266687560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ba8104460 msgr2=0x7f4ba8199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6a00640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ba8104460 0x7f4ba8199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6a00640 1 -- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8106d90 msgr2=0x7f4ba81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6a00640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8106d90 0x7f4ba81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6a00640 1 -- 172.21.15.60:0/266687560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba81a1200 con 0x7f4ba8101b30 2024-02-26T10:42:44.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba7400640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8106d90 0x7f4ba81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:44.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.445+0000 7f4ba6a00640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba8101b30 0x7f4ba81995e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b98005ab0 tx=0x7f4b9802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:44.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.446+0000 7f4b9f400640 1 -- 172.21.15.60:0/266687560 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98043020 con 0x7f4ba8101b30 2024-02-26T10:42:44.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.446+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba81a1490 con 0x7f4ba8101b30 2024-02-26T10:42:44.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.446+0000 7f4b9f400640 1 -- 172.21.15.60:0/266687560 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b980054e0 con 0x7f4ba8101b30 2024-02-26T10:42:44.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.446+0000 7f4b9f400640 1 -- 172.21.15.60:0/266687560 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98003ce0 con 0x7f4ba8101b30 2024-02-26T10:42:44.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.446+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba81a19e0 con 0x7f4ba8101b30 2024-02-26T10:42:44.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.447+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba81088e0 con 0x7f4ba8101b30 2024-02-26T10:42:44.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.449+0000 7f4b9f400640 1 -- 172.21.15.60:0/266687560 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4b98046020 con 0x7f4ba8101b30 2024-02-26T10:42:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.450+0000 7f4b9f400640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4b8c0a1d80 0x7f4b8c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.451+0000 7f4b9f400640 1 -- 172.21.15.60:0/266687560 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f4b980f01c0 con 0x7f4ba8101b30 2024-02-26T10:42:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.451+0000 7f4ba6000640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4b8c0a1d80 0x7f4b8c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.451+0000 7f4ba6000640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4b8c0a1d80 0x7f4b8c0a4240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ba819aaf0 tx=0x7f4b94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:44.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.453+0000 7f4b9f400640 1 -- 172.21.15.60:0/266687560 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b9804d020 con 0x7f4ba8101b30 2024-02-26T10:42:44.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:44.654+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4ba802ce30 con 0x7f4ba8101b30 2024-02-26T10:42:45.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:45 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] up:active 2024-02-26T10:42:45.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:45 smithi113 ceph-mon[36126]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:42:45.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:45 smithi113 ceph-mon[36126]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:42:45.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:45 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-26T10:42:45.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:45 smithi113 ceph-mon[36126]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T10:42:45.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:45 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/266687560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T10:42:45.335 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:45 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T10:42:45.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:45 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] up:active 2024-02-26T10:42:45.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:45 smithi060 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:42:45.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:45 smithi060 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:42:45.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:45 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-26T10:42:45.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:45 smithi060 ceph-mon[31539]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T10:42:45.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:45 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/266687560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T10:42:45.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:45 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T10:42:45.378 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:45 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] up:active 2024-02-26T10:42:45.378 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:45 smithi100 ceph-mon[36051]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:42:45.378 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:45 smithi100 ceph-mon[36051]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:42:45.378 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:45 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-26T10:42:45.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:45 smithi100 ceph-mon[36051]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T10:42:45.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:45 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/266687560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T10:42:45.379 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:45 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T10:42:46.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.199+0000 7f4b9f400640 1 -- 172.21.15.60:0/266687560 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v89) v1 ==== 228+0+0 (secure 0 0 0) 0x7f4b980bd560 con 0x7f4ba8101b30 2024-02-26T10:42:46.197 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data_ec' created 2024-02-26T10:42:46.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4b8c0a1d80 msgr2=0x7f4b8c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4b8c0a1d80 0x7f4b8c0a4240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ba819aaf0 tx=0x7f4b94005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:46.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba8101b30 msgr2=0x7f4ba81995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba8101b30 0x7f4ba81995e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b98005ab0 tx=0x7f4b9802c6c0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 shutdown_connections 2024-02-26T10:42:46.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4b8c0a1d80 0x7f4b8c0a4240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8106d90 0x7f4ba81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ba8104460 0x7f4ba8199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 --2- 172.21.15.60:0/266687560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba8101b30 0x7f4ba81995e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.205+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 >> 172.21.15.60:0/266687560 conn(0x7f4ba80fb540 msgr2=0x7f4ba81011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:46.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.206+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 shutdown_connections 2024-02-26T10:42:46.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.207+0000 7f4ba7e00640 1 -- 172.21.15.60:0/266687560 wait complete. 2024-02-26T10:42:46.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-02-26T10:42:46.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.423+0000 7fca6a400640 1 -- 172.21.15.60:0/891704333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64109330 msgr2=0x7fca6410fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.423+0000 7fca6a400640 1 --2- 172.21.15.60:0/891704333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64109330 0x7fca6410fbc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009be0 tx=0x7fca5c02bef0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.423+0000 7fca6a400640 1 -- 172.21.15.60:0/891704333 shutdown_connections 2024-02-26T10:42:46.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.423+0000 7fca6a400640 1 --2- 172.21.15.60:0/891704333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64109330 0x7fca6410fbc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.423+0000 7fca6a400640 1 --2- 172.21.15.60:0/891704333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca64108970 0x7fca64108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.423+0000 7fca6a400640 1 --2- 172.21.15.60:0/891704333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca64107780 0x7fca64107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.423+0000 7fca6a400640 1 -- 172.21.15.60:0/891704333 >> 172.21.15.60:0/891704333 conn(0x7fca64076730 msgr2=0x7fca64078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.423+0000 7fca6a400640 1 -- 172.21.15.60:0/891704333 shutdown_connections 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.423+0000 7fca6a400640 1 -- 172.21.15.60:0/891704333 wait complete. 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.424+0000 7fca6a400640 1 Processor -- start 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.424+0000 7fca6a400640 1 -- start start 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.424+0000 7fca6a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64107780 0x7fca6419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.424+0000 7fca6a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca64108970 0x7fca6419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.424+0000 7fca6a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca64109330 0x7fca641a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.424+0000 7fca6a400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca64113450 con 0x7fca64109330 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.424+0000 7fca6a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca641132d0 con 0x7fca64107780 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.424+0000 7fca6a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca641135d0 con 0x7fca64108970 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64107780 0x7fca6419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca64109330 0x7fca641a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64107780 0x7fca6419da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:43384/0 (socket says 172.21.15.60:43384) 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69000640 1 -- 172.21.15.60:0/3232161257 learned_addr learned my addr 172.21.15.60:0/3232161257 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca63e00640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca64108970 0x7fca6419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69000640 1 -- 172.21.15.60:0/3232161257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca64108970 msgr2=0x7fca6419df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69000640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca64108970 0x7fca6419df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69000640 1 -- 172.21.15.60:0/3232161257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca64109330 msgr2=0x7fca641a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69000640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca64109330 0x7fca641a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69000640 1 -- 172.21.15.60:0/3232161257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca641a5650 con 0x7fca64107780 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.425+0000 7fca69000640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64107780 0x7fca6419da30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca5800e9f0 tx=0x7fca5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.426+0000 7fca61600640 1 -- 172.21.15.60:0/3232161257 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800ccc0 con 0x7fca64107780 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.426+0000 7fca61600640 1 -- 172.21.15.60:0/3232161257 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca580035f0 con 0x7fca64107780 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.426+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca641a5940 con 0x7fca64107780 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.426+0000 7fca61600640 1 -- 172.21.15.60:0/3232161257 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800ce60 con 0x7fca64107780 2024-02-26T10:42:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.426+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca641a5e90 con 0x7fca64107780 2024-02-26T10:42:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.427+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6410eaa0 con 0x7fca64107780 2024-02-26T10:42:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.430+0000 7fca61600640 1 -- 172.21.15.60:0/3232161257 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fca580037a0 con 0x7fca64107780 2024-02-26T10:42:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.431+0000 7fca61600640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca4c0a1d80 0x7fca4c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.431+0000 7fca61600640 1 -- 172.21.15.60:0/3232161257 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fca580ce360 con 0x7fca64107780 2024-02-26T10:42:46.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.431+0000 7fca63e00640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca4c0a1d80 0x7fca4c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.432+0000 7fca63e00640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca4c0a1d80 0x7fca4c0a4240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca6419ef40 tx=0x7fca50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:46.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.432+0000 7fca61600640 1 -- 172.21.15.60:0/3232161257 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca5809b510 con 0x7fca64107780 2024-02-26T10:42:46.491 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:46 smithi100 ceph-mon[36051]: pgmap v321: 97 pgs: 8 creating+peering, 84 active+clean, 5 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-02-26T10:42:46.491 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:46 smithi100 ceph-mon[36051]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T10:42:46.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:46 smithi113 ceph-mon[36126]: pgmap v321: 97 pgs: 8 creating+peering, 84 active+clean, 5 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-02-26T10:42:46.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:46 smithi113 ceph-mon[36126]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T10:42:46.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:46 smithi060 ceph-mon[31539]: pgmap v321: 97 pgs: 8 creating+peering, 84 active+clean, 5 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-02-26T10:42:46.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:46 smithi060 ceph-mon[31539]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T10:42:46.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:46.632+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fca6402ce30 con 0x7fca64107780 2024-02-26T10:42:46.878 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:42:46 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b[66645]: 2024-02-26T10:42:46.492+0000 7efc85822700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:42:47.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.207+0000 7fca61600640 1 -- 172.21.15.60:0/3232161257 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7fca5809ae10 con 0x7fca64107780 2024-02-26T10:42:47.205 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 4 allow_ec_overwrites to true 2024-02-26T10:42:47.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca4c0a1d80 msgr2=0x7fca4c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca4c0a1d80 0x7fca4c0a4240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca6419ef40 tx=0x7fca50005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:47.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64107780 msgr2=0x7fca6419da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64107780 0x7fca6419da30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca5800e9f0 tx=0x7fca5800eeb0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 shutdown_connections 2024-02-26T10:42:47.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca4c0a1d80 0x7fca4c0a4240 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca64109330 0x7fca641a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca64108970 0x7fca6419df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 --2- 172.21.15.60:0/3232161257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca64107780 0x7fca6419da30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.213+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 >> 172.21.15.60:0/3232161257 conn(0x7fca64076730 msgr2=0x7fca6410bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:47.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.215+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 shutdown_connections 2024-02-26T10:42:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.215+0000 7fca6a400640 1 -- 172.21.15.60:0/3232161257 wait complete. 2024-02-26T10:42:47.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-02-26T10:42:47.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.443+0000 7f6c57600640 1 -- 172.21.15.60:0/1941478159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50104680 msgr2=0x7f6c50104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.443+0000 7f6c57600640 1 --2- 172.21.15.60:0/1941478159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50104680 0x7f6c50104b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009900 tx=0x7f6c3c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.443+0000 7f6c57600640 1 -- 172.21.15.60:0/1941478159 shutdown_connections 2024-02-26T10:42:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.443+0000 7f6c57600640 1 --2- 172.21.15.60:0/1941478159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c50105040 0x7f6c5010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.443+0000 7f6c57600640 1 --2- 172.21.15.60:0/1941478159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50104680 0x7f6c50104b00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.443+0000 7f6c57600640 1 --2- 172.21.15.60:0/1941478159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c50103490 0x7f6c501038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.443+0000 7f6c57600640 1 -- 172.21.15.60:0/1941478159 >> 172.21.15.60:0/1941478159 conn(0x7f6c500fea20 msgr2=0x7f6c50100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.443+0000 7f6c57600640 1 -- 172.21.15.60:0/1941478159 shutdown_connections 2024-02-26T10:42:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.443+0000 7f6c57600640 1 -- 172.21.15.60:0/1941478159 wait complete. 2024-02-26T10:42:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.444+0000 7f6c57600640 1 Processor -- start 2024-02-26T10:42:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.444+0000 7f6c57600640 1 -- start start 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.444+0000 7f6c57600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50103490 0x7f6c50199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.444+0000 7f6c57600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c50104680 0x7f6c50199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.444+0000 7f6c57600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c50105040 0x7f6c501a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c57600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5010f120 con 0x7f6c50103490 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c57600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5010efa0 con 0x7f6c50104680 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c57600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5010f2a0 con 0x7f6c50105040 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c54c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50103490 0x7f6c50199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c54c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50103490 0x7f6c50199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43396/0 (socket says 172.21.15.60:43396) 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c54c00640 1 -- 172.21.15.60:0/3433824177 learned_addr learned my addr 172.21.15.60:0/3433824177 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c4fe00640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c50104680 0x7f6c50199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c55600640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c50105040 0x7f6c501a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c54c00640 1 -- 172.21.15.60:0/3433824177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c50105040 msgr2=0x7f6c501a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c54c00640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c50105040 0x7f6c501a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c54c00640 1 -- 172.21.15.60:0/3433824177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c50104680 msgr2=0x7f6c50199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c54c00640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c50104680 0x7f6c50199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c54c00640 1 -- 172.21.15.60:0/3433824177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c501a1330 con 0x7f6c50103490 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c55600640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c50105040 0x7f6c501a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.445+0000 7f6c54c00640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50103490 0x7f6c50199710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400e9c0 tx=0x7f6c4400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.446+0000 7f6c4d600640 1 -- 172.21.15.60:0/3433824177 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400cde0 con 0x7f6c50103490 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.446+0000 7f6c4d600640 1 -- 172.21.15.60:0/3433824177 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c44010910 con 0x7f6c50103490 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.446+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c501a1620 con 0x7f6c50103490 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.446+0000 7f6c4d600640 1 -- 172.21.15.60:0/3433824177 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44018a00 con 0x7f6c50103490 2024-02-26T10:42:47.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.446+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c501a1b40 con 0x7f6c50103490 2024-02-26T10:42:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.447+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5010a7b0 con 0x7f6c50103490 2024-02-26T10:42:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.449+0000 7f6c4d600640 1 -- 172.21.15.60:0/3433824177 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6c4401f020 con 0x7f6c50103490 2024-02-26T10:42:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.451+0000 7f6c4d600640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c380a1d80 0x7f6c380a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.452+0000 7f6c4d600640 1 -- 172.21.15.60:0/3433824177 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f6c440ce2b0 con 0x7f6c50103490 2024-02-26T10:42:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.452+0000 7f6c4fe00640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c380a1d80 0x7f6c380a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.453+0000 7f6c4fe00640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c380a1d80 0x7f6c380a4240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02c7b0 tx=0x7f6c3c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.456+0000 7f6c4d600640 1 -- 172.21.15.60:0/3433824177 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c4409b460 con 0x7f6c50103490 2024-02-26T10:42:47.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:47 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T10:42:47.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:47 smithi113 ceph-mon[36126]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T10:42:47.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:47 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3232161257' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-26T10:42:47.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:47 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-26T10:42:47.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:47 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T10:42:47.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:47 smithi060 ceph-mon[31539]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T10:42:47.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:47 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3232161257' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-26T10:42:47.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:47 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-26T10:42:47.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:47 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T10:42:47.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:47 smithi100 ceph-mon[36051]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T10:42:47.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:47 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3232161257' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-26T10:42:47.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:47 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-26T10:42:47.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:47.703+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f6c5002ce30 con 0x7f6c50103490 2024-02-26T10:42:48.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.205+0000 7f6c4d600640 1 -- 172.21.15.60:0/3433824177 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f6c4409ad60 con 0x7f6c50103490 2024-02-26T10:42:48.203 INFO:teuthology.orchestra.run.smithi060.stderr:added data pool 4 to fsmap 2024-02-26T10:42:48.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.210+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c380a1d80 msgr2=0x7f6c380a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.210+0000 7f6c57600640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c380a1d80 0x7f6c380a4240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02c7b0 tx=0x7f6c3c005fd0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.210+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50103490 msgr2=0x7f6c50199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.210+0000 7f6c57600640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50103490 0x7f6c50199710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400e9c0 tx=0x7f6c4400ee80 comp rx=0 tx=0).stop 2024-02-26T10:42:48.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.211+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 shutdown_connections 2024-02-26T10:42:48.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.211+0000 7f6c57600640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6c380a1d80 0x7f6c380a4240 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.211+0000 7f6c57600640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c50104680 0x7f6c50199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.211+0000 7f6c57600640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c50105040 0x7f6c501a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.211+0000 7f6c57600640 1 --2- 172.21.15.60:0/3433824177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c50103490 0x7f6c50199710 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.211+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 >> 172.21.15.60:0/3433824177 conn(0x7f6c500fea20 msgr2=0x7f6c50107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:48.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.213+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 shutdown_connections 2024-02-26T10:42:48.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.213+0000 7f6c57600640 1 -- 172.21.15.60:0/3433824177 wait complete. 2024-02-26T10:42:48.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.443+0000 7f4cee200640 1 -- 172.21.15.60:0/2627893024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 msgr2=0x7f4ce8108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.443+0000 7f4cee200640 1 --2- 172.21.15.60:0/2627893024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 0x7f4ce8108f10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009900 tx=0x7f4cd002bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.443+0000 7f4cee200640 1 -- 172.21.15.60:0/2627893024 shutdown_connections 2024-02-26T10:42:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.443+0000 7f4cee200640 1 --2- 172.21.15.60:0/2627893024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce8109450 0x7f4ce810fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.443+0000 7f4cee200640 1 --2- 172.21.15.60:0/2627893024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 0x7f4ce8108f10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.443+0000 7f4cee200640 1 --2- 172.21.15.60:0/2627893024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce81078a0 0x7f4ce8107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.443+0000 7f4cee200640 1 -- 172.21.15.60:0/2627893024 >> 172.21.15.60:0/2627893024 conn(0x7f4ce8076870 msgr2=0x7f4ce8078c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.443+0000 7f4cee200640 1 -- 172.21.15.60:0/2627893024 shutdown_connections 2024-02-26T10:42:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.443+0000 7f4cee200640 1 -- 172.21.15.60:0/2627893024 wait complete. 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.444+0000 7f4cee200640 1 Processor -- start 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.444+0000 7f4cee200640 1 -- start start 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4cee200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce81078a0 0x7f4ce819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4cee200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 0x7f4ce819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4cee200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce8109450 0x7f4ce81a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4cee200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8113570 con 0x7f4ce8108a90 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4cee200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce81133f0 con 0x7f4ce8109450 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4cee200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce81136f0 con 0x7f4ce81078a0 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 0x7f4ce819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 0x7f4ce819e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43412/0 (socket says 172.21.15.60:43412) 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4cdea00640 1 -- 172.21.15.60:0/3398155674 learned_addr learned my addr 172.21.15.60:0/3398155674 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4ce7400640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce81078a0 0x7f4ce819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.445+0000 7f4ce7e00640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce8109450 0x7f4ce81a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4ce7e00640 1 -- 172.21.15.60:0/3398155674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce81078a0 msgr2=0x7f4ce819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4ce7e00640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce81078a0 0x7f4ce819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4ce7e00640 1 -- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 msgr2=0x7f4ce819e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4ce7e00640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 0x7f4ce819e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4ce7e00640 1 -- 172.21.15.60:0/3398155674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce81a57d0 con 0x7f4ce8109450 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4cdea00640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 0x7f4ce819e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4ce7e00640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce8109450 0x7f4ce81a5160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009fd0 tx=0x7f4ce000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4ce4c00640 1 -- 172.21.15.60:0/3398155674 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000ce20 con 0x7f4ce8109450 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4ce4c00640 1 -- 172.21.15.60:0/3398155674 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ce0005020 con 0x7f4ce8109450 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4ce4c00640 1 -- 172.21.15.60:0/3398155674 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0010720 con 0x7f4ce8109450 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce81a5ac0 con 0x7f4ce8109450 2024-02-26T10:42:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.446+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce81a5fe0 con 0x7f4ce8109450 2024-02-26T10:42:48.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.448+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce810ebc0 con 0x7f4ce8109450 2024-02-26T10:42:48.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.449+0000 7f4ce4c00640 1 -- 172.21.15.60:0/3398155674 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4ce0005540 con 0x7f4ce8109450 2024-02-26T10:42:48.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.451+0000 7f4ce4c00640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4ccc0a1e00 0x7f4ccc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.451+0000 7f4ce4c00640 1 -- 172.21.15.60:0/3398155674 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4ce00ce680 con 0x7f4ce8109450 2024-02-26T10:42:48.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.451+0000 7f4ce7400640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4ccc0a1e00 0x7f4ccc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.452+0000 7f4ce7400640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4ccc0a1e00 0x7f4ccc0a42c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ce81088f0 tx=0x7f4cd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.455+0000 7f4ce4c00640 1 -- 172.21.15.60:0/3398155674 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ce009b810 con 0x7f4ce8109450 2024-02-26T10:42:48.486 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:48 smithi113 ceph-mon[36126]: pgmap v324: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-02-26T10:42:48.486 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:48 smithi113 ceph-mon[36126]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T10:42:48.486 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:48 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-26T10:42:48.486 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:48 smithi113 ceph-mon[36126]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T10:42:48.486 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:48 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3433824177' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-26T10:42:48.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:48 smithi060 ceph-mon[31539]: pgmap v324: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-02-26T10:42:48.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:48 smithi060 ceph-mon[31539]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T10:42:48.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:48 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-26T10:42:48.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:48 smithi060 ceph-mon[31539]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T10:42:48.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:48 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3433824177' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-26T10:42:48.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:48 smithi100 ceph-mon[36051]: pgmap v324: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-02-26T10:42:48.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:48 smithi100 ceph-mon[36051]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T10:42:48.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:48 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-26T10:42:48.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:48 smithi100 ceph-mon[36051]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T10:42:48.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:48 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3433824177' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-26T10:42:48.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.694+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ce802ce30 con 0x7f4ce8109450 2024-02-26T10:42:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.695+0000 7f4ce4c00640 1 -- 172.21.15.60:0/3398155674 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11298 (secure 0 0 0) 0x7f4ce009b110 con 0x7f4ce8109450 2024-02-26T10:42:48.693 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:48.693 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:42:47.705287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24587},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-26T10:42:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 17 2024-02-26T10:42:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4ccc0a1e00 msgr2=0x7f4ccc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4ccc0a1e00 0x7f4ccc0a42c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ce81088f0 tx=0x7f4cd8005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce8109450 msgr2=0x7f4ce81a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce8109450 0x7f4ce81a5160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009fd0 tx=0x7f4ce000ef10 comp rx=0 tx=0).stop 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 shutdown_connections 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4ccc0a1e00 0x7f4ccc0a42c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce8109450 0x7f4ce81a5160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ce8108a90 0x7f4ce819e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 --2- 172.21.15.60:0/3398155674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce81078a0 0x7f4ce819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 >> 172.21.15.60:0/3398155674 conn(0x7f4ce8076870 msgr2=0x7f4ce810bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 shutdown_connections 2024-02-26T10:42:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.698+0000 7f4cee200640 1 -- 172.21.15.60:0/3398155674 wait complete. 2024-02-26T10:42:48.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.941+0000 7f8b01600640 1 -- 172.21.15.60:0/3975471789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 msgr2=0x7f8afc107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.941+0000 7f8b01600640 1 --2- 172.21.15.60:0/3975471789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 0x7f8afc107cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009ca0 tx=0x7f8aec02c100 comp rx=0 tx=0).stop 2024-02-26T10:42:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.941+0000 7f8b01600640 1 -- 172.21.15.60:0/3975471789 shutdown_connections 2024-02-26T10:42:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.941+0000 7f8b01600640 1 --2- 172.21.15.60:0/3975471789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc109450 0x7f8afc10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.941+0000 7f8b01600640 1 --2- 172.21.15.60:0/3975471789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc108a90 0x7f8afc108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.941+0000 7f8b01600640 1 --2- 172.21.15.60:0/3975471789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 0x7f8afc107cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.941+0000 7f8b01600640 1 -- 172.21.15.60:0/3975471789 >> 172.21.15.60:0/3975471789 conn(0x7f8afc076870 msgr2=0x7f8afc078c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.941+0000 7f8b01600640 1 -- 172.21.15.60:0/3975471789 shutdown_connections 2024-02-26T10:42:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.941+0000 7f8b01600640 1 -- 172.21.15.60:0/3975471789 wait complete. 2024-02-26T10:42:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.942+0000 7f8b01600640 1 Processor -- start 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.942+0000 7f8b01600640 1 -- start start 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.942+0000 7f8b01600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 0x7f8afc19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.942+0000 7f8b01600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc108a90 0x7f8afc19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8b01600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc109450 0x7f8afc1a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8b01600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc113510 con 0x7f8afc1078a0 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8b01600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc113390 con 0x7f8afc108a90 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8b01600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc113690 con 0x7f8afc109450 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 0x7f8afc19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 0x7f8afc19dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43434/0 (socket says 172.21.15.60:43434) 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afaa00640 1 -- 172.21.15.60:0/1956933789 learned_addr learned my addr 172.21.15.60:0/1956933789 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afa000640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc108a90 0x7f8afc19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afb400640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc109450 0x7f8afc1a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afaa00640 1 -- 172.21.15.60:0/1956933789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc109450 msgr2=0x7f8afc1a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afaa00640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc109450 0x7f8afc1a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afaa00640 1 -- 172.21.15.60:0/1956933789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc108a90 msgr2=0x7f8afc19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afaa00640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc108a90 0x7f8afc19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.943+0000 7f8afaa00640 1 -- 172.21.15.60:0/1956933789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc1a57f0 con 0x7f8afc1078a0 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.944+0000 7f8afaa00640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 0x7f8afc19dbd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8aec000f80 tx=0x7f8aec035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.944+0000 7f8af3400640 1 -- 172.21.15.60:0/1956933789 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec039040 con 0x7f8afc1078a0 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.944+0000 7f8af3400640 1 -- 172.21.15.60:0/1956933789 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8aec044070 con 0x7f8afc1078a0 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.944+0000 7f8af3400640 1 -- 172.21.15.60:0/1956933789 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0044a0 con 0x7f8afc1078a0 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.944+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc1a5a80 con 0x7f8afc1078a0 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.944+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc1a5f90 con 0x7f8afc1078a0 2024-02-26T10:42:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.945+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc10ebc0 con 0x7f8afc1078a0 2024-02-26T10:42:48.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.947+0000 7f8af3400640 1 -- 172.21.15.60:0/1956933789 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8aec004640 con 0x7f8afc1078a0 2024-02-26T10:42:48.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.949+0000 7f8af3400640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ae00a1e00 0x7f8ae00a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.949+0000 7f8af3400640 1 -- 172.21.15.60:0/1956933789 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8aec0f0600 con 0x7f8afc1078a0 2024-02-26T10:42:48.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.949+0000 7f8afa000640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ae00a1e00 0x7f8ae00a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.950+0000 7f8afa000640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ae00a1e00 0x7f8ae00a42c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8afc19f0e0 tx=0x7f8ae4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:48.953+0000 7f8af3400640 1 -- 172.21.15.60:0/1956933789 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8aec0bd790 con 0x7f8afc1078a0 2024-02-26T10:42:49.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.158+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8afc061e60 con 0x7f8afc1078a0 2024-02-26T10:42:49.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.162+0000 7f8af3400640 1 -- 172.21.15.60:0/1956933789 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7f8aec0bd090 con 0x7f8afc1078a0 2024-02-26T10:42:49.160 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:49.161 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:49.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.165+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ae00a1e00 msgr2=0x7f8ae00a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.165+0000 7f8b01600640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ae00a1e00 0x7f8ae00a42c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8afc19f0e0 tx=0x7f8ae4005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:49.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.165+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 msgr2=0x7f8afc19dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.165+0000 7f8b01600640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 0x7f8afc19dbd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8aec000f80 tx=0x7f8aec035d10 comp rx=0 tx=0).stop 2024-02-26T10:42:49.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.166+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 shutdown_connections 2024-02-26T10:42:49.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.166+0000 7f8b01600640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ae00a1e00 0x7f8ae00a42c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.166+0000 7f8b01600640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc108a90 0x7f8afc19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.166+0000 7f8b01600640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc109450 0x7f8afc1a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.166+0000 7f8b01600640 1 --2- 172.21.15.60:0/1956933789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8afc1078a0 0x7f8afc19dbd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.166+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 >> 172.21.15.60:0/1956933789 conn(0x7f8afc076870 msgr2=0x7f8afc10bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:49.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.166+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 shutdown_connections 2024-02-26T10:42:49.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.166+0000 7f8b01600640 1 -- 172.21.15.60:0/1956933789 wait complete. 2024-02-26T10:42:49.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:49.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.408+0000 7fc6b9a00640 1 -- 172.21.15.60:0/3312450744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b41044a0 msgr2=0x7fc6b4104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.408+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/3312450744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b41044a0 0x7fc6b4104920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009900 tx=0x7fc69c02bc40 comp rx=0 tx=0).stop 2024-02-26T10:42:49.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.408+0000 7fc6b9a00640 1 -- 172.21.15.60:0/3312450744 shutdown_connections 2024-02-26T10:42:49.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.408+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/3312450744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b4104e60 0x7fc6b410b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.408+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/3312450744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b41044a0 0x7fc6b4104920 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.408+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/3312450744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6b41032b0 0x7fc6b41036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.408+0000 7fc6b9a00640 1 -- 172.21.15.60:0/3312450744 >> 172.21.15.60:0/3312450744 conn(0x7fc6b40fe840 msgr2=0x7fc6b4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:49.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.408+0000 7fc6b9a00640 1 -- 172.21.15.60:0/3312450744 shutdown_connections 2024-02-26T10:42:49.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.409+0000 7fc6b9a00640 1 -- 172.21.15.60:0/3312450744 wait complete. 2024-02-26T10:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.410+0000 7fc6b9a00640 1 Processor -- start 2024-02-26T10:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.410+0000 7fc6b9a00640 1 -- start start 2024-02-26T10:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.410+0000 7fc6b9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6b41032b0 0x7fc6b4199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.410+0000 7fc6b9a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b41044a0 0x7fc6b4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b4104e60 0x7fc6b41a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b410ef90 con 0x7fc6b4104e60 2024-02-26T10:42:49.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b410ee10 con 0x7fc6b41044a0 2024-02-26T10:42:49.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b410f110 con 0x7fc6b41032b0 2024-02-26T10:42:49.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b4104e60 0x7fc6b41a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b4104e60 0x7fc6b41a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43448/0 (socket says 172.21.15.60:43448) 2024-02-26T10:42:49.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6b41032b0 0x7fc6b4199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9000640 1 -- 172.21.15.60:0/2976541781 learned_addr learned my addr 172.21.15.60:0/2976541781 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:49.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b3400640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b41044a0 0x7fc6b4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9000640 1 -- 172.21.15.60:0/2976541781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6b41032b0 msgr2=0x7fc6b4199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9000640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6b41032b0 0x7fc6b4199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9000640 1 -- 172.21.15.60:0/2976541781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b41044a0 msgr2=0x7fc6b4199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9000640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b41044a0 0x7fc6b4199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b9000640 1 -- 172.21.15.60:0/2976541781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b41a11b0 con 0x7fc6b4104e60 2024-02-26T10:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b3e00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6b41032b0 0x7fc6b4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.411+0000 7fc6b3400640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b41044a0 0x7fc6b4199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.412+0000 7fc6b9000640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b4104e60 0x7fc6b41a0b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00eb10 tx=0x7fc6ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.412+0000 7fc6b0c00640 1 -- 172.21.15.60:0/2976541781 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac004b20 con 0x7fc6b4104e60 2024-02-26T10:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.412+0000 7fc6b0c00640 1 -- 172.21.15.60:0/2976541781 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac004cc0 con 0x7fc6b4104e60 2024-02-26T10:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.412+0000 7fc6b0c00640 1 -- 172.21.15.60:0/2976541781 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac010760 con 0x7fc6b4104e60 2024-02-26T10:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.412+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b41a14a0 con 0x7fc6b4104e60 2024-02-26T10:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.412+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b41a19c0 con 0x7fc6b4104e60 2024-02-26T10:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.413+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b410a570 con 0x7fc6b4104e60 2024-02-26T10:42:49.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.415+0000 7fc6b0c00640 1 -- 172.21.15.60:0/2976541781 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc6ac005000 con 0x7fc6b4104e60 2024-02-26T10:42:49.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.417+0000 7fc6b0c00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc6980a1e00 0x7fc6980a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.417+0000 7fc6b0c00640 1 -- 172.21.15.60:0/2976541781 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc6ac0ce740 con 0x7fc6b4104e60 2024-02-26T10:42:49.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.417+0000 7fc6b3e00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc6980a1e00 0x7fc6980a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.418+0000 7fc6b3e00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc6980a1e00 0x7fc6980a42c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4104300 tx=0x7fc6a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.421+0000 7fc6b0c00640 1 -- 172.21.15.60:0/2976541781 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6ac052d20 con 0x7fc6b4104e60 2024-02-26T10:42:49.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:49 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3433824177' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-26T10:42:49.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:49 smithi113 ceph-mon[36126]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T10:42:49.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:49 smithi113 ceph-mon[36126]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:42:49.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:49 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3398155674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:49.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:49 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1956933789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:49.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:49 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3433824177' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-26T10:42:49.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:49 smithi060 ceph-mon[31539]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T10:42:49.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:49 smithi060 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:42:49.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:49 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3398155674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:49.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:49 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1956933789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:49.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.627+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6b4061e60 con 0x7fc6b4104e60 2024-02-26T10:42:49.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:49 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3433824177' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-26T10:42:49.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:49 smithi100 ceph-mon[36051]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T10:42:49.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:49 smithi100 ceph-mon[36051]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:42:49.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:49 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3398155674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:49.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:49 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1956933789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.631+0000 7fc6b0c00640 1 -- 172.21.15.60:0/2976541781 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7fc6ac09b8d0 con 0x7fc6b4104e60 2024-02-26T10:42:49.629 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:49.629 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc6980a1e00 msgr2=0x7fc6980a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc6980a1e00 0x7fc6980a42c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4104300 tx=0x7fc6a4005f90 comp rx=0 tx=0).stop 2024-02-26T10:42:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b4104e60 msgr2=0x7fc6b41a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b4104e60 0x7fc6b41a0b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00eb10 tx=0x7fc6ac00c2d0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 shutdown_connections 2024-02-26T10:42:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc6980a1e00 0x7fc6980a42c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b41044a0 0x7fc6b4199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b4104e60 0x7fc6b41a0b40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 --2- 172.21.15.60:0/2976541781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6b41032b0 0x7fc6b4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 >> 172.21.15.60:0/2976541781 conn(0x7fc6b40fe840 msgr2=0x7fc6b4107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.634+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 shutdown_connections 2024-02-26T10:42:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.635+0000 7fc6b9a00640 1 -- 172.21.15.60:0/2976541781 wait complete. 2024-02-26T10:42:49.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.884+0000 7ff1a6000640 1 -- 172.21.15.60:0/589812322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1a0104e60 msgr2=0x7ff1a010b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.884+0000 7ff1a6000640 1 --2- 172.21.15.60:0/589812322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1a0104e60 0x7ff1a010b6f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff198009be0 tx=0x7ff19802bef0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.885+0000 7ff1a6000640 1 -- 172.21.15.60:0/589812322 shutdown_connections 2024-02-26T10:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.885+0000 7ff1a6000640 1 --2- 172.21.15.60:0/589812322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1a0104e60 0x7ff1a010b6f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.885+0000 7ff1a6000640 1 --2- 172.21.15.60:0/589812322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a01044a0 0x7ff1a0104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.885+0000 7ff1a6000640 1 --2- 172.21.15.60:0/589812322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a01032b0 0x7ff1a01036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.885+0000 7ff1a6000640 1 -- 172.21.15.60:0/589812322 >> 172.21.15.60:0/589812322 conn(0x7ff1a00fe840 msgr2=0x7ff1a0100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.885+0000 7ff1a6000640 1 -- 172.21.15.60:0/589812322 shutdown_connections 2024-02-26T10:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.885+0000 7ff1a6000640 1 -- 172.21.15.60:0/589812322 wait complete. 2024-02-26T10:42:49.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.886+0000 7ff1a6000640 1 Processor -- start 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.886+0000 7ff1a6000640 1 -- start start 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.886+0000 7ff1a6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a01032b0 0x7ff1a01995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.886+0000 7ff1a6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1a01044a0 0x7ff1a0199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.886+0000 7ff1a6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a0104e60 0x7ff1a01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.886+0000 7ff1a6000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a010ef90 con 0x7ff1a0104e60 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.886+0000 7ff1a6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a010ee10 con 0x7ff1a01044a0 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.886+0000 7ff1a6000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a010f110 con 0x7ff1a01032b0 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff1a5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a0104e60 0x7ff1a01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff1a5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a0104e60 0x7ff1a01a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43472/0 (socket says 172.21.15.60:43472) 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff1a5600640 1 -- 172.21.15.60:0/2192443740 learned_addr learned my addr 172.21.15.60:0/2192443740 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff1a4c00640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a01032b0 0x7ff1a01995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff19fe00640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1a01044a0 0x7ff1a0199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff1a5600640 1 -- 172.21.15.60:0/2192443740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a01032b0 msgr2=0x7ff1a01995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff1a5600640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a01032b0 0x7ff1a01995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff1a5600640 1 -- 172.21.15.60:0/2192443740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1a01044a0 msgr2=0x7ff1a0199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff1a5600640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1a01044a0 0x7ff1a0199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.887+0000 7ff1a5600640 1 -- 172.21.15.60:0/2192443740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a01a1200 con 0x7ff1a0104e60 2024-02-26T10:42:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.888+0000 7ff1a5600640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a0104e60 0x7ff1a01a0b90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff198036040 tx=0x7ff1980337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.888+0000 7ff19d600640 1 -- 172.21.15.60:0/2192443740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198043020 con 0x7ff1a0104e60 2024-02-26T10:42:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.888+0000 7ff19d600640 1 -- 172.21.15.60:0/2192443740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff19802ca60 con 0x7ff1a0104e60 2024-02-26T10:42:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.888+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a01a1490 con 0x7ff1a0104e60 2024-02-26T10:42:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.888+0000 7ff19d600640 1 -- 172.21.15.60:0/2192443740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198034690 con 0x7ff1a0104e60 2024-02-26T10:42:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.889+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a01a1a30 con 0x7ff1a0104e60 2024-02-26T10:42:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.890+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a010a500 con 0x7ff1a0104e60 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.891+0000 7ff19d600640 1 -- 172.21.15.60:0/2192443740 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff198046020 con 0x7ff1a0104e60 2024-02-26T10:42:49.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.893+0000 7ff19d600640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff1840a1e00 0x7ff1840a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.893+0000 7ff19d600640 1 -- 172.21.15.60:0/2192443740 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff1980f05f0 con 0x7ff1a0104e60 2024-02-26T10:42:49.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.893+0000 7ff1a4c00640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff1840a1e00 0x7ff1840a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.894+0000 7ff1a4c00640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff1840a1e00 0x7ff1840a42c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff188009530 tx=0x7ff188009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:49.897+0000 7ff19d600640 1 -- 172.21.15.60:0/2192443740 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff19804d020 con 0x7ff1a0104e60 2024-02-26T10:42:50.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.139+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1a0061e60 con 0x7ff1a0104e60 2024-02-26T10:42:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.140+0000 7ff19d600640 1 -- 172.21.15.60:0/2192443740 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11298 (secure 0 0 0) 0x7ff1980bd780 con 0x7ff1a0104e60 2024-02-26T10:42:50.137 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:50.138 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:42:47.705287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24587},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-26T10:42:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 17 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff1840a1e00 msgr2=0x7ff1840a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff1840a1e00 0x7ff1840a42c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff188009530 tx=0x7ff188009260 comp rx=0 tx=0).stop 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a0104e60 msgr2=0x7ff1a01a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a0104e60 0x7ff1a01a0b90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff198036040 tx=0x7ff1980337b0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 shutdown_connections 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff1840a1e00 0x7ff1840a42c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1a01044a0 0x7ff1a0199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a0104e60 0x7ff1a01a0b90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 --2- 172.21.15.60:0/2192443740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a01032b0 0x7ff1a01995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 >> 172.21.15.60:0/2192443740 conn(0x7ff1a00fe840 msgr2=0x7ff1a01076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 shutdown_connections 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.143+0000 7ff1a6000640 1 -- 172.21.15.60:0/2192443740 wait complete. 2024-02-26T10:42:50.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:50.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.383+0000 7fb855800640 1 -- 172.21.15.60:0/1919130960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8501032b0 msgr2=0x7fb8501036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.383+0000 7fb855800640 1 --2- 172.21.15.60:0/1919130960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8501032b0 0x7fb8501036d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb840009900 tx=0x7fb84002bc50 comp rx=0 tx=0).stop 2024-02-26T10:42:50.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.384+0000 7fb855800640 1 -- 172.21.15.60:0/1919130960 shutdown_connections 2024-02-26T10:42:50.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.384+0000 7fb855800640 1 --2- 172.21.15.60:0/1919130960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb850104e60 0x7fb85010b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.384+0000 7fb855800640 1 --2- 172.21.15.60:0/1919130960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8501044a0 0x7fb850104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.384+0000 7fb855800640 1 --2- 172.21.15.60:0/1919130960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8501032b0 0x7fb8501036d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.384+0000 7fb855800640 1 -- 172.21.15.60:0/1919130960 >> 172.21.15.60:0/1919130960 conn(0x7fb8500fe840 msgr2=0x7fb850100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.384+0000 7fb855800640 1 -- 172.21.15.60:0/1919130960 shutdown_connections 2024-02-26T10:42:50.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.384+0000 7fb855800640 1 -- 172.21.15.60:0/1919130960 wait complete. 2024-02-26T10:42:50.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.385+0000 7fb855800640 1 Processor -- start 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.385+0000 7fb855800640 1 -- start start 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.385+0000 7fb855800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8501032b0 0x7fb850199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.385+0000 7fb855800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8501044a0 0x7fb850199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.385+0000 7fb855800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb850104e60 0x7fb8501a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.385+0000 7fb855800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85010ef40 con 0x7fb850104e60 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.385+0000 7fb855800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85010edc0 con 0x7fb8501032b0 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.385+0000 7fb855800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85010f0c0 con 0x7fb8501044a0 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb84fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8501032b0 0x7fb850199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb84fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8501032b0 0x7fb850199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:43500/0 (socket says 172.21.15.60:43500) 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb84fe00640 1 -- 172.21.15.60:0/1834064385 learned_addr learned my addr 172.21.15.60:0/1834064385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb854e00640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb850104e60 0x7fb8501a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb847e00640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8501044a0 0x7fb850199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb84fe00640 1 -- 172.21.15.60:0/1834064385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8501044a0 msgr2=0x7fb850199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb84fe00640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8501044a0 0x7fb850199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb84fe00640 1 -- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb850104e60 msgr2=0x7fb8501a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb84fe00640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb850104e60 0x7fb8501a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb84fe00640 1 -- 172.21.15.60:0/1834064385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8501a1240 con 0x7fb8501032b0 2024-02-26T10:42:50.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb854e00640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb850104e60 0x7fb8501a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:50.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.386+0000 7fb84fe00640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8501032b0 0x7fb850199620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb840000c00 tx=0x7fb840005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.387+0000 7fb84d600640 1 -- 172.21.15.60:0/1834064385 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb840042020 con 0x7fb8501032b0 2024-02-26T10:42:50.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.387+0000 7fb84d600640 1 -- 172.21.15.60:0/1834064385 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8400044d0 con 0x7fb8501032b0 2024-02-26T10:42:50.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.387+0000 7fb84d600640 1 -- 172.21.15.60:0/1834064385 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84003e460 con 0x7fb8501032b0 2024-02-26T10:42:50.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.387+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8501a14d0 con 0x7fb8501032b0 2024-02-26T10:42:50.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.387+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8501a1a70 con 0x7fb8501032b0 2024-02-26T10:42:50.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.388+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85010a5d0 con 0x7fb8501032b0 2024-02-26T10:42:50.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.390+0000 7fb84d600640 1 -- 172.21.15.60:0/1834064385 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb840045020 con 0x7fb8501032b0 2024-02-26T10:42:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.391+0000 7fb84d600640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8340a1e80 0x7fb8340a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.391+0000 7fb84d600640 1 -- 172.21.15.60:0/1834064385 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb8400f0230 con 0x7fb8501032b0 2024-02-26T10:42:50.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.392+0000 7fb847e00640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8340a1e80 0x7fb8340a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.392+0000 7fb847e00640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8340a1e80 0x7fb8340a4340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb838005fd0 tx=0x7fb838005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.394+0000 7fb84d600640 1 -- 172.21.15.60:0/1834064385 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb84004c020 con 0x7fb8501032b0 2024-02-26T10:42:50.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:50 smithi113 ceph-mon[36126]: pgmap v327: 129 pgs: 2 creating+peering, 12 unknown, 115 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2024-02-26T10:42:50.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:50 smithi113 ceph-mon[36126]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T10:42:50.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:50 smithi113 ceph-mon[36126]: Cluster is now healthy 2024-02-26T10:42:50.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:50 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2976541781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:50.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:50 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2192443740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:50.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.596+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb850061e60 con 0x7fb8501032b0 2024-02-26T10:42:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.598+0000 7fb84d600640 1 -- 172.21.15.60:0/1834064385 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7fb8400bd3c0 con 0x7fb8501032b0 2024-02-26T10:42:50.596 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:50.596 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.601+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8340a1e80 msgr2=0x7fb8340a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.601+0000 7fb855800640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8340a1e80 0x7fb8340a4340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb838005fd0 tx=0x7fb838005850 comp rx=0 tx=0).stop 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.601+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8501032b0 msgr2=0x7fb850199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.601+0000 7fb855800640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8501032b0 0x7fb850199620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb840000c00 tx=0x7fb840005d50 comp rx=0 tx=0).stop 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.601+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 shutdown_connections 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.601+0000 7fb855800640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8340a1e80 0x7fb8340a4340 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.602+0000 7fb855800640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb850104e60 0x7fb8501a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.602+0000 7fb855800640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8501044a0 0x7fb850199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.602+0000 7fb855800640 1 --2- 172.21.15.60:0/1834064385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8501032b0 0x7fb850199620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.602+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 >> 172.21.15.60:0/1834064385 conn(0x7fb8500fe840 msgr2=0x7fb8501077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.602+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 shutdown_connections 2024-02-26T10:42:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.602+0000 7fb855800640 1 -- 172.21.15.60:0/1834064385 wait complete. 2024-02-26T10:42:50.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:50 smithi060 ceph-mon[31539]: pgmap v327: 129 pgs: 2 creating+peering, 12 unknown, 115 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2024-02-26T10:42:50.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:50 smithi060 ceph-mon[31539]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T10:42:50.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:50 smithi060 ceph-mon[31539]: Cluster is now healthy 2024-02-26T10:42:50.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:50 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2976541781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:50.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:50 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2192443740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:50.619 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-26T10:42:42.076607+0000', 'modified': '2024-02-26T10:42:47.705287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24587}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6839/2377114018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 2377114018}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 2377114018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-02-26T10:42:50.619 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-26T10:42:50.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T10:42:50.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:50 smithi100 ceph-mon[36051]: pgmap v327: 129 pgs: 2 creating+peering, 12 unknown, 115 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2024-02-26T10:42:50.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:50 smithi100 ceph-mon[36051]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T10:42:50.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:50 smithi100 ceph-mon[36051]: Cluster is now healthy 2024-02-26T10:42:50.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:50 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2976541781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:50.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:50 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2192443740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:42:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.853+0000 7f9822600640 1 -- 172.21.15.60:0/3782455252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f981c103380 msgr2=0x7f981c1037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.853+0000 7f9822600640 1 --2- 172.21.15.60:0/3782455252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f981c103380 0x7f981c1037a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9810009900 tx=0x7f981002bc30 comp rx=0 tx=0).stop 2024-02-26T10:42:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.853+0000 7f9822600640 1 -- 172.21.15.60:0/3782455252 shutdown_connections 2024-02-26T10:42:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.853+0000 7f9822600640 1 --2- 172.21.15.60:0/3782455252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f981c104570 0x7f981c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.853+0000 7f9822600640 1 --2- 172.21.15.60:0/3782455252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981c104f30 0x7f981c10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.853+0000 7f9822600640 1 --2- 172.21.15.60:0/3782455252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f981c103380 0x7f981c1037a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.853+0000 7f9822600640 1 -- 172.21.15.60:0/3782455252 >> 172.21.15.60:0/3782455252 conn(0x7f981c0fe910 msgr2=0x7f981c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.853+0000 7f9822600640 1 -- 172.21.15.60:0/3782455252 shutdown_connections 2024-02-26T10:42:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.853+0000 7f9822600640 1 -- 172.21.15.60:0/3782455252 wait complete. 2024-02-26T10:42:50.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.854+0000 7f9822600640 1 Processor -- start 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.854+0000 7f9822600640 1 -- start start 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9822600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f981c103380 0x7f981c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9822600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981c104570 0x7f981c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9822600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f981c104f30 0x7f981c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9822600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c10f010 con 0x7f981c104570 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9822600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c10ee90 con 0x7f981c104f30 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9822600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c10f190 con 0x7f981c103380 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f981be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981c104570 0x7f981c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9821200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f981c103380 0x7f981c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f981be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981c104570 0x7f981c199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43502/0 (socket says 172.21.15.60:43502) 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f981be00640 1 -- 172.21.15.60:0/2809964609 learned_addr learned my addr 172.21.15.60:0/2809964609 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9821c00640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f981c104f30 0x7f981c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f981be00640 1 -- 172.21.15.60:0/2809964609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f981c103380 msgr2=0x7f981c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f981be00640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f981c103380 0x7f981c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f981be00640 1 -- 172.21.15.60:0/2809964609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f981c104f30 msgr2=0x7f981c1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f981be00640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f981c104f30 0x7f981c1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f981be00640 1 -- 172.21.15.60:0/2809964609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c1a12b0 con 0x7f981c104570 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9821c00640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f981c104f30 0x7f981c1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.855+0000 7f9821200640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f981c103380 0x7f981c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.856+0000 7f981be00640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981c104570 0x7f981c199bd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e9f0 tx=0x7f980c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.856+0000 7f9819600640 1 -- 172.21.15.60:0/2809964609 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c00cd80 con 0x7f981c104570 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.856+0000 7f9819600640 1 -- 172.21.15.60:0/2809964609 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f980c005500 con 0x7f981c104570 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.856+0000 7f9822600640 1 -- 172.21.15.60:0/2809964609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c1a15a0 con 0x7f981c104570 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.856+0000 7f9819600640 1 -- 172.21.15.60:0/2809964609 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c010690 con 0x7f981c104570 2024-02-26T10:42:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.856+0000 7f9822600640 1 -- 172.21.15.60:0/2809964609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c1a1af0 con 0x7f981c104570 2024-02-26T10:42:50.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9826bad740 1 Processor -- start 2024-02-26T10:42:50.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9819600640 1 -- 172.21.15.60:0/2809964609 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f980c0046c0 con 0x7f981c104570 2024-02-26T10:42:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9826bad740 1 -- start start 2024-02-26T10:42:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9826bad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bfbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9826bad740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c00da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9826bad740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 0x55e4e2c038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9826bad740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2b605d0 con 0x55e4e2bfc2a0 2024-02-26T10:42:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9826bad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2b09900 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9826bad740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2b5c670 con 0x55e4e2c01410 2024-02-26T10:42:50.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.860+0000 7f9821200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c00da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f981be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 0x55e4e2c038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f9821c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bfbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f9821200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c00da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43512/0 (socket says 172.21.15.60:43512) 2024-02-26T10:42:50.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f9821200640 1 -- 172.21.15.60:0/3510906889 learned_addr learned my addr 172.21.15.60:0/3510906889 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:50.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f9821c00640 1 -- 172.21.15.60:0/3510906889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 msgr2=0x55e4e2c038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f9821c00640 1 --2- 172.21.15.60:0/3510906889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 0x55e4e2c038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f9821c00640 1 -- 172.21.15.60:0/3510906889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfc2a0 msgr2=0x55e4e2c00da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f9821c00640 1 --2- 172.21.15.60:0/3510906889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c00da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f9821c00640 1 -- 172.21.15.60:0/3510906889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4e28752d0 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f9821c00640 1 --2- 172.21.15.60:0/3510906889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bfbcd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9814005fd0 tx=0x7f9814011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db182343af096ee2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.861+0000 7f980a000640 1 -- 172.21.15.60:0/3510906889 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814004d20 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.862+0000 7f980a000640 1 -- 172.21.15.60:0/3510906889 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f981400f9a0 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.862+0000 7f980a000640 1 -- 172.21.15.60:0/3510906889 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814004550 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.862+0000 7f9819600640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98040a1e00 0x7f98040a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.862+0000 7f9819600640 1 -- 172.21.15.60:0/2809964609 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f980c0ce380 con 0x7f981c104570 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.862+0000 7f9821200640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98040a1e00 0x7f98040a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.862+0000 7f9826bad740 1 -- 172.21.15.60:0/3510906889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 msgr2=0x55e4e2bfbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.862+0000 7f9826bad740 1 --2- 172.21.15.60:0/3510906889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bfbcd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9814005fd0 tx=0x7f9814011bf0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9826bad740 1 -- 172.21.15.60:0/3510906889 shutdown_connections 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9826bad740 1 --2- 172.21.15.60:0/3510906889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 0x55e4e2c038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9826bad740 1 --2- 172.21.15.60:0/3510906889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c00da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9826bad740 1 --2- 172.21.15.60:0/3510906889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bfbcd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9826bad740 1 -- 172.21.15.60:0/3510906889 >> 172.21.15.60:0/3510906889 conn(0x55e4e2bf6e20 msgr2=0x55e4e2bf7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9821200640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98040a1e00 0x7f98040a42c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f981002ce70 tx=0x7f9810005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9826bad740 1 -- 172.21.15.60:0/3510906889 shutdown_connections 2024-02-26T10:42:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9826bad740 1 -- 172.21.15.60:0/3510906889 wait complete. 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9826bad740 1 -- start start 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.863+0000 7f9826bad740 1 Processor -- start 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9826bad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2c05b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9826bad740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c06060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9826bad740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c0d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9826bad740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2c043c0 con 0x55e4e2c01410 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9826bad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2b1cd60 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9826bad740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2c04540 con 0x55e4e2bfc2a0 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9826bad740 10 client.0 _ll_register_callbacks cb 0x55e4e2b58320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9821c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c0d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9821200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2c05b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f981be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c06060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9821c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c0d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43522/0 (socket says 172.21.15.60:43522) 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.864+0000 7f9821c00640 1 -- 172.21.15.60:0/4230374446 learned_addr learned my addr 172.21.15.60:0/4230374446 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f9821c00640 1 -- 172.21.15.60:0/4230374446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfc2a0 msgr2=0x55e4e2c06060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f9821c00640 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c06060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f9821c00640 1 -- 172.21.15.60:0/4230374446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 msgr2=0x55e4e2c05b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f9821c00640 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2c05b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f9821c00640 1 -- 172.21.15.60:0/4230374446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4e293f6b0 con 0x55e4e2c01410 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f9821c00640 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c0d0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9814000f80 tx=0x7f9814003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b896be57fefcd7f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f97fea00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f97fea00640 1 -- 172.21.15.60:0/4230374446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814005000 con 0x55e4e2c01410 2024-02-26T10:42:50.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f97fea00640 1 -- 172.21.15.60:0/4230374446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f98140051a0 con 0x55e4e2c01410 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.865+0000 7f9826bad740 1 -- 172.21.15.60:0/4230374446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f981400f9a0 con 0x55e4e2c01410 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.866+0000 7f97fea00640 1 -- 172.21.15.60:0/4230374446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9814032050 con 0x55e4e2c01410 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.866+0000 7f9826bad740 10 client.14583 fetch_fsmap learned FSMap version 17 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.866+0000 7f9826bad740 1 -- 172.21.15.60:0/4230374446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55e4e2c10410 con 0x55e4e2c01410 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.866+0000 7f97fea00640 1 -- 172.21.15.60:0/4230374446 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f981400f9a0 con 0x55e4e2c01410 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.866+0000 7f97fea00640 1 -- 172.21.15.60:0/4230374446 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9814018070 con 0x55e4e2c01410 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.866+0000 7f9826bad740 10 client.14583 fetch_fsmap finished waiting for FSMap version 17 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.866+0000 7f9826bad740 10 client.14583 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.866+0000 7f9826bad740 20 client.14583 populate_metadata read hostname 'smithi060' 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.867+0000 7f9826bad740 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55e4e2c12570 0x55e4e2c11810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.867+0000 7f9826bad740 4 client.14583 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.867+0000 7f9826bad740 1 -- 172.21.15.60:0/4230374446 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4e2b6db10 con 0x55e4e2c12570 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.867+0000 7f9821200640 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55e4e2c12570 0x55e4e2c11810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.868+0000 7f9821200640 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55e4e2c12570 0x55e4e2c11810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1baf065e93591a server_cookie=1d82e6b903e7bc66 in_seq=0 out_seq=0 2024-02-26T10:42:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.868+0000 7f97fea00640 10 client.14583 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:42:50.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.868+0000 7f97fea00640 1 -- 172.21.15.60:0/4230374446 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e4e2b6db10 con 0x55e4e2c12570 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.868+0000 7f97fea00640 10 client.14583 handle_command_reply: tid=0 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.868+0000 7f9826bad740 1 client.14583 shutdown 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 -- 172.21.15.60:0/4230374446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 msgr2=0x55e4e2c0d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c0d0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9814000f80 tx=0x7f9814003ce0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 -- 172.21.15.60:0/4230374446 shutdown_connections 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55e4e2c12570 0x55e4e2c11810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c0d0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c06060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 --2- 172.21.15.60:0/4230374446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2c05b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 -- 172.21.15.60:0/4230374446 >> 172.21.15.60:0/4230374446 conn(0x55e4e2bf6e20 msgr2=0x55e4e2bf6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f97fea00640 0 client.14583 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 -- 172.21.15.60:0/4230374446 shutdown_connections 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 1 -- 172.21.15.60:0/4230374446 wait complete. 2024-02-26T10:42:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.869+0000 7f9826bad740 20 client.14583 trim_cache size 0 max 16384 2024-02-26T10:42:50.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.887+0000 7f9826bad740 1 Processor -- start 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.887+0000 7f9826bad740 1 -- start start 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.887+0000 7f9826bad740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bf7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.887+0000 7f9826bad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c0d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.887+0000 7f9826bad740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 0x55e4e2c1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.887+0000 7f9826bad740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2c11d90 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.887+0000 7f9826bad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2c123f0 con 0x55e4e2bfc2a0 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.887+0000 7f9826bad740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2c12a50 con 0x55e4e2c01410 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bf7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bf7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43524/0 (socket says 172.21.15.60:43524) 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821200640 1 -- 172.21.15.60:0/2546443245 learned_addr learned my addr 172.21.15.60:0/2546443245 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f981be00640 1 --2- 172.21.15.60:0/2546443245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c0d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821c00640 1 --2- 172.21.15.60:0/2546443245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 0x55e4e2c1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821200640 1 -- 172.21.15.60:0/2546443245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 msgr2=0x55e4e2c1aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821200640 1 --2- 172.21.15.60:0/2546443245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 0x55e4e2c1aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821200640 1 -- 172.21.15.60:0/2546443245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 msgr2=0x55e4e2c0d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821200640 1 --2- 172.21.15.60:0/2546443245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c0d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821200640 1 -- 172.21.15.60:0/2546443245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4e2bf7740 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.888+0000 7f9821200640 1 --2- 172.21.15.60:0/2546443245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bf7200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9810036040 tx=0x7f9810040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d89c9a1ac03a5c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9809600640 1 -- 172.21.15.60:0/2546443245 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810051b00 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9809600640 1 -- 172.21.15.60:0/2546443245 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9810057070 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9809600640 1 -- 172.21.15.60:0/2546443245 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810005460 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9826bad740 1 -- 172.21.15.60:0/2546443245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfb8b0 msgr2=0x55e4e2bf7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9826bad740 1 --2- 172.21.15.60:0/2546443245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bf7200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9810036040 tx=0x7f9810040000 comp rx=0 tx=0).stop 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9826bad740 1 -- 172.21.15.60:0/2546443245 shutdown_connections 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9826bad740 1 --2- 172.21.15.60:0/2546443245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c0d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9826bad740 1 --2- 172.21.15.60:0/2546443245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2c01410 0x55e4e2c1aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9826bad740 1 --2- 172.21.15.60:0/2546443245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2bf7200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.889+0000 7f9826bad740 1 -- 172.21.15.60:0/2546443245 >> 172.21.15.60:0/2546443245 conn(0x55e4e2bf6e20 msgr2=0x55e4e2bf1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.890+0000 7f9826bad740 1 -- 172.21.15.60:0/2546443245 shutdown_connections 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.890+0000 7f9826bad740 1 -- 172.21.15.60:0/2546443245 wait complete. 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.890+0000 7f9826bad740 1 -- start start 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.890+0000 7f9826bad740 1 Processor -- start 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.890+0000 7f9826bad740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2c0ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9826bad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c09950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9826bad740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9826bad740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2c13710 con 0x55e4e2c01410 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9826bad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2c130b0 con 0x55e4e2bfc2a0 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9826bad740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e2c13d70 con 0x55e4e2bfb8b0 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9826bad740 10 client.0 _ll_register_callbacks cb 0x7f9814009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9821c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9821200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2c0ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9821c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c09e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43534/0 (socket says 172.21.15.60:43534) 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9821c00640 1 -- 172.21.15.60:0/120171041 learned_addr learned my addr 172.21.15.60:0/120171041 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f981be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c09950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9821c00640 1 -- 172.21.15.60:0/120171041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfb8b0 msgr2=0x55e4e2c0ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9821c00640 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2c0ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9821c00640 1 -- 172.21.15.60:0/120171041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 msgr2=0x55e4e2c09950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9821c00640 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c09950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.891+0000 7f9821c00640 1 -- 172.21.15.60:0/120171041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4e2c68170 con 0x55e4e2c01410 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.892+0000 7f9821c00640 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c09e90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f981400f790 tx=0x7f9814005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931eb04b3b17fac7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.892+0000 7f97fea00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.892+0000 7f97fea00640 1 -- 172.21.15.60:0/120171041 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814011260 con 0x55e4e2c01410 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.892+0000 7f97fea00640 1 -- 172.21.15.60:0/120171041 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9814010430 con 0x55e4e2c01410 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.892+0000 7f9826bad740 1 -- 172.21.15.60:0/120171041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9810057070 con 0x55e4e2c01410 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.892+0000 7f97fea00640 1 -- 172.21.15.60:0/120171041 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9814044050 con 0x55e4e2c01410 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.892+0000 7f9826bad740 10 client.14589 fetch_fsmap learned FSMap version 17 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.892+0000 7f9826bad740 1 -- 172.21.15.60:0/120171041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55e4e2c692c0 con 0x55e4e2c01410 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.893+0000 7f97fea00640 1 -- 172.21.15.60:0/120171041 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9810057070 con 0x55e4e2c01410 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.893+0000 7f97fea00640 1 -- 172.21.15.60:0/120171041 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9814018070 con 0x55e4e2c01410 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.893+0000 7f9826bad740 10 client.14589 fetch_fsmap finished waiting for FSMap version 17 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.893+0000 7f9826bad740 10 client.14589 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.893+0000 7f9826bad740 20 client.14589 populate_metadata read hostname 'smithi060' 2024-02-26T10:42:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.893+0000 7f9826bad740 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55e4e2c6da60 0x55e4e2c698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.893+0000 7f9826bad740 4 client.14589 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:42:50.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.893+0000 7f9826bad740 1 -- 172.21.15.60:0/120171041 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e4e2b67e60 con 0x55e4e2c6da60 2024-02-26T10:42:50.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.893+0000 7f9821200640 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55e4e2c6da60 0x55e4e2c698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9821200640 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55e4e2c6da60 0x55e4e2c698b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8dd693bdcdd7ddf server_cookie=cbe13b42307148b2 in_seq=0 out_seq=0 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f97fea00640 10 client.14589 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f97fea00640 1 -- 172.21.15.60:0/120171041 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55e4e2b67e60 con 0x55e4e2c6da60 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f97fea00640 10 client.14589 handle_command_reply: tid=0 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 client.14589 shutdown 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 -- 172.21.15.60:0/120171041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 msgr2=0x55e4e2c09e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c09e90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f981400f790 tx=0x7f9814005c50 comp rx=0 tx=0).stop 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 -- 172.21.15.60:0/120171041 shutdown_connections 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55e4e2c6da60 0x55e4e2c698b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4e2bfc2a0 0x55e4e2c09950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e4e2c01410 0x55e4e2c09e90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f97fea00640 0 client.14589 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 --2- 172.21.15.60:0/120171041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e4e2bfb8b0 0x55e4e2c0ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 -- 172.21.15.60:0/120171041 >> 172.21.15.60:0/120171041 conn(0x55e4e2bf6e20 msgr2=0x55e4e2c1aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 -- 172.21.15.60:0/120171041 shutdown_connections 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.894+0000 7f9826bad740 1 -- 172.21.15.60:0/120171041 wait complete. 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9826bad740 20 client.14589 trim_cache size 0 max 16384 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-02-26T10:42:50.896 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24587, 2024-02-26T10:42:50.897 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T10:42:50.897 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T10:42:50.897 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T10:42:50.897 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 7.8294014980000002, 2024-02-26T10:42:50.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 17, 2024-02-26T10:42:50.897 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 91, 2024-02-26T10:42:50.897 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 90, 2024-02-26T10:42:50.897 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 14.773548513 2024-02-26T10:42:50.897 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 -- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98040a1e00 msgr2=0x7f98040a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98040a1e00 0x7f98040a42c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f981002ce70 tx=0x7f9810005fd0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 -- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981c104570 msgr2=0x7f981c199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981c104570 0x7f981c199bd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e9f0 tx=0x7f980c00edf0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 -- 172.21.15.60:0/2809964609 shutdown_connections 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f98040a1e00 0x7f98040a42c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f981c104f30 0x7f981c1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981c104570 0x7f981c199bd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 --2- 172.21.15.60:0/2809964609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f981c103380 0x7f981c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 -- 172.21.15.60:0/2809964609 >> 172.21.15.60:0/2809964609 conn(0x7f981c0fe910 msgr2=0x7f981c107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 -- 172.21.15.60:0/2809964609 shutdown_connections 2024-02-26T10:42:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:50.895+0000 7f9822600640 1 -- 172.21.15.60:0/2809964609 wait complete. 2024-02-26T10:42:50.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-26T10:42:50.910 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-26T10:42:50.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph fs ls 2024-02-26T10:42:51.368 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:42:51.515 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:51 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1834064385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:51.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:51 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1834064385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:51.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:51 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1834064385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:42:52.547 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:52 smithi060 ceph-mon[31539]: pgmap v328: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 538 B/s wr, 2 op/s 2024-02-26T10:42:52.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:52 smithi113 ceph-mon[36126]: pgmap v328: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 538 B/s wr, 2 op/s 2024-02-26T10:42:52.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:52 smithi100 ceph-mon[36051]: pgmap v328: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 538 B/s wr, 2 op/s 2024-02-26T10:42:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.688+0000 7fa884600700 1 -- 172.21.15.60:0/145209066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c06a3c0 msgr2=0x7fa87c10eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.688+0000 7fa884600700 1 --2- 172.21.15.60:0/145209066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c06a3c0 0x7fa87c10eba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b0a0 tx=0x7fa87400b3b0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.688+0000 7fa884600700 1 -- 172.21.15.60:0/145209066 shutdown_connections 2024-02-26T10:42:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.688+0000 7fa884600700 1 --2- 172.21.15.60:0/145209066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c06a3c0 0x7fa87c10eba0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.688+0000 7fa884600700 1 --2- 172.21.15.60:0/145209066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87c107d80 0x7fa87c108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.688+0000 7fa884600700 1 --2- 172.21.15.60:0/145209066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa87c069a00 0x7fa87c069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.688+0000 7fa884600700 1 -- 172.21.15.60:0/145209066 >> 172.21.15.60:0/145209066 conn(0x7fa87c077a00 msgr2=0x7fa87c077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.689+0000 7fa884600700 1 -- 172.21.15.60:0/145209066 shutdown_connections 2024-02-26T10:42:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.689+0000 7fa884600700 1 -- 172.21.15.60:0/145209066 wait complete. 2024-02-26T10:42:53.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.690+0000 7fa884600700 1 Processor -- start 2024-02-26T10:42:53.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.690+0000 7fa884600700 1 -- start start 2024-02-26T10:42:53.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa884600700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c069a00 0x7fa87c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa884600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87c06a3c0 0x7fa87c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa884600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa87c107d80 0x7fa87c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa884600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c112050 con 0x7fa87c107d80 2024-02-26T10:42:53.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa884600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c111ed0 con 0x7fa87c06a3c0 2024-02-26T10:42:53.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa884600700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c1121d0 con 0x7fa87c069a00 2024-02-26T10:42:53.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa881c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c069a00 0x7fa87c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa881c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c069a00 0x7fa87c19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:57334/0 (socket says 172.21.15.60:57334) 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa881c00700 1 -- 172.21.15.60:0/2743841279 learned_addr learned my addr 172.21.15.60:0/2743841279 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa882600700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa87c107d80 0x7fa87c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa881200700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87c06a3c0 0x7fa87c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa881c00700 1 -- 172.21.15.60:0/2743841279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87c06a3c0 msgr2=0x7fa87c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa881c00700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87c06a3c0 0x7fa87c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa881c00700 1 -- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa87c107d80 msgr2=0x7fa87c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa881c00700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa87c107d80 0x7fa87c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.691+0000 7fa881c00700 1 -- 172.21.15.60:0/2743841279 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c19f670 con 0x7fa87c069a00 2024-02-26T10:42:53.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.692+0000 7fa881c00700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c069a00 0x7fa87c19a660 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa87000e9a0 tx=0x7fa87000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.692+0000 7fa87a000700 1 -- 172.21.15.60:0/2743841279 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000ca10 con 0x7fa87c069a00 2024-02-26T10:42:53.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.692+0000 7fa87a000700 1 -- 172.21.15.60:0/2743841279 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa87000cbb0 con 0x7fa87c069a00 2024-02-26T10:42:53.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.692+0000 7fa87a000700 1 -- 172.21.15.60:0/2743841279 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870018680 con 0x7fa87c069a00 2024-02-26T10:42:53.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.692+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c19f960 con 0x7fa87c069a00 2024-02-26T10:42:53.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.692+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c1a71e0 con 0x7fa87c069a00 2024-02-26T10:42:53.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.694+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c068a10 con 0x7fa87c069a00 2024-02-26T10:42:53.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.695+0000 7fa87a000700 1 -- 172.21.15.60:0/2743841279 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa870010430 con 0x7fa87c069a00 2024-02-26T10:42:53.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.696+0000 7fa87a000700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8640a1e90 0x7fa8640a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.697+0000 7fa87a000700 1 -- 172.21.15.60:0/2743841279 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa8700ce190 con 0x7fa87c069a00 2024-02-26T10:42:53.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.697+0000 7fa881200700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8640a1e90 0x7fa8640a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.697+0000 7fa881200700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8640a1e90 0x7fa8640a4350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa87c102fa0 tx=0x7fa868009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.701+0000 7fa87a000700 1 -- 172.21.15.60:0/2743841279 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa87009b320 con 0x7fa87c069a00 2024-02-26T10:42:53.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.977+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa87c1a7410 con 0x7fa87c069a00 2024-02-26T10:42:53.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.978+0000 7fa87a000700 1 -- 172.21.15.60:0/2743841279 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa870085020 con 0x7fa87c069a00 2024-02-26T10:42:53.976 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-26T10:42:53.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8640a1e90 msgr2=0x7fa8640a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8640a1e90 0x7fa8640a4350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa87c102fa0 tx=0x7fa868009260 comp rx=0 tx=0).stop 2024-02-26T10:42:53.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c069a00 msgr2=0x7fa87c19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c069a00 0x7fa87c19a660 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa87000e9a0 tx=0x7fa87000ecb0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 shutdown_connections 2024-02-26T10:42:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8640a1e90 0x7fa8640a4350 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa87c107d80 0x7fa87c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c069a00 0x7fa87c19a660 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 --2- 172.21.15.60:0/2743841279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87c06a3c0 0x7fa87c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 >> 172.21.15.60:0/2743841279 conn(0x7fa87c077a00 msgr2=0x7fa87c10ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 shutdown_connections 2024-02-26T10:42:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:53.981+0000 7fa884600700 1 -- 172.21.15.60:0/2743841279 wait complete. 2024-02-26T10:42:54.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:54 smithi113 ceph-mon[36126]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 306 B/s wr, 1 op/s 2024-02-26T10:42:54.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:54 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2743841279' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T10:42:54.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:54 smithi060 ceph-mon[31539]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 306 B/s wr, 1 op/s 2024-02-26T10:42:54.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:54 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2743841279' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T10:42:54.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:54 smithi100 ceph-mon[36051]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 306 B/s wr, 1 op/s 2024-02-26T10:42:54.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:54 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2743841279' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T10:42:54.947 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-26T10:42:54.947 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:42:54.947 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-02-26T10:42:54.947 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-02-26T10:42:54.947 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-26T10:42:54.947 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-26T10:42:54.947 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-26T10:42:54.947 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:42:54.947 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-02-26T10:42:54.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:54.947 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-02-26T10:42:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:42:55.006 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:42:55.006 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41968sec preferred_lft 41968sec 2024-02-26T10:42:55.007 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-02-26T10:42:55.008 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:55.008 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:55.008 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-02-26T10:42:55.011 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-26T10:42:55.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:55.012 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:42:55.012 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:42:55.012 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-02-26T10:42:55.012 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-02-26T10:42:55.012 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-02-26T10:42:55.012 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-26T10:42:55.012 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:42:55.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:55.466 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:42:55.550 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-02-26T10:42:55.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:55.554 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-02-26T10:42:55.610 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-02-26T10:42:55.610 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:42:55.610 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-26T10:42:55.610 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-26T10:42:55.610 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:42:55.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:55.611 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:42:55.611 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:42:55.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-26T10:42:55.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-26T10:42:55.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-26T10:42:55.611 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:42:56.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:56.335 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-02-26T10:42:56.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:56.390 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-02-26T10:42:56.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:56.436 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:42:56.436 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:42:56.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T10:42:56.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-02-26T10:42:56.437 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:42:56.565 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:56 smithi060 ceph-mon[31539]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-26T10:42:56.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:56 smithi113 ceph-mon[36126]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-26T10:42:56.628 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-02-26T10:42:56.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:56.629 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:42:56.629 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:42:56.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-02-26T10:42:56.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-26T10:42:56.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-02-26T10:42:56.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-02-26T10:42:56.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-02-26T10:42:56.629 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:42:56.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:56 smithi100 ceph-mon[36051]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-26T10:42:56.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:56.960 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:42:56.960 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:42:56.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-02-26T10:42:56.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-02-26T10:42:56.961 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:42:57.131 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-02-26T10:42:57.131 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:57.131 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-02-26T10:42:57.174 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-02-26T10:42:57.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:57.174 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-02-26T10:42:57.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-02-26T10:42:57.381 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-02-26T10:42:57.427 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T10:42:57.428 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/e5a6a3e60f08ffabc67bde51f5ab3e6cb951febb92d0339a5aaed24f9bd43ce4/merged 2024-02-26T10:42:57.429 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/253346cbc385c073ac6a4e758d751db061614c97e9468042cadd4e85c852efcc/merged 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/0 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/5968bd15e2e17a7da34c339a27aec6445d79e494fa14e24057a93d18c55db7fa/merged 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/44542eaa013e36ad5fe2a9e94d1644c5153bc2b7503f2c40139155a5b4b93e9f/merged 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/6f03bbcd9f96db8db8eba6b64f88f78d3b2a7eb1900cde3bb579a40c2295a226/merged 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/6fdc2eac3ca88b63d2eaa8bdb04ce624c7f78459321234b3323415ae258cc352/merged 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/d308c945b169c9948d1a8d4250b3ce26905169c88f61290ddaad1fbea09232c3/merged 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/17694ec7e0747de64d1a13343c8351b46ee55b1e23bdc7a7f8325ba8fcb56a5d/merged 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/ce76e8d301ac2be53c72fb3c83f140e61071f479e87a17faa5e459f8fed3bf58/merged 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/da6919f976f6b017679e7e0151a951ca362a3f0c383d3d66b90ad2aff796cbeb/merged 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T10:42:57.430 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T10:42:57.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:57.432 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-02-26T10:42:57.524 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-26T10:42:57.524 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-02-26T10:42:57.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-02-26T10:42:57.607 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-02-26T10:42:57.649 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-02-26T10:42:57.650 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/e5a6a3e60f08ffabc67bde51f5ab3e6cb951febb92d0339a5aaed24f9bd43ce4/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/253346cbc385c073ac6a4e758d751db061614c97e9468042cadd4e85c852efcc/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/0 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/5968bd15e2e17a7da34c339a27aec6445d79e494fa14e24057a93d18c55db7fa/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/44542eaa013e36ad5fe2a9e94d1644c5153bc2b7503f2c40139155a5b4b93e9f/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/6f03bbcd9f96db8db8eba6b64f88f78d3b2a7eb1900cde3bb579a40c2295a226/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/6fdc2eac3ca88b63d2eaa8bdb04ce624c7f78459321234b3323415ae258cc352/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/d308c945b169c9948d1a8d4250b3ce26905169c88f61290ddaad1fbea09232c3/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/17694ec7e0747de64d1a13343c8351b46ee55b1e23bdc7a7f8325ba8fcb56a5d/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/ce76e8d301ac2be53c72fb3c83f140e61071f479e87a17faa5e459f8fed3bf58/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/da6919f976f6b017679e7e0151a951ca362a3f0c383d3d66b90ad2aff796cbeb/merged 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T10:42:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T10:42:57.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:57.652 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-02-26T10:42:57.748 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:2024-02-26T10:42:57.751+0000 7f313483a480 -1 init, newargv = 0x560a076f2600 newargc=15 2024-02-26T10:42:57.749 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[75410]: starting ceph client 2024-02-26T10:42:57.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[75410]: starting fuse 2024-02-26T10:42:58.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:42:58 smithi113 ceph-mon[36126]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 0 op/s 2024-02-26T10:42:58.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:42:58 smithi060 ceph-mon[31539]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 0 op/s 2024-02-26T10:42:58.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:42:58 smithi100 ceph-mon[36051]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 0 op/s 2024-02-26T10:42:58.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-02-26T10:42:58.949 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-02-26T10:42:59.006 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-02-26T10:42:59.007 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/e5a6a3e60f08ffabc67bde51f5ab3e6cb951febb92d0339a5aaed24f9bd43ce4/merged 2024-02-26T10:42:59.008 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/253346cbc385c073ac6a4e758d751db061614c97e9468042cadd4e85c852efcc/merged 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/0 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/5968bd15e2e17a7da34c339a27aec6445d79e494fa14e24057a93d18c55db7fa/merged 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/44542eaa013e36ad5fe2a9e94d1644c5153bc2b7503f2c40139155a5b4b93e9f/merged 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/6f03bbcd9f96db8db8eba6b64f88f78d3b2a7eb1900cde3bb579a40c2295a226/merged 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/6fdc2eac3ca88b63d2eaa8bdb04ce624c7f78459321234b3323415ae258cc352/merged 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/d308c945b169c9948d1a8d4250b3ce26905169c88f61290ddaad1fbea09232c3/merged 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/17694ec7e0747de64d1a13343c8351b46ee55b1e23bdc7a7f8325ba8fcb56a5d/merged 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/ce76e8d301ac2be53c72fb3c83f140e61071f479e87a17faa5e459f8fed3bf58/merged 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/da6919f976f6b017679e7e0151a951ca362a3f0c383d3d66b90ad2aff796cbeb/merged 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T10:42:59.009 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T10:42:59.010 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.admin 2024-02-26T10:42:59.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:59.011 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-02-26T10:42:59.065 INFO:teuthology.orchestra.run.smithi060.stdout:118 2024-02-26T10:42:59.066 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-02-26T10:42:59.066 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> 2024-02-26T10:42:59.067 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.admin")) 2024-02-26T10:42:59.068 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-02-26T10:42:59.248 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.admin.75410.asok 2024-02-26T10:42:59.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-26T10:42:59.266 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75410.asok 2024-02-26T10:42:59.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:59.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75410.asok status 2024-02-26T10:42:59.516 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:42:59.516 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-02-26T10:42:59.516 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "6085b91fefeb74bf4310344a87d81d3217ef7893", 2024-02-26T10:42:59.516 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1686-g6085b91f (6085b91fefeb74bf4310344a87d81d3217ef7893) squid (dev)", 2024-02-26T10:42:59.516 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "admin", 2024-02-26T10:42:59.516 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-02-26T10:42:59.516 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-02-26T10:42:59.516 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "75410", 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 14601, 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 14601 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3731183979 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:42:59.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3731183979 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.14601 192.168.0.1:0/3731183979", 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3731183979", 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 17, 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 91, 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-02-26T10:42:59.518 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:42:59.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:59.532 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-02-26T10:42:59.532 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-02-26T10:42:59.586 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-02-26T10:42:59.586 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-02-26T10:42:59.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:59.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-02-26T10:42:59.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:59.688 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-02-26T10:42:59.779 INFO:teuthology.orchestra.run.smithi060.stdout:# file: . 2024-02-26T10:42:59.779 INFO:teuthology.orchestra.run.smithi060.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-02-26T10:42:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:42:59.780 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:42:59.780 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:59.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:43:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.067+0000 7fe75b200640 1 -- 172.21.15.60:0/2098724607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe75c104460 msgr2=0x7fe75c106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.067+0000 7fe75b200640 1 --2- 172.21.15.60:0/2098724607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe75c104460 0x7fe75c106850 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe748009900 tx=0x7fe74802bc40 comp rx=0 tx=0).stop 2024-02-26T10:43:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.067+0000 7fe75b200640 1 -- 172.21.15.60:0/2098724607 shutdown_connections 2024-02-26T10:43:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.067+0000 7fe75b200640 1 --2- 172.21.15.60:0/2098724607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c106d90 0x7fe75c1091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.067+0000 7fe75b200640 1 --2- 172.21.15.60:0/2098724607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe75c104460 0x7fe75c106850 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.067+0000 7fe75b200640 1 --2- 172.21.15.60:0/2098724607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c101b30 0x7fe75c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.067+0000 7fe75b200640 1 -- 172.21.15.60:0/2098724607 >> 172.21.15.60:0/2098724607 conn(0x7fe75c0fb540 msgr2=0x7fe75c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.067+0000 7fe75b200640 1 -- 172.21.15.60:0/2098724607 shutdown_connections 2024-02-26T10:43:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.067+0000 7fe75b200640 1 -- 172.21.15.60:0/2098724607 wait complete. 2024-02-26T10:43:00.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.068+0000 7fe75b200640 1 Processor -- start 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.068+0000 7fe75b200640 1 -- start start 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.068+0000 7fe75b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe75c101b30 0x7fe75c1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.068+0000 7fe75b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c104460 0x7fe75c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.068+0000 7fe75b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c106d90 0x7fe75c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.068+0000 7fe75b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c10cd00 con 0x7fe75c101b30 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.068+0000 7fe75b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c10cb80 con 0x7fe75c106d90 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.068+0000 7fe75b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c10ce80 con 0x7fe75c104460 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe75a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c106d90 0x7fe75c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe75a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c106d90 0x7fe75c19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53544/0 (socket says 172.21.15.60:53544) 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe75a800640 1 -- 172.21.15.60:0/627934918 learned_addr learned my addr 172.21.15.60:0/627934918 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe759e00640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe75c101b30 0x7fe75c1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe759400640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c104460 0x7fe75c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe75a800640 1 -- 172.21.15.60:0/627934918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c104460 msgr2=0x7fe75c197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe75a800640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c104460 0x7fe75c197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe75a800640 1 -- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe75c101b30 msgr2=0x7fe75c1973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe75a800640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe75c101b30 0x7fe75c1973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.069+0000 7fe75a800640 1 -- 172.21.15.60:0/627934918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c0ff9a0 con 0x7fe75c106d90 2024-02-26T10:43:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.070+0000 7fe75a800640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c106d90 0x7fe75c19e980 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe75400eb10 tx=0x7fe75400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.070+0000 7fe752a00640 1 -- 172.21.15.60:0/627934918 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754004b20 con 0x7fe75c106d90 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.070+0000 7fe752a00640 1 -- 172.21.15.60:0/627934918 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe754004cc0 con 0x7fe75c106d90 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.070+0000 7fe752a00640 1 -- 172.21.15.60:0/627934918 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754010760 con 0x7fe75c106d90 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.070+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c0ffc90 con 0x7fe75c106d90 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.070+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c1001b0 con 0x7fe75c106d90 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.071+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c1015f0 con 0x7fe75c106d90 2024-02-26T10:43:00.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.073+0000 7fe752a00640 1 -- 172.21.15.60:0/627934918 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe754005000 con 0x7fe75c106d90 2024-02-26T10:43:00.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.075+0000 7fe752a00640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7400a1e00 0x7fe7400a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.075+0000 7fe752a00640 1 -- 172.21.15.60:0/627934918 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe7540ce740 con 0x7fe75c106d90 2024-02-26T10:43:00.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.075+0000 7fe759e00640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7400a1e00 0x7fe7400a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.076+0000 7fe759e00640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7400a1e00 0x7fe7400a42c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0095f0 tx=0x7fe74c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:00.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.077+0000 7fe752a00640 1 -- 172.21.15.60:0/627934918 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe754052d20 con 0x7fe75c106d90 2024-02-26T10:43:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.281+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe75c0fccf0 con 0x7fe75c106d90 2024-02-26T10:43:00.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.283+0000 7fe752a00640 1 -- 172.21.15.60:0/627934918 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7fe75409b8d0 con 0x7fe75c106d90 2024-02-26T10:43:00.281 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:00.281 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7400a1e00 msgr2=0x7fe7400a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7400a1e00 0x7fe7400a42c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0095f0 tx=0x7fe74c005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c106d90 msgr2=0x7fe75c19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c106d90 0x7fe75c19e980 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe75400eb10 tx=0x7fe75400c2d0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 shutdown_connections 2024-02-26T10:43:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7400a1e00 0x7fe7400a42c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c106d90 0x7fe75c19e980 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75c104460 0x7fe75c197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 --2- 172.21.15.60:0/627934918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe75c101b30 0x7fe75c1973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 >> 172.21.15.60:0/627934918 conn(0x7fe75c0fb540 msgr2=0x7fe75c102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 shutdown_connections 2024-02-26T10:43:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.286+0000 7fe75b200640 1 -- 172.21.15.60:0/627934918 wait complete. 2024-02-26T10:43:00.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:43:00.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:43:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.548+0000 7ff6e3400640 1 -- 172.21.15.60:0/57730347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc104690 msgr2=0x7ff6dc104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.548+0000 7ff6e3400640 1 --2- 172.21.15.60:0/57730347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc104690 0x7ff6dc104b10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009900 tx=0x7ff6d002bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.548+0000 7ff6e3400640 1 -- 172.21.15.60:0/57730347 shutdown_connections 2024-02-26T10:43:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.548+0000 7ff6e3400640 1 --2- 172.21.15.60:0/57730347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6dc105050 0x7ff6dc10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.548+0000 7ff6e3400640 1 --2- 172.21.15.60:0/57730347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc104690 0x7ff6dc104b10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.548+0000 7ff6e3400640 1 --2- 172.21.15.60:0/57730347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6dc1034a0 0x7ff6dc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.548+0000 7ff6e3400640 1 -- 172.21.15.60:0/57730347 >> 172.21.15.60:0/57730347 conn(0x7ff6dc0fea10 msgr2=0x7ff6dc100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.549+0000 7ff6e3400640 1 -- 172.21.15.60:0/57730347 shutdown_connections 2024-02-26T10:43:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.549+0000 7ff6e3400640 1 -- 172.21.15.60:0/57730347 wait complete. 2024-02-26T10:43:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.549+0000 7ff6e3400640 1 Processor -- start 2024-02-26T10:43:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e3400640 1 -- start start 2024-02-26T10:43:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc1034a0 0x7ff6dc199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6dc104690 0x7ff6dc199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6dc105050 0x7ff6dc1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e3400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc10f180 con 0x7ff6dc1034a0 2024-02-26T10:43:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e3400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc10f000 con 0x7ff6dc105050 2024-02-26T10:43:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e3400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc10f300 con 0x7ff6dc104690 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e0a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc1034a0 0x7ff6dc199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e0a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc1034a0 0x7ff6dc199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52908/0 (socket says 172.21.15.60:52908) 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e0a00640 1 -- 172.21.15.60:0/3631675686 learned_addr learned my addr 172.21.15.60:0/3631675686 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6dbe00640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6dc104690 0x7ff6dc199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.550+0000 7ff6e1400640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6dc105050 0x7ff6dc1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6e0a00640 1 -- 172.21.15.60:0/3631675686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6dc104690 msgr2=0x7ff6dc199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6e0a00640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6dc104690 0x7ff6dc199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6e0a00640 1 -- 172.21.15.60:0/3631675686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6dc105050 msgr2=0x7ff6dc1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6e0a00640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6dc105050 0x7ff6dc1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6e0a00640 1 -- 172.21.15.60:0/3631675686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc1a1390 con 0x7ff6dc1034a0 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6dbe00640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6dc104690 0x7ff6dc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6e0a00640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc1034a0 0x7ff6dc199770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00e9c0 tx=0x7ff6cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:00.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6d9600640 1 -- 172.21.15.60:0/3631675686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc00cde0 con 0x7ff6dc1034a0 2024-02-26T10:43:00.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6d9600640 1 -- 172.21.15.60:0/3631675686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6cc010910 con 0x7ff6dc1034a0 2024-02-26T10:43:00.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.551+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc1a1680 con 0x7ff6dc1034a0 2024-02-26T10:43:00.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.552+0000 7ff6d9600640 1 -- 172.21.15.60:0/3631675686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc018a00 con 0x7ff6dc1034a0 2024-02-26T10:43:00.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.552+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc1a1ba0 con 0x7ff6dc1034a0 2024-02-26T10:43:00.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.553+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc10a7c0 con 0x7ff6dc1034a0 2024-02-26T10:43:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.555+0000 7ff6d9600640 1 -- 172.21.15.60:0/3631675686 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff6cc01f020 con 0x7ff6dc1034a0 2024-02-26T10:43:00.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.556+0000 7ff6d9600640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff6c40a1d80 0x7ff6c40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.556+0000 7ff6d9600640 1 -- 172.21.15.60:0/3631675686 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff6cc0ce220 con 0x7ff6dc1034a0 2024-02-26T10:43:00.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.556+0000 7ff6dbe00640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff6c40a1d80 0x7ff6c40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.557+0000 7ff6dbe00640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff6c40a1d80 0x7ff6c40a4240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff6d002c240 tx=0x7ff6d002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:00.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.561+0000 7ff6d9600640 1 -- 172.21.15.60:0/3631675686 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6cc09b3b0 con 0x7ff6dc1034a0 2024-02-26T10:43:00.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:00 smithi113 ceph-mon[36126]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 95 B/s wr, 0 op/s 2024-02-26T10:43:00.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:00 smithi060 ceph-mon[31539]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 95 B/s wr, 0 op/s 2024-02-26T10:43:00.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:00 smithi100 ceph-mon[36051]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 95 B/s wr, 0 op/s 2024-02-26T10:43:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.760+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6dc061e60 con 0x7ff6dc1034a0 2024-02-26T10:43:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.761+0000 7ff6d9600640 1 -- 172.21.15.60:0/3631675686 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6cc09acb0 con 0x7ff6dc1034a0 2024-02-26T10:43:00.759 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff6c40a1d80 msgr2=0x7ff6c40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff6c40a1d80 0x7ff6c40a4240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff6d002c240 tx=0x7ff6d002df90 comp rx=0 tx=0).stop 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc1034a0 msgr2=0x7ff6dc199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc1034a0 0x7ff6dc199770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00e9c0 tx=0x7ff6cc00ee80 comp rx=0 tx=0).stop 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 shutdown_connections 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff6c40a1d80 0x7ff6c40a4240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6dc105050 0x7ff6dc1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6dc104690 0x7ff6dc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 --2- 172.21.15.60:0/3631675686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc1034a0 0x7ff6dc199770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 >> 172.21.15.60:0/3631675686 conn(0x7ff6dc0fea10 msgr2=0x7ff6dc107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 shutdown_connections 2024-02-26T10:43:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:00.764+0000 7ff6e3400640 1 -- 172.21.15.60:0/3631675686 wait complete. 2024-02-26T10:43:00.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:43:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.004+0000 7f80a4400640 1 -- 172.21.15.60:0/855795501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 msgr2=0x7f809c104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.004+0000 7f80a4400640 1 --2- 172.21.15.60:0/855795501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 0x7f809c104b10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8088009900 tx=0x7f808802bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.004+0000 7f80a4400640 1 -- 172.21.15.60:0/855795501 shutdown_connections 2024-02-26T10:43:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.004+0000 7f80a4400640 1 --2- 172.21.15.60:0/855795501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 0x7f809c104b10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.004+0000 7f80a4400640 1 --2- 172.21.15.60:0/855795501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f809c105050 0x7f809c10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.004+0000 7f80a4400640 1 --2- 172.21.15.60:0/855795501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f809c1034a0 0x7f809c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.004+0000 7f80a4400640 1 -- 172.21.15.60:0/855795501 >> 172.21.15.60:0/855795501 conn(0x7f809c0fea10 msgr2=0x7f809c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.005+0000 7f80a4400640 1 -- 172.21.15.60:0/855795501 shutdown_connections 2024-02-26T10:43:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.005+0000 7f80a4400640 1 -- 172.21.15.60:0/855795501 wait complete. 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.005+0000 7f80a4400640 1 Processor -- start 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a4400640 1 -- start start 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a4400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f809c1034a0 0x7f809c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a4400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 0x7f809c199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a4400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f809c105050 0x7f809c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a4400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c10f180 con 0x7f809c105050 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a4400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c10f000 con 0x7f809c104690 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a4400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c10f300 con 0x7f809c1034a0 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 0x7f809c199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 0x7f809c199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53584/0 (socket says 172.21.15.60:53584) 2024-02-26T10:43:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a1000640 1 -- 172.21.15.60:0/1524953188 learned_addr learned my addr 172.21.15.60:0/1524953188 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a2400640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f809c105050 0x7f809c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.006+0000 7f80a1a00640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f809c1034a0 0x7f809c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f80a1000640 1 -- 172.21.15.60:0/1524953188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f809c1034a0 msgr2=0x7f809c199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f80a1000640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f809c1034a0 0x7f809c199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f80a1000640 1 -- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f809c105050 msgr2=0x7f809c1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f80a1000640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f809c105050 0x7f809c1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f80a1000640 1 -- 172.21.15.60:0/1524953188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c1a13b0 con 0x7f809c104690 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f80a2400640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f809c105050 0x7f809c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f80a1000640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 0x7f809c199cd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8088000c00 tx=0x7f808802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f809a000640 1 -- 172.21.15.60:0/1524953188 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088043020 con 0x7f809c104690 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f809a000640 1 -- 172.21.15.60:0/1524953188 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80880054e0 con 0x7f809c104690 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f809a000640 1 -- 172.21.15.60:0/1524953188 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088003c40 con 0x7f809c104690 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.007+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c1a1640 con 0x7f809c104690 2024-02-26T10:43:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.008+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c1a1b90 con 0x7f809c104690 2024-02-26T10:43:01.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.009+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c10a7c0 con 0x7f809c104690 2024-02-26T10:43:01.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.011+0000 7f809a000640 1 -- 172.21.15.60:0/1524953188 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8088046020 con 0x7f809c104690 2024-02-26T10:43:01.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.012+0000 7f809a000640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f80840a1d80 0x7f80840a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.012+0000 7f809a000640 1 -- 172.21.15.60:0/1524953188 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f80880f0520 con 0x7f809c104690 2024-02-26T10:43:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.012+0000 7f80a1a00640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f80840a1d80 0x7f80840a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.013+0000 7f80a1a00640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f80840a1d80 0x7f80840a4240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f809c1044f0 tx=0x7f8090005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.014+0000 7f809a000640 1 -- 172.21.15.60:0/1524953188 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f808804d020 con 0x7f809c104690 2024-02-26T10:43:01.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.215+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f809c02d080 con 0x7f809c104690 2024-02-26T10:43:01.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.216+0000 7f809a000640 1 -- 172.21.15.60:0/1524953188 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80880bd6b0 con 0x7f809c104690 2024-02-26T10:43:01.214 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:43:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f80840a1d80 msgr2=0x7f80840a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f80840a1d80 0x7f80840a4240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f809c1044f0 tx=0x7f8090005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 msgr2=0x7f809c199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 0x7f809c199cd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8088000c00 tx=0x7f808802c650 comp rx=0 tx=0).stop 2024-02-26T10:43:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 shutdown_connections 2024-02-26T10:43:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f80840a1d80 0x7f80840a4240 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f809c104690 0x7f809c199cd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f809c105050 0x7f809c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 --2- 172.21.15.60:0/1524953188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f809c1034a0 0x7f809c199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 >> 172.21.15.60:0/1524953188 conn(0x7f809c0fea10 msgr2=0x7f809c107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 shutdown_connections 2024-02-26T10:43:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.219+0000 7f80a4400640 1 -- 172.21.15.60:0/1524953188 wait complete. 2024-02-26T10:43:01.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:43:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.458+0000 7f459a600640 1 -- 172.21.15.60:0/2952552341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4594101bc0 msgr2=0x7f4594103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.458+0000 7f459a600640 1 --2- 172.21.15.60:0/2952552341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4594101bc0 0x7f4594103fb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4584009ca0 tx=0x7f458402c0f0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.458+0000 7f459a600640 1 -- 172.21.15.60:0/2952552341 shutdown_connections 2024-02-26T10:43:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.458+0000 7f459a600640 1 --2- 172.21.15.60:0/2952552341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45941044f0 0x7f45941068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.458+0000 7f459a600640 1 --2- 172.21.15.60:0/2952552341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4594106e20 0x7f4594109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.458+0000 7f459a600640 1 --2- 172.21.15.60:0/2952552341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4594101bc0 0x7f4594103fb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.458+0000 7f459a600640 1 -- 172.21.15.60:0/2952552341 >> 172.21.15.60:0/2952552341 conn(0x7f45940fb5d0 msgr2=0x7f45940fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.459+0000 7f459a600640 1 -- 172.21.15.60:0/2952552341 shutdown_connections 2024-02-26T10:43:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.459+0000 7f459a600640 1 -- 172.21.15.60:0/2952552341 wait complete. 2024-02-26T10:43:01.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.460+0000 7f459a600640 1 Processor -- start 2024-02-26T10:43:01.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.460+0000 7f459a600640 1 -- start start 2024-02-26T10:43:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.460+0000 7f459a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4594101bc0 0x7f4594199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.460+0000 7f459a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45941044f0 0x7f4594199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.460+0000 7f459a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4594106e20 0x7f45941a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.460+0000 7f459a600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459410cdd0 con 0x7f4594106e20 2024-02-26T10:43:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f459a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459410cc50 con 0x7f45941044f0 2024-02-26T10:43:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f459a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459410cf50 con 0x7f4594101bc0 2024-02-26T10:43:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f458be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45941044f0 0x7f4594199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f458be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45941044f0 0x7f4594199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53598/0 (socket says 172.21.15.60:53598) 2024-02-26T10:43:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f458be00640 1 -- 172.21.15.60:0/3403600059 learned_addr learned my addr 172.21.15.60:0/3403600059 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f4599c00640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4594106e20 0x7f45941a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f4599200640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4594101bc0 0x7f4594199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f458be00640 1 -- 172.21.15.60:0/3403600059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4594101bc0 msgr2=0x7f4594199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f458be00640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4594101bc0 0x7f4594199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f458be00640 1 -- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4594106e20 msgr2=0x7f45941a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f458be00640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4594106e20 0x7f45941a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f458be00640 1 -- 172.21.15.60:0/3403600059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45941a1280 con 0x7f45941044f0 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.461+0000 7f4599200640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4594101bc0 0x7f4594199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.462+0000 7f458be00640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45941044f0 0x7f4594199ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e9a0 tx=0x7f457c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.462+0000 7f4592000640 1 -- 172.21.15.60:0/3403600059 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c00cd80 con 0x7f45941044f0 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.462+0000 7f4592000640 1 -- 172.21.15.60:0/3403600059 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f457c0035f0 con 0x7f45941044f0 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.462+0000 7f4592000640 1 -- 172.21.15.60:0/3403600059 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c010670 con 0x7f45941044f0 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.462+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45941a1570 con 0x7f45941044f0 2024-02-26T10:43:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.462+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45941a1a70 con 0x7f45941044f0 2024-02-26T10:43:01.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.463+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4594108970 con 0x7f45941044f0 2024-02-26T10:43:01.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.465+0000 7f4592000640 1 -- 172.21.15.60:0/3403600059 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f457c0037a0 con 0x7f45941044f0 2024-02-26T10:43:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.466+0000 7f4592000640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45780a1d80 0x7f45780a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.466+0000 7f4592000640 1 -- 172.21.15.60:0/3403600059 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f457c0ce260 con 0x7f45941044f0 2024-02-26T10:43:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.467+0000 7f4599200640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45780a1d80 0x7f45780a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.467+0000 7f4599200640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45780a1d80 0x7f45780a4240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f458402c700 tx=0x7f4584005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.470+0000 7f4592000640 1 -- 172.21.15.60:0/3403600059 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f457c09b3f0 con 0x7f45941044f0 2024-02-26T10:43:01.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:01 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/627934918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:01.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:01 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3631675686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:43:01.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:01 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1524953188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:43:01.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:01 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/627934918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:01.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:01 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3631675686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:43:01.618 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:01 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1524953188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:43:01.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:01 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/627934918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:01.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:01 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3631675686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:43:01.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:01 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1524953188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:43:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.675+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4594061e60 con 0x7f45941044f0 2024-02-26T10:43:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.676+0000 7f4592000640 1 -- 172.21.15.60:0/3403600059 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f457c09acf0 con 0x7f45941044f0 2024-02-26T10:43:01.673 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:43:01.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45780a1d80 msgr2=0x7f45780a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45780a1d80 0x7f45780a4240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f458402c700 tx=0x7f4584005ef0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45941044f0 msgr2=0x7f4594199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45941044f0 0x7f4594199ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e9a0 tx=0x7f457c00ee60 comp rx=0 tx=0).stop 2024-02-26T10:43:01.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 shutdown_connections 2024-02-26T10:43:01.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45780a1d80 0x7f45780a4240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45941044f0 0x7f4594199ba0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4594106e20 0x7f45941a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 --2- 172.21.15.60:0/3403600059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4594101bc0 0x7f4594199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 >> 172.21.15.60:0/3403600059 conn(0x7f45940fb5d0 msgr2=0x7f4594101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:01.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 shutdown_connections 2024-02-26T10:43:01.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.679+0000 7f459a600640 1 -- 172.21.15.60:0/3403600059 wait complete. 2024-02-26T10:43:01.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:43:01.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.923+0000 7f848d000640 1 -- 172.21.15.60:0/1844865845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8488108970 msgr2=0x7f8488108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.923+0000 7f848d000640 1 --2- 172.21.15.60:0/1844865845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8488108970 0x7f8488108df0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8470009900 tx=0x7f847002bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.923+0000 7f848d000640 1 -- 172.21.15.60:0/1844865845 shutdown_connections 2024-02-26T10:43:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.923+0000 7f848d000640 1 --2- 172.21.15.60:0/1844865845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8488108970 0x7f8488108df0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.923+0000 7f848d000640 1 --2- 172.21.15.60:0/1844865845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8488109330 0x7f848810fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.923+0000 7f848d000640 1 --2- 172.21.15.60:0/1844865845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8488107780 0x7f8488107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.923+0000 7f848d000640 1 -- 172.21.15.60:0/1844865845 >> 172.21.15.60:0/1844865845 conn(0x7f8488076730 msgr2=0x7f8488078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.923+0000 7f848d000640 1 -- 172.21.15.60:0/1844865845 shutdown_connections 2024-02-26T10:43:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.923+0000 7f848d000640 1 -- 172.21.15.60:0/1844865845 wait complete. 2024-02-26T10:43:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.924+0000 7f848d000640 1 Processor -- start 2024-02-26T10:43:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f848d000640 1 -- start start 2024-02-26T10:43:01.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f848d000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8488107780 0x7f848819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f848d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8488108970 0x7f848819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f848d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8488109330 0x7f84881a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f848d000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8488113400 con 0x7f8488107780 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f848d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8488113280 con 0x7f8488109330 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f848d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8488113580 con 0x7f8488108970 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f8487400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8488107780 0x7f848819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f847fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8488108970 0x7f848819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f8487400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8488107780 0x7f848819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52968/0 (socket says 172.21.15.60:52968) 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f8487400640 1 -- 172.21.15.60:0/4189786265 learned_addr learned my addr 172.21.15.60:0/4189786265 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f847fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8488108970 0x7f848819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:36594/0 (socket says 172.21.15.60:36594) 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.925+0000 7f8487e00640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8488109330 0x7f84881a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8487e00640 1 -- 172.21.15.60:0/4189786265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8488108970 msgr2=0x7f848819df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8487e00640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8488108970 0x7f848819df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8487e00640 1 -- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8488107780 msgr2=0x7f848819da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8487e00640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8488107780 0x7f848819da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8487e00640 1 -- 172.21.15.60:0/4189786265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84881a5640 con 0x7f8488109330 2024-02-26T10:43:01.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8487400640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8488107780 0x7f848819da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:01.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8487e00640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8488109330 0x7f84881a4fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8480009fd0 tx=0x7f848000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8484c00640 1 -- 172.21.15.60:0/4189786265 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848000ce20 con 0x7f8488109330 2024-02-26T10:43:01.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8484c00640 1 -- 172.21.15.60:0/4189786265 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8480005020 con 0x7f8488109330 2024-02-26T10:43:01.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84881a5930 con 0x7f8488109330 2024-02-26T10:43:01.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.926+0000 7f8484c00640 1 -- 172.21.15.60:0/4189786265 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480010720 con 0x7f8488109330 2024-02-26T10:43:01.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.927+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84881a5e50 con 0x7f8488109330 2024-02-26T10:43:01.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.928+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84880683c0 con 0x7f8488109330 2024-02-26T10:43:01.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.930+0000 7f8484c00640 1 -- 172.21.15.60:0/4189786265 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8480005540 con 0x7f8488109330 2024-02-26T10:43:01.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.931+0000 7f8484c00640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f846c0a1d80 0x7f846c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.931+0000 7f8484c00640 1 -- 172.21.15.60:0/4189786265 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f84800ce660 con 0x7f8488109330 2024-02-26T10:43:01.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.931+0000 7f8487400640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f846c0a1d80 0x7f846c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.932+0000 7f8487400640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f846c0a1d80 0x7f846c0a4240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f84780058c0 tx=0x7f8478005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:01.935+0000 7f8484c00640 1 -- 172.21.15.60:0/4189786265 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f848009b7f0 con 0x7f8488109330 2024-02-26T10:43:02.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.137+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8488061e60 con 0x7f8488109330 2024-02-26T10:43:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.138+0000 7f8484c00640 1 -- 172.21.15.60:0/4189786265 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f848009b0f0 con 0x7f8488109330 2024-02-26T10:43:02.135 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:43:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.140+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f846c0a1d80 msgr2=0x7f846c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.140+0000 7f848d000640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f846c0a1d80 0x7f846c0a4240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f84780058c0 tx=0x7f8478005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.140+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8488109330 msgr2=0x7f84881a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.140+0000 7f848d000640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8488109330 0x7f84881a4fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8480009fd0 tx=0x7f848000ef10 comp rx=0 tx=0).stop 2024-02-26T10:43:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.141+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 shutdown_connections 2024-02-26T10:43:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.141+0000 7f848d000640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f846c0a1d80 0x7f846c0a4240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.141+0000 7f848d000640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8488109330 0x7f84881a4fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.141+0000 7f848d000640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8488108970 0x7f848819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.141+0000 7f848d000640 1 --2- 172.21.15.60:0/4189786265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8488107780 0x7f848819da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.141+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 >> 172.21.15.60:0/4189786265 conn(0x7f8488076730 msgr2=0x7f848810bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.141+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 shutdown_connections 2024-02-26T10:43:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.141+0000 7f848d000640 1 -- 172.21.15.60:0/4189786265 wait complete. 2024-02-26T10:43:02.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:43:02.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.390+0000 7f9531600640 1 -- 172.21.15.60:0/2023609152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c108970 msgr2=0x7f952c108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.390+0000 7f9531600640 1 --2- 172.21.15.60:0/2023609152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c108970 0x7f952c108df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9514009900 tx=0x7f951402bc40 comp rx=0 tx=0).stop 2024-02-26T10:43:02.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.390+0000 7f9531600640 1 -- 172.21.15.60:0/2023609152 shutdown_connections 2024-02-26T10:43:02.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.390+0000 7f9531600640 1 --2- 172.21.15.60:0/2023609152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c108970 0x7f952c108df0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.390+0000 7f9531600640 1 --2- 172.21.15.60:0/2023609152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f952c109330 0x7f952c10fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.390+0000 7f9531600640 1 --2- 172.21.15.60:0/2023609152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c107780 0x7f952c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.390+0000 7f9531600640 1 -- 172.21.15.60:0/2023609152 >> 172.21.15.60:0/2023609152 conn(0x7f952c076730 msgr2=0x7f952c078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.390+0000 7f9531600640 1 -- 172.21.15.60:0/2023609152 shutdown_connections 2024-02-26T10:43:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.390+0000 7f9531600640 1 -- 172.21.15.60:0/2023609152 wait complete. 2024-02-26T10:43:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.391+0000 7f9531600640 1 Processor -- start 2024-02-26T10:43:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.391+0000 7f9531600640 1 -- start start 2024-02-26T10:43:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.391+0000 7f9531600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c107780 0x7f952c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.391+0000 7f9531600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c108970 0x7f952c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.391+0000 7f9531600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f952c109330 0x7f952c1a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.391+0000 7f9531600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c113430 con 0x7f952c108970 2024-02-26T10:43:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.391+0000 7f9531600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c1132b0 con 0x7f952c107780 2024-02-26T10:43:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.391+0000 7f9531600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c1135b0 con 0x7f952c109330 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f9523e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c108970 0x7f952c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f952be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c107780 0x7f952c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f9523e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c108970 0x7f952c19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52982/0 (socket says 172.21.15.60:52982) 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f9523e00640 1 -- 172.21.15.60:0/3683586138 learned_addr learned my addr 172.21.15.60:0/3683586138 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f9530c00640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f952c109330 0x7f952c1a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f9523e00640 1 -- 172.21.15.60:0/3683586138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f952c109330 msgr2=0x7f952c1a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f9523e00640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f952c109330 0x7f952c1a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f9523e00640 1 -- 172.21.15.60:0/3683586138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c107780 msgr2=0x7f952c19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f9523e00640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c107780 0x7f952c19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f9523e00640 1 -- 172.21.15.60:0/3683586138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c1a5670 con 0x7f952c108970 2024-02-26T10:43:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.392+0000 7f952be00640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c107780 0x7f952c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.393+0000 7f9523e00640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c108970 0x7f952c19df90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f951402c8f0 tx=0x7f9514005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.393+0000 7f9529600640 1 -- 172.21.15.60:0/3683586138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514041070 con 0x7f952c108970 2024-02-26T10:43:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.393+0000 7f9529600640 1 -- 172.21.15.60:0/3683586138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95140044d0 con 0x7f952c108970 2024-02-26T10:43:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.393+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c1a5900 con 0x7f952c108970 2024-02-26T10:43:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.393+0000 7f9529600640 1 -- 172.21.15.60:0/3683586138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514034440 con 0x7f952c108970 2024-02-26T10:43:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.393+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c1a5df0 con 0x7f952c108970 2024-02-26T10:43:02.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.394+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c10eaa0 con 0x7f952c108970 2024-02-26T10:43:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.396+0000 7f9529600640 1 -- 172.21.15.60:0/3683586138 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9514044070 con 0x7f952c108970 2024-02-26T10:43:02.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.397+0000 7f9529600640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f95100a1d80 0x7f95100a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.398+0000 7f9529600640 1 -- 172.21.15.60:0/3683586138 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f95140efbb0 con 0x7f952c108970 2024-02-26T10:43:02.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.398+0000 7f952be00640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f95100a1d80 0x7f95100a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.398+0000 7f952be00640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f95100a1d80 0x7f95100a4240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f952c1087d0 tx=0x7f951c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.402+0000 7f9529600640 1 -- 172.21.15.60:0/3683586138 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95140bc6f0 con 0x7f952c108970 2024-02-26T10:43:02.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:02 smithi113 ceph-mon[36126]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-02-26T10:43:02.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:02 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3403600059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:43:02.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:02 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/4189786265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:43:02.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.617+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f952c061e60 con 0x7f952c108970 2024-02-26T10:43:02.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.618+0000 7f9529600640 1 -- 172.21.15.60:0/3683586138 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+587 (secure 0 0 0) 0x7f95140c0790 con 0x7f952c108970 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/603652410 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3463459063 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2587364905 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/603652410 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/116674454 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/61419351 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3598708487 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1853267621 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/2948072642 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:02.616 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/2948072642 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:listed 10 entries 2024-02-26T10:43:02.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:02 smithi060 ceph-mon[31539]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-02-26T10:43:02.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:02 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3403600059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:43:02.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:02 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4189786265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:43:02.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.621+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f95100a1d80 msgr2=0x7f95100a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.621+0000 7f9531600640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f95100a1d80 0x7f95100a4240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f952c1087d0 tx=0x7f951c005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:02.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.621+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c108970 msgr2=0x7f952c19df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.621+0000 7f9531600640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c108970 0x7f952c19df90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f951402c8f0 tx=0x7f9514005d50 comp rx=0 tx=0).stop 2024-02-26T10:43:02.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.622+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 shutdown_connections 2024-02-26T10:43:02.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.622+0000 7f9531600640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f95100a1d80 0x7f95100a4240 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.622+0000 7f9531600640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f952c109330 0x7f952c1a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.622+0000 7f9531600640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c108970 0x7f952c19df90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.622+0000 7f9531600640 1 --2- 172.21.15.60:0/3683586138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c107780 0x7f952c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.622+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 >> 172.21.15.60:0/3683586138 conn(0x7f952c076730 msgr2=0x7f952c10bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:02.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.622+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 shutdown_connections 2024-02-26T10:43:02.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.622+0000 7f9531600640 1 -- 172.21.15.60:0/3683586138 wait complete. 2024-02-26T10:43:02.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:02 smithi100 ceph-mon[36051]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-02-26T10:43:02.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:02 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3403600059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:43:02.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:02 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/4189786265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:43:02.640 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:43:02.640 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:43:02.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:43:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.909+0000 7efd72200640 1 -- 172.21.15.60:0/540479113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c105030 msgr2=0x7efd6c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.909+0000 7efd72200640 1 --2- 172.21.15.60:0/540479113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c105030 0x7efd6c10b8c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd64009be0 tx=0x7efd6402bef0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.909+0000 7efd72200640 1 -- 172.21.15.60:0/540479113 shutdown_connections 2024-02-26T10:43:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.909+0000 7efd72200640 1 --2- 172.21.15.60:0/540479113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c105030 0x7efd6c10b8c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.909+0000 7efd72200640 1 --2- 172.21.15.60:0/540479113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c104670 0x7efd6c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.909+0000 7efd72200640 1 --2- 172.21.15.60:0/540479113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c103480 0x7efd6c1038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.909+0000 7efd72200640 1 -- 172.21.15.60:0/540479113 >> 172.21.15.60:0/540479113 conn(0x7efd6c0fea10 msgr2=0x7efd6c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.910+0000 7efd72200640 1 -- 172.21.15.60:0/540479113 shutdown_connections 2024-02-26T10:43:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.910+0000 7efd72200640 1 -- 172.21.15.60:0/540479113 wait complete. 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd72200640 1 Processor -- start 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd72200640 1 -- start start 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd72200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c103480 0x7efd6c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd72200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c104670 0x7efd6c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd72200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c105030 0x7efd6c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd72200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c10f130 con 0x7efd6c104670 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd72200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c10efb0 con 0x7efd6c103480 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd72200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c10f2b0 con 0x7efd6c105030 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd6aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c104670 0x7efd6c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd6b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c103480 0x7efd6c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd6b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c103480 0x7efd6c199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53650/0 (socket says 172.21.15.60:53650) 2024-02-26T10:43:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd6b400640 1 -- 172.21.15.60:0/1588227928 learned_addr learned my addr 172.21.15.60:0/1588227928 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.911+0000 7efd6be00640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c105030 0x7efd6c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.912+0000 7efd6b400640 1 -- 172.21.15.60:0/1588227928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c105030 msgr2=0x7efd6c1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.912+0000 7efd6b400640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c105030 0x7efd6c1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.912+0000 7efd6b400640 1 -- 172.21.15.60:0/1588227928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c104670 msgr2=0x7efd6c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.912+0000 7efd6b400640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c104670 0x7efd6c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.912+0000 7efd6b400640 1 -- 172.21.15.60:0/1588227928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c1a1390 con 0x7efd6c103480 2024-02-26T10:43:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.912+0000 7efd6b400640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c103480 0x7efd6c199770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00e9c0 tx=0x7efd5c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.912+0000 7efd63e00640 1 -- 172.21.15.60:0/1588227928 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c00ccc0 con 0x7efd6c103480 2024-02-26T10:43:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.912+0000 7efd63e00640 1 -- 172.21.15.60:0/1588227928 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd5c0035f0 con 0x7efd6c103480 2024-02-26T10:43:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.912+0000 7efd63e00640 1 -- 172.21.15.60:0/1588227928 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c00ce60 con 0x7efd6c103480 2024-02-26T10:43:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.913+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c1a1680 con 0x7efd6c103480 2024-02-26T10:43:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.913+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c1a1b80 con 0x7efd6c103480 2024-02-26T10:43:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.914+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c10a740 con 0x7efd6c103480 2024-02-26T10:43:02.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.916+0000 7efd63e00640 1 -- 172.21.15.60:0/1588227928 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7efd5c0037a0 con 0x7efd6c103480 2024-02-26T10:43:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.918+0000 7efd63e00640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efd500a1e00 0x7efd500a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.918+0000 7efd63e00640 1 -- 172.21.15.60:0/1588227928 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efd5c0ce440 con 0x7efd6c103480 2024-02-26T10:43:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.918+0000 7efd6aa00640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efd500a1e00 0x7efd500a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.919+0000 7efd6aa00640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efd500a1e00 0x7efd500a42c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efd6c19ac80 tx=0x7efd58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:02.920+0000 7efd63e00640 1 -- 172.21.15.60:0/1588227928 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd5c09b5d0 con 0x7efd6c103480 2024-02-26T10:43:03.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.121+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd6c02d020 con 0x7efd6c103480 2024-02-26T10:43:03.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.124+0000 7efd63e00640 1 -- 172.21.15.60:0/1588227928 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7efd5c09aed0 con 0x7efd6c103480 2024-02-26T10:43:03.121 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:03.121 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.126+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efd500a1e00 msgr2=0x7efd500a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.126+0000 7efd72200640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efd500a1e00 0x7efd500a42c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efd6c19ac80 tx=0x7efd58009260 comp rx=0 tx=0).stop 2024-02-26T10:43:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.126+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c103480 msgr2=0x7efd6c199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.126+0000 7efd72200640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c103480 0x7efd6c199770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00e9c0 tx=0x7efd5c00edc0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.127+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 shutdown_connections 2024-02-26T10:43:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.127+0000 7efd72200640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efd500a1e00 0x7efd500a42c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.127+0000 7efd72200640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c105030 0x7efd6c1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.127+0000 7efd72200640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c104670 0x7efd6c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.127+0000 7efd72200640 1 --2- 172.21.15.60:0/1588227928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c103480 0x7efd6c199770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.127+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 >> 172.21.15.60:0/1588227928 conn(0x7efd6c0fea10 msgr2=0x7efd6c107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.127+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 shutdown_connections 2024-02-26T10:43:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.127+0000 7efd72200640 1 -- 172.21.15.60:0/1588227928 wait complete. 2024-02-26T10:43:03.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:43:03.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:43:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.382+0000 7fcdea200640 1 -- 172.21.15.60:0/3017024139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde4100be0 msgr2=0x7fcde4101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.382+0000 7fcdea200640 1 --2- 172.21.15.60:0/3017024139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde4100be0 0x7fcde4101060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009930 tx=0x7fcdcc02bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.382+0000 7fcdea200640 1 -- 172.21.15.60:0/3017024139 shutdown_connections 2024-02-26T10:43:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.382+0000 7fcdea200640 1 --2- 172.21.15.60:0/3017024139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde41058e0 0x7fcde4107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.382+0000 7fcdea200640 1 --2- 172.21.15.60:0/3017024139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde4100be0 0x7fcde4101060 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.382+0000 7fcdea200640 1 --2- 172.21.15.60:0/3017024139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde40677c0 0x7fcde41006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.382+0000 7fcdea200640 1 -- 172.21.15.60:0/3017024139 >> 172.21.15.60:0/3017024139 conn(0x7fcde40faa20 msgr2=0x7fcde40fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.382+0000 7fcdea200640 1 -- 172.21.15.60:0/3017024139 shutdown_connections 2024-02-26T10:43:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.382+0000 7fcdea200640 1 -- 172.21.15.60:0/3017024139 wait complete. 2024-02-26T10:43:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.384+0000 7fcdea200640 1 Processor -- start 2024-02-26T10:43:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.384+0000 7fcdea200640 1 -- start start 2024-02-26T10:43:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.384+0000 7fcdea200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde40677c0 0x7fcde4199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.384+0000 7fcdea200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde4100be0 0x7fcde4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.384+0000 7fcdea200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde41058e0 0x7fcde41a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.384+0000 7fcdea200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde410ad30 con 0x7fcde41058e0 2024-02-26T10:43:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.384+0000 7fcdea200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde410abb0 con 0x7fcde4100be0 2024-02-26T10:43:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.384+0000 7fcdea200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde410aeb0 con 0x7fcde40677c0 2024-02-26T10:43:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde41058e0 0x7fcde41a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde40677c0 0x7fcde4199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde41058e0 0x7fcde41a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53002/0 (socket says 172.21.15.60:53002) 2024-02-26T10:43:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3e00640 1 -- 172.21.15.60:0/2808112266 learned_addr learned my addr 172.21.15.60:0/2808112266 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcddaa00640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde4100be0 0x7fcde4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3e00640 1 -- 172.21.15.60:0/2808112266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde40677c0 msgr2=0x7fcde4199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3e00640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde40677c0 0x7fcde4199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3e00640 1 -- 172.21.15.60:0/2808112266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde4100be0 msgr2=0x7fcde4199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3e00640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde4100be0 0x7fcde4199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3e00640 1 -- 172.21.15.60:0/2808112266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde41a1490 con 0x7fcde41058e0 2024-02-26T10:43:03.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.385+0000 7fcde3e00640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde41058e0 0x7fcde41a0e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e9c0 tx=0x7fcddc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.386+0000 7fcde0c00640 1 -- 172.21.15.60:0/2808112266 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00cce0 con 0x7fcde41058e0 2024-02-26T10:43:03.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.386+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde41a1780 con 0x7fcde41058e0 2024-02-26T10:43:03.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.386+0000 7fcde0c00640 1 -- 172.21.15.60:0/2808112266 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcddc005500 con 0x7fcde41058e0 2024-02-26T10:43:03.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.386+0000 7fcde0c00640 1 -- 172.21.15.60:0/2808112266 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc016670 con 0x7fcde41058e0 2024-02-26T10:43:03.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.386+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde41a1ca0 con 0x7fcde41058e0 2024-02-26T10:43:03.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.387+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde41046f0 con 0x7fcde41058e0 2024-02-26T10:43:03.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.389+0000 7fcde0c00640 1 -- 172.21.15.60:0/2808112266 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fcddc01d020 con 0x7fcde41058e0 2024-02-26T10:43:03.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.391+0000 7fcde0c00640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcdc80a1e00 0x7fcdc80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.391+0000 7fcde0c00640 1 -- 172.21.15.60:0/2808112266 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fcddc0cdac0 con 0x7fcde41058e0 2024-02-26T10:43:03.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.391+0000 7fcde3400640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcdc80a1e00 0x7fcdc80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.392+0000 7fcde3400640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcdc80a1e00 0x7fcdc80a42c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcde4100500 tx=0x7fcdd4009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.395+0000 7fcde0c00640 1 -- 172.21.15.60:0/2808112266 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcddc09ac50 con 0x7fcde41058e0 2024-02-26T10:43:03.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:03 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3683586138' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T10:43:03.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:03 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1588227928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:03.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.598+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcde4061e60 con 0x7fcde41058e0 2024-02-26T10:43:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.599+0000 7fcde0c00640 1 -- 172.21.15.60:0/2808112266 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcddc09a550 con 0x7fcde41058e0 2024-02-26T10:43:03.597 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:43:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.602+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcdc80a1e00 msgr2=0x7fcdc80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.602+0000 7fcdea200640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcdc80a1e00 0x7fcdc80a42c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcde4100500 tx=0x7fcdd4009720 comp rx=0 tx=0).stop 2024-02-26T10:43:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.602+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde41058e0 msgr2=0x7fcde41a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.602+0000 7fcdea200640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde41058e0 0x7fcde41a0e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e9c0 tx=0x7fcddc00ee80 comp rx=0 tx=0).stop 2024-02-26T10:43:03.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.602+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 shutdown_connections 2024-02-26T10:43:03.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.602+0000 7fcdea200640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcdc80a1e00 0x7fcdc80a42c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.602+0000 7fcdea200640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcde4100be0 0x7fcde4199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.602+0000 7fcdea200640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcde41058e0 0x7fcde41a0e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.603+0000 7fcdea200640 1 --2- 172.21.15.60:0/2808112266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde40677c0 0x7fcde4199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.603+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 >> 172.21.15.60:0/2808112266 conn(0x7fcde40faa20 msgr2=0x7fcde41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:03.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.603+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 shutdown_connections 2024-02-26T10:43:03.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.603+0000 7fcdea200640 1 -- 172.21.15.60:0/2808112266 wait complete. 2024-02-26T10:43:03.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:03 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3683586138' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T10:43:03.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:03 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1588227928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:03.621 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:43:03.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:03 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3683586138' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T10:43:03.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:03 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1588227928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:03.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.849+0000 7f2e8a400640 1 -- 172.21.15.60:0/2920841183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 msgr2=0x7f2e84074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.849+0000 7f2e8a400640 1 --2- 172.21.15.60:0/2920841183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 0x7f2e84074140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009930 tx=0x7f2e6c02bc20 comp rx=0 tx=0).stop 2024-02-26T10:43:03.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.849+0000 7f2e8a400640 1 -- 172.21.15.60:0/2920841183 shutdown_connections 2024-02-26T10:43:03.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.849+0000 7f2e8a400640 1 --2- 172.21.15.60:0/2920841183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 0x7f2e84074140 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.849+0000 7f2e8a400640 1 --2- 172.21.15.60:0/2920841183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8410d840 0x7f2e8410fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.849+0000 7f2e8a400640 1 --2- 172.21.15.60:0/2920841183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e840752a0 0x7f2e840736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.849+0000 7f2e8a400640 1 -- 172.21.15.60:0/2920841183 >> 172.21.15.60:0/2920841183 conn(0x7f2e840fc800 msgr2=0x7f2e840fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:03.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.849+0000 7f2e8a400640 1 -- 172.21.15.60:0/2920841183 shutdown_connections 2024-02-26T10:43:03.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.849+0000 7f2e8a400640 1 -- 172.21.15.60:0/2920841183 wait complete. 2024-02-26T10:43:03.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.850+0000 7f2e8a400640 1 Processor -- start 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.850+0000 7f2e8a400640 1 -- start start 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.850+0000 7f2e8a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 0x7f2e8419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.850+0000 7f2e8a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840752a0 0x7f2e8419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e8a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e8410d840 0x7f2e841a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e8a400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84113510 con 0x7f2e8410d840 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e8a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84113390 con 0x7f2e84073cc0 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e8a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84113690 con 0x7f2e840752a0 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e83400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 0x7f2e8419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e83400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 0x7f2e8419dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37766/0 (socket says 172.21.15.60:37766) 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e83400640 1 -- 172.21.15.60:0/3447105942 learned_addr learned my addr 172.21.15.60:0/3447105942 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e83e00640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e8410d840 0x7f2e841a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e82a00640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840752a0 0x7f2e8419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e82a00640 1 -- 172.21.15.60:0/3447105942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 msgr2=0x7f2e8419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e82a00640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 0x7f2e8419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e82a00640 1 -- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e8410d840 msgr2=0x7f2e841a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e82a00640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e8410d840 0x7f2e841a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e82a00640 1 -- 172.21.15.60:0/3447105942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e841a5800 con 0x7f2e840752a0 2024-02-26T10:43:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.851+0000 7f2e83400640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 0x7f2e8419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.852+0000 7f2e82a00640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840752a0 0x7f2e8419e120 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c02c160 tx=0x7f2e6c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.852+0000 7f2e7be00640 1 -- 172.21.15.60:0/3447105942 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c043020 con 0x7f2e840752a0 2024-02-26T10:43:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.852+0000 7f2e7be00640 1 -- 172.21.15.60:0/3447105942 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e6c004440 con 0x7f2e840752a0 2024-02-26T10:43:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.852+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e841a5a90 con 0x7f2e840752a0 2024-02-26T10:43:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.852+0000 7f2e7be00640 1 -- 172.21.15.60:0/3447105942 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c003d40 con 0x7f2e840752a0 2024-02-26T10:43:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.852+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e841a5fb0 con 0x7f2e840752a0 2024-02-26T10:43:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.854+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e84103a10 con 0x7f2e840752a0 2024-02-26T10:43:03.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.856+0000 7f2e7be00640 1 -- 172.21.15.60:0/3447105942 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2e6c046020 con 0x7f2e840752a0 2024-02-26T10:43:03.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.857+0000 7f2e7be00640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e680a1d80 0x7f2e680a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.857+0000 7f2e7be00640 1 -- 172.21.15.60:0/3447105942 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2e6c0f0290 con 0x7f2e840752a0 2024-02-26T10:43:03.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.857+0000 7f2e83400640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e680a1d80 0x7f2e680a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.858+0000 7f2e83400640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e680a1d80 0x7f2e680a4240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e74005fd0 tx=0x7f2e7400b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:03.861+0000 7f2e7be00640 1 -- 172.21.15.60:0/3447105942 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e6c04d020 con 0x7f2e840752a0 2024-02-26T10:43:04.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.070+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e84061e60 con 0x7f2e840752a0 2024-02-26T10:43:04.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.071+0000 7f2e7be00640 1 -- 172.21.15.60:0/3447105942 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2e6c0bd540 con 0x7f2e840752a0 2024-02-26T10:43:04.069 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:43:04.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e680a1d80 msgr2=0x7f2e680a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e680a1d80 0x7f2e680a4240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e74005fd0 tx=0x7f2e7400b480 comp rx=0 tx=0).stop 2024-02-26T10:43:04.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840752a0 msgr2=0x7f2e8419e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840752a0 0x7f2e8419e120 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c02c160 tx=0x7f2e6c004040 comp rx=0 tx=0).stop 2024-02-26T10:43:04.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 shutdown_connections 2024-02-26T10:43:04.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e680a1d80 0x7f2e680a4240 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e8410d840 0x7f2e841a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840752a0 0x7f2e8419e120 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 --2- 172.21.15.60:0/3447105942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e84073cc0 0x7f2e8419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 >> 172.21.15.60:0/3447105942 conn(0x7f2e840fc800 msgr2=0x7f2e84100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:04.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 shutdown_connections 2024-02-26T10:43:04.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.074+0000 7f2e8a400640 1 -- 172.21.15.60:0/3447105942 wait complete. 2024-02-26T10:43:04.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:43:04.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.328+0000 7f2e10e00640 1 -- 172.21.15.60:0/4125108514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102330 msgr2=0x7f2e0c1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.328+0000 7f2e10e00640 1 --2- 172.21.15.60:0/4125108514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102330 0x7f2e0c1027b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009900 tx=0x7f2df802bc40 comp rx=0 tx=0).stop 2024-02-26T10:43:04.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.328+0000 7f2e10e00640 1 -- 172.21.15.60:0/4125108514 shutdown_connections 2024-02-26T10:43:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.328+0000 7f2e10e00640 1 --2- 172.21.15.60:0/4125108514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e0c102cf0 0x7f2e0c109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.328+0000 7f2e10e00640 1 --2- 172.21.15.60:0/4125108514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102330 0x7f2e0c1027b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.328+0000 7f2e10e00640 1 --2- 172.21.15.60:0/4125108514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c101140 0x7f2e0c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.328+0000 7f2e10e00640 1 -- 172.21.15.60:0/4125108514 >> 172.21.15.60:0/4125108514 conn(0x7f2e0c0fc6d0 msgr2=0x7f2e0c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.328+0000 7f2e10e00640 1 -- 172.21.15.60:0/4125108514 shutdown_connections 2024-02-26T10:43:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.328+0000 7f2e10e00640 1 -- 172.21.15.60:0/4125108514 wait complete. 2024-02-26T10:43:04.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.329+0000 7f2e10e00640 1 Processor -- start 2024-02-26T10:43:04.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.329+0000 7f2e10e00640 1 -- start start 2024-02-26T10:43:04.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.329+0000 7f2e10e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c101140 0x7f2e0c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.329+0000 7f2e10e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e0c102330 0x7f2e0c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e10e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102cf0 0x7f2e0c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e10e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c10cdd0 con 0x7f2e0c102cf0 2024-02-26T10:43:04.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e10e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c10cc50 con 0x7f2e0c101140 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e10e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c10cf50 con 0x7f2e0c102330 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102cf0 0x7f2e0c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c101140 0x7f2e0c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102cf0 0x7f2e0c1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49364/0 (socket says 172.21.15.60:49364) 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c101140 0x7f2e0c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37794/0 (socket says 172.21.15.60:37794) 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0be00640 1 -- 172.21.15.60:0/383475999 learned_addr learned my addr 172.21.15.60:0/383475999 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0aa00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e0c102330 0x7f2e0c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0be00640 1 -- 172.21.15.60:0/383475999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e0c102330 msgr2=0x7f2e0c199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0be00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e0c102330 0x7f2e0c199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0be00640 1 -- 172.21.15.60:0/383475999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c101140 msgr2=0x7f2e0c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0be00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c101140 0x7f2e0c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.330+0000 7f2e0be00640 1 -- 172.21.15.60:0/383475999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c1a1230 con 0x7f2e0c102cf0 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.331+0000 7f2e0be00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102cf0 0x7f2e0c1a0bc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400eaa0 tx=0x7f2e0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.331+0000 7f2e03e00640 1 -- 172.21.15.60:0/383475999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0400cdb0 con 0x7f2e0c102cf0 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.331+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c1a1520 con 0x7f2e0c102cf0 2024-02-26T10:43:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.331+0000 7f2e03e00640 1 -- 172.21.15.60:0/383475999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e04005020 con 0x7f2e0c102cf0 2024-02-26T10:43:04.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.331+0000 7f2e03e00640 1 -- 172.21.15.60:0/383475999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e040116b0 con 0x7f2e0c102cf0 2024-02-26T10:43:04.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.331+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c1a19f0 con 0x7f2e0c102cf0 2024-02-26T10:43:04.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.332+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0c1a1cb0 con 0x7f2e0c102cf0 2024-02-26T10:43:04.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.334+0000 7f2e03e00640 1 -- 172.21.15.60:0/383475999 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2e04005560 con 0x7f2e0c102cf0 2024-02-26T10:43:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.336+0000 7f2e03e00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2df00a1d80 0x7f2df00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.336+0000 7f2e03e00640 1 -- 172.21.15.60:0/383475999 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2e040ce220 con 0x7f2e0c102cf0 2024-02-26T10:43:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.336+0000 7f2e0b400640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2df00a1d80 0x7f2df00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.337+0000 7f2e0b400640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2df00a1d80 0x7f2df00a4240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c102190 tx=0x7f2dfc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.340+0000 7f2e03e00640 1 -- 172.21.15.60:0/383475999 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e04016030 con 0x7f2e0c102cf0 2024-02-26T10:43:04.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:04 smithi060 ceph-mon[31539]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-02-26T10:43:04.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:04 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2808112266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:43:04.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:04 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3447105942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:43:04.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.545+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e0c1a1f20 con 0x7f2e0c102cf0 2024-02-26T10:43:04.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.546+0000 7f2e03e00640 1 -- 172.21.15.60:0/383475999 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e0409ad60 con 0x7f2e0c102cf0 2024-02-26T10:43:04.543 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:43:04.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2df00a1d80 msgr2=0x7f2df00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2df00a1d80 0x7f2df00a4240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c102190 tx=0x7f2dfc005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102cf0 msgr2=0x7f2e0c1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102cf0 0x7f2e0c1a0bc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400eaa0 tx=0x7f2e0400ef60 comp rx=0 tx=0).stop 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 shutdown_connections 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2df00a1d80 0x7f2df00a4240 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e0c102cf0 0x7f2e0c1a0bc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e0c102330 0x7f2e0c199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 --2- 172.21.15.60:0/383475999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c101140 0x7f2e0c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 >> 172.21.15.60:0/383475999 conn(0x7f2e0c0fc6d0 msgr2=0x7f2e0c073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 shutdown_connections 2024-02-26T10:43:04.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.549+0000 7f2e10e00640 1 -- 172.21.15.60:0/383475999 wait complete. 2024-02-26T10:43:04.566 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:43:04.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:04 smithi113 ceph-mon[36126]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-02-26T10:43:04.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:04 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2808112266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:43:04.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:04 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3447105942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:43:04.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:04 smithi100 ceph-mon[36051]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-02-26T10:43:04.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:04 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2808112266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:43:04.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:04 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3447105942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:43:04.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.797+0000 7fd766600640 1 -- 172.21.15.60:0/1852238073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd760104550 msgr2=0x7fd7601049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.797+0000 7fd766600640 1 --2- 172.21.15.60:0/1852238073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd760104550 0x7fd7601049d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd748009860 tx=0x7fd74802bb70 comp rx=0 tx=0).stop 2024-02-26T10:43:04.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.797+0000 7fd766600640 1 -- 172.21.15.60:0/1852238073 shutdown_connections 2024-02-26T10:43:04.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.797+0000 7fd766600640 1 --2- 172.21.15.60:0/1852238073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd760104f10 0x7fd76010b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.797+0000 7fd766600640 1 --2- 172.21.15.60:0/1852238073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd760104550 0x7fd7601049d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.797+0000 7fd766600640 1 --2- 172.21.15.60:0/1852238073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd760103360 0x7fd760103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.797+0000 7fd766600640 1 -- 172.21.15.60:0/1852238073 >> 172.21.15.60:0/1852238073 conn(0x7fd7600fe910 msgr2=0x7fd760100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:04.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.797+0000 7fd766600640 1 -- 172.21.15.60:0/1852238073 shutdown_connections 2024-02-26T10:43:04.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.797+0000 7fd766600640 1 -- 172.21.15.60:0/1852238073 wait complete. 2024-02-26T10:43:04.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.799+0000 7fd766600640 1 Processor -- start 2024-02-26T10:43:04.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.799+0000 7fd766600640 1 -- start start 2024-02-26T10:43:04.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.799+0000 7fd766600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd760103360 0x7fd760199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.799+0000 7fd766600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd760104550 0x7fd760199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.799+0000 7fd766600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd760104f10 0x7fd7601a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.799+0000 7fd766600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76010eff0 con 0x7fd760104f10 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.799+0000 7fd766600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76010ee70 con 0x7fd760104550 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.799+0000 7fd766600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76010f170 con 0x7fd760103360 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd757e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd760104550 0x7fd760199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd757e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd760104550 0x7fd760199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37806/0 (socket says 172.21.15.60:37806) 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd757e00640 1 -- 172.21.15.60:0/757198367 learned_addr learned my addr 172.21.15.60:0/757198367 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd765200640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd760103360 0x7fd760199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd765c00640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd760104f10 0x7fd7601a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd757e00640 1 -- 172.21.15.60:0/757198367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd760103360 msgr2=0x7fd760199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd757e00640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd760103360 0x7fd760199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd757e00640 1 -- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd760104f10 msgr2=0x7fd7601a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd757e00640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd760104f10 0x7fd7601a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd757e00640 1 -- 172.21.15.60:0/757198367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7601a1270 con 0x7fd760104550 2024-02-26T10:43:04.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.800+0000 7fd757e00640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd760104550 0x7fd760199b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7480349e0 tx=0x7fd748032450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.801+0000 7fd75e000640 1 -- 172.21.15.60:0/757198367 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748041030 con 0x7fd760104550 2024-02-26T10:43:04.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.801+0000 7fd75e000640 1 -- 172.21.15.60:0/757198367 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd74802c510 con 0x7fd760104550 2024-02-26T10:43:04.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.801+0000 7fd75e000640 1 -- 172.21.15.60:0/757198367 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748039040 con 0x7fd760104550 2024-02-26T10:43:04.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.801+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7601a1500 con 0x7fd760104550 2024-02-26T10:43:04.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.801+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7601a1a40 con 0x7fd760104550 2024-02-26T10:43:04.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.802+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76010a750 con 0x7fd760104550 2024-02-26T10:43:04.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.804+0000 7fd75e000640 1 -- 172.21.15.60:0/757198367 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd748044070 con 0x7fd760104550 2024-02-26T10:43:04.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.805+0000 7fd75e000640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd7440a1e00 0x7fd7440a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.805+0000 7fd75e000640 1 -- 172.21.15.60:0/757198367 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd7480ef500 con 0x7fd760104550 2024-02-26T10:43:04.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.806+0000 7fd765200640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd7440a1e00 0x7fd7440a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.806+0000 7fd765200640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd7440a1e00 0x7fd7440a42c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd7601043b0 tx=0x7fd750009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:04.810+0000 7fd75e000640 1 -- 172.21.15.60:0/757198367 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7480bc690 con 0x7fd760104550 2024-02-26T10:43:05.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.010+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd760061e60 con 0x7fd760104550 2024-02-26T10:43:05.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.011+0000 7fd75e000640 1 -- 172.21.15.60:0/757198367 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd7480bbf90 con 0x7fd760104550 2024-02-26T10:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd7440a1e00 msgr2=0x7fd7440a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd7440a1e00 0x7fd7440a42c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd7601043b0 tx=0x7fd750009500 comp rx=0 tx=0).stop 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd760104550 msgr2=0x7fd760199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd760104550 0x7fd760199b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7480349e0 tx=0x7fd748032450 comp rx=0 tx=0).stop 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 shutdown_connections 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd7440a1e00 0x7fd7440a42c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd760104550 0x7fd760199b90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd760104f10 0x7fd7601a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 --2- 172.21.15.60:0/757198367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd760103360 0x7fd760199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.014+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 >> 172.21.15.60:0/757198367 conn(0x7fd7600fe910 msgr2=0x7fd760107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.015+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 shutdown_connections 2024-02-26T10:43:05.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.015+0000 7fd766600640 1 -- 172.21.15.60:0/757198367 wait complete. 2024-02-26T10:43:05.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:43:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.265+0000 7fc30a400640 1 -- 172.21.15.60:0/3901751264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304102350 msgr2=0x7fc3041027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.265+0000 7fc30a400640 1 --2- 172.21.15.60:0/3901751264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304102350 0x7fc3041027d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009900 tx=0x7fc2ec02bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.265+0000 7fc30a400640 1 -- 172.21.15.60:0/3901751264 shutdown_connections 2024-02-26T10:43:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.265+0000 7fc30a400640 1 --2- 172.21.15.60:0/3901751264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc304102d10 0x7fc3041095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.265+0000 7fc30a400640 1 --2- 172.21.15.60:0/3901751264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304102350 0x7fc3041027d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.265+0000 7fc30a400640 1 --2- 172.21.15.60:0/3901751264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc304101160 0x7fc304101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.265+0000 7fc30a400640 1 -- 172.21.15.60:0/3901751264 >> 172.21.15.60:0/3901751264 conn(0x7fc3040fc6d0 msgr2=0x7fc3040feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.266+0000 7fc30a400640 1 -- 172.21.15.60:0/3901751264 shutdown_connections 2024-02-26T10:43:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.266+0000 7fc30a400640 1 -- 172.21.15.60:0/3901751264 wait complete. 2024-02-26T10:43:05.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.267+0000 7fc30a400640 1 Processor -- start 2024-02-26T10:43:05.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.267+0000 7fc30a400640 1 -- start start 2024-02-26T10:43:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.267+0000 7fc30a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304101160 0x7fc304197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.267+0000 7fc30a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc304102350 0x7fc304197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.267+0000 7fc30a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc304102d10 0x7fc30419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.267+0000 7fc30a400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30410cdf0 con 0x7fc304102350 2024-02-26T10:43:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.267+0000 7fc30a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30410cc70 con 0x7fc304102d10 2024-02-26T10:43:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.267+0000 7fc30a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30410cf70 con 0x7fc304101160 2024-02-26T10:43:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc304102350 0x7fc304197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc309000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304101160 0x7fc304197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc309000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304101160 0x7fc304197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:38490/0 (socket says 172.21.15.60:38490) 2024-02-26T10:43:05.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc309000640 1 -- 172.21.15.60:0/1391439706 learned_addr learned my addr 172.21.15.60:0/1391439706 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:05.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc309a00640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc304102d10 0x7fc30419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc309000640 1 -- 172.21.15.60:0/1391439706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc304102d10 msgr2=0x7fc30419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc309000640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc304102d10 0x7fc30419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc309000640 1 -- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc304102350 msgr2=0x7fc304197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc309000640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc304102350 0x7fc304197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc309000640 1 -- 172.21.15.60:0/1391439706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30419f030 con 0x7fc304101160 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.268+0000 7fc2fbe00640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc304102350 0x7fc304197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.269+0000 7fc309000640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304101160 0x7fc304197410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400eb70 tx=0x7fc2f400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.269+0000 7fc302000640 1 -- 172.21.15.60:0/1391439706 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4003b70 con 0x7fc304101160 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.269+0000 7fc302000640 1 -- 172.21.15.60:0/1391439706 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2f400ee70 con 0x7fc304101160 2024-02-26T10:43:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.269+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30419f320 con 0x7fc304101160 2024-02-26T10:43:05.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.269+0000 7fc302000640 1 -- 172.21.15.60:0/1391439706 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4010830 con 0x7fc304101160 2024-02-26T10:43:05.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.269+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30419f840 con 0x7fc304101160 2024-02-26T10:43:05.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.271+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc304108480 con 0x7fc304101160 2024-02-26T10:43:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.273+0000 7fc302000640 1 -- 172.21.15.60:0/1391439706 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc2f4004020 con 0x7fc304101160 2024-02-26T10:43:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.274+0000 7fc302000640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc2e80a1e00 0x7fc2e80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.274+0000 7fc302000640 1 -- 172.21.15.60:0/1391439706 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc2f40ce650 con 0x7fc304101160 2024-02-26T10:43:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.274+0000 7fc2fbe00640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc2e80a1e00 0x7fc2e80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.275+0000 7fc2fbe00640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc2e80a1e00 0x7fc2e80a42c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec02c7b0 tx=0x7fc2ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.278+0000 7fc302000640 1 -- 172.21.15.60:0/1391439706 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc2f409b7e0 con 0x7fc304101160 2024-02-26T10:43:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.495+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc30402ce30 con 0x7fc304101160 2024-02-26T10:43:05.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.496+0000 7fc302000640 1 -- 172.21.15.60:0/1391439706 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+587 (secure 0 0 0) 0x7fc2f409b0e0 con 0x7fc304101160 2024-02-26T10:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/603652410 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3463459063 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2587364905 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/603652410 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/116674454 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/61419351 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3598708487 2024-02-27T10:34:36.733932+0000 2024-02-26T10:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1853267621 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/2948072642 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/2948072642 2024-02-27T10:34:01.770755+0000 2024-02-26T10:43:05.494 INFO:teuthology.orchestra.run.smithi060.stderr:listed 10 entries 2024-02-26T10:43:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.498+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc2e80a1e00 msgr2=0x7fc2e80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.498+0000 7fc30a400640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc2e80a1e00 0x7fc2e80a42c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec02c7b0 tx=0x7fc2ec005fd0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.498+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304101160 msgr2=0x7fc304197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.498+0000 7fc30a400640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304101160 0x7fc304197410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400eb70 tx=0x7fc2f400c6a0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.499+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 shutdown_connections 2024-02-26T10:43:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.499+0000 7fc30a400640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc2e80a1e00 0x7fc2e80a42c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.499+0000 7fc30a400640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc304102d10 0x7fc30419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.499+0000 7fc30a400640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc304102350 0x7fc304197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.499+0000 7fc30a400640 1 --2- 172.21.15.60:0/1391439706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc304101160 0x7fc304197410 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.499+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 >> 172.21.15.60:0/1391439706 conn(0x7fc3040fc6d0 msgr2=0x7fc304105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.499+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 shutdown_connections 2024-02-26T10:43:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:05.499+0000 7fc30a400640 1 -- 172.21.15.60:0/1391439706 wait complete. 2024-02-26T10:43:05.516 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-02-26T10:43:05.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:43:05.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-02-26T10:43:05.553 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:05 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/383475999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:43:05.553 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:05 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/757198367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:43:05.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:05 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/383475999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:43:05.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:05 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/757198367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:43:05.609 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[75410]: fuse finished with error 0 and tester_r 0 2024-02-26T10:43:05.616 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:05.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:05 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/383475999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:43:05.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:05 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/757198367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:43:06.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:06 smithi113 ceph-mon[36126]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:06.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:06 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1391439706' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T10:43:06.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:06 smithi060 ceph-mon[31539]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:06.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:06 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1391439706' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T10:43:06.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:06 smithi100 ceph-mon[36051]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:06.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:06 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1391439706' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T10:43:08.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:08 smithi060 ceph-mon[31539]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:08.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:08 smithi113 ceph-mon[36126]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:08.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:08 smithi100 ceph-mon[36051]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:10.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:10 smithi113 ceph-mon[36126]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T10:43:10.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:10 smithi060 ceph-mon[31539]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T10:43:10.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:10 smithi100 ceph-mon[36051]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T10:43:11.718 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T10:43:11.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:43:11.718 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-02-26T10:43:11.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:43:12.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.006+0000 7ff917e00640 1 -- 172.21.15.60:0/1818913610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9181076c0 msgr2=0x7ff918107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.006+0000 7ff917e00640 1 --2- 172.21.15.60:0/1818913610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9181076c0 0x7ff918107ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff908009900 tx=0x7ff90802bc50 comp rx=0 tx=0).stop 2024-02-26T10:43:12.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.007+0000 7ff917e00640 1 -- 172.21.15.60:0/1818913610 shutdown_connections 2024-02-26T10:43:12.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.007+0000 7ff917e00640 1 --2- 172.21.15.60:0/1818913610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff918109270 0x7ff91810fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.007+0000 7ff917e00640 1 --2- 172.21.15.60:0/1818913610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9181088b0 0x7ff918108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.007+0000 7ff917e00640 1 --2- 172.21.15.60:0/1818913610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9181076c0 0x7ff918107ae0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.007+0000 7ff917e00640 1 -- 172.21.15.60:0/1818913610 >> 172.21.15.60:0/1818913610 conn(0x7ff918076670 msgr2=0x7ff918078a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.007+0000 7ff917e00640 1 -- 172.21.15.60:0/1818913610 shutdown_connections 2024-02-26T10:43:12.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.007+0000 7ff917e00640 1 -- 172.21.15.60:0/1818913610 wait complete. 2024-02-26T10:43:12.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.008+0000 7ff917e00640 1 Processor -- start 2024-02-26T10:43:12.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.008+0000 7ff917e00640 1 -- start start 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff917e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9181088b0 0x7ff91819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff917e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff918109270 0x7ff91819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff917e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9181a3160 0x7ff9181a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff917e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff918113360 con 0x7ff9181088b0 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff917e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9181131e0 con 0x7ff9181a3160 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff917e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9181134e0 con 0x7ff918109270 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff916a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9181088b0 0x7ff91819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff916a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9181088b0 0x7ff91819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49400/0 (socket says 172.21.15.60:49400) 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff916a00640 1 -- 172.21.15.60:0/1659849794 learned_addr learned my addr 172.21.15.60:0/1659849794 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff917400640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9181a3160 0x7ff9181a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff916000640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff918109270 0x7ff91819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff916a00640 1 -- 172.21.15.60:0/1659849794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff918109270 msgr2=0x7ff91819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff916a00640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff918109270 0x7ff91819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff916a00640 1 -- 172.21.15.60:0/1659849794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9181a3160 msgr2=0x7ff9181a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff916a00640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9181a3160 0x7ff9181a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.009+0000 7ff916a00640 1 -- 172.21.15.60:0/1659849794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9181a5bc0 con 0x7ff9181088b0 2024-02-26T10:43:12.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.010+0000 7ff916a00640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9181088b0 0x7ff91819dbc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff908036040 tx=0x7ff90802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.010+0000 7ff90f400640 1 -- 172.21.15.60:0/1659849794 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908041070 con 0x7ff9181088b0 2024-02-26T10:43:12.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.010+0000 7ff90f400640 1 -- 172.21.15.60:0/1659849794 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff908004450 con 0x7ff9181088b0 2024-02-26T10:43:12.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.010+0000 7ff90f400640 1 -- 172.21.15.60:0/1659849794 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908003e00 con 0x7ff9181088b0 2024-02-26T10:43:12.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.010+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9181a5e50 con 0x7ff9181088b0 2024-02-26T10:43:12.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.010+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9181a6260 con 0x7ff9181088b0 2024-02-26T10:43:12.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.012+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91810ea60 con 0x7ff9181088b0 2024-02-26T10:43:12.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.015+0000 7ff90f400640 1 -- 172.21.15.60:0/1659849794 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff908044070 con 0x7ff9181088b0 2024-02-26T10:43:12.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.016+0000 7ff90f400640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8fc0a1e00 0x7ff8fc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.017+0000 7ff90f400640 1 -- 172.21.15.60:0/1659849794 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff9080efba0 con 0x7ff9181088b0 2024-02-26T10:43:12.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.017+0000 7ff916000640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8fc0a1e00 0x7ff8fc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.017+0000 7ff916000640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8fc0a1e00 0x7ff8fc0a42c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff91819f0d0 tx=0x7ff904005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.019+0000 7ff90f400640 1 -- 172.21.15.60:0/1659849794 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9080bc6e0 con 0x7ff9181088b0 2024-02-26T10:43:12.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.223+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff918061f00 con 0x7ff9181088b0 2024-02-26T10:43:12.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.225+0000 7ff90f400640 1 -- 172.21.15.60:0/1659849794 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7ff9080c0780 con 0x7ff9181088b0 2024-02-26T10:43:12.223 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:12.223 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:12.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8fc0a1e00 msgr2=0x7ff8fc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8fc0a1e00 0x7ff8fc0a42c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff91819f0d0 tx=0x7ff904005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:12.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9181088b0 msgr2=0x7ff91819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9181088b0 0x7ff91819dbc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff908036040 tx=0x7ff90802c6e0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 shutdown_connections 2024-02-26T10:43:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8fc0a1e00 0x7ff8fc0a42c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9181a3160 0x7ff9181a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff918109270 0x7ff91819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 --2- 172.21.15.60:0/1659849794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9181088b0 0x7ff91819dbc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 >> 172.21.15.60:0/1659849794 conn(0x7ff918076670 msgr2=0x7ff91810bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 shutdown_connections 2024-02-26T10:43:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.228+0000 7ff917e00640 1 -- 172.21.15.60:0/1659849794 wait complete. 2024-02-26T10:43:12.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:43:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.461+0000 7f70b1e00640 1 -- 172.21.15.60:0/1355010210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac073c40 msgr2=0x7f70ac0740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.461+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1355010210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac073c40 0x7f70ac0740a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7098009900 tx=0x7f709802bcd0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.461+0000 7f70b1e00640 1 -- 172.21.15.60:0/1355010210 shutdown_connections 2024-02-26T10:43:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.461+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1355010210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70ac108f50 0x7f70ac10b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.461+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1355010210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac073c40 0x7f70ac0740a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.461+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1355010210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac075220 0x7f70ac073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.461+0000 7f70b1e00640 1 -- 172.21.15.60:0/1355010210 >> 172.21.15.60:0/1355010210 conn(0x7f70ac0fc5f0 msgr2=0x7f70ac0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.461+0000 7f70b1e00640 1 -- 172.21.15.60:0/1355010210 shutdown_connections 2024-02-26T10:43:12.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.461+0000 7f70b1e00640 1 -- 172.21.15.60:0/1355010210 wait complete. 2024-02-26T10:43:12.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.462+0000 7f70b1e00640 1 Processor -- start 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.462+0000 7f70b1e00640 1 -- start start 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70ac073c40 0x7f70ac1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac075220 0x7f70ac199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac108f50 0x7f70ac1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b1e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac10ef50 con 0x7f70ac075220 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b1e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac10edd0 con 0x7f70ac108f50 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b1e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac10f0d0 con 0x7f70ac073c40 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac075220 0x7f70ac199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac075220 0x7f70ac199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49426/0 (socket says 172.21.15.60:49426) 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70abe00640 1 -- 172.21.15.60:0/1130828433 learned_addr learned my addr 172.21.15.60:0/1130828433 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b0a00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70ac073c40 0x7f70ac1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b1400640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac108f50 0x7f70ac1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70abe00640 1 -- 172.21.15.60:0/1130828433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70ac073c40 msgr2=0x7f70ac1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70abe00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70ac073c40 0x7f70ac1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70abe00640 1 -- 172.21.15.60:0/1130828433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac108f50 msgr2=0x7f70ac1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70abe00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac108f50 0x7f70ac1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70abe00640 1 -- 172.21.15.60:0/1130828433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac1a11e0 con 0x7f70ac075220 2024-02-26T10:43:12.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b1400640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac108f50 0x7f70ac1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:43:12.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.463+0000 7f70b0a00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70ac073c40 0x7f70ac1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:12.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.464+0000 7f70abe00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac075220 0x7f70ac199b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7098005ab0 tx=0x7f709802c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.464+0000 7f70a9600640 1 -- 172.21.15.60:0/1130828433 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098043020 con 0x7f70ac075220 2024-02-26T10:43:12.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.464+0000 7f70a9600640 1 -- 172.21.15.60:0/1130828433 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70980054e0 con 0x7f70ac075220 2024-02-26T10:43:12.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.464+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac1a1470 con 0x7f70ac075220 2024-02-26T10:43:12.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.464+0000 7f70a9600640 1 -- 172.21.15.60:0/1130828433 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098003ca0 con 0x7f70ac075220 2024-02-26T10:43:12.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.464+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac1a19c0 con 0x7f70ac075220 2024-02-26T10:43:12.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.465+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac107d10 con 0x7f70ac075220 2024-02-26T10:43:12.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.468+0000 7f70a9600640 1 -- 172.21.15.60:0/1130828433 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f7098046020 con 0x7f70ac075220 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.469+0000 7f70a9600640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f70940a1d80 0x7f70940a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.469+0000 7f70a9600640 1 -- 172.21.15.60:0/1130828433 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f70980f01d0 con 0x7f70ac075220 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.469+0000 7f70b0a00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f70940a1d80 0x7f70940a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.470+0000 7f70b0a00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f70940a1d80 0x7f70940a4240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009fc0 tx=0x7f70a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.473+0000 7f70a9600640 1 -- 172.21.15.60:0/1130828433 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f709804d020 con 0x7f70ac075220 2024-02-26T10:43:12.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:12 smithi113 ceph-mon[36126]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:12.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:12 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1659849794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:12.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:12 smithi060 ceph-mon[31539]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:12.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:12 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1659849794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:12.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:12 smithi100 ceph-mon[36051]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:12.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:12 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1659849794' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:12.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.676+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70ac061e60 con 0x7f70ac075220 2024-02-26T10:43:12.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.679+0000 7f70a9600640 1 -- 172.21.15.60:0/1130828433 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7f70980bd360 con 0x7f70ac075220 2024-02-26T10:43:12.677 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:12.677 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:12.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f70940a1d80 msgr2=0x7f70940a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f70940a1d80 0x7f70940a4240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009fc0 tx=0x7f70a0005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac075220 msgr2=0x7f70ac199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac075220 0x7f70ac199b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7098005ab0 tx=0x7f709802c760 comp rx=0 tx=0).stop 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 shutdown_connections 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f70940a1d80 0x7f70940a4240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac108f50 0x7f70ac1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70ac075220 0x7f70ac199b00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 --2- 172.21.15.60:0/1130828433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70ac073c40 0x7f70ac1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 >> 172.21.15.60:0/1130828433 conn(0x7f70ac0fc5f0 msgr2=0x7f70ac104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 shutdown_connections 2024-02-26T10:43:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.682+0000 7f70b1e00640 1 -- 172.21.15.60:0/1130828433 wait complete. 2024-02-26T10:43:12.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-26T10:43:12.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.937+0000 7fe643200640 1 -- 172.21.15.60:0/3209721314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104f10 msgr2=0x7fe64410b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.937+0000 7fe643200640 1 --2- 172.21.15.60:0/3209721314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104f10 0x7fe64410b7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009ca0 tx=0x7fe63c02c100 comp rx=0 tx=0).stop 2024-02-26T10:43:12.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.937+0000 7fe643200640 1 -- 172.21.15.60:0/3209721314 shutdown_connections 2024-02-26T10:43:12.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.937+0000 7fe643200640 1 --2- 172.21.15.60:0/3209721314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe644104550 0x7fe6441049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.937+0000 7fe643200640 1 --2- 172.21.15.60:0/3209721314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104f10 0x7fe64410b7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.937+0000 7fe643200640 1 --2- 172.21.15.60:0/3209721314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe644103360 0x7fe644103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.937+0000 7fe643200640 1 -- 172.21.15.60:0/3209721314 >> 172.21.15.60:0/3209721314 conn(0x7fe6440fe8d0 msgr2=0x7fe644100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.937+0000 7fe643200640 1 -- 172.21.15.60:0/3209721314 shutdown_connections 2024-02-26T10:43:12.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.937+0000 7fe643200640 1 -- 172.21.15.60:0/3209721314 wait complete. 2024-02-26T10:43:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.938+0000 7fe643200640 1 Processor -- start 2024-02-26T10:43:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.938+0000 7fe643200640 1 -- start start 2024-02-26T10:43:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe643200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe644103360 0x7fe6441995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe643200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104550 0x7fe644199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe643200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe644104f10 0x7fe6441a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe643200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64410eff0 con 0x7fe644104550 2024-02-26T10:43:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe643200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64410ee70 con 0x7fe644104f10 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe643200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64410f170 con 0x7fe644103360 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe641400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104550 0x7fe644199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe641400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104550 0x7fe644199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49438/0 (socket says 172.21.15.60:49438) 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe641400640 1 -- 172.21.15.60:0/2277620851 learned_addr learned my addr 172.21.15.60:0/2277620851 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe641e00640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe644103360 0x7fe6441995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe642800640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe644104f10 0x7fe6441a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe641400640 1 -- 172.21.15.60:0/2277620851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe644103360 msgr2=0x7fe6441995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe641400640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe644103360 0x7fe6441995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.939+0000 7fe641400640 1 -- 172.21.15.60:0/2277620851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe644104f10 msgr2=0x7fe6441a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.940+0000 7fe641400640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe644104f10 0x7fe6441a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.940+0000 7fe641400640 1 -- 172.21.15.60:0/2277620851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6441a11d0 con 0x7fe644104550 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.940+0000 7fe641400640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104550 0x7fe644199af0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe63000d950 tx=0x7fe63000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.940+0000 7fe63aa00640 1 -- 172.21.15.60:0/2277620851 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe630017030 con 0x7fe644104550 2024-02-26T10:43:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.940+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6441a14c0 con 0x7fe644104550 2024-02-26T10:43:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.940+0000 7fe63aa00640 1 -- 172.21.15.60:0/2277620851 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe630004020 con 0x7fe644104550 2024-02-26T10:43:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.940+0000 7fe63aa00640 1 -- 172.21.15.60:0/2277620851 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6300047a0 con 0x7fe644104550 2024-02-26T10:43:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.940+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6441a1a10 con 0x7fe644104550 2024-02-26T10:43:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.941+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64410a620 con 0x7fe644104550 2024-02-26T10:43:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.943+0000 7fe63aa00640 1 -- 172.21.15.60:0/2277620851 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe6300041d0 con 0x7fe644104550 2024-02-26T10:43:12.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.945+0000 7fe63aa00640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6280a1d80 0x7fe6280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.945+0000 7fe63aa00640 1 -- 172.21.15.60:0/2277620851 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe6300cd9c0 con 0x7fe644104550 2024-02-26T10:43:12.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.945+0000 7fe641e00640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6280a1d80 0x7fe6280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.946+0000 7fe641e00640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6280a1d80 0x7fe6280a4240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe6441043b0 tx=0x7fe634005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:12.950+0000 7fe63aa00640 1 -- 172.21.15.60:0/2277620851 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe63009ab50 con 0x7fe644104550 2024-02-26T10:43:13.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.193+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe64402ce30 con 0x7fe644104550 2024-02-26T10:43:13.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.310+0000 7fe63aa00640 1 -- 172.21.15.60:0/2277620851 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe63009a450 con 0x7fe644104550 2024-02-26T10:43:13.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6280a1d80 msgr2=0x7fe6280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:13.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6280a1d80 0x7fe6280a4240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe6441043b0 tx=0x7fe634005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:13.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104550 msgr2=0x7fe644199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:13.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104550 0x7fe644199af0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe63000d950 tx=0x7fe63000de10 comp rx=0 tx=0).stop 2024-02-26T10:43:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 shutdown_connections 2024-02-26T10:43:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6280a1d80 0x7fe6280a4240 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe644104f10 0x7fe6441a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe644104550 0x7fe644199af0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 --2- 172.21.15.60:0/2277620851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe644103360 0x7fe6441995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.314+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 >> 172.21.15.60:0/2277620851 conn(0x7fe6440fe8d0 msgr2=0x7fe6441077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.315+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 shutdown_connections 2024-02-26T10:43:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.315+0000 7fe643200640 1 -- 172.21.15.60:0/2277620851 wait complete. 2024-02-26T10:43:13.334 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-02-26T10:43:13.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-02-26T10:43:13.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.572+0000 7fef6c200640 1 -- 172.21.15.60:0/1016580467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64105030 msgr2=0x7fef6410b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:13.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.572+0000 7fef6c200640 1 --2- 172.21.15.60:0/1016580467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64105030 0x7fef6410b8c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009900 tx=0x7fef5c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:13.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.572+0000 7fef6c200640 1 -- 172.21.15.60:0/1016580467 shutdown_connections 2024-02-26T10:43:13.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.572+0000 7fef6c200640 1 --2- 172.21.15.60:0/1016580467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64105030 0x7fef6410b8c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:13.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.572+0000 7fef6c200640 1 --2- 172.21.15.60:0/1016580467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64104670 0x7fef64104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:13.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.572+0000 7fef6c200640 1 --2- 172.21.15.60:0/1016580467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef64103480 0x7fef641038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:13.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.572+0000 7fef6c200640 1 -- 172.21.15.60:0/1016580467 >> 172.21.15.60:0/1016580467 conn(0x7fef640fea10 msgr2=0x7fef64100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:13.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.572+0000 7fef6c200640 1 -- 172.21.15.60:0/1016580467 shutdown_connections 2024-02-26T10:43:13.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.572+0000 7fef6c200640 1 -- 172.21.15.60:0/1016580467 wait complete. 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.573+0000 7fef6c200640 1 Processor -- start 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.573+0000 7fef6c200640 1 -- start start 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.573+0000 7fef6c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef64103480 0x7fef641996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.573+0000 7fef6c200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64104670 0x7fef64199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.573+0000 7fef6c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64105030 0x7fef641a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.573+0000 7fef6c200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6410f160 con 0x7fef64104670 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.573+0000 7fef6c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6410efe0 con 0x7fef64103480 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.573+0000 7fef6c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6410f2e0 con 0x7fef64105030 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef69800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef64103480 0x7fef641996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef69800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef64103480 0x7fef641996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37898/0 (socket says 172.21.15.60:37898) 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef69800640 1 -- 172.21.15.60:0/3755216869 learned_addr learned my addr 172.21.15.60:0/3755216869 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef68e00640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64104670 0x7fef64199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef6a200640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64105030 0x7fef641a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef69800640 1 -- 172.21.15.60:0/3755216869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64105030 msgr2=0x7fef641a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef69800640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64105030 0x7fef641a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef69800640 1 -- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64104670 msgr2=0x7fef64199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef69800640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64104670 0x7fef64199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef69800640 1 -- 172.21.15.60:0/3755216869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef641a12f0 con 0x7fef64103480 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.574+0000 7fef69800640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef64103480 0x7fef641996d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef5800ebc0 tx=0x7fef5800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.575+0000 7fef62000640 1 -- 172.21.15.60:0/3755216869 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58003b70 con 0x7fef64103480 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.575+0000 7fef62000640 1 -- 172.21.15.60:0/3755216869 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef58003d10 con 0x7fef64103480 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.575+0000 7fef62000640 1 -- 172.21.15.60:0/3755216869 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58019980 con 0x7fef64103480 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.575+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef641a15e0 con 0x7fef64103480 2024-02-26T10:43:13.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.575+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef641a1ae0 con 0x7fef64103480 2024-02-26T10:43:13.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.576+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6410a740 con 0x7fef64103480 2024-02-26T10:43:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.578+0000 7fef62000640 1 -- 172.21.15.60:0/3755216869 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fef58019b20 con 0x7fef64103480 2024-02-26T10:43:13.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.579+0000 7fef62000640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef4c0a1e00 0x7fef4c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:13.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.579+0000 7fef62000640 1 -- 172.21.15.60:0/3755216869 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fef580ce7d0 con 0x7fef64103480 2024-02-26T10:43:13.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.580+0000 7fef68e00640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef4c0a1e00 0x7fef4c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:13.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.580+0000 7fef68e00640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef4c0a1e00 0x7fef4c0a42c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef540095f0 tx=0x7fef54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:13.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.583+0000 7fef62000640 1 -- 172.21.15.60:0/3755216869 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef5809b960 con 0x7fef64103480 2024-02-26T10:43:13.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:13 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1130828433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:13.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:13 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2277620851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T10:43:13.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:13 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1130828433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:13.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:13 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2277620851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T10:43:13.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:13 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1130828433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:13.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:13 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2277620851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T10:43:13.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:13.831+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fef64061e90 con 0x7fef64103480 2024-02-26T10:43:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.326+0000 7fef62000640 1 -- 172.21.15.60:0/3755216869 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fef5809b260 con 0x7fef64103480 2024-02-26T10:43:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef4c0a1e00 msgr2=0x7fef4c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef4c0a1e00 0x7fef4c0a42c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef540095f0 tx=0x7fef54005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef64103480 msgr2=0x7fef641996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef64103480 0x7fef641996d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef5800ebc0 tx=0x7fef5800c6a0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 shutdown_connections 2024-02-26T10:43:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef4c0a1e00 0x7fef4c0a42c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64105030 0x7fef641a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64104670 0x7fef64199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 --2- 172.21.15.60:0/3755216869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef64103480 0x7fef641996d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 >> 172.21.15.60:0/3755216869 conn(0x7fef640fea10 msgr2=0x7fef64107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 shutdown_connections 2024-02-26T10:43:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.330+0000 7fef6c200640 1 -- 172.21.15.60:0/3755216869 wait complete. 2024-02-26T10:43:14.354 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-26T10:43:14.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.564+0000 7f44c2a00640 1 -- 172.21.15.60:0/1065729768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc073cc0 msgr2=0x7f44bc074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.564+0000 7f44c2a00640 1 --2- 172.21.15.60:0/1065729768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc073cc0 0x7f44bc074140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009930 tx=0x7f44a802bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:14.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.564+0000 7f44c2a00640 1 -- 172.21.15.60:0/1065729768 shutdown_connections 2024-02-26T10:43:14.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.564+0000 7f44c2a00640 1 --2- 172.21.15.60:0/1065729768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44bc10d710 0x7f44bc10fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.564+0000 7f44c2a00640 1 --2- 172.21.15.60:0/1065729768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc073cc0 0x7f44bc074140 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.564+0000 7f44c2a00640 1 --2- 172.21.15.60:0/1065729768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc0752a0 0x7f44bc0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.564+0000 7f44c2a00640 1 -- 172.21.15.60:0/1065729768 >> 172.21.15.60:0/1065729768 conn(0x7f44bc0fc6d0 msgr2=0x7f44bc0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:14.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.564+0000 7f44c2a00640 1 -- 172.21.15.60:0/1065729768 shutdown_connections 2024-02-26T10:43:14.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.564+0000 7f44c2a00640 1 -- 172.21.15.60:0/1065729768 wait complete. 2024-02-26T10:43:14.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.565+0000 7f44c2a00640 1 Processor -- start 2024-02-26T10:43:14.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.566+0000 7f44c2a00640 1 -- start start 2024-02-26T10:43:14.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.566+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc073cc0 0x7f44bc19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.566+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc0752a0 0x7f44bc19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.566+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44bc10d710 0x7f44bc1a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.566+0000 7f44c2a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc113390 con 0x7f44bc0752a0 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.566+0000 7f44c2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc113210 con 0x7f44bc10d710 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.566+0000 7f44c2a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc113510 con 0x7f44bc073cc0 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.566+0000 7f44c0c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc0752a0 0x7f44bc19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c0c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc0752a0 0x7f44bc19dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39466/0 (socket says 172.21.15.60:39466) 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c1600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc073cc0 0x7f44bc19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c0c00640 1 -- 172.21.15.60:0/2981562391 learned_addr learned my addr 172.21.15.60:0/2981562391 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:14.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c2000640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44bc10d710 0x7f44bc1a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c0c00640 1 -- 172.21.15.60:0/2981562391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc073cc0 msgr2=0x7f44bc19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c0c00640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc073cc0 0x7f44bc19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c0c00640 1 -- 172.21.15.60:0/2981562391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44bc10d710 msgr2=0x7f44bc1a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c0c00640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44bc10d710 0x7f44bc1a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c0c00640 1 -- 172.21.15.60:0/2981562391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc1a56d0 con 0x7f44bc0752a0 2024-02-26T10:43:14.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c1600640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc073cc0 0x7f44bc19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:14.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.567+0000 7f44c0c00640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc0752a0 0x7f44bc19dff0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44a8000c00 tx=0x7f44a802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.568+0000 7f44ba000640 1 -- 172.21.15.60:0/2981562391 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8041070 con 0x7f44bc0752a0 2024-02-26T10:43:14.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.568+0000 7f44ba000640 1 -- 172.21.15.60:0/2981562391 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44a8004450 con 0x7f44bc0752a0 2024-02-26T10:43:14.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.568+0000 7f44ba000640 1 -- 172.21.15.60:0/2981562391 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8003df0 con 0x7f44bc0752a0 2024-02-26T10:43:14.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.568+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc1a5960 con 0x7f44bc0752a0 2024-02-26T10:43:14.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.568+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc1a5ea0 con 0x7f44bc0752a0 2024-02-26T10:43:14.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.569+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc1038e0 con 0x7f44bc0752a0 2024-02-26T10:43:14.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.571+0000 7f44ba000640 1 -- 172.21.15.60:0/2981562391 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f44a8044070 con 0x7f44bc0752a0 2024-02-26T10:43:14.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.573+0000 7f44ba000640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44a40a1e00 0x7f44a40a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.574+0000 7f44ba000640 1 -- 172.21.15.60:0/2981562391 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f44a80efa70 con 0x7f44bc0752a0 2024-02-26T10:43:14.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.574+0000 7f44c1600640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44a40a1e00 0x7f44a40a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.575+0000 7f44c1600640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44a40a1e00 0x7f44a40a42c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44b00099a0 tx=0x7f44b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.578+0000 7f44ba000640 1 -- 172.21.15.60:0/2981562391 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f44a80bcc90 con 0x7f44bc0752a0 2024-02-26T10:43:14.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:14 smithi113 ceph-mon[36126]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:14.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:14 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2277620851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T10:43:14.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:14 smithi113 ceph-mon[36126]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:43:14.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:14 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3755216869' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T10:43:14.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:14 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T10:43:14.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:14 smithi060 ceph-mon[31539]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:14.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:14 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2277620851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T10:43:14.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:14 smithi060 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:43:14.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:14 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3755216869' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T10:43:14.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:14 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T10:43:14.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:14 smithi100 ceph-mon[36051]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T10:43:14.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:14 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2277620851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T10:43:14.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:14 smithi100 ceph-mon[36051]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:43:14.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:14 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3755216869' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T10:43:14.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:14 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T10:43:14.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:14.823+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f44bc061e60 con 0x7f44bc0752a0 2024-02-26T10:43:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.331+0000 7f44ba000640 1 -- 172.21.15.60:0/2981562391 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f44a80bc590 con 0x7f44bc0752a0 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.335+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44a40a1e00 msgr2=0x7f44a40a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.335+0000 7f44c2a00640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44a40a1e00 0x7f44a40a42c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44b00099a0 tx=0x7f44b0009500 comp rx=0 tx=0).stop 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.335+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc0752a0 msgr2=0x7f44bc19dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.335+0000 7f44c2a00640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc0752a0 0x7f44bc19dff0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44a8000c00 tx=0x7f44a802c640 comp rx=0 tx=0).stop 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.336+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 shutdown_connections 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.336+0000 7f44c2a00640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44a40a1e00 0x7f44a40a42c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.336+0000 7f44c2a00640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44bc10d710 0x7f44bc1a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.336+0000 7f44c2a00640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44bc0752a0 0x7f44bc19dff0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.336+0000 7f44c2a00640 1 --2- 172.21.15.60:0/2981562391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44bc073cc0 0x7f44bc19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.336+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 >> 172.21.15.60:0/2981562391 conn(0x7f44bc0fc6d0 msgr2=0x7f44bc100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.336+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 shutdown_connections 2024-02-26T10:43:15.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.336+0000 7f44c2a00640 1 -- 172.21.15.60:0/2981562391 wait complete. 2024-02-26T10:43:15.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-26T10:43:15.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:15 smithi113 ceph-mon[36126]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T10:43:15.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:15 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-26T10:43:15.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:15 smithi113 ceph-mon[36126]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T10:43:15.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:15 smithi113 ceph-mon[36126]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:43:15.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:15 smithi113 ceph-mon[36126]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-26T10:43:15.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:15 smithi113 ceph-mon[36126]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-26T10:43:15.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:15 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2981562391' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T10:43:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.589+0000 7f8282c00640 1 -- 172.21.15.60:0/3356069451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c101160 msgr2=0x7f827c101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.589+0000 7f8282c00640 1 --2- 172.21.15.60:0/3356069451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c101160 0x7f827c101580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8270009900 tx=0x7f827002bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.589+0000 7f8282c00640 1 -- 172.21.15.60:0/3356069451 shutdown_connections 2024-02-26T10:43:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.589+0000 7f8282c00640 1 --2- 172.21.15.60:0/3356069451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c102350 0x7f827c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.589+0000 7f8282c00640 1 --2- 172.21.15.60:0/3356069451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c102d10 0x7f827c1095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.589+0000 7f8282c00640 1 --2- 172.21.15.60:0/3356069451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c101160 0x7f827c101580 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.589+0000 7f8282c00640 1 -- 172.21.15.60:0/3356069451 >> 172.21.15.60:0/3356069451 conn(0x7f827c0fc6d0 msgr2=0x7f827c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:15.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.589+0000 7f8282c00640 1 -- 172.21.15.60:0/3356069451 shutdown_connections 2024-02-26T10:43:15.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.589+0000 7f8282c00640 1 -- 172.21.15.60:0/3356069451 wait complete. 2024-02-26T10:43:15.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.590+0000 7f8282c00640 1 Processor -- start 2024-02-26T10:43:15.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.590+0000 7f8282c00640 1 -- start start 2024-02-26T10:43:15.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8282c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c101160 0x7f827c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8282c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c102350 0x7f827c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8282c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c102d10 0x7f827c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8282c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c10ce40 con 0x7f827c102350 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8282c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c10ccc0 con 0x7f827c101160 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8282c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c10cfc0 con 0x7f827c102d10 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8280e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c102350 0x7f827c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8280e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c102350 0x7f827c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39490/0 (socket says 172.21.15.60:39490) 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8280e00640 1 -- 172.21.15.60:0/3264400064 learned_addr learned my addr 172.21.15.60:0/3264400064 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8282200640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c102d10 0x7f827c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8281800640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c101160 0x7f827c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8280e00640 1 -- 172.21.15.60:0/3264400064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c102d10 msgr2=0x7f827c1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8280e00640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c102d10 0x7f827c1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8280e00640 1 -- 172.21.15.60:0/3264400064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c101160 msgr2=0x7f827c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.591+0000 7f8280e00640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c101160 0x7f827c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.592+0000 7f8280e00640 1 -- 172.21.15.60:0/3264400064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c1a12a0 con 0x7f827c102350 2024-02-26T10:43:15.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.592+0000 7f8280e00640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c102350 0x7f827c199bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f826c00e9f0 tx=0x7f826c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.592+0000 7f827a000640 1 -- 172.21.15.60:0/3264400064 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c00cd80 con 0x7f827c102350 2024-02-26T10:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.592+0000 7f827a000640 1 -- 172.21.15.60:0/3264400064 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f826c010910 con 0x7f827c102350 2024-02-26T10:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.592+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c1a1590 con 0x7f827c102350 2024-02-26T10:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.592+0000 7f827a000640 1 -- 172.21.15.60:0/3264400064 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c018990 con 0x7f827c102350 2024-02-26T10:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.592+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c1a1ae0 con 0x7f827c102350 2024-02-26T10:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.593+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c1a1d40 con 0x7f827c102350 2024-02-26T10:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.596+0000 7f827a000640 1 -- 172.21.15.60:0/3264400064 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f826c010ac0 con 0x7f827c102350 2024-02-26T10:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.597+0000 7f827a000640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f82640a1e00 0x7f82640a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.597+0000 7f827a000640 1 -- 172.21.15.60:0/3264400064 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f826c0ce750 con 0x7f827c102350 2024-02-26T10:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.597+0000 7f8281800640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f82640a1e00 0x7f82640a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.598+0000 7f8281800640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f82640a1e00 0x7f82640a42c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f827002c170 tx=0x7f827002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.600+0000 7f827a000640 1 -- 172.21.15.60:0/3264400064 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f826c09b290 con 0x7f827c102350 2024-02-26T10:43:15.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:15 smithi060 ceph-mon[31539]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T10:43:15.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:15 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-26T10:43:15.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:15 smithi060 ceph-mon[31539]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T10:43:15.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:15 smithi060 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:43:15.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:15 smithi060 ceph-mon[31539]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-26T10:43:15.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:15 smithi060 ceph-mon[31539]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-26T10:43:15.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:15 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2981562391' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T10:43:15.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:15 smithi100 ceph-mon[36051]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T10:43:15.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:15 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-26T10:43:15.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:15 smithi100 ceph-mon[36051]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T10:43:15.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:15 smithi100 ceph-mon[36051]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T10:43:15.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:15 smithi100 ceph-mon[36051]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-26T10:43:15.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:15 smithi100 ceph-mon[36051]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-26T10:43:15.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:15 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2981562391' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T10:43:15.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:15.846+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f827c1a1f10 con 0x7f827c102350 2024-02-26T10:43:16.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.032+0000 7f827a000640 1 -- 172.21.15.60:0/3264400064 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f826c09f330 con 0x7f827c102350 2024-02-26T10:43:16.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f82640a1e00 msgr2=0x7f82640a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f82640a1e00 0x7f82640a42c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f827002c170 tx=0x7f827002df90 comp rx=0 tx=0).stop 2024-02-26T10:43:16.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c102350 msgr2=0x7f827c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c102350 0x7f827c199bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f826c00e9f0 tx=0x7f826c00eeb0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 shutdown_connections 2024-02-26T10:43:16.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f82640a1e00 0x7f82640a42c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c102d10 0x7f827c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f827c102350 0x7f827c199bc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 --2- 172.21.15.60:0/3264400064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f827c101160 0x7f827c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.036+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 >> 172.21.15.60:0/3264400064 conn(0x7f827c0fc6d0 msgr2=0x7f827c073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:16.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.037+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 shutdown_connections 2024-02-26T10:43:16.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.037+0000 7f8282c00640 1 -- 172.21.15.60:0/3264400064 wait complete. 2024-02-26T10:43:16.054 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-02-26T10:43:16.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-02-26T10:43:16.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.300+0000 7fae81200640 1 -- 172.21.15.60:0/60055309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c108a70 msgr2=0x7fae7c108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.300+0000 7fae81200640 1 --2- 172.21.15.60:0/60055309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c108a70 0x7fae7c108ef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae64009860 tx=0x7fae6402bb70 comp rx=0 tx=0).stop 2024-02-26T10:43:16.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.300+0000 7fae81200640 1 -- 172.21.15.60:0/60055309 shutdown_connections 2024-02-26T10:43:16.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.300+0000 7fae81200640 1 --2- 172.21.15.60:0/60055309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7c109430 0x7fae7c10fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.300+0000 7fae81200640 1 --2- 172.21.15.60:0/60055309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c108a70 0x7fae7c108ef0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.300+0000 7fae81200640 1 --2- 172.21.15.60:0/60055309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7c107880 0x7fae7c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.300+0000 7fae81200640 1 -- 172.21.15.60:0/60055309 >> 172.21.15.60:0/60055309 conn(0x7fae7c076820 msgr2=0x7fae7c078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.300+0000 7fae81200640 1 -- 172.21.15.60:0/60055309 shutdown_connections 2024-02-26T10:43:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.300+0000 7fae81200640 1 -- 172.21.15.60:0/60055309 wait complete. 2024-02-26T10:43:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.301+0000 7fae81200640 1 Processor -- start 2024-02-26T10:43:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.301+0000 7fae81200640 1 -- start start 2024-02-26T10:43:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae81200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c107880 0x7fae7c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae81200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7c108a70 0x7fae7c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae81200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7c109430 0x7fae7c1a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae81200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c1134e0 con 0x7fae7c109430 2024-02-26T10:43:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae81200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c113360 con 0x7fae7c108a70 2024-02-26T10:43:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae81200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c113660 con 0x7fae7c107880 2024-02-26T10:43:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae79e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c107880 0x7fae7c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae79e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c107880 0x7fae7c19dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:45210/0 (socket says 172.21.15.60:45210) 2024-02-26T10:43:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae79e00640 1 -- 172.21.15.60:0/867643515 learned_addr learned my addr 172.21.15.60:0/867643515 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae7a800640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7c109430 0x7fae7c1a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.302+0000 7fae71400640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7c108a70 0x7fae7c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae79e00640 1 -- 172.21.15.60:0/867643515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7c108a70 msgr2=0x7fae7c19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae79e00640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7c108a70 0x7fae7c19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae79e00640 1 -- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7c109430 msgr2=0x7fae7c1a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae79e00640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7c109430 0x7fae7c1a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae79e00640 1 -- 172.21.15.60:0/867643515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c1a57f0 con 0x7fae7c107880 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae79e00640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c107880 0x7fae7c19dbd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00e9f0 tx=0x7fae6c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae73400640 1 -- 172.21.15.60:0/867643515 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00ccc0 con 0x7fae7c107880 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae73400640 1 -- 172.21.15.60:0/867643515 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae6c0035f0 con 0x7fae7c107880 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae73400640 1 -- 172.21.15.60:0/867643515 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00ce60 con 0x7fae7c107880 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c1a5ae0 con 0x7fae7c107880 2024-02-26T10:43:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.303+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c1a6000 con 0x7fae7c107880 2024-02-26T10:43:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.305+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c10eba0 con 0x7fae7c107880 2024-02-26T10:43:16.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.306+0000 7fae73400640 1 -- 172.21.15.60:0/867643515 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fae6c0037a0 con 0x7fae7c107880 2024-02-26T10:43:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.308+0000 7fae73400640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fae600a1d80 0x7fae600a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.308+0000 7fae73400640 1 -- 172.21.15.60:0/867643515 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fae6c0ce230 con 0x7fae7c107880 2024-02-26T10:43:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.308+0000 7fae71400640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fae600a1d80 0x7fae600a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.309+0000 7fae71400640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fae600a1d80 0x7fae600a4240 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fae64009830 tx=0x7fae64034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.312+0000 7fae73400640 1 -- 172.21.15.60:0/867643515 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae6c09b3c0 con 0x7fae7c107880 2024-02-26T10:43:16.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.543+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fae7c0616a0 con 0x7fae600a1d80 2024-02-26T10:43:16.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.576+0000 7fae73400640 1 -- 172.21.15.60:0/867643515 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fae7c0616a0 con 0x7fae600a1d80 2024-02-26T10:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fae600a1d80 msgr2=0x7fae600a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fae600a1d80 0x7fae600a4240 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fae64009830 tx=0x7fae64034ab0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c107880 msgr2=0x7fae7c19dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c107880 0x7fae7c19dbd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00e9f0 tx=0x7fae6c00edf0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 shutdown_connections 2024-02-26T10:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fae600a1d80 0x7fae600a4240 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7c108a70 0x7fae7c19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7c109430 0x7fae7c1a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 --2- 172.21.15.60:0/867643515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae7c107880 0x7fae7c19dbd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 >> 172.21.15.60:0/867643515 conn(0x7fae7c076820 msgr2=0x7fae7c10bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:16.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 shutdown_connections 2024-02-26T10:43:16.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.579+0000 7fae81200640 1 -- 172.21.15.60:0/867643515 wait complete. 2024-02-26T10:43:16.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-02-26T10:43:16.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2981562391' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] up:active 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: Cluster is now healthy 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3264400064' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3264400064' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] up:active 2024-02-26T10:43:16.585 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:16 smithi113 ceph-mon[36126]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2981562391' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] up:active 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: Cluster is now healthy 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3264400064' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T10:43:16.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3264400064' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T10:43:16.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] up:active 2024-02-26T10:43:16.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:43:16.594 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:43:16.556+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:43:16.594 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:43:16.556+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:43:16.594 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:43:16.556+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:43:16.594 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:43:16.556+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:43:16.594 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:43:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:43:16.556+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:43:16.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b[66645]: 2024-02-26T10:43:16.494+0000 7efc85822700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b[66645]: 2024-02-26T10:43:16.494+0000 7efc85822700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2981562391' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] up:active 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: Cluster is now healthy 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-26T10:43:16.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-02-26T10:43:16.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-02-26T10:43:16.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3264400064' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T10:43:16.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3264400064' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T10:43:16.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] up:active 2024-02-26T10:43:16.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:16 smithi100 ceph-mon[36051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:43:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.809+0000 7fca0a200640 1 -- 172.21.15.60:0/4185726592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04107880 msgr2=0x7fca04107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.809+0000 7fca0a200640 1 --2- 172.21.15.60:0/4185726592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04107880 0x7fca04107ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009900 tx=0x7fc9f402bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.809+0000 7fca0a200640 1 -- 172.21.15.60:0/4185726592 shutdown_connections 2024-02-26T10:43:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.809+0000 7fca0a200640 1 --2- 172.21.15.60:0/4185726592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04108a70 0x7fca04108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.809+0000 7fca0a200640 1 --2- 172.21.15.60:0/4185726592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca04109430 0x7fca0410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.809+0000 7fca0a200640 1 --2- 172.21.15.60:0/4185726592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04107880 0x7fca04107ca0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.809+0000 7fca0a200640 1 -- 172.21.15.60:0/4185726592 >> 172.21.15.60:0/4185726592 conn(0x7fca04076820 msgr2=0x7fca04078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.809+0000 7fca0a200640 1 -- 172.21.15.60:0/4185726592 shutdown_connections 2024-02-26T10:43:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.809+0000 7fca0a200640 1 -- 172.21.15.60:0/4185726592 wait complete. 2024-02-26T10:43:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.810+0000 7fca0a200640 1 Processor -- start 2024-02-26T10:43:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.810+0000 7fca0a200640 1 -- start start 2024-02-26T10:43:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.810+0000 7fca0a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04107880 0x7fca0419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.810+0000 7fca0a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04108a70 0x7fca0419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.810+0000 7fca0a200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca04109430 0x7fca041a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.810+0000 7fca0a200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04113540 con 0x7fca04109430 2024-02-26T10:43:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.810+0000 7fca0a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca041133c0 con 0x7fca04107880 2024-02-26T10:43:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.810+0000 7fca0a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca041136c0 con 0x7fca04108a70 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca02a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04108a70 0x7fca0419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca02a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04108a70 0x7fca0419e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:45238/0 (socket says 172.21.15.60:45238) 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca02a00640 1 -- 172.21.15.60:0/1936305310 learned_addr learned my addr 172.21.15.60:0/1936305310 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca03e00640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca04109430 0x7fca041a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca03400640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04107880 0x7fca0419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca02a00640 1 -- 172.21.15.60:0/1936305310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04107880 msgr2=0x7fca0419db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca02a00640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04107880 0x7fca0419db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca02a00640 1 -- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca04109430 msgr2=0x7fca041a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca02a00640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca04109430 0x7fca041a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.811+0000 7fca02a00640 1 -- 172.21.15.60:0/1936305310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca041a57b0 con 0x7fca04108a70 2024-02-26T10:43:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.812+0000 7fca02a00640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04108a70 0x7fca0419e0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00e9f0 tx=0x7fc9ec00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.812+0000 7fc9fbe00640 1 -- 172.21.15.60:0/1936305310 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00cd80 con 0x7fca04108a70 2024-02-26T10:43:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.812+0000 7fc9fbe00640 1 -- 172.21.15.60:0/1936305310 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9ec005500 con 0x7fca04108a70 2024-02-26T10:43:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.812+0000 7fc9fbe00640 1 -- 172.21.15.60:0/1936305310 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec010690 con 0x7fca04108a70 2024-02-26T10:43:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.812+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca041a5aa0 con 0x7fca04108a70 2024-02-26T10:43:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.812+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca041a5fa0 con 0x7fca04108a70 2024-02-26T10:43:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.813+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0410eba0 con 0x7fca04108a70 2024-02-26T10:43:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.815+0000 7fc9fbe00640 1 -- 172.21.15.60:0/1936305310 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc9ec0046c0 con 0x7fca04108a70 2024-02-26T10:43:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.817+0000 7fc9fbe00640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9e80a1e00 0x7fc9e80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.817+0000 7fc9fbe00640 1 -- 172.21.15.60:0/1936305310 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc9ec0ce3b0 con 0x7fca04108a70 2024-02-26T10:43:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.817+0000 7fca03400640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9e80a1e00 0x7fc9e80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.818+0000 7fca03400640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9e80a1e00 0x7fc9e80a42c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9f402c170 tx=0x7fc9f402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:16.821+0000 7fc9fbe00640 1 -- 172.21.15.60:0/1936305310 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9ec09b540 con 0x7fca04108a70 2024-02-26T10:43:17.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.053+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fca040616a0 con 0x7fc9e80a1e00 2024-02-26T10:43:17.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.086+0000 7fc9fbe00640 1 -- 172.21.15.60:0/1936305310 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fca040616a0 con 0x7fc9e80a1e00 2024-02-26T10:43:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9e80a1e00 msgr2=0x7fc9e80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9e80a1e00 0x7fc9e80a42c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9f402c170 tx=0x7fc9f402df90 comp rx=0 tx=0).stop 2024-02-26T10:43:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04108a70 msgr2=0x7fca0419e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04108a70 0x7fca0419e0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00e9f0 tx=0x7fc9ec00edf0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 shutdown_connections 2024-02-26T10:43:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9e80a1e00 0x7fc9e80a42c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca04109430 0x7fca041a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca04108a70 0x7fca0419e0d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 --2- 172.21.15.60:0/1936305310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca04107880 0x7fca0419db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 >> 172.21.15.60:0/1936305310 conn(0x7fca04076820 msgr2=0x7fca0410bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 shutdown_connections 2024-02-26T10:43:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.089+0000 7fca0a200640 1 -- 172.21.15.60:0/1936305310 wait complete. 2024-02-26T10:43:17.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-02-26T10:43:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.361+0000 7fa521e00640 1 -- 172.21.15.60:0/3940035789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51c104690 msgr2=0x7fa51c104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.361+0000 7fa521e00640 1 --2- 172.21.15.60:0/3940035789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51c104690 0x7fa51c104b10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa508009900 tx=0x7fa50802bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.361+0000 7fa521e00640 1 -- 172.21.15.60:0/3940035789 shutdown_connections 2024-02-26T10:43:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.361+0000 7fa521e00640 1 --2- 172.21.15.60:0/3940035789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51c104690 0x7fa51c104b10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.361+0000 7fa521e00640 1 --2- 172.21.15.60:0/3940035789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51c105050 0x7fa51c10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.361+0000 7fa521e00640 1 --2- 172.21.15.60:0/3940035789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c1034a0 0x7fa51c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.361+0000 7fa521e00640 1 -- 172.21.15.60:0/3940035789 >> 172.21.15.60:0/3940035789 conn(0x7fa51c0fea10 msgr2=0x7fa51c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.361+0000 7fa521e00640 1 -- 172.21.15.60:0/3940035789 shutdown_connections 2024-02-26T10:43:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.361+0000 7fa521e00640 1 -- 172.21.15.60:0/3940035789 wait complete. 2024-02-26T10:43:17.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.363+0000 7fa521e00640 1 Processor -- start 2024-02-26T10:43:17.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.363+0000 7fa521e00640 1 -- start start 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.363+0000 7fa521e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51c1034a0 0x7fa51c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.363+0000 7fa521e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c104690 0x7fa51c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa521e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51c105050 0x7fa51c1a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa521e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c10f260 con 0x7fa51c105050 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa521e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c10f0e0 con 0x7fa51c1034a0 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa521e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c10f3e0 con 0x7fa51c104690 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51c1034a0 0x7fa51c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c104690 0x7fa51c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c104690 0x7fa51c199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:45264/0 (socket says 172.21.15.60:45264) 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51c1034a0 0x7fa51c1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37824/0 (socket says 172.21.15.60:37824) 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51c105050 0x7fa51c1a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51a000640 1 -- 172.21.15.60:0/1016247516 learned_addr learned my addr 172.21.15.60:0/1016247516 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:17.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51a000640 1 -- 172.21.15.60:0/1016247516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51c1034a0 msgr2=0x7fa51c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51a000640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51c1034a0 0x7fa51c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51a000640 1 -- 172.21.15.60:0/1016247516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51c105050 msgr2=0x7fa51c1a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51a000640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51c105050 0x7fa51c1a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.364+0000 7fa51a000640 1 -- 172.21.15.60:0/1016247516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c1a14d0 con 0x7fa51c104690 2024-02-26T10:43:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.365+0000 7fa51a000640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c104690 0x7fa51c199df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa508000c00 tx=0x7fa508004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.365+0000 7fa520a00640 1 -- 172.21.15.60:0/1016247516 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508043020 con 0x7fa51c104690 2024-02-26T10:43:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.365+0000 7fa520a00640 1 -- 172.21.15.60:0/1016247516 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa508004440 con 0x7fa51c104690 2024-02-26T10:43:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.365+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c1a1760 con 0x7fa51c104690 2024-02-26T10:43:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.365+0000 7fa520a00640 1 -- 172.21.15.60:0/1016247516 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50803e450 con 0x7fa51c104690 2024-02-26T10:43:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.365+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c1a1d00 con 0x7fa51c104690 2024-02-26T10:43:17.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.367+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51c10a810 con 0x7fa51c104690 2024-02-26T10:43:17.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.368+0000 7fa520a00640 1 -- 172.21.15.60:0/1016247516 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa508046020 con 0x7fa51c104690 2024-02-26T10:43:17.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:17 smithi060 ceph-mon[31539]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:17.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.370+0000 7fa520a00640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5000a1e90 0x7fa5000a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.370+0000 7fa520a00640 1 -- 172.21.15.60:0/1016247516 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa5080f0460 con 0x7fa51c104690 2024-02-26T10:43:17.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.370+0000 7fa51aa00640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5000a1e90 0x7fa5000a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.371+0000 7fa51aa00640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5000a1e90 0x7fa5000a4350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa51c1044f0 tx=0x7fa50c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.374+0000 7fa520a00640 1 -- 172.21.15.60:0/1016247516 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa50804d020 con 0x7fa51c104690 2024-02-26T10:43:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.605+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fa51c0616a0 con 0x7fa5000a1e90 2024-02-26T10:43:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.611+0000 7fa520a00640 1 -- 172.21.15.60:0/1016247516 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fa51c0616a0 con 0x7fa5000a1e90 2024-02-26T10:43:17.608 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/qa/sv_0/ae36f7a0-8c24-4d4b-8949-7a7c389315fa 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5000a1e90 msgr2=0x7fa5000a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5000a1e90 0x7fa5000a4350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa51c1044f0 tx=0x7fa50c009110 comp rx=0 tx=0).stop 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c104690 msgr2=0x7fa51c199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c104690 0x7fa51c199df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa508000c00 tx=0x7fa508004040 comp rx=0 tx=0).stop 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 shutdown_connections 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5000a1e90 0x7fa5000a4350 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51c105050 0x7fa51c1a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c104690 0x7fa51c199df0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 --2- 172.21.15.60:0/1016247516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51c1034a0 0x7fa51c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 >> 172.21.15.60:0/1016247516 conn(0x7fa51c0fea10 msgr2=0x7fa51c1079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 shutdown_connections 2024-02-26T10:43:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.614+0000 7fa521e00640 1 -- 172.21.15.60:0/1016247516 wait complete. 2024-02-26T10:43:17.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:17 smithi100 ceph-mon[36051]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:17.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-02-26T10:43:17.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:17 smithi113 ceph-mon[36126]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.876+0000 7f8d8d800640 1 -- 172.21.15.60:0/3056352446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d880752a0 msgr2=0x7f8d880736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.876+0000 7f8d8d800640 1 --2- 172.21.15.60:0/3056352446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d880752a0 0x7f8d880736f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009900 tx=0x7f8d7802bc50 comp rx=0 tx=0).stop 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.876+0000 7f8d8d800640 1 -- 172.21.15.60:0/3056352446 shutdown_connections 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.876+0000 7f8d8d800640 1 --2- 172.21.15.60:0/3056352446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d88109000 0x7f8d8810b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.876+0000 7f8d8d800640 1 --2- 172.21.15.60:0/3056352446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d88073cc0 0x7f8d88074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.876+0000 7f8d8d800640 1 --2- 172.21.15.60:0/3056352446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d880752a0 0x7f8d880736f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.876+0000 7f8d8d800640 1 -- 172.21.15.60:0/3056352446 >> 172.21.15.60:0/3056352446 conn(0x7f8d880fc6d0 msgr2=0x7f8d880feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.876+0000 7f8d8d800640 1 -- 172.21.15.60:0/3056352446 shutdown_connections 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.876+0000 7f8d8d800640 1 -- 172.21.15.60:0/3056352446 wait complete. 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.877+0000 7f8d8d800640 1 Processor -- start 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.877+0000 7f8d8d800640 1 -- start start 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.877+0000 7f8d8d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d88073cc0 0x7f8d88199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.877+0000 7f8d8d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d880752a0 0x7f8d88199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.877+0000 7f8d8d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d88109000 0x7f8d881a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.877+0000 7f8d8d800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8810efd0 con 0x7f8d88073cc0 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.877+0000 7f8d8d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8810ee50 con 0x7f8d880752a0 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.877+0000 7f8d8d800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8810f150 con 0x7f8d88109000 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d88073cc0 0x7f8d88199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d88073cc0 0x7f8d88199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39606/0 (socket says 172.21.15.60:39606) 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87e00640 1 -- 172.21.15.60:0/312146780 learned_addr learned my addr 172.21.15.60:0/312146780 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87400640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d880752a0 0x7f8d88199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d8ce00640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d88109000 0x7f8d881a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87400640 1 -- 172.21.15.60:0/312146780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d88109000 msgr2=0x7f8d881a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87400640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d88109000 0x7f8d881a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87400640 1 -- 172.21.15.60:0/312146780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d88073cc0 msgr2=0x7f8d88199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87400640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d88073cc0 0x7f8d88199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87400640 1 -- 172.21.15.60:0/312146780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d881a1280 con 0x7f8d880752a0 2024-02-26T10:43:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.878+0000 7f8d87400640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d880752a0 0x7f8d88199ba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400e9f0 tx=0x7f8d7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.879+0000 7f8d84c00640 1 -- 172.21.15.60:0/312146780 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400ccc0 con 0x7f8d880752a0 2024-02-26T10:43:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.879+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d881a1570 con 0x7f8d880752a0 2024-02-26T10:43:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.879+0000 7f8d84c00640 1 -- 172.21.15.60:0/312146780 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d74010910 con 0x7f8d880752a0 2024-02-26T10:43:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.879+0000 7f8d84c00640 1 -- 172.21.15.60:0/312146780 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74018960 con 0x7f8d880752a0 2024-02-26T10:43:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.879+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d881a1b10 con 0x7f8d880752a0 2024-02-26T10:43:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.880+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d88107d30 con 0x7f8d880752a0 2024-02-26T10:43:17.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.882+0000 7f8d84c00640 1 -- 172.21.15.60:0/312146780 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f8d74010ac0 con 0x7f8d880752a0 2024-02-26T10:43:17.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.883+0000 7f8d84c00640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d6c0a1e10 0x7f8d6c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.884+0000 7f8d84c00640 1 -- 172.21.15.60:0/312146780 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8d740ce520 con 0x7f8d880752a0 2024-02-26T10:43:17.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.884+0000 7f8d87e00640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d6c0a1e10 0x7f8d6c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.885+0000 7f8d87e00640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d6c0a1e10 0x7f8d6c0a42d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8d780096c0 tx=0x7f8d78000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:17.886+0000 7f8d84c00640 1 -- 172.21.15.60:0/312146780 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d7409af00 con 0x7f8d880752a0 2024-02-26T10:43:18.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.118+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d880616a0 con 0x7f8d6c0a1e10 2024-02-26T10:43:18.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.150+0000 7f8d84c00640 1 -- 172.21.15.60:0/312146780 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8d880616a0 con 0x7f8d6c0a1e10 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.152+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d6c0a1e10 msgr2=0x7f8d6c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.152+0000 7f8d8d800640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d6c0a1e10 0x7f8d6c0a42d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8d780096c0 tx=0x7f8d78000bc0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.152+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d880752a0 msgr2=0x7f8d88199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.152+0000 7f8d8d800640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d880752a0 0x7f8d88199ba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400e9f0 tx=0x7f8d7400eeb0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.153+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 shutdown_connections 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.153+0000 7f8d8d800640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d6c0a1e10 0x7f8d6c0a42d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.153+0000 7f8d8d800640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d880752a0 0x7f8d88199ba0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.153+0000 7f8d8d800640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d88109000 0x7f8d881a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.153+0000 7f8d8d800640 1 --2- 172.21.15.60:0/312146780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d88073cc0 0x7f8d88199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.153+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 >> 172.21.15.60:0/312146780 conn(0x7f8d880fc6d0 msgr2=0x7f8d88104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.153+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 shutdown_connections 2024-02-26T10:43:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.153+0000 7f8d8d800640 1 -- 172.21.15.60:0/312146780 wait complete. 2024-02-26T10:43:18.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-02-26T10:43:18.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.408+0000 7f4935200640 1 -- 172.21.15.60:0/3857164342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4930104510 msgr2=0x7f4930106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.408+0000 7f4935200640 1 --2- 172.21.15.60:0/3857164342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4930104510 0x7f4930106900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f491c009860 tx=0x7f491c02bb70 comp rx=0 tx=0).stop 2024-02-26T10:43:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.408+0000 7f4935200640 1 -- 172.21.15.60:0/3857164342 shutdown_connections 2024-02-26T10:43:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.408+0000 7f4935200640 1 --2- 172.21.15.60:0/3857164342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930106e40 0x7f4930109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.408+0000 7f4935200640 1 --2- 172.21.15.60:0/3857164342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4930104510 0x7f4930106900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.408+0000 7f4935200640 1 --2- 172.21.15.60:0/3857164342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930101be0 0x7f4930103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.408+0000 7f4935200640 1 -- 172.21.15.60:0/3857164342 >> 172.21.15.60:0/3857164342 conn(0x7f49300fb5d0 msgr2=0x7f49300fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.408+0000 7f4935200640 1 -- 172.21.15.60:0/3857164342 shutdown_connections 2024-02-26T10:43:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.408+0000 7f4935200640 1 -- 172.21.15.60:0/3857164342 wait complete. 2024-02-26T10:43:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.410+0000 7f4935200640 1 Processor -- start 2024-02-26T10:43:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.410+0000 7f4935200640 1 -- start start 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.410+0000 7f4935200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930101be0 0x7f4930199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.410+0000 7f4935200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4930104510 0x7f4930199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.410+0000 7f4935200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930106e40 0x7f49301a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.410+0000 7f4935200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493010cdf0 con 0x7f4930101be0 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.410+0000 7f4935200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493010cc70 con 0x7f4930106e40 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.410+0000 7f4935200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493010cf70 con 0x7f4930104510 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930101be0 0x7f4930199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930101be0 0x7f4930199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39624/0 (socket says 172.21.15.60:39624) 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492f400640 1 -- 172.21.15.60:0/2853661040 learned_addr learned my addr 172.21.15.60:0/2853661040 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492ea00640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4930104510 0x7f4930199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492fe00640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930106e40 0x7f49301a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492ea00640 1 -- 172.21.15.60:0/2853661040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930106e40 msgr2=0x7f49301a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492ea00640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930106e40 0x7f49301a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492ea00640 1 -- 172.21.15.60:0/2853661040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930101be0 msgr2=0x7f4930199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492ea00640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930101be0 0x7f4930199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492ea00640 1 -- 172.21.15.60:0/2853661040 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49301a12a0 con 0x7f4930104510 2024-02-26T10:43:18.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.411+0000 7f492ea00640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4930104510 0x7f4930199bc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f491c009780 tx=0x7f491c032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.412+0000 7f4927e00640 1 -- 172.21.15.60:0/2853661040 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c041020 con 0x7f4930104510 2024-02-26T10:43:18.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.412+0000 7f4927e00640 1 -- 172.21.15.60:0/2853661040 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f491c02c9f0 con 0x7f4930104510 2024-02-26T10:43:18.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.412+0000 7f4927e00640 1 -- 172.21.15.60:0/2853661040 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c039040 con 0x7f4930104510 2024-02-26T10:43:18.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.412+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49301a1530 con 0x7f4930104510 2024-02-26T10:43:18.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.412+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49301a19a0 con 0x7f4930104510 2024-02-26T10:43:18.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.414+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4930108990 con 0x7f4930104510 2024-02-26T10:43:18.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.416+0000 7f4927e00640 1 -- 172.21.15.60:0/2853661040 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f491c044020 con 0x7f4930104510 2024-02-26T10:43:18.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.417+0000 7f4927e00640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f49140a1e90 0x7f49140a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.417+0000 7f4927e00640 1 -- 172.21.15.60:0/2853661040 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f491c0ef7d0 con 0x7f4930104510 2024-02-26T10:43:18.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.418+0000 7f492f400640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f49140a1e90 0x7f49140a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.418+0000 7f492f400640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f49140a1e90 0x7f49140a4350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f49200058c0 tx=0x7f4920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.421+0000 7f4927e00640 1 -- 172.21.15.60:0/2853661040 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f491c04b020 con 0x7f4930104510 2024-02-26T10:43:18.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:18 smithi060 ceph-mon[31539]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-02-26T10:43:18.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:18 smithi060 ceph-mon[31539]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:18.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:18 smithi060 ceph-mon[31539]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-02-26T10:43:18.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:18 smithi060 ceph-mon[31539]: from='client.24850 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:18.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:18 smithi100 ceph-mon[36051]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-02-26T10:43:18.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:18 smithi100 ceph-mon[36051]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:18.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:18 smithi100 ceph-mon[36051]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-02-26T10:43:18.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:18 smithi100 ceph-mon[36051]: from='client.24850 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:18.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.651+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4930061730 con 0x7f49140a1e90 2024-02-26T10:43:18.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.660+0000 7f4927e00640 1 -- 172.21.15.60:0/2853661040 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f4930061730 con 0x7f49140a1e90 2024-02-26T10:43:18.657 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72 2024-02-26T10:43:18.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f49140a1e90 msgr2=0x7f49140a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f49140a1e90 0x7f49140a4350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f49200058c0 tx=0x7f4920009260 comp rx=0 tx=0).stop 2024-02-26T10:43:18.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4930104510 msgr2=0x7f4930199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4930104510 0x7f4930199bc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f491c009780 tx=0x7f491c032430 comp rx=0 tx=0).stop 2024-02-26T10:43:18.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 shutdown_connections 2024-02-26T10:43:18.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f49140a1e90 0x7f49140a4350 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930106e40 0x7f49301a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4930104510 0x7f4930199bc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 --2- 172.21.15.60:0/2853661040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930101be0 0x7f4930199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 >> 172.21.15.60:0/2853661040 conn(0x7f49300fb5d0 msgr2=0x7f49301012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 shutdown_connections 2024-02-26T10:43:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.663+0000 7f4935200640 1 -- 172.21.15.60:0/2853661040 wait complete. 2024-02-26T10:43:18.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:43:18.736 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:18 smithi113 ceph-mon[36126]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-02-26T10:43:18.736 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:18 smithi113 ceph-mon[36126]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:18.736 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:18 smithi113 ceph-mon[36126]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-02-26T10:43:18.736 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:18 smithi113 ceph-mon[36126]: from='client.24850 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.899+0000 7ff84f400640 1 -- 172.21.15.60:0/3637045560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 msgr2=0x7ff848104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.899+0000 7ff84f400640 1 --2- 172.21.15.60:0/3637045560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 0x7ff848104b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff834009900 tx=0x7ff83402bc30 comp rx=0 tx=0).stop 2024-02-26T10:43:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.900+0000 7ff84f400640 1 -- 172.21.15.60:0/3637045560 shutdown_connections 2024-02-26T10:43:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.900+0000 7ff84f400640 1 --2- 172.21.15.60:0/3637045560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 0x7ff848104b10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.900+0000 7ff84f400640 1 --2- 172.21.15.60:0/3637045560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848105050 0x7ff84810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.900+0000 7ff84f400640 1 --2- 172.21.15.60:0/3637045560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8481034a0 0x7ff8481038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.900+0000 7ff84f400640 1 -- 172.21.15.60:0/3637045560 >> 172.21.15.60:0/3637045560 conn(0x7ff8480fea10 msgr2=0x7ff848100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.900+0000 7ff84f400640 1 -- 172.21.15.60:0/3637045560 shutdown_connections 2024-02-26T10:43:18.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.900+0000 7ff84f400640 1 -- 172.21.15.60:0/3637045560 wait complete. 2024-02-26T10:43:18.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.901+0000 7ff84f400640 1 Processor -- start 2024-02-26T10:43:18.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.901+0000 7ff84f400640 1 -- start start 2024-02-26T10:43:18.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.901+0000 7ff84f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8481034a0 0x7ff848199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff84f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 0x7ff848199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff84f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848105050 0x7ff8481a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff84f400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84810f130 con 0x7ff848105050 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff84f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84810efb0 con 0x7ff848104690 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff84f400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84810f2b0 con 0x7ff8481034a0 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff847e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 0x7ff848199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff84d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848105050 0x7ff8481a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff847e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 0x7ff848199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37884/0 (socket says 172.21.15.60:37884) 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff847e00640 1 -- 172.21.15.60:0/1483061683 learned_addr learned my addr 172.21.15.60:0/1483061683 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff84ca00640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8481034a0 0x7ff848199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff847e00640 1 -- 172.21.15.60:0/1483061683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8481034a0 msgr2=0x7ff848199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff847e00640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8481034a0 0x7ff848199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff847e00640 1 -- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848105050 msgr2=0x7ff8481a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff847e00640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848105050 0x7ff8481a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff847e00640 1 -- 172.21.15.60:0/1483061683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8481a1370 con 0x7ff848104690 2024-02-26T10:43:18.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.902+0000 7ff84d400640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848105050 0x7ff8481a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:18.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.903+0000 7ff847e00640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 0x7ff848199c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff834009230 tx=0x7ff83402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.903+0000 7ff845600640 1 -- 172.21.15.60:0/1483061683 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff834041070 con 0x7ff848104690 2024-02-26T10:43:18.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.903+0000 7ff845600640 1 -- 172.21.15.60:0/1483061683 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff834004450 con 0x7ff848104690 2024-02-26T10:43:18.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.903+0000 7ff845600640 1 -- 172.21.15.60:0/1483061683 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff834003e20 con 0x7ff848104690 2024-02-26T10:43:18.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.903+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8481a1600 con 0x7ff848104690 2024-02-26T10:43:18.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.903+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8481a1b40 con 0x7ff848104690 2024-02-26T10:43:18.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.904+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84810a760 con 0x7ff848104690 2024-02-26T10:43:18.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.907+0000 7ff845600640 1 -- 172.21.15.60:0/1483061683 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff834044070 con 0x7ff848104690 2024-02-26T10:43:18.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.908+0000 7ff845600640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8300a1e90 0x7ff8300a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.908+0000 7ff845600640 1 -- 172.21.15.60:0/1483061683 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff8340eff90 con 0x7ff848104690 2024-02-26T10:43:18.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.908+0000 7ff84ca00640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8300a1e90 0x7ff8300a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.909+0000 7ff84ca00640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8300a1e90 0x7ff8300a4350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8481044f0 tx=0x7ff83c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:18.910+0000 7ff845600640 1 -- 172.21.15.60:0/1483061683 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff8340bca80 con 0x7ff848104690 2024-02-26T10:43:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.152+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff84802cff0 con 0x7ff848104690 2024-02-26T10:43:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.153+0000 7ff845600640 1 -- 172.21.15.60:0/1483061683 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11359 (secure 0 0 0) 0x7ff8340c0b20 con 0x7ff848104690 2024-02-26T10:43:19.150 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:19.151 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:43:16.029212+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:43:19.151 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8300a1e90 msgr2=0x7ff8300a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8300a1e90 0x7ff8300a4350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8481044f0 tx=0x7ff83c005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 msgr2=0x7ff848199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 0x7ff848199c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff834009230 tx=0x7ff83402c650 comp rx=0 tx=0).stop 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 shutdown_connections 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff8300a1e90 0x7ff8300a4350 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff848104690 0x7ff848199c90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848105050 0x7ff8481a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 --2- 172.21.15.60:0/1483061683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8481034a0 0x7ff848199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 >> 172.21.15.60:0/1483061683 conn(0x7ff8480fea10 msgr2=0x7ff848107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:19.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 shutdown_connections 2024-02-26T10:43:19.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.156+0000 7ff84f400640 1 -- 172.21.15.60:0/1483061683 wait complete. 2024-02-26T10:43:19.173 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:43:19.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.406+0000 7fec82400640 1 -- 172.21.15.60:0/2683362178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c1044a0 msgr2=0x7fec7c104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.406+0000 7fec82400640 1 --2- 172.21.15.60:0/2683362178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c1044a0 0x7fec7c104920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fec68009900 tx=0x7fec6802bc40 comp rx=0 tx=0).stop 2024-02-26T10:43:19.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.407+0000 7fec82400640 1 -- 172.21.15.60:0/2683362178 shutdown_connections 2024-02-26T10:43:19.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.407+0000 7fec82400640 1 --2- 172.21.15.60:0/2683362178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec7c104e60 0x7fec7c10b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.407+0000 7fec82400640 1 --2- 172.21.15.60:0/2683362178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c1044a0 0x7fec7c104920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.407+0000 7fec82400640 1 --2- 172.21.15.60:0/2683362178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7c1032b0 0x7fec7c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.407+0000 7fec82400640 1 -- 172.21.15.60:0/2683362178 >> 172.21.15.60:0/2683362178 conn(0x7fec7c0fe840 msgr2=0x7fec7c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:19.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.407+0000 7fec82400640 1 -- 172.21.15.60:0/2683362178 shutdown_connections 2024-02-26T10:43:19.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.407+0000 7fec82400640 1 -- 172.21.15.60:0/2683362178 wait complete. 2024-02-26T10:43:19.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.408+0000 7fec82400640 1 Processor -- start 2024-02-26T10:43:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.408+0000 7fec82400640 1 -- start start 2024-02-26T10:43:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec82400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7c1032b0 0x7fec7c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec82400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec7c1044a0 0x7fec7c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec82400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c104e60 0x7fec7c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec82400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c10ef90 con 0x7fec7c1044a0 2024-02-26T10:43:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec82400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c10ee10 con 0x7fec7c1032b0 2024-02-26T10:43:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec82400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c10f110 con 0x7fec7c104e60 2024-02-26T10:43:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec7be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec7c1044a0 0x7fec7c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec7be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec7c1044a0 0x7fec7c199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39660/0 (socket says 172.21.15.60:39660) 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec7be00640 1 -- 172.21.15.60:0/1636377300 learned_addr learned my addr 172.21.15.60:0/1636377300 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec81a00640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c104e60 0x7fec7c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.409+0000 7fec81000640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7c1032b0 0x7fec7c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec81a00640 1 -- 172.21.15.60:0/1636377300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7c1032b0 msgr2=0x7fec7c199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec81a00640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7c1032b0 0x7fec7c199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec81a00640 1 -- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec7c1044a0 msgr2=0x7fec7c199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec81a00640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec7c1044a0 0x7fec7c199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec81a00640 1 -- 172.21.15.60:0/1636377300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c1a1180 con 0x7fec7c104e60 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec7be00640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec7c1044a0 0x7fec7c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec81a00640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c104e60 0x7fec7c1a0b10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fec74009fd0 tx=0x7fec7400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec79600640 1 -- 172.21.15.60:0/1636377300 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400ce20 con 0x7fec7c104e60 2024-02-26T10:43:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c1a1470 con 0x7fec7c104e60 2024-02-26T10:43:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec79600640 1 -- 172.21.15.60:0/1636377300 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec74005020 con 0x7fec7c104e60 2024-02-26T10:43:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec79600640 1 -- 172.21.15.60:0/1636377300 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74010720 con 0x7fec7c104e60 2024-02-26T10:43:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.410+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c1a1990 con 0x7fec7c104e60 2024-02-26T10:43:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.411+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c10a570 con 0x7fec7c104e60 2024-02-26T10:43:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.413+0000 7fec79600640 1 -- 172.21.15.60:0/1636377300 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fec74005540 con 0x7fec7c104e60 2024-02-26T10:43:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.414+0000 7fec79600640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec640a1e10 0x7fec640a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.415+0000 7fec79600640 1 -- 172.21.15.60:0/1636377300 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fec740ce8c0 con 0x7fec7c104e60 2024-02-26T10:43:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.415+0000 7fec81000640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec640a1e10 0x7fec640a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.415+0000 7fec81000640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec640a1e10 0x7fec640a42d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fec7c104300 tx=0x7fec7000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.419+0000 7fec79600640 1 -- 172.21.15.60:0/1636377300 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec7409b8f0 con 0x7fec7c104e60 2024-02-26T10:43:19.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:19 smithi060 ceph-mon[31539]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:19.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:19 smithi060 ceph-mon[31539]: from='client.24862 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:19.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:19 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1483061683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:43:19.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.622+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec7c061e60 con 0x7fec7c104e60 2024-02-26T10:43:19.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.624+0000 7fec79600640 1 -- 172.21.15.60:0/1636377300 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7fec7409b1f0 con 0x7fec7c104e60 2024-02-26T10:43:19.621 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:19.622 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec640a1e10 msgr2=0x7fec640a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec640a1e10 0x7fec640a42d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fec7c104300 tx=0x7fec7000b390 comp rx=0 tx=0).stop 2024-02-26T10:43:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c104e60 msgr2=0x7fec7c1a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c104e60 0x7fec7c1a0b10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fec74009fd0 tx=0x7fec7400ef10 comp rx=0 tx=0).stop 2024-02-26T10:43:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 shutdown_connections 2024-02-26T10:43:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec640a1e10 0x7fec640a42d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c104e60 0x7fec7c1a0b10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec7c1044a0 0x7fec7c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 --2- 172.21.15.60:0/1636377300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec7c1032b0 0x7fec7c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 >> 172.21.15.60:0/1636377300 conn(0x7fec7c0fe840 msgr2=0x7fec7c107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 shutdown_connections 2024-02-26T10:43:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.627+0000 7fec82400640 1 -- 172.21.15.60:0/1636377300 wait complete. 2024-02-26T10:43:19.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:19 smithi100 ceph-mon[36051]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:19.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:19 smithi100 ceph-mon[36051]: from='client.24862 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:19.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:19 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1483061683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:43:19.645 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T10:43:19.645 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:43:19.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:19 smithi113 ceph-mon[36126]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:19.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:19 smithi113 ceph-mon[36126]: from='client.24862 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:43:19.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:19 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1483061683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:43:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.869+0000 7faa55600640 1 -- 172.21.15.60:0/1400239486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 msgr2=0x7faa500772c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.869+0000 7faa55600640 1 --2- 172.21.15.60:0/1400239486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 0x7faa500772c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faa38009ca0 tx=0x7faa3802c100 comp rx=0 tx=0).stop 2024-02-26T10:43:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.869+0000 7faa55600640 1 -- 172.21.15.60:0/1400239486 shutdown_connections 2024-02-26T10:43:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.869+0000 7faa55600640 1 --2- 172.21.15.60:0/1400239486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa50077800 0x7faa500780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.869+0000 7faa55600640 1 --2- 172.21.15.60:0/1400239486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 0x7faa500772c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.869+0000 7faa55600640 1 --2- 172.21.15.60:0/1400239486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa50075c20 0x7faa50076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.869+0000 7faa55600640 1 -- 172.21.15.60:0/1400239486 >> 172.21.15.60:0/1400239486 conn(0x7faa500fea10 msgr2=0x7faa50100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:19.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.869+0000 7faa55600640 1 -- 172.21.15.60:0/1400239486 shutdown_connections 2024-02-26T10:43:19.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.869+0000 7faa55600640 1 -- 172.21.15.60:0/1400239486 wait complete. 2024-02-26T10:43:19.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.871+0000 7faa55600640 1 Processor -- start 2024-02-26T10:43:19.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.871+0000 7faa55600640 1 -- start start 2024-02-26T10:43:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.871+0000 7faa55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa50075c20 0x7faa5019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.871+0000 7faa55600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 0x7faa5019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.871+0000 7faa55600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa50077800 0x7faa501a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.871+0000 7faa55600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa501134f0 con 0x7faa50076e60 2024-02-26T10:43:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.871+0000 7faa55600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50113370 con 0x7faa50075c20 2024-02-26T10:43:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.871+0000 7faa55600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50113670 con 0x7faa50077800 2024-02-26T10:43:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa46000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 0x7faa5019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa4ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa50075c20 0x7faa5019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa46000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 0x7faa5019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39684/0 (socket says 172.21.15.60:39684) 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa46000640 1 -- 172.21.15.60:0/983605652 learned_addr learned my addr 172.21.15.60:0/983605652 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa4ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa50075c20 0x7faa5019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37934/0 (socket says 172.21.15.60:37934) 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa4f400640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa50077800 0x7faa501a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa46000640 1 -- 172.21.15.60:0/983605652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa50077800 msgr2=0x7faa501a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa46000640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa50077800 0x7faa501a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa46000640 1 -- 172.21.15.60:0/983605652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa50075c20 msgr2=0x7faa5019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa46000640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa50075c20 0x7faa5019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa46000640 1 -- 172.21.15.60:0/983605652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa501a57c0 con 0x7faa50076e60 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.872+0000 7faa4f400640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa50077800 0x7faa501a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.873+0000 7faa46000640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 0x7faa5019e0e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faa38009dc0 tx=0x7faa38035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.873+0000 7faa47e00640 1 -- 172.21.15.60:0/983605652 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38039040 con 0x7faa50076e60 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.873+0000 7faa47e00640 1 -- 172.21.15.60:0/983605652 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa38044070 con 0x7faa50076e60 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.873+0000 7faa47e00640 1 -- 172.21.15.60:0/983605652 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38004600 con 0x7faa50076e60 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.873+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa501a5a50 con 0x7faa50076e60 2024-02-26T10:43:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.873+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa501a5f40 con 0x7faa50076e60 2024-02-26T10:43:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.874+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5010fff0 con 0x7faa50076e60 2024-02-26T10:43:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.876+0000 7faa47e00640 1 -- 172.21.15.60:0/983605652 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7faa38032890 con 0x7faa50076e60 2024-02-26T10:43:19.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.877+0000 7faa47e00640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa340a1e10 0x7faa340a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.878+0000 7faa47e00640 1 -- 172.21.15.60:0/983605652 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7faa380f0840 con 0x7faa50076e60 2024-02-26T10:43:19.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.878+0000 7faa4ea00640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa340a1e10 0x7faa340a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.879+0000 7faa4ea00640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa340a1e10 0x7faa340a42d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faa40009bd0 tx=0x7faa40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:19.882+0000 7faa47e00640 1 -- 172.21.15.60:0/983605652 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa380bd980 con 0x7faa50076e60 2024-02-26T10:43:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.080+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faa500616a0 con 0x7faa340a1e10 2024-02-26T10:43:20.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.118+0000 7faa47e00640 1 -- 172.21.15.60:0/983605652 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340645 (secure 0 0 0) 0x7faa500616a0 con 0x7faa340a1e10 2024-02-26T10:43:20.120 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T10:43:20.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.124+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa340a1e10 msgr2=0x7faa340a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.124+0000 7faa55600640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa340a1e10 0x7faa340a42d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faa40009bd0 tx=0x7faa40005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:20.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.124+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 msgr2=0x7faa5019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.125+0000 7faa55600640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 0x7faa5019e0e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faa38009dc0 tx=0x7faa38035d90 comp rx=0 tx=0).stop 2024-02-26T10:43:20.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.125+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 shutdown_connections 2024-02-26T10:43:20.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.125+0000 7faa55600640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa340a1e10 0x7faa340a42d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.125+0000 7faa55600640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa50077800 0x7faa501a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.125+0000 7faa55600640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa50076e60 0x7faa5019e0e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.125+0000 7faa55600640 1 --2- 172.21.15.60:0/983605652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa50075c20 0x7faa5019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.125+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 >> 172.21.15.60:0/983605652 conn(0x7faa500fea10 msgr2=0x7faa50100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:20.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.125+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 shutdown_connections 2024-02-26T10:43:20.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.125+0000 7faa55600640 1 -- 172.21.15.60:0/983605652 wait complete. 2024-02-26T10:43:20.144 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-02-26T10:43:18.895503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632064,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":159,"num_read_kb":156,"num_write":302,"num_write_kb":3012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369452,"kb_used_data":11960,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572372,"statfs":{"total":1151940427776,"available":1151562108928,"internally_reserved":0,"allocated":12247040,"data_stored":7731031,"data_compressed":287540,"data_compressed_allocated":385024,"data_compressed_original":2269184,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37197,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":33,"num_read_kb":47,"num_write":73,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006326"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799086+0000","last_change":"2024-02-26T10:42:42.176634+0000","last_active":"2024-02-26T10:42:48.799086+0000","last_peered":"2024-02-26T10:42:48.799086+0000","last_clean":"2024-02-26T10:42:48.799086+0000","last_became_active":"2024-02-26T10:42:42.176386+0000","last_became_peered":"2024-02-26T10:42:42.176386+0000","last_unstale":"2024-02-26T10:42:48.799086+0000","last_undegraded":"2024-02-26T10:42:48.799086+0000","last_fullsized":"2024-02-26T10:42:48.799086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:45.005506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799615+0000","last_change":"2024-02-26T10:42:42.176769+0000","last_active":"2024-02-26T10:42:48.799615+0000","last_peered":"2024-02-26T10:42:48.799615+0000","last_clean":"2024-02-26T10:42:48.799615+0000","last_became_active":"2024-02-26T10:42:42.174868+0000","last_became_peered":"2024-02-26T10:42:42.174868+0000","last_unstale":"2024-02-26T10:42:48.799615+0000","last_undegraded":"2024-02-26T10:42:48.799615+0000","last_fullsized":"2024-02-26T10:42:48.799615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:16.202371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800123+0000","last_change":"2024-02-26T10:42:42.184410+0000","last_active":"2024-02-26T10:42:48.800123+0000","last_peered":"2024-02-26T10:42:48.800123+0000","last_clean":"2024-02-26T10:42:48.800123+0000","last_became_active":"2024-02-26T10:42:42.182986+0000","last_became_peered":"2024-02-26T10:42:42.182986+0000","last_unstale":"2024-02-26T10:42:48.800123+0000","last_undegraded":"2024-02-26T10:42:48.800123+0000","last_fullsized":"2024-02-26T10:42:48.800123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:28:45.445325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518257+0000","last_change":"2024-02-26T10:42:42.168283+0000","last_active":"2024-02-26T10:42:48.518257+0000","last_peered":"2024-02-26T10:42:48.518257+0000","last_clean":"2024-02-26T10:42:48.518257+0000","last_became_active":"2024-02-26T10:42:42.167565+0000","last_became_peered":"2024-02-26T10:42:42.167565+0000","last_unstale":"2024-02-26T10:42:48.518257+0000","last_undegraded":"2024-02-26T10:42:48.518257+0000","last_fullsized":"2024-02-26T10:42:48.518257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:16:37.766158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797743+0000","last_change":"2024-02-26T10:42:42.192448+0000","last_active":"2024-02-26T10:42:48.797743+0000","last_peered":"2024-02-26T10:42:48.797743+0000","last_clean":"2024-02-26T10:42:48.797743+0000","last_became_active":"2024-02-26T10:42:42.192133+0000","last_became_peered":"2024-02-26T10:42:42.192133+0000","last_unstale":"2024-02-26T10:42:48.797743+0000","last_undegraded":"2024-02-26T10:42:48.797743+0000","last_fullsized":"2024-02-26T10:42:48.797743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:43:49.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797683+0000","last_change":"2024-02-26T10:42:42.169999+0000","last_active":"2024-02-26T10:42:48.797683+0000","last_peered":"2024-02-26T10:42:48.797683+0000","last_clean":"2024-02-26T10:42:48.797683+0000","last_became_active":"2024-02-26T10:42:42.169678+0000","last_became_peered":"2024-02-26T10:42:42.169678+0000","last_unstale":"2024-02-26T10:42:48.797683+0000","last_undegraded":"2024-02-26T10:42:48.797683+0000","last_fullsized":"2024-02-26T10:42:48.797683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:54.435203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797671+0000","last_change":"2024-02-26T10:42:42.177538+0000","last_active":"2024-02-26T10:42:48.797671+0000","last_peered":"2024-02-26T10:42:48.797671+0000","last_clean":"2024-02-26T10:42:48.797671+0000","last_became_active":"2024-02-26T10:42:42.177204+0000","last_became_peered":"2024-02-26T10:42:42.177204+0000","last_unstale":"2024-02-26T10:42:48.797671+0000","last_undegraded":"2024-02-26T10:42:48.797671+0000","last_fullsized":"2024-02-26T10:42:48.797671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:09.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799743+0000","last_change":"2024-02-26T10:42:42.191342+0000","last_active":"2024-02-26T10:42:48.799743+0000","last_peered":"2024-02-26T10:42:48.799743+0000","last_clean":"2024-02-26T10:42:48.799743+0000","last_became_active":"2024-02-26T10:42:42.190990+0000","last_became_peered":"2024-02-26T10:42:42.190990+0000","last_unstale":"2024-02-26T10:42:48.799743+0000","last_undegraded":"2024-02-26T10:42:48.799743+0000","last_fullsized":"2024-02-26T10:42:48.799743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:20:45.190300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799562+0000","last_change":"2024-02-26T10:42:42.173175+0000","last_active":"2024-02-26T10:42:48.799562+0000","last_peered":"2024-02-26T10:42:48.799562+0000","last_clean":"2024-02-26T10:42:48.799562+0000","last_became_active":"2024-02-26T10:42:42.172828+0000","last_became_peered":"2024-02-26T10:42:42.172828+0000","last_unstale":"2024-02-26T10:42:48.799562+0000","last_undegraded":"2024-02-26T10:42:48.799562+0000","last_fullsized":"2024-02-26T10:42:48.799562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:17:34.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798681+0000","last_change":"2024-02-26T10:42:42.167267+0000","last_active":"2024-02-26T10:42:48.798681+0000","last_peered":"2024-02-26T10:42:48.798681+0000","last_clean":"2024-02-26T10:42:48.798681+0000","last_became_active":"2024-02-26T10:42:42.166948+0000","last_became_peered":"2024-02-26T10:42:42.166948+0000","last_unstale":"2024-02-26T10:42:48.798681+0000","last_undegraded":"2024-02-26T10:42:48.798681+0000","last_fullsized":"2024-02-26T10:42:48.798681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:52.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798497+0000","last_change":"2024-02-26T10:42:42.171299+0000","last_active":"2024-02-26T10:42:48.798497+0000","last_peered":"2024-02-26T10:42:48.798497+0000","last_clean":"2024-02-26T10:42:48.798497+0000","last_became_active":"2024-02-26T10:42:42.170616+0000","last_became_peered":"2024-02-26T10:42:42.170616+0000","last_unstale":"2024-02-26T10:42:48.798497+0000","last_undegraded":"2024-02-26T10:42:48.798497+0000","last_fullsized":"2024-02-26T10:42:48.798497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:50.320859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518171+0000","last_change":"2024-02-26T10:42:42.168408+0000","last_active":"2024-02-26T10:42:48.518171+0000","last_peered":"2024-02-26T10:42:48.518171+0000","last_clean":"2024-02-26T10:42:48.518171+0000","last_became_active":"2024-02-26T10:42:42.167150+0000","last_became_peered":"2024-02-26T10:42:42.167150+0000","last_unstale":"2024-02-26T10:42:48.518171+0000","last_undegraded":"2024-02-26T10:42:48.518171+0000","last_fullsized":"2024-02-26T10:42:48.518171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:34.763331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212669+0000","last_change":"2024-02-26T10:42:42.171720+0000","last_active":"2024-02-26T10:42:48.212669+0000","last_peered":"2024-02-26T10:42:48.212669+0000","last_clean":"2024-02-26T10:42:48.212669+0000","last_became_active":"2024-02-26T10:42:42.171309+0000","last_became_peered":"2024-02-26T10:42:42.171309+0000","last_unstale":"2024-02-26T10:42:48.212669+0000","last_undegraded":"2024-02-26T10:42:48.212669+0000","last_fullsized":"2024-02-26T10:42:48.212669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:20:52.151553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799200+0000","last_change":"2024-02-26T10:42:42.176857+0000","last_active":"2024-02-26T10:42:48.799200+0000","last_peered":"2024-02-26T10:42:48.799200+0000","last_clean":"2024-02-26T10:42:48.799200+0000","last_became_active":"2024-02-26T10:42:42.176385+0000","last_became_peered":"2024-02-26T10:42:42.176385+0000","last_unstale":"2024-02-26T10:42:48.799200+0000","last_undegraded":"2024-02-26T10:42:48.799200+0000","last_fullsized":"2024-02-26T10:42:48.799200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:45:51.852212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801034+0000","last_change":"2024-02-26T10:42:42.174682+0000","last_active":"2024-02-26T10:42:48.801034+0000","last_peered":"2024-02-26T10:42:48.801034+0000","last_clean":"2024-02-26T10:42:48.801034+0000","last_became_active":"2024-02-26T10:42:42.174328+0000","last_became_peered":"2024-02-26T10:42:42.174328+0000","last_unstale":"2024-02-26T10:42:48.801034+0000","last_undegraded":"2024-02-26T10:42:48.801034+0000","last_fullsized":"2024-02-26T10:42:48.801034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:47.161020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798763+0000","last_change":"2024-02-26T10:42:42.185289+0000","last_active":"2024-02-26T10:42:48.798763+0000","last_peered":"2024-02-26T10:42:48.798763+0000","last_clean":"2024-02-26T10:42:48.798763+0000","last_became_active":"2024-02-26T10:42:42.185000+0000","last_became_peered":"2024-02-26T10:42:42.185000+0000","last_unstale":"2024-02-26T10:42:48.798763+0000","last_undegraded":"2024-02-26T10:42:48.798763+0000","last_fullsized":"2024-02-26T10:42:48.798763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:48.552584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798900+0000","last_change":"2024-02-26T10:42:42.176558+0000","last_active":"2024-02-26T10:42:48.798900+0000","last_peered":"2024-02-26T10:42:48.798900+0000","last_clean":"2024-02-26T10:42:48.798900+0000","last_became_active":"2024-02-26T10:42:42.176334+0000","last_became_peered":"2024-02-26T10:42:42.176334+0000","last_unstale":"2024-02-26T10:42:48.798900+0000","last_undegraded":"2024-02-26T10:42:48.798900+0000","last_fullsized":"2024-02-26T10:42:48.798900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:59.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212806+0000","last_change":"2024-02-26T10:42:42.168534+0000","last_active":"2024-02-26T10:42:48.212806+0000","last_peered":"2024-02-26T10:42:48.212806+0000","last_clean":"2024-02-26T10:42:48.212806+0000","last_became_active":"2024-02-26T10:42:42.168179+0000","last_became_peered":"2024-02-26T10:42:42.168179+0000","last_unstale":"2024-02-26T10:42:48.212806+0000","last_undegraded":"2024-02-26T10:42:48.212806+0000","last_fullsized":"2024-02-26T10:42:48.212806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:09.625969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209473+0000","last_change":"2024-02-26T10:42:42.192446+0000","last_active":"2024-02-26T10:42:48.209473+0000","last_peered":"2024-02-26T10:42:48.209473+0000","last_clean":"2024-02-26T10:42:48.209473+0000","last_became_active":"2024-02-26T10:42:42.191863+0000","last_became_peered":"2024-02-26T10:42:42.191863+0000","last_unstale":"2024-02-26T10:42:48.209473+0000","last_undegraded":"2024-02-26T10:42:48.209473+0000","last_fullsized":"2024-02-26T10:42:48.209473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:30:19.477529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209130+0000","last_change":"2024-02-26T10:42:47.257605+0000","last_active":"2024-02-26T10:42:48.209130+0000","last_peered":"2024-02-26T10:42:48.209130+0000","last_clean":"2024-02-26T10:42:48.209130+0000","last_became_active":"2024-02-26T10:42:47.257206+0000","last_became_peered":"2024-02-26T10:42:47.257206+0000","last_unstale":"2024-02-26T10:42:48.209130+0000","last_undegraded":"2024-02-26T10:42:48.209130+0000","last_fullsized":"2024-02-26T10:42:48.209130+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:26:17.681862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213163+0000","last_change":"2024-02-26T10:42:41.613087+0000","last_active":"2024-02-26T10:42:48.213163+0000","last_peered":"2024-02-26T10:42:48.213163+0000","last_clean":"2024-02-26T10:42:48.213163+0000","last_became_active":"2024-02-26T10:42:41.612664+0000","last_became_peered":"2024-02-26T10:42:41.612664+0000","last_unstale":"2024-02-26T10:42:48.213163+0000","last_undegraded":"2024-02-26T10:42:48.213163+0000","last_fullsized":"2024-02-26T10:42:48.213163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:00.352166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801675+0000","last_change":"2024-02-26T10:42:42.191438+0000","last_active":"2024-02-26T10:42:48.801675+0000","last_peered":"2024-02-26T10:42:48.801675+0000","last_clean":"2024-02-26T10:42:48.801675+0000","last_became_active":"2024-02-26T10:42:42.191095+0000","last_became_peered":"2024-02-26T10:42:42.191095+0000","last_unstale":"2024-02-26T10:42:48.801675+0000","last_undegraded":"2024-02-26T10:42:48.801675+0000","last_fullsized":"2024-02-26T10:42:48.801675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:20.918715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.517742+0000","last_change":"2024-02-26T10:42:47.247503+0000","last_active":"2024-02-26T10:42:48.517742+0000","last_peered":"2024-02-26T10:42:48.517742+0000","last_clean":"2024-02-26T10:42:48.517742+0000","last_became_active":"2024-02-26T10:42:47.247121+0000","last_became_peered":"2024-02-26T10:42:47.247121+0000","last_unstale":"2024-02-26T10:42:48.517742+0000","last_undegraded":"2024-02-26T10:42:48.517742+0000","last_fullsized":"2024-02-26T10:42:48.517742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:01:38.733131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.802175+0000","last_change":"2024-02-26T10:42:41.626227+0000","last_active":"2024-02-26T10:42:48.802175+0000","last_peered":"2024-02-26T10:42:48.802175+0000","last_clean":"2024-02-26T10:42:48.802175+0000","last_became_active":"2024-02-26T10:42:41.625760+0000","last_became_peered":"2024-02-26T10:42:41.625760+0000","last_unstale":"2024-02-26T10:42:48.802175+0000","last_undegraded":"2024-02-26T10:42:48.802175+0000","last_fullsized":"2024-02-26T10:42:48.802175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:04.600151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800269+0000","last_change":"2024-02-26T10:42:42.182808+0000","last_active":"2024-02-26T10:42:48.800269+0000","last_peered":"2024-02-26T10:42:48.800269+0000","last_clean":"2024-02-26T10:42:48.800269+0000","last_became_active":"2024-02-26T10:42:42.182485+0000","last_became_peered":"2024-02-26T10:42:42.182485+0000","last_unstale":"2024-02-26T10:42:48.800269+0000","last_undegraded":"2024-02-26T10:42:48.800269+0000","last_fullsized":"2024-02-26T10:42:48.800269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:45.205969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.517885+0000","last_change":"2024-02-26T10:42:47.250694+0000","last_active":"2024-02-26T10:42:48.517885+0000","last_peered":"2024-02-26T10:42:48.517885+0000","last_clean":"2024-02-26T10:42:48.517885+0000","last_became_active":"2024-02-26T10:42:47.249566+0000","last_became_peered":"2024-02-26T10:42:47.249566+0000","last_unstale":"2024-02-26T10:42:48.517885+0000","last_undegraded":"2024-02-26T10:42:48.517885+0000","last_fullsized":"2024-02-26T10:42:48.517885+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:44.016413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798585+0000","last_change":"2024-02-26T10:42:41.606849+0000","last_active":"2024-02-26T10:42:48.798585+0000","last_peered":"2024-02-26T10:42:48.798585+0000","last_clean":"2024-02-26T10:42:48.798585+0000","last_became_active":"2024-02-26T10:42:41.606494+0000","last_became_peered":"2024-02-26T10:42:41.606494+0000","last_unstale":"2024-02-26T10:42:48.798585+0000","last_undegraded":"2024-02-26T10:42:48.798585+0000","last_fullsized":"2024-02-26T10:42:48.798585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:36.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799191+0000","last_change":"2024-02-26T10:42:42.175926+0000","last_active":"2024-02-26T10:42:48.799191+0000","last_peered":"2024-02-26T10:42:48.799191+0000","last_clean":"2024-02-26T10:42:48.799191+0000","last_became_active":"2024-02-26T10:42:42.175614+0000","last_became_peered":"2024-02-26T10:42:42.175614+0000","last_unstale":"2024-02-26T10:42:48.799191+0000","last_undegraded":"2024-02-26T10:42:48.799191+0000","last_fullsized":"2024-02-26T10:42:48.799191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:20:20.829849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.517723+0000","last_change":"2024-02-26T10:42:47.227215+0000","last_active":"2024-02-26T10:42:48.517723+0000","last_peered":"2024-02-26T10:42:48.517723+0000","last_clean":"2024-02-26T10:42:48.517723+0000","last_became_active":"2024-02-26T10:42:47.226771+0000","last_became_peered":"2024-02-26T10:42:47.226771+0000","last_unstale":"2024-02-26T10:42:48.517723+0000","last_undegraded":"2024-02-26T10:42:48.517723+0000","last_fullsized":"2024-02-26T10:42:48.517723+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:12:58.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797601+0000","last_change":"2024-02-26T10:42:41.606407+0000","last_active":"2024-02-26T10:42:48.797601+0000","last_peered":"2024-02-26T10:42:48.797601+0000","last_clean":"2024-02-26T10:42:48.797601+0000","last_became_active":"2024-02-26T10:42:41.606094+0000","last_became_peered":"2024-02-26T10:42:41.606094+0000","last_unstale":"2024-02-26T10:42:48.797601+0000","last_undegraded":"2024-02-26T10:42:48.797601+0000","last_fullsized":"2024-02-26T10:42:48.797601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:53.673017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798116+0000","last_change":"2024-02-26T10:42:42.172967+0000","last_active":"2024-02-26T10:42:48.798116+0000","last_peered":"2024-02-26T10:42:48.798116+0000","last_clean":"2024-02-26T10:42:48.798116+0000","last_became_active":"2024-02-26T10:42:42.172675+0000","last_became_peered":"2024-02-26T10:42:42.172675+0000","last_unstale":"2024-02-26T10:42:48.798116+0000","last_undegraded":"2024-02-26T10:42:48.798116+0000","last_fullsized":"2024-02-26T10:42:48.798116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209528+0000","last_change":"2024-02-26T10:42:47.229375+0000","last_active":"2024-02-26T10:42:48.209528+0000","last_peered":"2024-02-26T10:42:48.209528+0000","last_clean":"2024-02-26T10:42:48.209528+0000","last_became_active":"2024-02-26T10:42:47.228874+0000","last_became_peered":"2024-02-26T10:42:47.228874+0000","last_unstale":"2024-02-26T10:42:48.209528+0000","last_undegraded":"2024-02-26T10:42:48.209528+0000","last_fullsized":"2024-02-26T10:42:48.209528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:02.188519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:16.041683+0000","last_change":"2024-02-26T10:42:41.602783+0000","last_active":"2024-02-26T10:43:16.041683+0000","last_peered":"2024-02-26T10:43:16.041683+0000","last_clean":"2024-02-26T10:43:16.041683+0000","last_became_active":"2024-02-26T10:42:41.602491+0000","last_became_peered":"2024-02-26T10:42:41.602491+0000","last_unstale":"2024-02-26T10:43:16.041683+0000","last_undegraded":"2024-02-26T10:43:16.041683+0000","last_fullsized":"2024-02-26T10:43:16.041683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:23:33.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797452+0000","last_change":"2024-02-26T10:42:42.170018+0000","last_active":"2024-02-26T10:42:48.797452+0000","last_peered":"2024-02-26T10:42:48.797452+0000","last_clean":"2024-02-26T10:42:48.797452+0000","last_became_active":"2024-02-26T10:42:42.169596+0000","last_became_peered":"2024-02-26T10:42:42.169596+0000","last_unstale":"2024-02-26T10:42:48.797452+0000","last_undegraded":"2024-02-26T10:42:48.797452+0000","last_fullsized":"2024-02-26T10:42:48.797452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:20:43.275319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209462+0000","last_change":"2024-02-26T10:42:47.250104+0000","last_active":"2024-02-26T10:42:48.209462+0000","last_peered":"2024-02-26T10:42:48.209462+0000","last_clean":"2024-02-26T10:42:48.209462+0000","last_became_active":"2024-02-26T10:42:47.249560+0000","last_became_peered":"2024-02-26T10:42:47.249560+0000","last_unstale":"2024-02-26T10:42:48.209462+0000","last_undegraded":"2024-02-26T10:42:48.209462+0000","last_fullsized":"2024-02-26T10:42:48.209462+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:11:12.056434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:15.359888+0000","last_change":"2024-02-26T10:42:41.602887+0000","last_active":"2024-02-26T10:43:15.359888+0000","last_peered":"2024-02-26T10:43:15.359888+0000","last_clean":"2024-02-26T10:43:15.359888+0000","last_became_active":"2024-02-26T10:42:41.602564+0000","last_became_peered":"2024-02-26T10:42:41.602564+0000","last_unstale":"2024-02-26T10:43:15.359888+0000","last_undegraded":"2024-02-26T10:43:15.359888+0000","last_fullsized":"2024-02-26T10:43:15.359888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:22:41.711283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798636+0000","last_change":"2024-02-26T10:42:42.192700+0000","last_active":"2024-02-26T10:42:48.798636+0000","last_peered":"2024-02-26T10:42:48.798636+0000","last_clean":"2024-02-26T10:42:48.798636+0000","last_became_active":"2024-02-26T10:42:42.192387+0000","last_became_peered":"2024-02-26T10:42:42.192387+0000","last_unstale":"2024-02-26T10:42:48.798636+0000","last_undegraded":"2024-02-26T10:42:48.798636+0000","last_fullsized":"2024-02-26T10:42:48.798636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:21:15.104872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798996+0000","last_change":"2024-02-26T10:42:47.247000+0000","last_active":"2024-02-26T10:42:48.798996+0000","last_peered":"2024-02-26T10:42:48.798996+0000","last_clean":"2024-02-26T10:42:48.798996+0000","last_became_active":"2024-02-26T10:42:47.246520+0000","last_became_peered":"2024-02-26T10:42:47.246520+0000","last_unstale":"2024-02-26T10:42:48.798996+0000","last_undegraded":"2024-02-26T10:42:48.798996+0000","last_fullsized":"2024-02-26T10:42:48.798996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:29.977858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:15.359585+0000","last_change":"2024-02-26T10:42:41.609460+0000","last_active":"2024-02-26T10:43:15.359585+0000","last_peered":"2024-02-26T10:43:15.359585+0000","last_clean":"2024-02-26T10:43:15.359585+0000","last_became_active":"2024-02-26T10:42:41.608981+0000","last_became_peered":"2024-02-26T10:42:41.608981+0000","last_unstale":"2024-02-26T10:43:15.359585+0000","last_undegraded":"2024-02-26T10:43:15.359585+0000","last_fullsized":"2024-02-26T10:43:15.359585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:04.599910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799489+0000","last_change":"2024-02-26T10:42:42.185359+0000","last_active":"2024-02-26T10:42:48.799489+0000","last_peered":"2024-02-26T10:42:48.799489+0000","last_clean":"2024-02-26T10:42:48.799489+0000","last_became_active":"2024-02-26T10:42:42.184221+0000","last_became_peered":"2024-02-26T10:42:42.184221+0000","last_unstale":"2024-02-26T10:42:48.799489+0000","last_undegraded":"2024-02-26T10:42:48.799489+0000","last_fullsized":"2024-02-26T10:42:48.799489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:53.907290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800252+0000","last_change":"2024-02-26T10:42:47.241700+0000","last_active":"2024-02-26T10:42:48.800252+0000","last_peered":"2024-02-26T10:42:48.800252+0000","last_clean":"2024-02-26T10:42:48.800252+0000","last_became_active":"2024-02-26T10:42:47.240667+0000","last_became_peered":"2024-02-26T10:42:47.240667+0000","last_unstale":"2024-02-26T10:42:48.800252+0000","last_undegraded":"2024-02-26T10:42:48.800252+0000","last_fullsized":"2024-02-26T10:42:48.800252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:19.071244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797703+0000","last_change":"2024-02-26T10:42:41.604519+0000","last_active":"2024-02-26T10:42:48.797703+0000","last_peered":"2024-02-26T10:42:48.797703+0000","last_clean":"2024-02-26T10:42:48.797703+0000","last_became_active":"2024-02-26T10:42:41.603596+0000","last_became_peered":"2024-02-26T10:42:41.603596+0000","last_unstale":"2024-02-26T10:42:48.797703+0000","last_undegraded":"2024-02-26T10:42:48.797703+0000","last_fullsized":"2024-02-26T10:42:48.797703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:39.764730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.210647+0000","last_change":"2024-02-26T10:42:42.187532+0000","last_active":"2024-02-26T10:42:48.210647+0000","last_peered":"2024-02-26T10:42:48.210647+0000","last_clean":"2024-02-26T10:42:48.210647+0000","last_became_active":"2024-02-26T10:42:42.185635+0000","last_became_peered":"2024-02-26T10:42:42.185635+0000","last_unstale":"2024-02-26T10:42:48.210647+0000","last_undegraded":"2024-02-26T10:42:48.210647+0000","last_fullsized":"2024-02-26T10:42:48.210647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:16.894747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801774+0000","last_change":"2024-02-26T10:42:47.233507+0000","last_active":"2024-02-26T10:42:48.801774+0000","last_peered":"2024-02-26T10:42:48.801774+0000","last_clean":"2024-02-26T10:42:48.801774+0000","last_became_active":"2024-02-26T10:42:47.229589+0000","last_became_peered":"2024-02-26T10:42:47.229589+0000","last_unstale":"2024-02-26T10:42:48.801774+0000","last_undegraded":"2024-02-26T10:42:48.801774+0000","last_fullsized":"2024-02-26T10:42:48.801774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:02:42.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:17.042679+0000","last_change":"2024-02-26T10:42:41.613952+0000","last_active":"2024-02-26T10:43:17.042679+0000","last_peered":"2024-02-26T10:43:17.042679+0000","last_clean":"2024-02-26T10:43:17.042679+0000","last_became_active":"2024-02-26T10:42:41.613656+0000","last_became_peered":"2024-02-26T10:42:41.613656+0000","last_unstale":"2024-02-26T10:43:17.042679+0000","last_undegraded":"2024-02-26T10:43:17.042679+0000","last_fullsized":"2024-02-26T10:43:17.042679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:14.090821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799493+0000","last_change":"2024-02-26T10:42:42.178357+0000","last_active":"2024-02-26T10:42:48.799493+0000","last_peered":"2024-02-26T10:42:48.799493+0000","last_clean":"2024-02-26T10:42:48.799493+0000","last_became_active":"2024-02-26T10:42:42.178051+0000","last_became_peered":"2024-02-26T10:42:42.178051+0000","last_unstale":"2024-02-26T10:42:48.799493+0000","last_undegraded":"2024-02-26T10:42:48.799493+0000","last_fullsized":"2024-02-26T10:42:48.799493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:13:01.326122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.210857+0000","last_change":"2024-02-26T10:42:47.246231+0000","last_active":"2024-02-26T10:42:48.210857+0000","last_peered":"2024-02-26T10:42:48.210857+0000","last_clean":"2024-02-26T10:42:48.210857+0000","last_became_active":"2024-02-26T10:42:47.245739+0000","last_became_peered":"2024-02-26T10:42:47.245739+0000","last_unstale":"2024-02-26T10:42:48.210857+0000","last_undegraded":"2024-02-26T10:42:48.210857+0000","last_fullsized":"2024-02-26T10:42:48.210857+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:30.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.339654+0000","last_change":"2024-02-26T10:42:41.603126+0000","last_active":"2024-02-26T10:43:14.339654+0000","last_peered":"2024-02-26T10:43:14.339654+0000","last_clean":"2024-02-26T10:43:14.339654+0000","last_became_active":"2024-02-26T10:42:41.602790+0000","last_became_peered":"2024-02-26T10:42:41.602790+0000","last_unstale":"2024-02-26T10:43:14.339654+0000","last_undegraded":"2024-02-26T10:43:14.339654+0000","last_fullsized":"2024-02-26T10:43:14.339654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:31:58.908209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800535+0000","last_change":"2024-02-26T10:42:42.185034+0000","last_active":"2024-02-26T10:42:48.800535+0000","last_peered":"2024-02-26T10:42:48.800535+0000","last_clean":"2024-02-26T10:42:48.800535+0000","last_became_active":"2024-02-26T10:42:42.181894+0000","last_became_peered":"2024-02-26T10:42:42.181894+0000","last_unstale":"2024-02-26T10:42:48.800535+0000","last_undegraded":"2024-02-26T10:42:48.800535+0000","last_fullsized":"2024-02-26T10:42:48.800535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:22.885475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518247+0000","last_change":"2024-02-26T10:42:47.250075+0000","last_active":"2024-02-26T10:42:48.518247+0000","last_peered":"2024-02-26T10:42:48.518247+0000","last_clean":"2024-02-26T10:42:48.518247+0000","last_became_active":"2024-02-26T10:42:47.249490+0000","last_became_peered":"2024-02-26T10:42:47.249490+0000","last_unstale":"2024-02-26T10:42:48.518247+0000","last_undegraded":"2024-02-26T10:42:48.518247+0000","last_fullsized":"2024-02-26T10:42:48.518247+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:42.659282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.342063+0000","last_change":"2024-02-26T10:42:41.615314+0000","last_active":"2024-02-26T10:43:14.342063+0000","last_peered":"2024-02-26T10:43:14.342063+0000","last_clean":"2024-02-26T10:43:14.342063+0000","last_became_active":"2024-02-26T10:42:41.615091+0000","last_became_peered":"2024-02-26T10:42:41.615091+0000","last_unstale":"2024-02-26T10:43:14.342063+0000","last_undegraded":"2024-02-26T10:43:14.342063+0000","last_fullsized":"2024-02-26T10:43:14.342063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:41:52.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799389+0000","last_change":"2024-02-26T10:42:42.176609+0000","last_active":"2024-02-26T10:42:48.799389+0000","last_peered":"2024-02-26T10:42:48.799389+0000","last_clean":"2024-02-26T10:42:48.799389+0000","last_became_active":"2024-02-26T10:42:42.176311+0000","last_became_peered":"2024-02-26T10:42:42.176311+0000","last_unstale":"2024-02-26T10:42:48.799389+0000","last_undegraded":"2024-02-26T10:42:48.799389+0000","last_fullsized":"2024-02-26T10:42:48.799389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:25:38.415311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.211107+0000","last_change":"2024-02-26T10:42:47.250255+0000","last_active":"2024-02-26T10:42:48.211107+0000","last_peered":"2024-02-26T10:42:48.211107+0000","last_clean":"2024-02-26T10:42:48.211107+0000","last_became_active":"2024-02-26T10:42:47.249465+0000","last_became_peered":"2024-02-26T10:42:47.249465+0000","last_unstale":"2024-02-26T10:42:48.211107+0000","last_undegraded":"2024-02-26T10:42:48.211107+0000","last_fullsized":"2024-02-26T10:42:48.211107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:23.173863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'23","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:18.132992+0000","last_change":"2024-02-26T10:42:41.602049+0000","last_active":"2024-02-26T10:43:18.132992+0000","last_peered":"2024-02-26T10:43:18.132992+0000","last_clean":"2024-02-26T10:43:18.132992+0000","last_became_active":"2024-02-26T10:42:41.601720+0000","last_became_peered":"2024-02-26T10:42:41.601720+0000","last_unstale":"2024-02-26T10:43:18.132992+0000","last_undegraded":"2024-02-26T10:43:18.132992+0000","last_fullsized":"2024-02-26T10:43:18.132992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:31.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37807,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":28,"num_write":24,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798370+0000","last_change":"2024-02-26T10:42:42.168174+0000","last_active":"2024-02-26T10:42:48.798370+0000","last_peered":"2024-02-26T10:42:48.798370+0000","last_clean":"2024-02-26T10:42:48.798370+0000","last_became_active":"2024-02-26T10:42:42.167840+0000","last_became_peered":"2024-02-26T10:42:42.167840+0000","last_unstale":"2024-02-26T10:42:48.798370+0000","last_undegraded":"2024-02-26T10:42:48.798370+0000","last_fullsized":"2024-02-26T10:42:48.798370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:43.079139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798497+0000","last_change":"2024-02-26T10:42:47.230048+0000","last_active":"2024-02-26T10:42:48.798497+0000","last_peered":"2024-02-26T10:42:48.798497+0000","last_clean":"2024-02-26T10:42:48.798497+0000","last_became_active":"2024-02-26T10:42:47.229396+0000","last_became_peered":"2024-02-26T10:42:47.229396+0000","last_unstale":"2024-02-26T10:42:48.798497+0000","last_undegraded":"2024-02-26T10:42:48.798497+0000","last_fullsized":"2024-02-26T10:42:48.798497+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:57.804297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:15.355234+0000","last_change":"2024-02-26T10:42:41.614413+0000","last_active":"2024-02-26T10:43:15.355234+0000","last_peered":"2024-02-26T10:43:15.355234+0000","last_clean":"2024-02-26T10:43:15.355234+0000","last_became_active":"2024-02-26T10:42:41.614065+0000","last_became_peered":"2024-02-26T10:42:41.614065+0000","last_unstale":"2024-02-26T10:43:15.355234+0000","last_undegraded":"2024-02-26T10:43:15.355234+0000","last_fullsized":"2024-02-26T10:43:15.355234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:50:20.552340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212358+0000","last_change":"2024-02-26T10:42:42.188140+0000","last_active":"2024-02-26T10:42:48.212358+0000","last_peered":"2024-02-26T10:42:48.212358+0000","last_clean":"2024-02-26T10:42:48.212358+0000","last_became_active":"2024-02-26T10:42:42.187055+0000","last_became_peered":"2024-02-26T10:42:42.187055+0000","last_unstale":"2024-02-26T10:42:48.212358+0000","last_undegraded":"2024-02-26T10:42:48.212358+0000","last_fullsized":"2024-02-26T10:42:48.212358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:49:43.568562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798410+0000","last_change":"2024-02-26T10:42:47.251998+0000","last_active":"2024-02-26T10:42:48.798410+0000","last_peered":"2024-02-26T10:42:48.798410+0000","last_clean":"2024-02-26T10:42:48.798410+0000","last_became_active":"2024-02-26T10:42:47.251574+0000","last_became_peered":"2024-02-26T10:42:47.251574+0000","last_unstale":"2024-02-26T10:42:48.798410+0000","last_undegraded":"2024-02-26T10:42:48.798410+0000","last_fullsized":"2024-02-26T10:42:48.798410+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:07.122542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.343549+0000","last_change":"2024-02-26T10:42:41.612337+0000","last_active":"2024-02-26T10:43:14.343549+0000","last_peered":"2024-02-26T10:43:14.343549+0000","last_clean":"2024-02-26T10:43:14.343549+0000","last_became_active":"2024-02-26T10:42:41.611920+0000","last_became_peered":"2024-02-26T10:42:41.611920+0000","last_unstale":"2024-02-26T10:43:14.343549+0000","last_undegraded":"2024-02-26T10:43:14.343549+0000","last_fullsized":"2024-02-26T10:43:14.343549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:29:13.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798055+0000","last_change":"2024-02-26T10:42:42.171497+0000","last_active":"2024-02-26T10:42:48.798055+0000","last_peered":"2024-02-26T10:42:48.798055+0000","last_clean":"2024-02-26T10:42:48.798055+0000","last_became_active":"2024-02-26T10:42:42.171176+0000","last_became_peered":"2024-02-26T10:42:42.171176+0000","last_unstale":"2024-02-26T10:42:48.798055+0000","last_undegraded":"2024-02-26T10:42:48.798055+0000","last_fullsized":"2024-02-26T10:42:48.798055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:05:41.193025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212442+0000","last_change":"2024-02-26T10:42:47.254585+0000","last_active":"2024-02-26T10:42:48.212442+0000","last_peered":"2024-02-26T10:42:48.212442+0000","last_clean":"2024-02-26T10:42:48.212442+0000","last_became_active":"2024-02-26T10:42:47.253870+0000","last_became_peered":"2024-02-26T10:42:47.253870+0000","last_unstale":"2024-02-26T10:42:48.212442+0000","last_undegraded":"2024-02-26T10:42:48.212442+0000","last_fullsized":"2024-02-26T10:42:48.212442+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:49.276906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:17.044948+0000","last_change":"2024-02-26T10:42:41.600901+0000","last_active":"2024-02-26T10:43:17.044948+0000","last_peered":"2024-02-26T10:43:17.044948+0000","last_clean":"2024-02-26T10:43:17.044948+0000","last_became_active":"2024-02-26T10:42:41.600018+0000","last_became_peered":"2024-02-26T10:42:41.600018+0000","last_unstale":"2024-02-26T10:43:17.044948+0000","last_undegraded":"2024-02-26T10:43:17.044948+0000","last_fullsized":"2024-02-26T10:43:17.044948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:07.876694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798237+0000","last_change":"2024-02-26T10:42:42.168702+0000","last_active":"2024-02-26T10:42:48.798237+0000","last_peered":"2024-02-26T10:42:48.798237+0000","last_clean":"2024-02-26T10:42:48.798237+0000","last_became_active":"2024-02-26T10:42:42.168420+0000","last_became_peered":"2024-02-26T10:42:42.168420+0000","last_unstale":"2024-02-26T10:42:48.798237+0000","last_undegraded":"2024-02-26T10:42:48.798237+0000","last_fullsized":"2024-02-26T10:42:48.798237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:35.111699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213885+0000","last_change":"2024-02-26T10:42:47.250714+0000","last_active":"2024-02-26T10:42:48.213885+0000","last_peered":"2024-02-26T10:42:48.213885+0000","last_clean":"2024-02-26T10:42:48.213885+0000","last_became_active":"2024-02-26T10:42:47.249506+0000","last_became_peered":"2024-02-26T10:42:47.249506+0000","last_unstale":"2024-02-26T10:42:48.213885+0000","last_undegraded":"2024-02-26T10:42:48.213885+0000","last_fullsized":"2024-02-26T10:42:48.213885+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:05.058052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.339670+0000","last_change":"2024-02-26T10:42:41.620841+0000","last_active":"2024-02-26T10:43:14.339670+0000","last_peered":"2024-02-26T10:43:14.339670+0000","last_clean":"2024-02-26T10:43:14.339670+0000","last_became_active":"2024-02-26T10:42:41.620530+0000","last_became_peered":"2024-02-26T10:42:41.620530+0000","last_unstale":"2024-02-26T10:43:14.339670+0000","last_undegraded":"2024-02-26T10:43:14.339670+0000","last_fullsized":"2024-02-26T10:43:14.339670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:51:15.851893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797959+0000","last_change":"2024-02-26T10:42:42.172845+0000","last_active":"2024-02-26T10:42:48.797959+0000","last_peered":"2024-02-26T10:42:48.797959+0000","last_clean":"2024-02-26T10:42:48.797959+0000","last_became_active":"2024-02-26T10:42:42.171263+0000","last_became_peered":"2024-02-26T10:42:42.171263+0000","last_unstale":"2024-02-26T10:42:48.797959+0000","last_undegraded":"2024-02-26T10:42:48.797959+0000","last_fullsized":"2024-02-26T10:42:48.797959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:21.439474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797847+0000","last_change":"2024-02-26T10:42:47.236434+0000","last_active":"2024-02-26T10:42:48.797847+0000","last_peered":"2024-02-26T10:42:48.797847+0000","last_clean":"2024-02-26T10:42:48.797847+0000","last_became_active":"2024-02-26T10:42:47.236040+0000","last_became_peered":"2024-02-26T10:42:47.236040+0000","last_unstale":"2024-02-26T10:42:48.797847+0000","last_undegraded":"2024-02-26T10:42:48.797847+0000","last_fullsized":"2024-02-26T10:42:48.797847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:20:23.230348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798303+0000","last_change":"2024-02-26T10:42:41.604208+0000","last_active":"2024-02-26T10:42:48.798303+0000","last_peered":"2024-02-26T10:42:48.798303+0000","last_clean":"2024-02-26T10:42:48.798303+0000","last_became_active":"2024-02-26T10:42:41.603894+0000","last_became_peered":"2024-02-26T10:42:41.603894+0000","last_unstale":"2024-02-26T10:42:48.798303+0000","last_undegraded":"2024-02-26T10:42:48.798303+0000","last_fullsized":"2024-02-26T10:42:48.798303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:57:13.775868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798751+0000","last_change":"2024-02-26T10:42:42.169395+0000","last_active":"2024-02-26T10:42:48.798751+0000","last_peered":"2024-02-26T10:42:48.798751+0000","last_clean":"2024-02-26T10:42:48.798751+0000","last_became_active":"2024-02-26T10:42:42.168686+0000","last_became_peered":"2024-02-26T10:42:42.168686+0000","last_unstale":"2024-02-26T10:42:48.798751+0000","last_undegraded":"2024-02-26T10:42:48.798751+0000","last_fullsized":"2024-02-26T10:42:48.798751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:21:35.085188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797583+0000","last_change":"2024-02-26T10:42:47.252119+0000","last_active":"2024-02-26T10:42:48.797583+0000","last_peered":"2024-02-26T10:42:48.797583+0000","last_clean":"2024-02-26T10:42:48.797583+0000","last_became_active":"2024-02-26T10:42:47.251723+0000","last_became_peered":"2024-02-26T10:42:47.251723+0000","last_unstale":"2024-02-26T10:42:48.797583+0000","last_undegraded":"2024-02-26T10:42:48.797583+0000","last_fullsized":"2024-02-26T10:42:48.797583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:17.130806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.210414+0000","last_change":"2024-02-26T10:42:41.621393+0000","last_active":"2024-02-26T10:42:48.210414+0000","last_peered":"2024-02-26T10:42:48.210414+0000","last_clean":"2024-02-26T10:42:48.210414+0000","last_became_active":"2024-02-26T10:42:41.621008+0000","last_became_peered":"2024-02-26T10:42:41.621008+0000","last_unstale":"2024-02-26T10:42:48.210414+0000","last_undegraded":"2024-02-26T10:42:48.210414+0000","last_fullsized":"2024-02-26T10:42:48.210414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:46.979854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797989+0000","last_change":"2024-02-26T10:42:42.194296+0000","last_active":"2024-02-26T10:42:48.797989+0000","last_peered":"2024-02-26T10:42:48.797989+0000","last_clean":"2024-02-26T10:42:48.797989+0000","last_became_active":"2024-02-26T10:42:42.193987+0000","last_became_peered":"2024-02-26T10:42:42.193987+0000","last_unstale":"2024-02-26T10:42:48.797989+0000","last_undegraded":"2024-02-26T10:42:48.797989+0000","last_fullsized":"2024-02-26T10:42:48.797989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:35:52.677918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213098+0000","last_change":"2024-02-26T10:42:47.238520+0000","last_active":"2024-02-26T10:42:48.213098+0000","last_peered":"2024-02-26T10:42:48.213098+0000","last_clean":"2024-02-26T10:42:48.213098+0000","last_became_active":"2024-02-26T10:42:47.237213+0000","last_became_peered":"2024-02-26T10:42:47.237213+0000","last_unstale":"2024-02-26T10:42:48.213098+0000","last_undegraded":"2024-02-26T10:42:48.213098+0000","last_fullsized":"2024-02-26T10:42:48.213098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:29.283850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798040+0000","last_change":"2024-02-26T10:42:41.609479+0000","last_active":"2024-02-26T10:42:48.798040+0000","last_peered":"2024-02-26T10:42:48.798040+0000","last_clean":"2024-02-26T10:42:48.798040+0000","last_became_active":"2024-02-26T10:42:41.609084+0000","last_became_peered":"2024-02-26T10:42:41.609084+0000","last_unstale":"2024-02-26T10:42:48.798040+0000","last_undegraded":"2024-02-26T10:42:48.798040+0000","last_fullsized":"2024-02-26T10:42:48.798040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:32.412507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797661+0000","last_change":"2024-02-26T10:42:42.174985+0000","last_active":"2024-02-26T10:42:48.797661+0000","last_peered":"2024-02-26T10:42:48.797661+0000","last_clean":"2024-02-26T10:42:48.797661+0000","last_became_active":"2024-02-26T10:42:42.174694+0000","last_became_peered":"2024-02-26T10:42:42.174694+0000","last_unstale":"2024-02-26T10:42:48.797661+0000","last_undegraded":"2024-02-26T10:42:48.797661+0000","last_fullsized":"2024-02-26T10:42:48.797661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:10.000154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213503+0000","last_change":"2024-02-26T10:42:47.242587+0000","last_active":"2024-02-26T10:42:48.213503+0000","last_peered":"2024-02-26T10:42:48.213503+0000","last_clean":"2024-02-26T10:42:48.213503+0000","last_became_active":"2024-02-26T10:42:47.242099+0000","last_became_peered":"2024-02-26T10:42:47.242099+0000","last_unstale":"2024-02-26T10:42:48.213503+0000","last_undegraded":"2024-02-26T10:42:48.213503+0000","last_fullsized":"2024-02-26T10:42:48.213503+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:38:42.852080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797910+0000","last_change":"2024-02-26T10:42:42.170613+0000","last_active":"2024-02-26T10:42:48.797910+0000","last_peered":"2024-02-26T10:42:48.797910+0000","last_clean":"2024-02-26T10:42:48.797910+0000","last_became_active":"2024-02-26T10:42:42.170317+0000","last_became_peered":"2024-02-26T10:42:42.170317+0000","last_unstale":"2024-02-26T10:42:48.797910+0000","last_undegraded":"2024-02-26T10:42:48.797910+0000","last_fullsized":"2024-02-26T10:42:48.797910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:29:44.276718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:17.044347+0000","last_change":"2024-02-26T10:42:41.617447+0000","last_active":"2024-02-26T10:43:17.044347+0000","last_peered":"2024-02-26T10:43:17.044347+0000","last_clean":"2024-02-26T10:43:17.044347+0000","last_became_active":"2024-02-26T10:42:41.617107+0000","last_became_peered":"2024-02-26T10:42:41.617107+0000","last_unstale":"2024-02-26T10:43:17.044347+0000","last_undegraded":"2024-02-26T10:43:17.044347+0000","last_fullsized":"2024-02-26T10:43:17.044347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:06.052364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798268+0000","last_change":"2024-02-26T10:42:47.251971+0000","last_active":"2024-02-26T10:42:48.798268+0000","last_peered":"2024-02-26T10:42:48.798268+0000","last_clean":"2024-02-26T10:42:48.798268+0000","last_became_active":"2024-02-26T10:42:47.251506+0000","last_became_peered":"2024-02-26T10:42:47.251506+0000","last_unstale":"2024-02-26T10:42:48.798268+0000","last_undegraded":"2024-02-26T10:42:48.798268+0000","last_fullsized":"2024-02-26T10:42:48.798268+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:37.029348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800528+0000","last_change":"2024-02-26T10:42:42.194414+0000","last_active":"2024-02-26T10:42:48.800528+0000","last_peered":"2024-02-26T10:42:48.800528+0000","last_clean":"2024-02-26T10:42:48.800528+0000","last_became_active":"2024-02-26T10:42:42.193959+0000","last_became_peered":"2024-02-26T10:42:42.193959+0000","last_unstale":"2024-02-26T10:42:48.800528+0000","last_undegraded":"2024-02-26T10:42:48.800528+0000","last_fullsized":"2024-02-26T10:42:48.800528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:50.398674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:16.038406+0000","last_change":"2024-02-26T10:42:41.607947+0000","last_active":"2024-02-26T10:43:16.038406+0000","last_peered":"2024-02-26T10:43:16.038406+0000","last_clean":"2024-02-26T10:43:16.038406+0000","last_became_active":"2024-02-26T10:42:41.607522+0000","last_became_peered":"2024-02-26T10:42:41.607522+0000","last_unstale":"2024-02-26T10:43:16.038406+0000","last_undegraded":"2024-02-26T10:43:16.038406+0000","last_fullsized":"2024-02-26T10:43:16.038406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:32:36.647621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798097+0000","last_change":"2024-02-26T10:41:21.177202+0000","last_active":"2024-02-26T10:42:48.798097+0000","last_peered":"2024-02-26T10:42:48.798097+0000","last_clean":"2024-02-26T10:42:48.798097+0000","last_became_active":"2024-02-26T10:41:21.176762+0000","last_became_peered":"2024-02-26T10:41:21.176762+0000","last_unstale":"2024-02-26T10:42:48.798097+0000","last_undegraded":"2024-02-26T10:42:48.798097+0000","last_fullsized":"2024-02-26T10:42:48.798097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_clean_scrub_stamp":"2024-02-26T10:37:25.424790+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:19.619104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798723+0000","last_change":"2024-02-26T10:42:47.244346+0000","last_active":"2024-02-26T10:42:48.798723+0000","last_peered":"2024-02-26T10:42:48.798723+0000","last_clean":"2024-02-26T10:42:48.798723+0000","last_became_active":"2024-02-26T10:42:47.243872+0000","last_became_peered":"2024-02-26T10:42:47.243872+0000","last_unstale":"2024-02-26T10:42:48.798723+0000","last_undegraded":"2024-02-26T10:42:48.798723+0000","last_fullsized":"2024-02-26T10:42:48.798723+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:36.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.211674+0000","last_change":"2024-02-26T10:42:42.166266+0000","last_active":"2024-02-26T10:42:48.211674+0000","last_peered":"2024-02-26T10:42:48.211674+0000","last_clean":"2024-02-26T10:42:48.211674+0000","last_became_active":"2024-02-26T10:42:42.165721+0000","last_became_peered":"2024-02-26T10:42:42.165721+0000","last_unstale":"2024-02-26T10:42:48.211674+0000","last_undegraded":"2024-02-26T10:42:48.211674+0000","last_fullsized":"2024-02-26T10:42:48.211674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:32.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798192+0000","last_change":"2024-02-26T10:42:41.606268+0000","last_active":"2024-02-26T10:42:48.798192+0000","last_peered":"2024-02-26T10:42:48.798192+0000","last_clean":"2024-02-26T10:42:48.798192+0000","last_became_active":"2024-02-26T10:42:41.605950+0000","last_became_peered":"2024-02-26T10:42:41.605950+0000","last_unstale":"2024-02-26T10:42:48.798192+0000","last_undegraded":"2024-02-26T10:42:48.798192+0000","last_fullsized":"2024-02-26T10:42:48.798192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:40:03.616987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798718+0000","last_change":"2024-02-26T10:42:47.233203+0000","last_active":"2024-02-26T10:42:48.798718+0000","last_peered":"2024-02-26T10:42:48.798718+0000","last_clean":"2024-02-26T10:42:48.798718+0000","last_became_active":"2024-02-26T10:42:47.232705+0000","last_became_peered":"2024-02-26T10:42:47.232705+0000","last_unstale":"2024-02-26T10:42:48.798718+0000","last_undegraded":"2024-02-26T10:42:48.798718+0000","last_fullsized":"2024-02-26T10:42:48.798718+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:53.818901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798521+0000","last_change":"2024-02-26T10:42:41.623086+0000","last_active":"2024-02-26T10:42:48.798521+0000","last_peered":"2024-02-26T10:42:48.798521+0000","last_clean":"2024-02-26T10:42:48.798521+0000","last_became_active":"2024-02-26T10:42:41.622813+0000","last_became_peered":"2024-02-26T10:42:41.622813+0000","last_unstale":"2024-02-26T10:42:48.798521+0000","last_undegraded":"2024-02-26T10:42:48.798521+0000","last_fullsized":"2024-02-26T10:42:48.798521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:49:41.928829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213451+0000","last_change":"2024-02-26T10:42:42.167674+0000","last_active":"2024-02-26T10:42:48.213451+0000","last_peered":"2024-02-26T10:42:48.213451+0000","last_clean":"2024-02-26T10:42:48.213451+0000","last_became_active":"2024-02-26T10:42:42.167224+0000","last_became_peered":"2024-02-26T10:42:42.167224+0000","last_unstale":"2024-02-26T10:42:48.213451+0000","last_undegraded":"2024-02-26T10:42:48.213451+0000","last_fullsized":"2024-02-26T10:42:48.213451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:49:20.433913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799407+0000","last_change":"2024-02-26T10:42:47.244076+0000","last_active":"2024-02-26T10:42:48.799407+0000","last_peered":"2024-02-26T10:42:48.799407+0000","last_clean":"2024-02-26T10:42:48.799407+0000","last_became_active":"2024-02-26T10:42:47.243569+0000","last_became_peered":"2024-02-26T10:42:47.243569+0000","last_unstale":"2024-02-26T10:42:48.799407+0000","last_undegraded":"2024-02-26T10:42:48.799407+0000","last_fullsized":"2024-02-26T10:42:48.799407+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:46:19.855621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212742+0000","last_change":"2024-02-26T10:42:41.623604+0000","last_active":"2024-02-26T10:42:48.212742+0000","last_peered":"2024-02-26T10:42:48.212742+0000","last_clean":"2024-02-26T10:42:48.212742+0000","last_became_active":"2024-02-26T10:42:41.623272+0000","last_became_peered":"2024-02-26T10:42:41.623272+0000","last_unstale":"2024-02-26T10:42:48.212742+0000","last_undegraded":"2024-02-26T10:42:48.212742+0000","last_fullsized":"2024-02-26T10:42:48.212742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:13:57.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801436+0000","last_change":"2024-02-26T10:42:42.186300+0000","last_active":"2024-02-26T10:42:48.801436+0000","last_peered":"2024-02-26T10:42:48.801436+0000","last_clean":"2024-02-26T10:42:48.801436+0000","last_became_active":"2024-02-26T10:42:42.185965+0000","last_became_peered":"2024-02-26T10:42:42.185965+0000","last_unstale":"2024-02-26T10:42:48.801436+0000","last_undegraded":"2024-02-26T10:42:48.801436+0000","last_fullsized":"2024-02-26T10:42:48.801436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:06:50.352308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209143+0000","last_change":"2024-02-26T10:42:47.227753+0000","last_active":"2024-02-26T10:42:48.209143+0000","last_peered":"2024-02-26T10:42:48.209143+0000","last_clean":"2024-02-26T10:42:48.209143+0000","last_became_active":"2024-02-26T10:42:47.227300+0000","last_became_peered":"2024-02-26T10:42:47.227300+0000","last_unstale":"2024-02-26T10:42:48.209143+0000","last_undegraded":"2024-02-26T10:42:48.209143+0000","last_fullsized":"2024-02-26T10:42:48.209143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:08.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.342146+0000","last_change":"2024-02-26T10:42:41.602960+0000","last_active":"2024-02-26T10:43:14.342146+0000","last_peered":"2024-02-26T10:43:14.342146+0000","last_clean":"2024-02-26T10:43:14.342146+0000","last_became_active":"2024-02-26T10:42:41.602676+0000","last_became_peered":"2024-02-26T10:42:41.602676+0000","last_unstale":"2024-02-26T10:43:14.342146+0000","last_undegraded":"2024-02-26T10:43:14.342146+0000","last_fullsized":"2024-02-26T10:43:14.342146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:57.997612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799808+0000","last_change":"2024-02-26T10:42:42.185451+0000","last_active":"2024-02-26T10:42:48.799808+0000","last_peered":"2024-02-26T10:42:48.799808+0000","last_clean":"2024-02-26T10:42:48.799808+0000","last_became_active":"2024-02-26T10:42:42.185116+0000","last_became_peered":"2024-02-26T10:42:42.185116+0000","last_unstale":"2024-02-26T10:42:48.799808+0000","last_undegraded":"2024-02-26T10:42:48.799808+0000","last_fullsized":"2024-02-26T10:42:48.799808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:43:39.823370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518143+0000","last_change":"2024-02-26T10:42:47.250682+0000","last_active":"2024-02-26T10:42:48.518143+0000","last_peered":"2024-02-26T10:42:48.518143+0000","last_clean":"2024-02-26T10:42:48.518143+0000","last_became_active":"2024-02-26T10:42:47.249524+0000","last_became_peered":"2024-02-26T10:42:47.249524+0000","last_unstale":"2024-02-26T10:42:48.518143+0000","last_undegraded":"2024-02-26T10:42:48.518143+0000","last_fullsized":"2024-02-26T10:42:48.518143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:07:01.307206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.340265+0000","last_change":"2024-02-26T10:42:41.620764+0000","last_active":"2024-02-26T10:43:14.340265+0000","last_peered":"2024-02-26T10:43:14.340265+0000","last_clean":"2024-02-26T10:43:14.340265+0000","last_became_active":"2024-02-26T10:42:41.620500+0000","last_became_peered":"2024-02-26T10:42:41.620500+0000","last_unstale":"2024-02-26T10:43:14.340265+0000","last_undegraded":"2024-02-26T10:43:14.340265+0000","last_fullsized":"2024-02-26T10:43:14.340265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:15.542396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797244+0000","last_change":"2024-02-26T10:42:42.165060+0000","last_active":"2024-02-26T10:42:48.797244+0000","last_peered":"2024-02-26T10:42:48.797244+0000","last_clean":"2024-02-26T10:42:48.797244+0000","last_became_active":"2024-02-26T10:42:42.164733+0000","last_became_peered":"2024-02-26T10:42:42.164733+0000","last_unstale":"2024-02-26T10:42:48.797244+0000","last_undegraded":"2024-02-26T10:42:48.797244+0000","last_fullsized":"2024-02-26T10:42:48.797244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:34.190747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213000+0000","last_change":"2024-02-26T10:42:47.246110+0000","last_active":"2024-02-26T10:42:48.213000+0000","last_peered":"2024-02-26T10:42:48.213000+0000","last_clean":"2024-02-26T10:42:48.213000+0000","last_became_active":"2024-02-26T10:42:47.245630+0000","last_became_peered":"2024-02-26T10:42:47.245630+0000","last_unstale":"2024-02-26T10:42:48.213000+0000","last_undegraded":"2024-02-26T10:42:48.213000+0000","last_fullsized":"2024-02-26T10:42:48.213000+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:54:19.108602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.345234+0000","last_change":"2024-02-26T10:42:41.624940+0000","last_active":"2024-02-26T10:43:14.345234+0000","last_peered":"2024-02-26T10:43:14.345234+0000","last_clean":"2024-02-26T10:43:14.345234+0000","last_became_active":"2024-02-26T10:42:41.624538+0000","last_became_peered":"2024-02-26T10:42:41.624538+0000","last_unstale":"2024-02-26T10:43:14.345234+0000","last_undegraded":"2024-02-26T10:43:14.345234+0000","last_fullsized":"2024-02-26T10:43:14.345234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:10.052806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798026+0000","last_change":"2024-02-26T10:42:42.174932+0000","last_active":"2024-02-26T10:42:48.798026+0000","last_peered":"2024-02-26T10:42:48.798026+0000","last_clean":"2024-02-26T10:42:48.798026+0000","last_became_active":"2024-02-26T10:42:42.174566+0000","last_became_peered":"2024-02-26T10:42:42.174566+0000","last_unstale":"2024-02-26T10:42:48.798026+0000","last_undegraded":"2024-02-26T10:42:48.798026+0000","last_fullsized":"2024-02-26T10:42:48.798026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:03.242487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.517970+0000","last_change":"2024-02-26T10:42:47.250023+0000","last_active":"2024-02-26T10:42:48.517970+0000","last_peered":"2024-02-26T10:42:48.517970+0000","last_clean":"2024-02-26T10:42:48.517970+0000","last_became_active":"2024-02-26T10:42:47.249571+0000","last_became_peered":"2024-02-26T10:42:47.249571+0000","last_unstale":"2024-02-26T10:42:48.517970+0000","last_undegraded":"2024-02-26T10:42:48.517970+0000","last_fullsized":"2024-02-26T10:42:48.517970+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:53.666184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209074+0000","last_change":"2024-02-26T10:42:41.614403+0000","last_active":"2024-02-26T10:42:48.209074+0000","last_peered":"2024-02-26T10:42:48.209074+0000","last_clean":"2024-02-26T10:42:48.209074+0000","last_became_active":"2024-02-26T10:42:41.614057+0000","last_became_peered":"2024-02-26T10:42:41.614057+0000","last_unstale":"2024-02-26T10:42:48.209074+0000","last_undegraded":"2024-02-26T10:42:48.209074+0000","last_fullsized":"2024-02-26T10:42:48.209074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:16:54.739090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801000+0000","last_change":"2024-02-26T10:42:42.187465+0000","last_active":"2024-02-26T10:42:48.801000+0000","last_peered":"2024-02-26T10:42:48.801000+0000","last_clean":"2024-02-26T10:42:48.801000+0000","last_became_active":"2024-02-26T10:42:42.182310+0000","last_became_peered":"2024-02-26T10:42:42.182310+0000","last_unstale":"2024-02-26T10:42:48.801000+0000","last_undegraded":"2024-02-26T10:42:48.801000+0000","last_fullsized":"2024-02-26T10:42:48.801000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:01:56.645330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518070+0000","last_change":"2024-02-26T10:42:47.241751+0000","last_active":"2024-02-26T10:42:48.518070+0000","last_peered":"2024-02-26T10:42:48.518070+0000","last_clean":"2024-02-26T10:42:48.518070+0000","last_became_active":"2024-02-26T10:42:47.241271+0000","last_became_peered":"2024-02-26T10:42:47.241271+0000","last_unstale":"2024-02-26T10:42:48.518070+0000","last_undegraded":"2024-02-26T10:42:48.518070+0000","last_fullsized":"2024-02-26T10:42:48.518070+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:40:39.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.339681+0000","last_change":"2024-02-26T10:42:41.607243+0000","last_active":"2024-02-26T10:43:14.339681+0000","last_peered":"2024-02-26T10:43:14.339681+0000","last_clean":"2024-02-26T10:43:14.339681+0000","last_became_active":"2024-02-26T10:42:41.606921+0000","last_became_peered":"2024-02-26T10:42:41.606921+0000","last_unstale":"2024-02-26T10:43:14.339681+0000","last_undegraded":"2024-02-26T10:43:14.339681+0000","last_fullsized":"2024-02-26T10:43:14.339681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:04.626391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798186+0000","last_change":"2024-02-26T10:42:42.178240+0000","last_active":"2024-02-26T10:42:48.798186+0000","last_peered":"2024-02-26T10:42:48.798186+0000","last_clean":"2024-02-26T10:42:48.798186+0000","last_became_active":"2024-02-26T10:42:42.177790+0000","last_became_peered":"2024-02-26T10:42:42.177790+0000","last_unstale":"2024-02-26T10:42:48.798186+0000","last_undegraded":"2024-02-26T10:42:48.798186+0000","last_fullsized":"2024-02-26T10:42:48.798186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:24.110406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799122+0000","last_change":"2024-02-26T10:42:47.233120+0000","last_active":"2024-02-26T10:42:48.799122+0000","last_peered":"2024-02-26T10:42:48.799122+0000","last_clean":"2024-02-26T10:42:48.799122+0000","last_became_active":"2024-02-26T10:42:47.232630+0000","last_became_peered":"2024-02-26T10:42:47.232630+0000","last_unstale":"2024-02-26T10:42:48.799122+0000","last_undegraded":"2024-02-26T10:42:48.799122+0000","last_fullsized":"2024-02-26T10:42:48.799122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:05:03.023227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:15.356910+0000","last_change":"2024-02-26T10:42:41.621269+0000","last_active":"2024-02-26T10:43:15.356910+0000","last_peered":"2024-02-26T10:43:15.356910+0000","last_clean":"2024-02-26T10:43:15.356910+0000","last_became_active":"2024-02-26T10:42:41.620923+0000","last_became_peered":"2024-02-26T10:42:41.620923+0000","last_unstale":"2024-02-26T10:43:15.356910+0000","last_undegraded":"2024-02-26T10:43:15.356910+0000","last_fullsized":"2024-02-26T10:43:15.356910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:06:25.650313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798425+0000","last_change":"2024-02-26T10:42:42.171821+0000","last_active":"2024-02-26T10:42:48.798425+0000","last_peered":"2024-02-26T10:42:48.798425+0000","last_clean":"2024-02-26T10:42:48.798425+0000","last_became_active":"2024-02-26T10:42:42.171523+0000","last_became_peered":"2024-02-26T10:42:42.171523+0000","last_unstale":"2024-02-26T10:42:48.798425+0000","last_undegraded":"2024-02-26T10:42:48.798425+0000","last_fullsized":"2024-02-26T10:42:48.798425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:00.877998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799776+0000","last_change":"2024-02-26T10:42:47.250072+0000","last_active":"2024-02-26T10:42:48.799776+0000","last_peered":"2024-02-26T10:42:48.799776+0000","last_clean":"2024-02-26T10:42:48.799776+0000","last_became_active":"2024-02-26T10:42:47.249570+0000","last_became_peered":"2024-02-26T10:42:47.249570+0000","last_unstale":"2024-02-26T10:42:48.799776+0000","last_undegraded":"2024-02-26T10:42:48.799776+0000","last_fullsized":"2024-02-26T10:42:48.799776+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:43:03.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798199+0000","last_change":"2024-02-26T10:42:41.614686+0000","last_active":"2024-02-26T10:42:48.798199+0000","last_peered":"2024-02-26T10:42:48.798199+0000","last_clean":"2024-02-26T10:42:48.798199+0000","last_became_active":"2024-02-26T10:42:41.614019+0000","last_became_peered":"2024-02-26T10:42:41.614019+0000","last_unstale":"2024-02-26T10:42:48.798199+0000","last_undegraded":"2024-02-26T10:42:48.798199+0000","last_fullsized":"2024-02-26T10:42:48.798199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:13:28.127288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797634+0000","last_change":"2024-02-26T10:42:42.181774+0000","last_active":"2024-02-26T10:42:48.797634+0000","last_peered":"2024-02-26T10:42:48.797634+0000","last_clean":"2024-02-26T10:42:48.797634+0000","last_became_active":"2024-02-26T10:42:42.181412+0000","last_became_peered":"2024-02-26T10:42:42.181412+0000","last_unstale":"2024-02-26T10:42:48.797634+0000","last_undegraded":"2024-02-26T10:42:48.797634+0000","last_fullsized":"2024-02-26T10:42:48.797634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:09.159964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798329+0000","last_change":"2024-02-26T10:42:47.228247+0000","last_active":"2024-02-26T10:42:48.798329+0000","last_peered":"2024-02-26T10:42:48.798329+0000","last_clean":"2024-02-26T10:42:48.798329+0000","last_became_active":"2024-02-26T10:42:47.227569+0000","last_became_peered":"2024-02-26T10:42:47.227569+0000","last_unstale":"2024-02-26T10:42:48.798329+0000","last_undegraded":"2024-02-26T10:42:48.798329+0000","last_fullsized":"2024-02-26T10:42:48.798329+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:15.001264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213970+0000","last_change":"2024-02-26T10:42:41.626573+0000","last_active":"2024-02-26T10:42:48.213970+0000","last_peered":"2024-02-26T10:42:48.213970+0000","last_clean":"2024-02-26T10:42:48.213970+0000","last_became_active":"2024-02-26T10:42:41.624091+0000","last_became_peered":"2024-02-26T10:42:41.624091+0000","last_unstale":"2024-02-26T10:42:48.213970+0000","last_undegraded":"2024-02-26T10:42:48.213970+0000","last_fullsized":"2024-02-26T10:42:48.213970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:24:14.077234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797993+0000","last_change":"2024-02-26T10:42:42.166426+0000","last_active":"2024-02-26T10:42:48.797993+0000","last_peered":"2024-02-26T10:42:48.797993+0000","last_clean":"2024-02-26T10:42:48.797993+0000","last_became_active":"2024-02-26T10:42:42.166082+0000","last_became_peered":"2024-02-26T10:42:42.166082+0000","last_unstale":"2024-02-26T10:42:48.797993+0000","last_undegraded":"2024-02-26T10:42:48.797993+0000","last_fullsized":"2024-02-26T10:42:48.797993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:17.136568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798136+0000","last_change":"2024-02-26T10:42:42.168227+0000","last_active":"2024-02-26T10:42:48.798136+0000","last_peered":"2024-02-26T10:42:48.798136+0000","last_clean":"2024-02-26T10:42:48.798136+0000","last_became_active":"2024-02-26T10:42:42.167897+0000","last_became_peered":"2024-02-26T10:42:42.167897+0000","last_unstale":"2024-02-26T10:42:48.798136+0000","last_undegraded":"2024-02-26T10:42:48.798136+0000","last_fullsized":"2024-02-26T10:42:48.798136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:52:37.391022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209260+0000","last_change":"2024-02-26T10:42:42.168640+0000","last_active":"2024-02-26T10:42:48.209260+0000","last_peered":"2024-02-26T10:42:48.209260+0000","last_clean":"2024-02-26T10:42:48.209260+0000","last_became_active":"2024-02-26T10:42:42.168237+0000","last_became_peered":"2024-02-26T10:42:42.168237+0000","last_unstale":"2024-02-26T10:42:48.209260+0000","last_undegraded":"2024-02-26T10:42:48.209260+0000","last_fullsized":"2024-02-26T10:42:48.209260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:14.853730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.208878+0000","last_change":"2024-02-26T10:42:42.192703+0000","last_active":"2024-02-26T10:42:48.208878+0000","last_peered":"2024-02-26T10:42:48.208878+0000","last_clean":"2024-02-26T10:42:48.208878+0000","last_became_active":"2024-02-26T10:42:42.191899+0000","last_became_peered":"2024-02-26T10:42:42.191899+0000","last_unstale":"2024-02-26T10:42:48.208878+0000","last_undegraded":"2024-02-26T10:42:48.208878+0000","last_fullsized":"2024-02-26T10:42:48.208878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:40.479523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799926+0000","last_change":"2024-02-26T10:42:42.188709+0000","last_active":"2024-02-26T10:42:48.799926+0000","last_peered":"2024-02-26T10:42:48.799926+0000","last_clean":"2024-02-26T10:42:48.799926+0000","last_became_active":"2024-02-26T10:42:42.188418+0000","last_became_peered":"2024-02-26T10:42:42.188418+0000","last_unstale":"2024-02-26T10:42:48.799926+0000","last_undegraded":"2024-02-26T10:42:48.799926+0000","last_fullsized":"2024-02-26T10:42:48.799926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:54:15.697746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213288+0000","last_change":"2024-02-26T10:42:42.174221+0000","last_active":"2024-02-26T10:42:48.213288+0000","last_peered":"2024-02-26T10:42:48.213288+0000","last_clean":"2024-02-26T10:42:48.213288+0000","last_became_active":"2024-02-26T10:42:42.171308+0000","last_became_peered":"2024-02-26T10:42:42.171308+0000","last_unstale":"2024-02-26T10:42:48.213288+0000","last_undegraded":"2024-02-26T10:42:48.213288+0000","last_fullsized":"2024-02-26T10:42:48.213288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:46.765856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798848+0000","last_change":"2024-02-26T10:42:42.170749+0000","last_active":"2024-02-26T10:42:48.798848+0000","last_peered":"2024-02-26T10:42:48.798848+0000","last_clean":"2024-02-26T10:42:48.798848+0000","last_became_active":"2024-02-26T10:42:42.169966+0000","last_became_peered":"2024-02-26T10:42:42.169966+0000","last_unstale":"2024-02-26T10:42:48.798848+0000","last_undegraded":"2024-02-26T10:42:48.798848+0000","last_fullsized":"2024-02-26T10:42:48.798848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:02.633195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209608+0000","last_change":"2024-02-26T10:42:42.166529+0000","last_active":"2024-02-26T10:42:48.209608+0000","last_peered":"2024-02-26T10:42:48.209608+0000","last_clean":"2024-02-26T10:42:48.209608+0000","last_became_active":"2024-02-26T10:42:42.166235+0000","last_became_peered":"2024-02-26T10:42:42.166235+0000","last_unstale":"2024-02-26T10:42:48.209608+0000","last_undegraded":"2024-02-26T10:42:48.209608+0000","last_fullsized":"2024-02-26T10:42:48.209608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:03.068328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801713+0000","last_change":"2024-02-26T10:42:42.191312+0000","last_active":"2024-02-26T10:42:48.801713+0000","last_peered":"2024-02-26T10:42:48.801713+0000","last_clean":"2024-02-26T10:42:48.801713+0000","last_became_active":"2024-02-26T10:42:42.190964+0000","last_became_peered":"2024-02-26T10:42:42.190964+0000","last_unstale":"2024-02-26T10:42:48.801713+0000","last_undegraded":"2024-02-26T10:42:48.801713+0000","last_fullsized":"2024-02-26T10:42:48.801713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:24:04.697129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797798+0000","last_change":"2024-02-26T10:42:42.182882+0000","last_active":"2024-02-26T10:42:48.797798+0000","last_peered":"2024-02-26T10:42:48.797798+0000","last_clean":"2024-02-26T10:42:48.797798+0000","last_became_active":"2024-02-26T10:42:42.182551+0000","last_became_peered":"2024-02-26T10:42:42.182551+0000","last_unstale":"2024-02-26T10:42:48.797798+0000","last_undegraded":"2024-02-26T10:42:48.797798+0000","last_fullsized":"2024-02-26T10:42:48.797798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:11:58.578832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800871+0000","last_change":"2024-02-26T10:42:42.193353+0000","last_active":"2024-02-26T10:42:48.800871+0000","last_peered":"2024-02-26T10:42:48.800871+0000","last_clean":"2024-02-26T10:42:48.800871+0000","last_became_active":"2024-02-26T10:42:42.190935+0000","last_became_peered":"2024-02-26T10:42:42.190935+0000","last_unstale":"2024-02-26T10:42:48.800871+0000","last_undegraded":"2024-02-26T10:42:48.800871+0000","last_fullsized":"2024-02-26T10:42:48.800871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:03:14.319769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799191+0000","last_change":"2024-02-26T10:42:42.194529+0000","last_active":"2024-02-26T10:42:48.799191+0000","last_peered":"2024-02-26T10:42:48.799191+0000","last_clean":"2024-02-26T10:42:48.799191+0000","last_became_active":"2024-02-26T10:42:42.194212+0000","last_became_peered":"2024-02-26T10:42:42.194212+0000","last_unstale":"2024-02-26T10:42:48.799191+0000","last_undegraded":"2024-02-26T10:42:48.799191+0000","last_fullsized":"2024-02-26T10:42:48.799191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:36.876672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212786+0000","last_change":"2024-02-26T10:42:42.178119+0000","last_active":"2024-02-26T10:42:48.212786+0000","last_peered":"2024-02-26T10:42:48.212786+0000","last_clean":"2024-02-26T10:42:48.212786+0000","last_became_active":"2024-02-26T10:42:42.177798+0000","last_became_peered":"2024-02-26T10:42:42.177798+0000","last_unstale":"2024-02-26T10:42:48.212786+0000","last_undegraded":"2024-02-26T10:42:48.212786+0000","last_fullsized":"2024-02-26T10:42:48.212786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:52:29.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798471+0000","last_change":"2024-02-26T10:42:42.167380+0000","last_active":"2024-02-26T10:42:48.798471+0000","last_peered":"2024-02-26T10:42:48.798471+0000","last_clean":"2024-02-26T10:42:48.798471+0000","last_became_active":"2024-02-26T10:42:42.167079+0000","last_became_peered":"2024-02-26T10:42:42.167079+0000","last_unstale":"2024-02-26T10:42:48.798471+0000","last_undegraded":"2024-02-26T10:42:48.798471+0000","last_fullsized":"2024-02-26T10:42:48.798471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:14:25.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250986,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41696,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":33,"num_read_kb":47,"num_write":121,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":93067,"data_compressed":3664,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":489888,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612634,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1105366,"data_compressed":51052,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":490672,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037798,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":490514,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1080838,"data_compressed":51052,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462961,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":489978,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":493730,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":489978,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":533144,"data_compressed":16560,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":490172,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":490923,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1085828,"data_compressed":51052,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":43256,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":39776,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:43:20.152 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:43:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.421+0000 7f9de6400640 1 -- 172.21.15.60:0/4219926077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 msgr2=0x7f9de01036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.421+0000 7f9de6400640 1 --2- 172.21.15.60:0/4219926077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 0x7f9de01036d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009ca0 tx=0x7f9dd002c0f0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.422+0000 7f9de6400640 1 -- 172.21.15.60:0/4219926077 shutdown_connections 2024-02-26T10:43:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.422+0000 7f9de6400640 1 --2- 172.21.15.60:0/4219926077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9de01044a0 0x7f9de0104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.422+0000 7f9de6400640 1 --2- 172.21.15.60:0/4219926077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9de0104e60 0x7f9de010b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.422+0000 7f9de6400640 1 --2- 172.21.15.60:0/4219926077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 0x7f9de01036d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.422+0000 7f9de6400640 1 -- 172.21.15.60:0/4219926077 >> 172.21.15.60:0/4219926077 conn(0x7f9de00fe840 msgr2=0x7f9de0100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.422+0000 7f9de6400640 1 -- 172.21.15.60:0/4219926077 shutdown_connections 2024-02-26T10:43:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.422+0000 7f9de6400640 1 -- 172.21.15.60:0/4219926077 wait complete. 2024-02-26T10:43:20.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.423+0000 7f9de6400640 1 Processor -- start 2024-02-26T10:43:20.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.423+0000 7f9de6400640 1 -- start start 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de6400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 0x7f9de0199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9de01044a0 0x7f9de0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9de0104e60 0x7f9de01a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de6400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de010ef40 con 0x7f9de01032b0 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de010edc0 con 0x7f9de01044a0 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de6400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de010f0c0 con 0x7f9de0104e60 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 0x7f9de0199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 0x7f9de0199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39702/0 (socket says 172.21.15.60:39702) 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de5000640 1 -- 172.21.15.60:0/1683609208 learned_addr learned my addr 172.21.15.60:0/1683609208 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9ddfe00640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9de01044a0 0x7f9de0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de5a00640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9de0104e60 0x7f9de01a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de5000640 1 -- 172.21.15.60:0/1683609208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9de0104e60 msgr2=0x7f9de01a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de5000640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9de0104e60 0x7f9de01a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de5000640 1 -- 172.21.15.60:0/1683609208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9de01044a0 msgr2=0x7f9de0199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de5000640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9de01044a0 0x7f9de0199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.424+0000 7f9de5000640 1 -- 172.21.15.60:0/1683609208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de01a1180 con 0x7f9de01032b0 2024-02-26T10:43:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.425+0000 7f9de5a00640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9de0104e60 0x7f9de01a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.425+0000 7f9ddfe00640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9de01044a0 0x7f9de0199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.425+0000 7f9de5000640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 0x7f9de0199560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0000f80 tx=0x7f9dd0032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.425+0000 7f9ddd600640 1 -- 172.21.15.60:0/1683609208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0041020 con 0x7f9de01032b0 2024-02-26T10:43:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.425+0000 7f9ddd600640 1 -- 172.21.15.60:0/1683609208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9dd0034cd0 con 0x7f9de01032b0 2024-02-26T10:43:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.425+0000 7f9ddd600640 1 -- 172.21.15.60:0/1683609208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0039040 con 0x7f9de01032b0 2024-02-26T10:43:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.425+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de01a1410 con 0x7f9de01032b0 2024-02-26T10:43:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.425+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de01a1880 con 0x7f9de01032b0 2024-02-26T10:43:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.427+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de010a570 con 0x7f9de01032b0 2024-02-26T10:43:20.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.429+0000 7f9ddd600640 1 -- 172.21.15.60:0/1683609208 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f9dd0044020 con 0x7f9de01032b0 2024-02-26T10:43:20.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.430+0000 7f9ddd600640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9dc40a1e90 0x7f9dc40a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.430+0000 7f9ddd600640 1 -- 172.21.15.60:0/1683609208 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9dd00efdf0 con 0x7f9de01032b0 2024-02-26T10:43:20.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.430+0000 7f9ddfe00640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9dc40a1e90 0x7f9dc40a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.431+0000 7f9ddfe00640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9dc40a1e90 0x7f9dc40a4350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0095f0 tx=0x7f9dcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:20.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.435+0000 7f9ddd600640 1 -- 172.21.15.60:0/1683609208 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9dd004b020 con 0x7f9de01032b0 2024-02-26T10:43:20.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:20 smithi060 ceph-mon[31539]: pgmap v342: 129 pgs: 129 active+clean; 617 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-02-26T10:43:20.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:20 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1636377300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:20.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:20 smithi100 ceph-mon[36051]: pgmap v342: 129 pgs: 129 active+clean; 617 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-02-26T10:43:20.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:20 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1636377300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:20.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.637+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9de00616a0 con 0x7f9dc40a1e90 2024-02-26T10:43:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.654+0000 7f9ddd600640 1 -- 172.21.15.60:0/1683609208 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340645 (secure 0 0 0) 0x7f9de00616a0 con 0x7f9dc40a1e90 2024-02-26T10:43:20.655 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:20.657 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T10:43:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.660+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9dc40a1e90 msgr2=0x7f9dc40a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.660+0000 7f9de6400640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9dc40a1e90 0x7f9dc40a4350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0095f0 tx=0x7f9dcc005f90 comp rx=0 tx=0).stop 2024-02-26T10:43:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.660+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 msgr2=0x7f9de0199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.660+0000 7f9de6400640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 0x7f9de0199560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0000f80 tx=0x7f9dd0032ec0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.661+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 shutdown_connections 2024-02-26T10:43:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.661+0000 7f9de6400640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f9dc40a1e90 0x7f9dc40a4350 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.661+0000 7f9de6400640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9de01044a0 0x7f9de0199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.661+0000 7f9de6400640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9de0104e60 0x7f9de01a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.661+0000 7f9de6400640 1 --2- 172.21.15.60:0/1683609208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9de01032b0 0x7f9de0199560 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.661+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 >> 172.21.15.60:0/1683609208 conn(0x7f9de00fe840 msgr2=0x7f9de0107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.661+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 shutdown_connections 2024-02-26T10:43:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:20.661+0000 7f9de6400640 1 -- 172.21.15.60:0/1683609208 wait complete. 2024-02-26T10:43:20.677 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-02-26T10:43:18.895503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632064,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":159,"num_read_kb":156,"num_write":302,"num_write_kb":3012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369452,"kb_used_data":11960,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572372,"statfs":{"total":1151940427776,"available":1151562108928,"internally_reserved":0,"allocated":12247040,"data_stored":7731031,"data_compressed":287540,"data_compressed_allocated":385024,"data_compressed_original":2269184,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37197,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":33,"num_read_kb":47,"num_write":73,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006326"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799086+0000","last_change":"2024-02-26T10:42:42.176634+0000","last_active":"2024-02-26T10:42:48.799086+0000","last_peered":"2024-02-26T10:42:48.799086+0000","last_clean":"2024-02-26T10:42:48.799086+0000","last_became_active":"2024-02-26T10:42:42.176386+0000","last_became_peered":"2024-02-26T10:42:42.176386+0000","last_unstale":"2024-02-26T10:42:48.799086+0000","last_undegraded":"2024-02-26T10:42:48.799086+0000","last_fullsized":"2024-02-26T10:42:48.799086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:45.005506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799615+0000","last_change":"2024-02-26T10:42:42.176769+0000","last_active":"2024-02-26T10:42:48.799615+0000","last_peered":"2024-02-26T10:42:48.799615+0000","last_clean":"2024-02-26T10:42:48.799615+0000","last_became_active":"2024-02-26T10:42:42.174868+0000","last_became_peered":"2024-02-26T10:42:42.174868+0000","last_unstale":"2024-02-26T10:42:48.799615+0000","last_undegraded":"2024-02-26T10:42:48.799615+0000","last_fullsized":"2024-02-26T10:42:48.799615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:16.202371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800123+0000","last_change":"2024-02-26T10:42:42.184410+0000","last_active":"2024-02-26T10:42:48.800123+0000","last_peered":"2024-02-26T10:42:48.800123+0000","last_clean":"2024-02-26T10:42:48.800123+0000","last_became_active":"2024-02-26T10:42:42.182986+0000","last_became_peered":"2024-02-26T10:42:42.182986+0000","last_unstale":"2024-02-26T10:42:48.800123+0000","last_undegraded":"2024-02-26T10:42:48.800123+0000","last_fullsized":"2024-02-26T10:42:48.800123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:28:45.445325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518257+0000","last_change":"2024-02-26T10:42:42.168283+0000","last_active":"2024-02-26T10:42:48.518257+0000","last_peered":"2024-02-26T10:42:48.518257+0000","last_clean":"2024-02-26T10:42:48.518257+0000","last_became_active":"2024-02-26T10:42:42.167565+0000","last_became_peered":"2024-02-26T10:42:42.167565+0000","last_unstale":"2024-02-26T10:42:48.518257+0000","last_undegraded":"2024-02-26T10:42:48.518257+0000","last_fullsized":"2024-02-26T10:42:48.518257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:16:37.766158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797743+0000","last_change":"2024-02-26T10:42:42.192448+0000","last_active":"2024-02-26T10:42:48.797743+0000","last_peered":"2024-02-26T10:42:48.797743+0000","last_clean":"2024-02-26T10:42:48.797743+0000","last_became_active":"2024-02-26T10:42:42.192133+0000","last_became_peered":"2024-02-26T10:42:42.192133+0000","last_unstale":"2024-02-26T10:42:48.797743+0000","last_undegraded":"2024-02-26T10:42:48.797743+0000","last_fullsized":"2024-02-26T10:42:48.797743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:43:49.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797683+0000","last_change":"2024-02-26T10:42:42.169999+0000","last_active":"2024-02-26T10:42:48.797683+0000","last_peered":"2024-02-26T10:42:48.797683+0000","last_clean":"2024-02-26T10:42:48.797683+0000","last_became_active":"2024-02-26T10:42:42.169678+0000","last_became_peered":"2024-02-26T10:42:42.169678+0000","last_unstale":"2024-02-26T10:42:48.797683+0000","last_undegraded":"2024-02-26T10:42:48.797683+0000","last_fullsized":"2024-02-26T10:42:48.797683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:54.435203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797671+0000","last_change":"2024-02-26T10:42:42.177538+0000","last_active":"2024-02-26T10:42:48.797671+0000","last_peered":"2024-02-26T10:42:48.797671+0000","last_clean":"2024-02-26T10:42:48.797671+0000","last_became_active":"2024-02-26T10:42:42.177204+0000","last_became_peered":"2024-02-26T10:42:42.177204+0000","last_unstale":"2024-02-26T10:42:48.797671+0000","last_undegraded":"2024-02-26T10:42:48.797671+0000","last_fullsized":"2024-02-26T10:42:48.797671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:09.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799743+0000","last_change":"2024-02-26T10:42:42.191342+0000","last_active":"2024-02-26T10:42:48.799743+0000","last_peered":"2024-02-26T10:42:48.799743+0000","last_clean":"2024-02-26T10:42:48.799743+0000","last_became_active":"2024-02-26T10:42:42.190990+0000","last_became_peered":"2024-02-26T10:42:42.190990+0000","last_unstale":"2024-02-26T10:42:48.799743+0000","last_undegraded":"2024-02-26T10:42:48.799743+0000","last_fullsized":"2024-02-26T10:42:48.799743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:20:45.190300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799562+0000","last_change":"2024-02-26T10:42:42.173175+0000","last_active":"2024-02-26T10:42:48.799562+0000","last_peered":"2024-02-26T10:42:48.799562+0000","last_clean":"2024-02-26T10:42:48.799562+0000","last_became_active":"2024-02-26T10:42:42.172828+0000","last_became_peered":"2024-02-26T10:42:42.172828+0000","last_unstale":"2024-02-26T10:42:48.799562+0000","last_undegraded":"2024-02-26T10:42:48.799562+0000","last_fullsized":"2024-02-26T10:42:48.799562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:17:34.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798681+0000","last_change":"2024-02-26T10:42:42.167267+0000","last_active":"2024-02-26T10:42:48.798681+0000","last_peered":"2024-02-26T10:42:48.798681+0000","last_clean":"2024-02-26T10:42:48.798681+0000","last_became_active":"2024-02-26T10:42:42.166948+0000","last_became_peered":"2024-02-26T10:42:42.166948+0000","last_unstale":"2024-02-26T10:42:48.798681+0000","last_undegraded":"2024-02-26T10:42:48.798681+0000","last_fullsized":"2024-02-26T10:42:48.798681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:52.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798497+0000","last_change":"2024-02-26T10:42:42.171299+0000","last_active":"2024-02-26T10:42:48.798497+0000","last_peered":"2024-02-26T10:42:48.798497+0000","last_clean":"2024-02-26T10:42:48.798497+0000","last_became_active":"2024-02-26T10:42:42.170616+0000","last_became_peered":"2024-02-26T10:42:42.170616+0000","last_unstale":"2024-02-26T10:42:48.798497+0000","last_undegraded":"2024-02-26T10:42:48.798497+0000","last_fullsized":"2024-02-26T10:42:48.798497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:50.320859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518171+0000","last_change":"2024-02-26T10:42:42.168408+0000","last_active":"2024-02-26T10:42:48.518171+0000","last_peered":"2024-02-26T10:42:48.518171+0000","last_clean":"2024-02-26T10:42:48.518171+0000","last_became_active":"2024-02-26T10:42:42.167150+0000","last_became_peered":"2024-02-26T10:42:42.167150+0000","last_unstale":"2024-02-26T10:42:48.518171+0000","last_undegraded":"2024-02-26T10:42:48.518171+0000","last_fullsized":"2024-02-26T10:42:48.518171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:34.763331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212669+0000","last_change":"2024-02-26T10:42:42.171720+0000","last_active":"2024-02-26T10:42:48.212669+0000","last_peered":"2024-02-26T10:42:48.212669+0000","last_clean":"2024-02-26T10:42:48.212669+0000","last_became_active":"2024-02-26T10:42:42.171309+0000","last_became_peered":"2024-02-26T10:42:42.171309+0000","last_unstale":"2024-02-26T10:42:48.212669+0000","last_undegraded":"2024-02-26T10:42:48.212669+0000","last_fullsized":"2024-02-26T10:42:48.212669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:20:52.151553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799200+0000","last_change":"2024-02-26T10:42:42.176857+0000","last_active":"2024-02-26T10:42:48.799200+0000","last_peered":"2024-02-26T10:42:48.799200+0000","last_clean":"2024-02-26T10:42:48.799200+0000","last_became_active":"2024-02-26T10:42:42.176385+0000","last_became_peered":"2024-02-26T10:42:42.176385+0000","last_unstale":"2024-02-26T10:42:48.799200+0000","last_undegraded":"2024-02-26T10:42:48.799200+0000","last_fullsized":"2024-02-26T10:42:48.799200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:45:51.852212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801034+0000","last_change":"2024-02-26T10:42:42.174682+0000","last_active":"2024-02-26T10:42:48.801034+0000","last_peered":"2024-02-26T10:42:48.801034+0000","last_clean":"2024-02-26T10:42:48.801034+0000","last_became_active":"2024-02-26T10:42:42.174328+0000","last_became_peered":"2024-02-26T10:42:42.174328+0000","last_unstale":"2024-02-26T10:42:48.801034+0000","last_undegraded":"2024-02-26T10:42:48.801034+0000","last_fullsized":"2024-02-26T10:42:48.801034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:47.161020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798763+0000","last_change":"2024-02-26T10:42:42.185289+0000","last_active":"2024-02-26T10:42:48.798763+0000","last_peered":"2024-02-26T10:42:48.798763+0000","last_clean":"2024-02-26T10:42:48.798763+0000","last_became_active":"2024-02-26T10:42:42.185000+0000","last_became_peered":"2024-02-26T10:42:42.185000+0000","last_unstale":"2024-02-26T10:42:48.798763+0000","last_undegraded":"2024-02-26T10:42:48.798763+0000","last_fullsized":"2024-02-26T10:42:48.798763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:48.552584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798900+0000","last_change":"2024-02-26T10:42:42.176558+0000","last_active":"2024-02-26T10:42:48.798900+0000","last_peered":"2024-02-26T10:42:48.798900+0000","last_clean":"2024-02-26T10:42:48.798900+0000","last_became_active":"2024-02-26T10:42:42.176334+0000","last_became_peered":"2024-02-26T10:42:42.176334+0000","last_unstale":"2024-02-26T10:42:48.798900+0000","last_undegraded":"2024-02-26T10:42:48.798900+0000","last_fullsized":"2024-02-26T10:42:48.798900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:59.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212806+0000","last_change":"2024-02-26T10:42:42.168534+0000","last_active":"2024-02-26T10:42:48.212806+0000","last_peered":"2024-02-26T10:42:48.212806+0000","last_clean":"2024-02-26T10:42:48.212806+0000","last_became_active":"2024-02-26T10:42:42.168179+0000","last_became_peered":"2024-02-26T10:42:42.168179+0000","last_unstale":"2024-02-26T10:42:48.212806+0000","last_undegraded":"2024-02-26T10:42:48.212806+0000","last_fullsized":"2024-02-26T10:42:48.212806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:09.625969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209473+0000","last_change":"2024-02-26T10:42:42.192446+0000","last_active":"2024-02-26T10:42:48.209473+0000","last_peered":"2024-02-26T10:42:48.209473+0000","last_clean":"2024-02-26T10:42:48.209473+0000","last_became_active":"2024-02-26T10:42:42.191863+0000","last_became_peered":"2024-02-26T10:42:42.191863+0000","last_unstale":"2024-02-26T10:42:48.209473+0000","last_undegraded":"2024-02-26T10:42:48.209473+0000","last_fullsized":"2024-02-26T10:42:48.209473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:30:19.477529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209130+0000","last_change":"2024-02-26T10:42:47.257605+0000","last_active":"2024-02-26T10:42:48.209130+0000","last_peered":"2024-02-26T10:42:48.209130+0000","last_clean":"2024-02-26T10:42:48.209130+0000","last_became_active":"2024-02-26T10:42:47.257206+0000","last_became_peered":"2024-02-26T10:42:47.257206+0000","last_unstale":"2024-02-26T10:42:48.209130+0000","last_undegraded":"2024-02-26T10:42:48.209130+0000","last_fullsized":"2024-02-26T10:42:48.209130+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:26:17.681862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213163+0000","last_change":"2024-02-26T10:42:41.613087+0000","last_active":"2024-02-26T10:42:48.213163+0000","last_peered":"2024-02-26T10:42:48.213163+0000","last_clean":"2024-02-26T10:42:48.213163+0000","last_became_active":"2024-02-26T10:42:41.612664+0000","last_became_peered":"2024-02-26T10:42:41.612664+0000","last_unstale":"2024-02-26T10:42:48.213163+0000","last_undegraded":"2024-02-26T10:42:48.213163+0000","last_fullsized":"2024-02-26T10:42:48.213163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:00.352166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801675+0000","last_change":"2024-02-26T10:42:42.191438+0000","last_active":"2024-02-26T10:42:48.801675+0000","last_peered":"2024-02-26T10:42:48.801675+0000","last_clean":"2024-02-26T10:42:48.801675+0000","last_became_active":"2024-02-26T10:42:42.191095+0000","last_became_peered":"2024-02-26T10:42:42.191095+0000","last_unstale":"2024-02-26T10:42:48.801675+0000","last_undegraded":"2024-02-26T10:42:48.801675+0000","last_fullsized":"2024-02-26T10:42:48.801675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:20.918715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.517742+0000","last_change":"2024-02-26T10:42:47.247503+0000","last_active":"2024-02-26T10:42:48.517742+0000","last_peered":"2024-02-26T10:42:48.517742+0000","last_clean":"2024-02-26T10:42:48.517742+0000","last_became_active":"2024-02-26T10:42:47.247121+0000","last_became_peered":"2024-02-26T10:42:47.247121+0000","last_unstale":"2024-02-26T10:42:48.517742+0000","last_undegraded":"2024-02-26T10:42:48.517742+0000","last_fullsized":"2024-02-26T10:42:48.517742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:01:38.733131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.802175+0000","last_change":"2024-02-26T10:42:41.626227+0000","last_active":"2024-02-26T10:42:48.802175+0000","last_peered":"2024-02-26T10:42:48.802175+0000","last_clean":"2024-02-26T10:42:48.802175+0000","last_became_active":"2024-02-26T10:42:41.625760+0000","last_became_peered":"2024-02-26T10:42:41.625760+0000","last_unstale":"2024-02-26T10:42:48.802175+0000","last_undegraded":"2024-02-26T10:42:48.802175+0000","last_fullsized":"2024-02-26T10:42:48.802175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:04.600151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800269+0000","last_change":"2024-02-26T10:42:42.182808+0000","last_active":"2024-02-26T10:42:48.800269+0000","last_peered":"2024-02-26T10:42:48.800269+0000","last_clean":"2024-02-26T10:42:48.800269+0000","last_became_active":"2024-02-26T10:42:42.182485+0000","last_became_peered":"2024-02-26T10:42:42.182485+0000","last_unstale":"2024-02-26T10:42:48.800269+0000","last_undegraded":"2024-02-26T10:42:48.800269+0000","last_fullsized":"2024-02-26T10:42:48.800269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:45.205969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.517885+0000","last_change":"2024-02-26T10:42:47.250694+0000","last_active":"2024-02-26T10:42:48.517885+0000","last_peered":"2024-02-26T10:42:48.517885+0000","last_clean":"2024-02-26T10:42:48.517885+0000","last_became_active":"2024-02-26T10:42:47.249566+0000","last_became_peered":"2024-02-26T10:42:47.249566+0000","last_unstale":"2024-02-26T10:42:48.517885+0000","last_undegraded":"2024-02-26T10:42:48.517885+0000","last_fullsized":"2024-02-26T10:42:48.517885+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:44.016413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798585+0000","last_change":"2024-02-26T10:42:41.606849+0000","last_active":"2024-02-26T10:42:48.798585+0000","last_peered":"2024-02-26T10:42:48.798585+0000","last_clean":"2024-02-26T10:42:48.798585+0000","last_became_active":"2024-02-26T10:42:41.606494+0000","last_became_peered":"2024-02-26T10:42:41.606494+0000","last_unstale":"2024-02-26T10:42:48.798585+0000","last_undegraded":"2024-02-26T10:42:48.798585+0000","last_fullsized":"2024-02-26T10:42:48.798585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:36.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799191+0000","last_change":"2024-02-26T10:42:42.175926+0000","last_active":"2024-02-26T10:42:48.799191+0000","last_peered":"2024-02-26T10:42:48.799191+0000","last_clean":"2024-02-26T10:42:48.799191+0000","last_became_active":"2024-02-26T10:42:42.175614+0000","last_became_peered":"2024-02-26T10:42:42.175614+0000","last_unstale":"2024-02-26T10:42:48.799191+0000","last_undegraded":"2024-02-26T10:42:48.799191+0000","last_fullsized":"2024-02-26T10:42:48.799191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:20:20.829849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.517723+0000","last_change":"2024-02-26T10:42:47.227215+0000","last_active":"2024-02-26T10:42:48.517723+0000","last_peered":"2024-02-26T10:42:48.517723+0000","last_clean":"2024-02-26T10:42:48.517723+0000","last_became_active":"2024-02-26T10:42:47.226771+0000","last_became_peered":"2024-02-26T10:42:47.226771+0000","last_unstale":"2024-02-26T10:42:48.517723+0000","last_undegraded":"2024-02-26T10:42:48.517723+0000","last_fullsized":"2024-02-26T10:42:48.517723+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:12:58.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797601+0000","last_change":"2024-02-26T10:42:41.606407+0000","last_active":"2024-02-26T10:42:48.797601+0000","last_peered":"2024-02-26T10:42:48.797601+0000","last_clean":"2024-02-26T10:42:48.797601+0000","last_became_active":"2024-02-26T10:42:41.606094+0000","last_became_peered":"2024-02-26T10:42:41.606094+0000","last_unstale":"2024-02-26T10:42:48.797601+0000","last_undegraded":"2024-02-26T10:42:48.797601+0000","last_fullsized":"2024-02-26T10:42:48.797601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:53.673017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798116+0000","last_change":"2024-02-26T10:42:42.172967+0000","last_active":"2024-02-26T10:42:48.798116+0000","last_peered":"2024-02-26T10:42:48.798116+0000","last_clean":"2024-02-26T10:42:48.798116+0000","last_became_active":"2024-02-26T10:42:42.172675+0000","last_became_peered":"2024-02-26T10:42:42.172675+0000","last_unstale":"2024-02-26T10:42:48.798116+0000","last_undegraded":"2024-02-26T10:42:48.798116+0000","last_fullsized":"2024-02-26T10:42:48.798116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209528+0000","last_change":"2024-02-26T10:42:47.229375+0000","last_active":"2024-02-26T10:42:48.209528+0000","last_peered":"2024-02-26T10:42:48.209528+0000","last_clean":"2024-02-26T10:42:48.209528+0000","last_became_active":"2024-02-26T10:42:47.228874+0000","last_became_peered":"2024-02-26T10:42:47.228874+0000","last_unstale":"2024-02-26T10:42:48.209528+0000","last_undegraded":"2024-02-26T10:42:48.209528+0000","last_fullsized":"2024-02-26T10:42:48.209528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:02.188519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:16.041683+0000","last_change":"2024-02-26T10:42:41.602783+0000","last_active":"2024-02-26T10:43:16.041683+0000","last_peered":"2024-02-26T10:43:16.041683+0000","last_clean":"2024-02-26T10:43:16.041683+0000","last_became_active":"2024-02-26T10:42:41.602491+0000","last_became_peered":"2024-02-26T10:42:41.602491+0000","last_unstale":"2024-02-26T10:43:16.041683+0000","last_undegraded":"2024-02-26T10:43:16.041683+0000","last_fullsized":"2024-02-26T10:43:16.041683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:23:33.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797452+0000","last_change":"2024-02-26T10:42:42.170018+0000","last_active":"2024-02-26T10:42:48.797452+0000","last_peered":"2024-02-26T10:42:48.797452+0000","last_clean":"2024-02-26T10:42:48.797452+0000","last_became_active":"2024-02-26T10:42:42.169596+0000","last_became_peered":"2024-02-26T10:42:42.169596+0000","last_unstale":"2024-02-26T10:42:48.797452+0000","last_undegraded":"2024-02-26T10:42:48.797452+0000","last_fullsized":"2024-02-26T10:42:48.797452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:20:43.275319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209462+0000","last_change":"2024-02-26T10:42:47.250104+0000","last_active":"2024-02-26T10:42:48.209462+0000","last_peered":"2024-02-26T10:42:48.209462+0000","last_clean":"2024-02-26T10:42:48.209462+0000","last_became_active":"2024-02-26T10:42:47.249560+0000","last_became_peered":"2024-02-26T10:42:47.249560+0000","last_unstale":"2024-02-26T10:42:48.209462+0000","last_undegraded":"2024-02-26T10:42:48.209462+0000","last_fullsized":"2024-02-26T10:42:48.209462+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:11:12.056434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:15.359888+0000","last_change":"2024-02-26T10:42:41.602887+0000","last_active":"2024-02-26T10:43:15.359888+0000","last_peered":"2024-02-26T10:43:15.359888+0000","last_clean":"2024-02-26T10:43:15.359888+0000","last_became_active":"2024-02-26T10:42:41.602564+0000","last_became_peered":"2024-02-26T10:42:41.602564+0000","last_unstale":"2024-02-26T10:43:15.359888+0000","last_undegraded":"2024-02-26T10:43:15.359888+0000","last_fullsized":"2024-02-26T10:43:15.359888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:22:41.711283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798636+0000","last_change":"2024-02-26T10:42:42.192700+0000","last_active":"2024-02-26T10:42:48.798636+0000","last_peered":"2024-02-26T10:42:48.798636+0000","last_clean":"2024-02-26T10:42:48.798636+0000","last_became_active":"2024-02-26T10:42:42.192387+0000","last_became_peered":"2024-02-26T10:42:42.192387+0000","last_unstale":"2024-02-26T10:42:48.798636+0000","last_undegraded":"2024-02-26T10:42:48.798636+0000","last_fullsized":"2024-02-26T10:42:48.798636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:21:15.104872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798996+0000","last_change":"2024-02-26T10:42:47.247000+0000","last_active":"2024-02-26T10:42:48.798996+0000","last_peered":"2024-02-26T10:42:48.798996+0000","last_clean":"2024-02-26T10:42:48.798996+0000","last_became_active":"2024-02-26T10:42:47.246520+0000","last_became_peered":"2024-02-26T10:42:47.246520+0000","last_unstale":"2024-02-26T10:42:48.798996+0000","last_undegraded":"2024-02-26T10:42:48.798996+0000","last_fullsized":"2024-02-26T10:42:48.798996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:29.977858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:15.359585+0000","last_change":"2024-02-26T10:42:41.609460+0000","last_active":"2024-02-26T10:43:15.359585+0000","last_peered":"2024-02-26T10:43:15.359585+0000","last_clean":"2024-02-26T10:43:15.359585+0000","last_became_active":"2024-02-26T10:42:41.608981+0000","last_became_peered":"2024-02-26T10:42:41.608981+0000","last_unstale":"2024-02-26T10:43:15.359585+0000","last_undegraded":"2024-02-26T10:43:15.359585+0000","last_fullsized":"2024-02-26T10:43:15.359585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:04.599910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799489+0000","last_change":"2024-02-26T10:42:42.185359+0000","last_active":"2024-02-26T10:42:48.799489+0000","last_peered":"2024-02-26T10:42:48.799489+0000","last_clean":"2024-02-26T10:42:48.799489+0000","last_became_active":"2024-02-26T10:42:42.184221+0000","last_became_peered":"2024-02-26T10:42:42.184221+0000","last_unstale":"2024-02-26T10:42:48.799489+0000","last_undegraded":"2024-02-26T10:42:48.799489+0000","last_fullsized":"2024-02-26T10:42:48.799489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:53.907290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800252+0000","last_change":"2024-02-26T10:42:47.241700+0000","last_active":"2024-02-26T10:42:48.800252+0000","last_peered":"2024-02-26T10:42:48.800252+0000","last_clean":"2024-02-26T10:42:48.800252+0000","last_became_active":"2024-02-26T10:42:47.240667+0000","last_became_peered":"2024-02-26T10:42:47.240667+0000","last_unstale":"2024-02-26T10:42:48.800252+0000","last_undegraded":"2024-02-26T10:42:48.800252+0000","last_fullsized":"2024-02-26T10:42:48.800252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:19.071244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797703+0000","last_change":"2024-02-26T10:42:41.604519+0000","last_active":"2024-02-26T10:42:48.797703+0000","last_peered":"2024-02-26T10:42:48.797703+0000","last_clean":"2024-02-26T10:42:48.797703+0000","last_became_active":"2024-02-26T10:42:41.603596+0000","last_became_peered":"2024-02-26T10:42:41.603596+0000","last_unstale":"2024-02-26T10:42:48.797703+0000","last_undegraded":"2024-02-26T10:42:48.797703+0000","last_fullsized":"2024-02-26T10:42:48.797703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:39.764730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.210647+0000","last_change":"2024-02-26T10:42:42.187532+0000","last_active":"2024-02-26T10:42:48.210647+0000","last_peered":"2024-02-26T10:42:48.210647+0000","last_clean":"2024-02-26T10:42:48.210647+0000","last_became_active":"2024-02-26T10:42:42.185635+0000","last_became_peered":"2024-02-26T10:42:42.185635+0000","last_unstale":"2024-02-26T10:42:48.210647+0000","last_undegraded":"2024-02-26T10:42:48.210647+0000","last_fullsized":"2024-02-26T10:42:48.210647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:16.894747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801774+0000","last_change":"2024-02-26T10:42:47.233507+0000","last_active":"2024-02-26T10:42:48.801774+0000","last_peered":"2024-02-26T10:42:48.801774+0000","last_clean":"2024-02-26T10:42:48.801774+0000","last_became_active":"2024-02-26T10:42:47.229589+0000","last_became_peered":"2024-02-26T10:42:47.229589+0000","last_unstale":"2024-02-26T10:42:48.801774+0000","last_undegraded":"2024-02-26T10:42:48.801774+0000","last_fullsized":"2024-02-26T10:42:48.801774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:02:42.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:17.042679+0000","last_change":"2024-02-26T10:42:41.613952+0000","last_active":"2024-02-26T10:43:17.042679+0000","last_peered":"2024-02-26T10:43:17.042679+0000","last_clean":"2024-02-26T10:43:17.042679+0000","last_became_active":"2024-02-26T10:42:41.613656+0000","last_became_peered":"2024-02-26T10:42:41.613656+0000","last_unstale":"2024-02-26T10:43:17.042679+0000","last_undegraded":"2024-02-26T10:43:17.042679+0000","last_fullsized":"2024-02-26T10:43:17.042679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:14.090821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799493+0000","last_change":"2024-02-26T10:42:42.178357+0000","last_active":"2024-02-26T10:42:48.799493+0000","last_peered":"2024-02-26T10:42:48.799493+0000","last_clean":"2024-02-26T10:42:48.799493+0000","last_became_active":"2024-02-26T10:42:42.178051+0000","last_became_peered":"2024-02-26T10:42:42.178051+0000","last_unstale":"2024-02-26T10:42:48.799493+0000","last_undegraded":"2024-02-26T10:42:48.799493+0000","last_fullsized":"2024-02-26T10:42:48.799493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:13:01.326122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.210857+0000","last_change":"2024-02-26T10:42:47.246231+0000","last_active":"2024-02-26T10:42:48.210857+0000","last_peered":"2024-02-26T10:42:48.210857+0000","last_clean":"2024-02-26T10:42:48.210857+0000","last_became_active":"2024-02-26T10:42:47.245739+0000","last_became_peered":"2024-02-26T10:42:47.245739+0000","last_unstale":"2024-02-26T10:42:48.210857+0000","last_undegraded":"2024-02-26T10:42:48.210857+0000","last_fullsized":"2024-02-26T10:42:48.210857+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:30.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.339654+0000","last_change":"2024-02-26T10:42:41.603126+0000","last_active":"2024-02-26T10:43:14.339654+0000","last_peered":"2024-02-26T10:43:14.339654+0000","last_clean":"2024-02-26T10:43:14.339654+0000","last_became_active":"2024-02-26T10:42:41.602790+0000","last_became_peered":"2024-02-26T10:42:41.602790+0000","last_unstale":"2024-02-26T10:43:14.339654+0000","last_undegraded":"2024-02-26T10:43:14.339654+0000","last_fullsized":"2024-02-26T10:43:14.339654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:31:58.908209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800535+0000","last_change":"2024-02-26T10:42:42.185034+0000","last_active":"2024-02-26T10:42:48.800535+0000","last_peered":"2024-02-26T10:42:48.800535+0000","last_clean":"2024-02-26T10:42:48.800535+0000","last_became_active":"2024-02-26T10:42:42.181894+0000","last_became_peered":"2024-02-26T10:42:42.181894+0000","last_unstale":"2024-02-26T10:42:48.800535+0000","last_undegraded":"2024-02-26T10:42:48.800535+0000","last_fullsized":"2024-02-26T10:42:48.800535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:22.885475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518247+0000","last_change":"2024-02-26T10:42:47.250075+0000","last_active":"2024-02-26T10:42:48.518247+0000","last_peered":"2024-02-26T10:42:48.518247+0000","last_clean":"2024-02-26T10:42:48.518247+0000","last_became_active":"2024-02-26T10:42:47.249490+0000","last_became_peered":"2024-02-26T10:42:47.249490+0000","last_unstale":"2024-02-26T10:42:48.518247+0000","last_undegraded":"2024-02-26T10:42:48.518247+0000","last_fullsized":"2024-02-26T10:42:48.518247+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:42.659282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.342063+0000","last_change":"2024-02-26T10:42:41.615314+0000","last_active":"2024-02-26T10:43:14.342063+0000","last_peered":"2024-02-26T10:43:14.342063+0000","last_clean":"2024-02-26T10:43:14.342063+0000","last_became_active":"2024-02-26T10:42:41.615091+0000","last_became_peered":"2024-02-26T10:42:41.615091+0000","last_unstale":"2024-02-26T10:43:14.342063+0000","last_undegraded":"2024-02-26T10:43:14.342063+0000","last_fullsized":"2024-02-26T10:43:14.342063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:41:52.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799389+0000","last_change":"2024-02-26T10:42:42.176609+0000","last_active":"2024-02-26T10:42:48.799389+0000","last_peered":"2024-02-26T10:42:48.799389+0000","last_clean":"2024-02-26T10:42:48.799389+0000","last_became_active":"2024-02-26T10:42:42.176311+0000","last_became_peered":"2024-02-26T10:42:42.176311+0000","last_unstale":"2024-02-26T10:42:48.799389+0000","last_undegraded":"2024-02-26T10:42:48.799389+0000","last_fullsized":"2024-02-26T10:42:48.799389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:25:38.415311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.211107+0000","last_change":"2024-02-26T10:42:47.250255+0000","last_active":"2024-02-26T10:42:48.211107+0000","last_peered":"2024-02-26T10:42:48.211107+0000","last_clean":"2024-02-26T10:42:48.211107+0000","last_became_active":"2024-02-26T10:42:47.249465+0000","last_became_peered":"2024-02-26T10:42:47.249465+0000","last_unstale":"2024-02-26T10:42:48.211107+0000","last_undegraded":"2024-02-26T10:42:48.211107+0000","last_fullsized":"2024-02-26T10:42:48.211107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:23.173863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'23","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:18.132992+0000","last_change":"2024-02-26T10:42:41.602049+0000","last_active":"2024-02-26T10:43:18.132992+0000","last_peered":"2024-02-26T10:43:18.132992+0000","last_clean":"2024-02-26T10:43:18.132992+0000","last_became_active":"2024-02-26T10:42:41.601720+0000","last_became_peered":"2024-02-26T10:42:41.601720+0000","last_unstale":"2024-02-26T10:43:18.132992+0000","last_undegraded":"2024-02-26T10:43:18.132992+0000","last_fullsized":"2024-02-26T10:43:18.132992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:31.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37807,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":28,"num_write":24,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798370+0000","last_change":"2024-02-26T10:42:42.168174+0000","last_active":"2024-02-26T10:42:48.798370+0000","last_peered":"2024-02-26T10:42:48.798370+0000","last_clean":"2024-02-26T10:42:48.798370+0000","last_became_active":"2024-02-26T10:42:42.167840+0000","last_became_peered":"2024-02-26T10:42:42.167840+0000","last_unstale":"2024-02-26T10:42:48.798370+0000","last_undegraded":"2024-02-26T10:42:48.798370+0000","last_fullsized":"2024-02-26T10:42:48.798370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:43.079139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798497+0000","last_change":"2024-02-26T10:42:47.230048+0000","last_active":"2024-02-26T10:42:48.798497+0000","last_peered":"2024-02-26T10:42:48.798497+0000","last_clean":"2024-02-26T10:42:48.798497+0000","last_became_active":"2024-02-26T10:42:47.229396+0000","last_became_peered":"2024-02-26T10:42:47.229396+0000","last_unstale":"2024-02-26T10:42:48.798497+0000","last_undegraded":"2024-02-26T10:42:48.798497+0000","last_fullsized":"2024-02-26T10:42:48.798497+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:57.804297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:15.355234+0000","last_change":"2024-02-26T10:42:41.614413+0000","last_active":"2024-02-26T10:43:15.355234+0000","last_peered":"2024-02-26T10:43:15.355234+0000","last_clean":"2024-02-26T10:43:15.355234+0000","last_became_active":"2024-02-26T10:42:41.614065+0000","last_became_peered":"2024-02-26T10:42:41.614065+0000","last_unstale":"2024-02-26T10:43:15.355234+0000","last_undegraded":"2024-02-26T10:43:15.355234+0000","last_fullsized":"2024-02-26T10:43:15.355234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:50:20.552340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212358+0000","last_change":"2024-02-26T10:42:42.188140+0000","last_active":"2024-02-26T10:42:48.212358+0000","last_peered":"2024-02-26T10:42:48.212358+0000","last_clean":"2024-02-26T10:42:48.212358+0000","last_became_active":"2024-02-26T10:42:42.187055+0000","last_became_peered":"2024-02-26T10:42:42.187055+0000","last_unstale":"2024-02-26T10:42:48.212358+0000","last_undegraded":"2024-02-26T10:42:48.212358+0000","last_fullsized":"2024-02-26T10:42:48.212358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:49:43.568562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798410+0000","last_change":"2024-02-26T10:42:47.251998+0000","last_active":"2024-02-26T10:42:48.798410+0000","last_peered":"2024-02-26T10:42:48.798410+0000","last_clean":"2024-02-26T10:42:48.798410+0000","last_became_active":"2024-02-26T10:42:47.251574+0000","last_became_peered":"2024-02-26T10:42:47.251574+0000","last_unstale":"2024-02-26T10:42:48.798410+0000","last_undegraded":"2024-02-26T10:42:48.798410+0000","last_fullsized":"2024-02-26T10:42:48.798410+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:07.122542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.343549+0000","last_change":"2024-02-26T10:42:41.612337+0000","last_active":"2024-02-26T10:43:14.343549+0000","last_peered":"2024-02-26T10:43:14.343549+0000","last_clean":"2024-02-26T10:43:14.343549+0000","last_became_active":"2024-02-26T10:42:41.611920+0000","last_became_peered":"2024-02-26T10:42:41.611920+0000","last_unstale":"2024-02-26T10:43:14.343549+0000","last_undegraded":"2024-02-26T10:43:14.343549+0000","last_fullsized":"2024-02-26T10:43:14.343549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:29:13.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798055+0000","last_change":"2024-02-26T10:42:42.171497+0000","last_active":"2024-02-26T10:42:48.798055+0000","last_peered":"2024-02-26T10:42:48.798055+0000","last_clean":"2024-02-26T10:42:48.798055+0000","last_became_active":"2024-02-26T10:42:42.171176+0000","last_became_peered":"2024-02-26T10:42:42.171176+0000","last_unstale":"2024-02-26T10:42:48.798055+0000","last_undegraded":"2024-02-26T10:42:48.798055+0000","last_fullsized":"2024-02-26T10:42:48.798055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:05:41.193025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212442+0000","last_change":"2024-02-26T10:42:47.254585+0000","last_active":"2024-02-26T10:42:48.212442+0000","last_peered":"2024-02-26T10:42:48.212442+0000","last_clean":"2024-02-26T10:42:48.212442+0000","last_became_active":"2024-02-26T10:42:47.253870+0000","last_became_peered":"2024-02-26T10:42:47.253870+0000","last_unstale":"2024-02-26T10:42:48.212442+0000","last_undegraded":"2024-02-26T10:42:48.212442+0000","last_fullsized":"2024-02-26T10:42:48.212442+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:49.276906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:17.044948+0000","last_change":"2024-02-26T10:42:41.600901+0000","last_active":"2024-02-26T10:43:17.044948+0000","last_peered":"2024-02-26T10:43:17.044948+0000","last_clean":"2024-02-26T10:43:17.044948+0000","last_became_active":"2024-02-26T10:42:41.600018+0000","last_became_peered":"2024-02-26T10:42:41.600018+0000","last_unstale":"2024-02-26T10:43:17.044948+0000","last_undegraded":"2024-02-26T10:43:17.044948+0000","last_fullsized":"2024-02-26T10:43:17.044948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:07.876694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798237+0000","last_change":"2024-02-26T10:42:42.168702+0000","last_active":"2024-02-26T10:42:48.798237+0000","last_peered":"2024-02-26T10:42:48.798237+0000","last_clean":"2024-02-26T10:42:48.798237+0000","last_became_active":"2024-02-26T10:42:42.168420+0000","last_became_peered":"2024-02-26T10:42:42.168420+0000","last_unstale":"2024-02-26T10:42:48.798237+0000","last_undegraded":"2024-02-26T10:42:48.798237+0000","last_fullsized":"2024-02-26T10:42:48.798237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:35.111699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213885+0000","last_change":"2024-02-26T10:42:47.250714+0000","last_active":"2024-02-26T10:42:48.213885+0000","last_peered":"2024-02-26T10:42:48.213885+0000","last_clean":"2024-02-26T10:42:48.213885+0000","last_became_active":"2024-02-26T10:42:47.249506+0000","last_became_peered":"2024-02-26T10:42:47.249506+0000","last_unstale":"2024-02-26T10:42:48.213885+0000","last_undegraded":"2024-02-26T10:42:48.213885+0000","last_fullsized":"2024-02-26T10:42:48.213885+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:05.058052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.339670+0000","last_change":"2024-02-26T10:42:41.620841+0000","last_active":"2024-02-26T10:43:14.339670+0000","last_peered":"2024-02-26T10:43:14.339670+0000","last_clean":"2024-02-26T10:43:14.339670+0000","last_became_active":"2024-02-26T10:42:41.620530+0000","last_became_peered":"2024-02-26T10:42:41.620530+0000","last_unstale":"2024-02-26T10:43:14.339670+0000","last_undegraded":"2024-02-26T10:43:14.339670+0000","last_fullsized":"2024-02-26T10:43:14.339670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:51:15.851893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797959+0000","last_change":"2024-02-26T10:42:42.172845+0000","last_active":"2024-02-26T10:42:48.797959+0000","last_peered":"2024-02-26T10:42:48.797959+0000","last_clean":"2024-02-26T10:42:48.797959+0000","last_became_active":"2024-02-26T10:42:42.171263+0000","last_became_peered":"2024-02-26T10:42:42.171263+0000","last_unstale":"2024-02-26T10:42:48.797959+0000","last_undegraded":"2024-02-26T10:42:48.797959+0000","last_fullsized":"2024-02-26T10:42:48.797959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:21.439474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797847+0000","last_change":"2024-02-26T10:42:47.236434+0000","last_active":"2024-02-26T10:42:48.797847+0000","last_peered":"2024-02-26T10:42:48.797847+0000","last_clean":"2024-02-26T10:42:48.797847+0000","last_became_active":"2024-02-26T10:42:47.236040+0000","last_became_peered":"2024-02-26T10:42:47.236040+0000","last_unstale":"2024-02-26T10:42:48.797847+0000","last_undegraded":"2024-02-26T10:42:48.797847+0000","last_fullsized":"2024-02-26T10:42:48.797847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:20:23.230348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798303+0000","last_change":"2024-02-26T10:42:41.604208+0000","last_active":"2024-02-26T10:42:48.798303+0000","last_peered":"2024-02-26T10:42:48.798303+0000","last_clean":"2024-02-26T10:42:48.798303+0000","last_became_active":"2024-02-26T10:42:41.603894+0000","last_became_peered":"2024-02-26T10:42:41.603894+0000","last_unstale":"2024-02-26T10:42:48.798303+0000","last_undegraded":"2024-02-26T10:42:48.798303+0000","last_fullsized":"2024-02-26T10:42:48.798303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:57:13.775868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798751+0000","last_change":"2024-02-26T10:42:42.169395+0000","last_active":"2024-02-26T10:42:48.798751+0000","last_peered":"2024-02-26T10:42:48.798751+0000","last_clean":"2024-02-26T10:42:48.798751+0000","last_became_active":"2024-02-26T10:42:42.168686+0000","last_became_peered":"2024-02-26T10:42:42.168686+0000","last_unstale":"2024-02-26T10:42:48.798751+0000","last_undegraded":"2024-02-26T10:42:48.798751+0000","last_fullsized":"2024-02-26T10:42:48.798751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:21:35.085188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797583+0000","last_change":"2024-02-26T10:42:47.252119+0000","last_active":"2024-02-26T10:42:48.797583+0000","last_peered":"2024-02-26T10:42:48.797583+0000","last_clean":"2024-02-26T10:42:48.797583+0000","last_became_active":"2024-02-26T10:42:47.251723+0000","last_became_peered":"2024-02-26T10:42:47.251723+0000","last_unstale":"2024-02-26T10:42:48.797583+0000","last_undegraded":"2024-02-26T10:42:48.797583+0000","last_fullsized":"2024-02-26T10:42:48.797583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:17.130806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.210414+0000","last_change":"2024-02-26T10:42:41.621393+0000","last_active":"2024-02-26T10:42:48.210414+0000","last_peered":"2024-02-26T10:42:48.210414+0000","last_clean":"2024-02-26T10:42:48.210414+0000","last_became_active":"2024-02-26T10:42:41.621008+0000","last_became_peered":"2024-02-26T10:42:41.621008+0000","last_unstale":"2024-02-26T10:42:48.210414+0000","last_undegraded":"2024-02-26T10:42:48.210414+0000","last_fullsized":"2024-02-26T10:42:48.210414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:46.979854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797989+0000","last_change":"2024-02-26T10:42:42.194296+0000","last_active":"2024-02-26T10:42:48.797989+0000","last_peered":"2024-02-26T10:42:48.797989+0000","last_clean":"2024-02-26T10:42:48.797989+0000","last_became_active":"2024-02-26T10:42:42.193987+0000","last_became_peered":"2024-02-26T10:42:42.193987+0000","last_unstale":"2024-02-26T10:42:48.797989+0000","last_undegraded":"2024-02-26T10:42:48.797989+0000","last_fullsized":"2024-02-26T10:42:48.797989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:35:52.677918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213098+0000","last_change":"2024-02-26T10:42:47.238520+0000","last_active":"2024-02-26T10:42:48.213098+0000","last_peered":"2024-02-26T10:42:48.213098+0000","last_clean":"2024-02-26T10:42:48.213098+0000","last_became_active":"2024-02-26T10:42:47.237213+0000","last_became_peered":"2024-02-26T10:42:47.237213+0000","last_unstale":"2024-02-26T10:42:48.213098+0000","last_undegraded":"2024-02-26T10:42:48.213098+0000","last_fullsized":"2024-02-26T10:42:48.213098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:29.283850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798040+0000","last_change":"2024-02-26T10:42:41.609479+0000","last_active":"2024-02-26T10:42:48.798040+0000","last_peered":"2024-02-26T10:42:48.798040+0000","last_clean":"2024-02-26T10:42:48.798040+0000","last_became_active":"2024-02-26T10:42:41.609084+0000","last_became_peered":"2024-02-26T10:42:41.609084+0000","last_unstale":"2024-02-26T10:42:48.798040+0000","last_undegraded":"2024-02-26T10:42:48.798040+0000","last_fullsized":"2024-02-26T10:42:48.798040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:32.412507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797661+0000","last_change":"2024-02-26T10:42:42.174985+0000","last_active":"2024-02-26T10:42:48.797661+0000","last_peered":"2024-02-26T10:42:48.797661+0000","last_clean":"2024-02-26T10:42:48.797661+0000","last_became_active":"2024-02-26T10:42:42.174694+0000","last_became_peered":"2024-02-26T10:42:42.174694+0000","last_unstale":"2024-02-26T10:42:48.797661+0000","last_undegraded":"2024-02-26T10:42:48.797661+0000","last_fullsized":"2024-02-26T10:42:48.797661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:10.000154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213503+0000","last_change":"2024-02-26T10:42:47.242587+0000","last_active":"2024-02-26T10:42:48.213503+0000","last_peered":"2024-02-26T10:42:48.213503+0000","last_clean":"2024-02-26T10:42:48.213503+0000","last_became_active":"2024-02-26T10:42:47.242099+0000","last_became_peered":"2024-02-26T10:42:47.242099+0000","last_unstale":"2024-02-26T10:42:48.213503+0000","last_undegraded":"2024-02-26T10:42:48.213503+0000","last_fullsized":"2024-02-26T10:42:48.213503+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:38:42.852080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797910+0000","last_change":"2024-02-26T10:42:42.170613+0000","last_active":"2024-02-26T10:42:48.797910+0000","last_peered":"2024-02-26T10:42:48.797910+0000","last_clean":"2024-02-26T10:42:48.797910+0000","last_became_active":"2024-02-26T10:42:42.170317+0000","last_became_peered":"2024-02-26T10:42:42.170317+0000","last_unstale":"2024-02-26T10:42:48.797910+0000","last_undegraded":"2024-02-26T10:42:48.797910+0000","last_fullsized":"2024-02-26T10:42:48.797910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:29:44.276718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:17.044347+0000","last_change":"2024-02-26T10:42:41.617447+0000","last_active":"2024-02-26T10:43:17.044347+0000","last_peered":"2024-02-26T10:43:17.044347+0000","last_clean":"2024-02-26T10:43:17.044347+0000","last_became_active":"2024-02-26T10:42:41.617107+0000","last_became_peered":"2024-02-26T10:42:41.617107+0000","last_unstale":"2024-02-26T10:43:17.044347+0000","last_undegraded":"2024-02-26T10:43:17.044347+0000","last_fullsized":"2024-02-26T10:43:17.044347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:06.052364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798268+0000","last_change":"2024-02-26T10:42:47.251971+0000","last_active":"2024-02-26T10:42:48.798268+0000","last_peered":"2024-02-26T10:42:48.798268+0000","last_clean":"2024-02-26T10:42:48.798268+0000","last_became_active":"2024-02-26T10:42:47.251506+0000","last_became_peered":"2024-02-26T10:42:47.251506+0000","last_unstale":"2024-02-26T10:42:48.798268+0000","last_undegraded":"2024-02-26T10:42:48.798268+0000","last_fullsized":"2024-02-26T10:42:48.798268+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:37.029348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800528+0000","last_change":"2024-02-26T10:42:42.194414+0000","last_active":"2024-02-26T10:42:48.800528+0000","last_peered":"2024-02-26T10:42:48.800528+0000","last_clean":"2024-02-26T10:42:48.800528+0000","last_became_active":"2024-02-26T10:42:42.193959+0000","last_became_peered":"2024-02-26T10:42:42.193959+0000","last_unstale":"2024-02-26T10:42:48.800528+0000","last_undegraded":"2024-02-26T10:42:48.800528+0000","last_fullsized":"2024-02-26T10:42:48.800528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:50.398674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:16.038406+0000","last_change":"2024-02-26T10:42:41.607947+0000","last_active":"2024-02-26T10:43:16.038406+0000","last_peered":"2024-02-26T10:43:16.038406+0000","last_clean":"2024-02-26T10:43:16.038406+0000","last_became_active":"2024-02-26T10:42:41.607522+0000","last_became_peered":"2024-02-26T10:42:41.607522+0000","last_unstale":"2024-02-26T10:43:16.038406+0000","last_undegraded":"2024-02-26T10:43:16.038406+0000","last_fullsized":"2024-02-26T10:43:16.038406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:32:36.647621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798097+0000","last_change":"2024-02-26T10:41:21.177202+0000","last_active":"2024-02-26T10:42:48.798097+0000","last_peered":"2024-02-26T10:42:48.798097+0000","last_clean":"2024-02-26T10:42:48.798097+0000","last_became_active":"2024-02-26T10:41:21.176762+0000","last_became_peered":"2024-02-26T10:41:21.176762+0000","last_unstale":"2024-02-26T10:42:48.798097+0000","last_undegraded":"2024-02-26T10:42:48.798097+0000","last_fullsized":"2024-02-26T10:42:48.798097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_clean_scrub_stamp":"2024-02-26T10:37:25.424790+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:19.619104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798723+0000","last_change":"2024-02-26T10:42:47.244346+0000","last_active":"2024-02-26T10:42:48.798723+0000","last_peered":"2024-02-26T10:42:48.798723+0000","last_clean":"2024-02-26T10:42:48.798723+0000","last_became_active":"2024-02-26T10:42:47.243872+0000","last_became_peered":"2024-02-26T10:42:47.243872+0000","last_unstale":"2024-02-26T10:42:48.798723+0000","last_undegraded":"2024-02-26T10:42:48.798723+0000","last_fullsized":"2024-02-26T10:42:48.798723+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:36.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.211674+0000","last_change":"2024-02-26T10:42:42.166266+0000","last_active":"2024-02-26T10:42:48.211674+0000","last_peered":"2024-02-26T10:42:48.211674+0000","last_clean":"2024-02-26T10:42:48.211674+0000","last_became_active":"2024-02-26T10:42:42.165721+0000","last_became_peered":"2024-02-26T10:42:42.165721+0000","last_unstale":"2024-02-26T10:42:48.211674+0000","last_undegraded":"2024-02-26T10:42:48.211674+0000","last_fullsized":"2024-02-26T10:42:48.211674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:32.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798192+0000","last_change":"2024-02-26T10:42:41.606268+0000","last_active":"2024-02-26T10:42:48.798192+0000","last_peered":"2024-02-26T10:42:48.798192+0000","last_clean":"2024-02-26T10:42:48.798192+0000","last_became_active":"2024-02-26T10:42:41.605950+0000","last_became_peered":"2024-02-26T10:42:41.605950+0000","last_unstale":"2024-02-26T10:42:48.798192+0000","last_undegraded":"2024-02-26T10:42:48.798192+0000","last_fullsized":"2024-02-26T10:42:48.798192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:40:03.616987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798718+0000","last_change":"2024-02-26T10:42:47.233203+0000","last_active":"2024-02-26T10:42:48.798718+0000","last_peered":"2024-02-26T10:42:48.798718+0000","last_clean":"2024-02-26T10:42:48.798718+0000","last_became_active":"2024-02-26T10:42:47.232705+0000","last_became_peered":"2024-02-26T10:42:47.232705+0000","last_unstale":"2024-02-26T10:42:48.798718+0000","last_undegraded":"2024-02-26T10:42:48.798718+0000","last_fullsized":"2024-02-26T10:42:48.798718+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:53.818901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798521+0000","last_change":"2024-02-26T10:42:41.623086+0000","last_active":"2024-02-26T10:42:48.798521+0000","last_peered":"2024-02-26T10:42:48.798521+0000","last_clean":"2024-02-26T10:42:48.798521+0000","last_became_active":"2024-02-26T10:42:41.622813+0000","last_became_peered":"2024-02-26T10:42:41.622813+0000","last_unstale":"2024-02-26T10:42:48.798521+0000","last_undegraded":"2024-02-26T10:42:48.798521+0000","last_fullsized":"2024-02-26T10:42:48.798521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:49:41.928829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213451+0000","last_change":"2024-02-26T10:42:42.167674+0000","last_active":"2024-02-26T10:42:48.213451+0000","last_peered":"2024-02-26T10:42:48.213451+0000","last_clean":"2024-02-26T10:42:48.213451+0000","last_became_active":"2024-02-26T10:42:42.167224+0000","last_became_peered":"2024-02-26T10:42:42.167224+0000","last_unstale":"2024-02-26T10:42:48.213451+0000","last_undegraded":"2024-02-26T10:42:48.213451+0000","last_fullsized":"2024-02-26T10:42:48.213451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:49:20.433913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799407+0000","last_change":"2024-02-26T10:42:47.244076+0000","last_active":"2024-02-26T10:42:48.799407+0000","last_peered":"2024-02-26T10:42:48.799407+0000","last_clean":"2024-02-26T10:42:48.799407+0000","last_became_active":"2024-02-26T10:42:47.243569+0000","last_became_peered":"2024-02-26T10:42:47.243569+0000","last_unstale":"2024-02-26T10:42:48.799407+0000","last_undegraded":"2024-02-26T10:42:48.799407+0000","last_fullsized":"2024-02-26T10:42:48.799407+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:46:19.855621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212742+0000","last_change":"2024-02-26T10:42:41.623604+0000","last_active":"2024-02-26T10:42:48.212742+0000","last_peered":"2024-02-26T10:42:48.212742+0000","last_clean":"2024-02-26T10:42:48.212742+0000","last_became_active":"2024-02-26T10:42:41.623272+0000","last_became_peered":"2024-02-26T10:42:41.623272+0000","last_unstale":"2024-02-26T10:42:48.212742+0000","last_undegraded":"2024-02-26T10:42:48.212742+0000","last_fullsized":"2024-02-26T10:42:48.212742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:13:57.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801436+0000","last_change":"2024-02-26T10:42:42.186300+0000","last_active":"2024-02-26T10:42:48.801436+0000","last_peered":"2024-02-26T10:42:48.801436+0000","last_clean":"2024-02-26T10:42:48.801436+0000","last_became_active":"2024-02-26T10:42:42.185965+0000","last_became_peered":"2024-02-26T10:42:42.185965+0000","last_unstale":"2024-02-26T10:42:48.801436+0000","last_undegraded":"2024-02-26T10:42:48.801436+0000","last_fullsized":"2024-02-26T10:42:48.801436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:06:50.352308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209143+0000","last_change":"2024-02-26T10:42:47.227753+0000","last_active":"2024-02-26T10:42:48.209143+0000","last_peered":"2024-02-26T10:42:48.209143+0000","last_clean":"2024-02-26T10:42:48.209143+0000","last_became_active":"2024-02-26T10:42:47.227300+0000","last_became_peered":"2024-02-26T10:42:47.227300+0000","last_unstale":"2024-02-26T10:42:48.209143+0000","last_undegraded":"2024-02-26T10:42:48.209143+0000","last_fullsized":"2024-02-26T10:42:48.209143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:08.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.342146+0000","last_change":"2024-02-26T10:42:41.602960+0000","last_active":"2024-02-26T10:43:14.342146+0000","last_peered":"2024-02-26T10:43:14.342146+0000","last_clean":"2024-02-26T10:43:14.342146+0000","last_became_active":"2024-02-26T10:42:41.602676+0000","last_became_peered":"2024-02-26T10:42:41.602676+0000","last_unstale":"2024-02-26T10:43:14.342146+0000","last_undegraded":"2024-02-26T10:43:14.342146+0000","last_fullsized":"2024-02-26T10:43:14.342146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:57.997612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799808+0000","last_change":"2024-02-26T10:42:42.185451+0000","last_active":"2024-02-26T10:42:48.799808+0000","last_peered":"2024-02-26T10:42:48.799808+0000","last_clean":"2024-02-26T10:42:48.799808+0000","last_became_active":"2024-02-26T10:42:42.185116+0000","last_became_peered":"2024-02-26T10:42:42.185116+0000","last_unstale":"2024-02-26T10:42:48.799808+0000","last_undegraded":"2024-02-26T10:42:48.799808+0000","last_fullsized":"2024-02-26T10:42:48.799808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:43:39.823370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518143+0000","last_change":"2024-02-26T10:42:47.250682+0000","last_active":"2024-02-26T10:42:48.518143+0000","last_peered":"2024-02-26T10:42:48.518143+0000","last_clean":"2024-02-26T10:42:48.518143+0000","last_became_active":"2024-02-26T10:42:47.249524+0000","last_became_peered":"2024-02-26T10:42:47.249524+0000","last_unstale":"2024-02-26T10:42:48.518143+0000","last_undegraded":"2024-02-26T10:42:48.518143+0000","last_fullsized":"2024-02-26T10:42:48.518143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:07:01.307206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.340265+0000","last_change":"2024-02-26T10:42:41.620764+0000","last_active":"2024-02-26T10:43:14.340265+0000","last_peered":"2024-02-26T10:43:14.340265+0000","last_clean":"2024-02-26T10:43:14.340265+0000","last_became_active":"2024-02-26T10:42:41.620500+0000","last_became_peered":"2024-02-26T10:42:41.620500+0000","last_unstale":"2024-02-26T10:43:14.340265+0000","last_undegraded":"2024-02-26T10:43:14.340265+0000","last_fullsized":"2024-02-26T10:43:14.340265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:15.542396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797244+0000","last_change":"2024-02-26T10:42:42.165060+0000","last_active":"2024-02-26T10:42:48.797244+0000","last_peered":"2024-02-26T10:42:48.797244+0000","last_clean":"2024-02-26T10:42:48.797244+0000","last_became_active":"2024-02-26T10:42:42.164733+0000","last_became_peered":"2024-02-26T10:42:42.164733+0000","last_unstale":"2024-02-26T10:42:48.797244+0000","last_undegraded":"2024-02-26T10:42:48.797244+0000","last_fullsized":"2024-02-26T10:42:48.797244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:34.190747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213000+0000","last_change":"2024-02-26T10:42:47.246110+0000","last_active":"2024-02-26T10:42:48.213000+0000","last_peered":"2024-02-26T10:42:48.213000+0000","last_clean":"2024-02-26T10:42:48.213000+0000","last_became_active":"2024-02-26T10:42:47.245630+0000","last_became_peered":"2024-02-26T10:42:47.245630+0000","last_unstale":"2024-02-26T10:42:48.213000+0000","last_undegraded":"2024-02-26T10:42:48.213000+0000","last_fullsized":"2024-02-26T10:42:48.213000+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:54:19.108602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.345234+0000","last_change":"2024-02-26T10:42:41.624940+0000","last_active":"2024-02-26T10:43:14.345234+0000","last_peered":"2024-02-26T10:43:14.345234+0000","last_clean":"2024-02-26T10:43:14.345234+0000","last_became_active":"2024-02-26T10:42:41.624538+0000","last_became_peered":"2024-02-26T10:42:41.624538+0000","last_unstale":"2024-02-26T10:43:14.345234+0000","last_undegraded":"2024-02-26T10:43:14.345234+0000","last_fullsized":"2024-02-26T10:43:14.345234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:10.052806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798026+0000","last_change":"2024-02-26T10:42:42.174932+0000","last_active":"2024-02-26T10:42:48.798026+0000","last_peered":"2024-02-26T10:42:48.798026+0000","last_clean":"2024-02-26T10:42:48.798026+0000","last_became_active":"2024-02-26T10:42:42.174566+0000","last_became_peered":"2024-02-26T10:42:42.174566+0000","last_unstale":"2024-02-26T10:42:48.798026+0000","last_undegraded":"2024-02-26T10:42:48.798026+0000","last_fullsized":"2024-02-26T10:42:48.798026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:03.242487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.517970+0000","last_change":"2024-02-26T10:42:47.250023+0000","last_active":"2024-02-26T10:42:48.517970+0000","last_peered":"2024-02-26T10:42:48.517970+0000","last_clean":"2024-02-26T10:42:48.517970+0000","last_became_active":"2024-02-26T10:42:47.249571+0000","last_became_peered":"2024-02-26T10:42:47.249571+0000","last_unstale":"2024-02-26T10:42:48.517970+0000","last_undegraded":"2024-02-26T10:42:48.517970+0000","last_fullsized":"2024-02-26T10:42:48.517970+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:53.666184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209074+0000","last_change":"2024-02-26T10:42:41.614403+0000","last_active":"2024-02-26T10:42:48.209074+0000","last_peered":"2024-02-26T10:42:48.209074+0000","last_clean":"2024-02-26T10:42:48.209074+0000","last_became_active":"2024-02-26T10:42:41.614057+0000","last_became_peered":"2024-02-26T10:42:41.614057+0000","last_unstale":"2024-02-26T10:42:48.209074+0000","last_undegraded":"2024-02-26T10:42:48.209074+0000","last_fullsized":"2024-02-26T10:42:48.209074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:16:54.739090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801000+0000","last_change":"2024-02-26T10:42:42.187465+0000","last_active":"2024-02-26T10:42:48.801000+0000","last_peered":"2024-02-26T10:42:48.801000+0000","last_clean":"2024-02-26T10:42:48.801000+0000","last_became_active":"2024-02-26T10:42:42.182310+0000","last_became_peered":"2024-02-26T10:42:42.182310+0000","last_unstale":"2024-02-26T10:42:48.801000+0000","last_undegraded":"2024-02-26T10:42:48.801000+0000","last_fullsized":"2024-02-26T10:42:48.801000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:01:56.645330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.518070+0000","last_change":"2024-02-26T10:42:47.241751+0000","last_active":"2024-02-26T10:42:48.518070+0000","last_peered":"2024-02-26T10:42:48.518070+0000","last_clean":"2024-02-26T10:42:48.518070+0000","last_became_active":"2024-02-26T10:42:47.241271+0000","last_became_peered":"2024-02-26T10:42:47.241271+0000","last_unstale":"2024-02-26T10:42:48.518070+0000","last_undegraded":"2024-02-26T10:42:48.518070+0000","last_fullsized":"2024-02-26T10:42:48.518070+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:40:39.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:14.339681+0000","last_change":"2024-02-26T10:42:41.607243+0000","last_active":"2024-02-26T10:43:14.339681+0000","last_peered":"2024-02-26T10:43:14.339681+0000","last_clean":"2024-02-26T10:43:14.339681+0000","last_became_active":"2024-02-26T10:42:41.606921+0000","last_became_peered":"2024-02-26T10:42:41.606921+0000","last_unstale":"2024-02-26T10:43:14.339681+0000","last_undegraded":"2024-02-26T10:43:14.339681+0000","last_fullsized":"2024-02-26T10:43:14.339681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:04.626391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798186+0000","last_change":"2024-02-26T10:42:42.178240+0000","last_active":"2024-02-26T10:42:48.798186+0000","last_peered":"2024-02-26T10:42:48.798186+0000","last_clean":"2024-02-26T10:42:48.798186+0000","last_became_active":"2024-02-26T10:42:42.177790+0000","last_became_peered":"2024-02-26T10:42:42.177790+0000","last_unstale":"2024-02-26T10:42:48.798186+0000","last_undegraded":"2024-02-26T10:42:48.798186+0000","last_fullsized":"2024-02-26T10:42:48.798186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:24.110406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799122+0000","last_change":"2024-02-26T10:42:47.233120+0000","last_active":"2024-02-26T10:42:48.799122+0000","last_peered":"2024-02-26T10:42:48.799122+0000","last_clean":"2024-02-26T10:42:48.799122+0000","last_became_active":"2024-02-26T10:42:47.232630+0000","last_became_peered":"2024-02-26T10:42:47.232630+0000","last_unstale":"2024-02-26T10:42:48.799122+0000","last_undegraded":"2024-02-26T10:42:48.799122+0000","last_fullsized":"2024-02-26T10:42:48.799122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:05:03.023227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:43:15.356910+0000","last_change":"2024-02-26T10:42:41.621269+0000","last_active":"2024-02-26T10:43:15.356910+0000","last_peered":"2024-02-26T10:43:15.356910+0000","last_clean":"2024-02-26T10:43:15.356910+0000","last_became_active":"2024-02-26T10:42:41.620923+0000","last_became_peered":"2024-02-26T10:42:41.620923+0000","last_unstale":"2024-02-26T10:43:15.356910+0000","last_undegraded":"2024-02-26T10:43:15.356910+0000","last_fullsized":"2024-02-26T10:43:15.356910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:06:25.650313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798425+0000","last_change":"2024-02-26T10:42:42.171821+0000","last_active":"2024-02-26T10:42:48.798425+0000","last_peered":"2024-02-26T10:42:48.798425+0000","last_clean":"2024-02-26T10:42:48.798425+0000","last_became_active":"2024-02-26T10:42:42.171523+0000","last_became_peered":"2024-02-26T10:42:42.171523+0000","last_unstale":"2024-02-26T10:42:48.798425+0000","last_undegraded":"2024-02-26T10:42:48.798425+0000","last_fullsized":"2024-02-26T10:42:48.798425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:00.877998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799776+0000","last_change":"2024-02-26T10:42:47.250072+0000","last_active":"2024-02-26T10:42:48.799776+0000","last_peered":"2024-02-26T10:42:48.799776+0000","last_clean":"2024-02-26T10:42:48.799776+0000","last_became_active":"2024-02-26T10:42:47.249570+0000","last_became_peered":"2024-02-26T10:42:47.249570+0000","last_unstale":"2024-02-26T10:42:48.799776+0000","last_undegraded":"2024-02-26T10:42:48.799776+0000","last_fullsized":"2024-02-26T10:42:48.799776+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:43:03.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798199+0000","last_change":"2024-02-26T10:42:41.614686+0000","last_active":"2024-02-26T10:42:48.798199+0000","last_peered":"2024-02-26T10:42:48.798199+0000","last_clean":"2024-02-26T10:42:48.798199+0000","last_became_active":"2024-02-26T10:42:41.614019+0000","last_became_peered":"2024-02-26T10:42:41.614019+0000","last_unstale":"2024-02-26T10:42:48.798199+0000","last_undegraded":"2024-02-26T10:42:48.798199+0000","last_fullsized":"2024-02-26T10:42:48.798199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:13:28.127288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797634+0000","last_change":"2024-02-26T10:42:42.181774+0000","last_active":"2024-02-26T10:42:48.797634+0000","last_peered":"2024-02-26T10:42:48.797634+0000","last_clean":"2024-02-26T10:42:48.797634+0000","last_became_active":"2024-02-26T10:42:42.181412+0000","last_became_peered":"2024-02-26T10:42:42.181412+0000","last_unstale":"2024-02-26T10:42:48.797634+0000","last_undegraded":"2024-02-26T10:42:48.797634+0000","last_fullsized":"2024-02-26T10:42:48.797634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:09.159964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798329+0000","last_change":"2024-02-26T10:42:47.228247+0000","last_active":"2024-02-26T10:42:48.798329+0000","last_peered":"2024-02-26T10:42:48.798329+0000","last_clean":"2024-02-26T10:42:48.798329+0000","last_became_active":"2024-02-26T10:42:47.227569+0000","last_became_peered":"2024-02-26T10:42:47.227569+0000","last_unstale":"2024-02-26T10:42:48.798329+0000","last_undegraded":"2024-02-26T10:42:48.798329+0000","last_fullsized":"2024-02-26T10:42:48.798329+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:15.001264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213970+0000","last_change":"2024-02-26T10:42:41.626573+0000","last_active":"2024-02-26T10:42:48.213970+0000","last_peered":"2024-02-26T10:42:48.213970+0000","last_clean":"2024-02-26T10:42:48.213970+0000","last_became_active":"2024-02-26T10:42:41.624091+0000","last_became_peered":"2024-02-26T10:42:41.624091+0000","last_unstale":"2024-02-26T10:42:48.213970+0000","last_undegraded":"2024-02-26T10:42:48.213970+0000","last_fullsized":"2024-02-26T10:42:48.213970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:24:14.077234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797993+0000","last_change":"2024-02-26T10:42:42.166426+0000","last_active":"2024-02-26T10:42:48.797993+0000","last_peered":"2024-02-26T10:42:48.797993+0000","last_clean":"2024-02-26T10:42:48.797993+0000","last_became_active":"2024-02-26T10:42:42.166082+0000","last_became_peered":"2024-02-26T10:42:42.166082+0000","last_unstale":"2024-02-26T10:42:48.797993+0000","last_undegraded":"2024-02-26T10:42:48.797993+0000","last_fullsized":"2024-02-26T10:42:48.797993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:17.136568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798136+0000","last_change":"2024-02-26T10:42:42.168227+0000","last_active":"2024-02-26T10:42:48.798136+0000","last_peered":"2024-02-26T10:42:48.798136+0000","last_clean":"2024-02-26T10:42:48.798136+0000","last_became_active":"2024-02-26T10:42:42.167897+0000","last_became_peered":"2024-02-26T10:42:42.167897+0000","last_unstale":"2024-02-26T10:42:48.798136+0000","last_undegraded":"2024-02-26T10:42:48.798136+0000","last_fullsized":"2024-02-26T10:42:48.798136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:52:37.391022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209260+0000","last_change":"2024-02-26T10:42:42.168640+0000","last_active":"2024-02-26T10:42:48.209260+0000","last_peered":"2024-02-26T10:42:48.209260+0000","last_clean":"2024-02-26T10:42:48.209260+0000","last_became_active":"2024-02-26T10:42:42.168237+0000","last_became_peered":"2024-02-26T10:42:42.168237+0000","last_unstale":"2024-02-26T10:42:48.209260+0000","last_undegraded":"2024-02-26T10:42:48.209260+0000","last_fullsized":"2024-02-26T10:42:48.209260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:14.853730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.208878+0000","last_change":"2024-02-26T10:42:42.192703+0000","last_active":"2024-02-26T10:42:48.208878+0000","last_peered":"2024-02-26T10:42:48.208878+0000","last_clean":"2024-02-26T10:42:48.208878+0000","last_became_active":"2024-02-26T10:42:42.191899+0000","last_became_peered":"2024-02-26T10:42:42.191899+0000","last_unstale":"2024-02-26T10:42:48.208878+0000","last_undegraded":"2024-02-26T10:42:48.208878+0000","last_fullsized":"2024-02-26T10:42:48.208878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:40.479523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799926+0000","last_change":"2024-02-26T10:42:42.188709+0000","last_active":"2024-02-26T10:42:48.799926+0000","last_peered":"2024-02-26T10:42:48.799926+0000","last_clean":"2024-02-26T10:42:48.799926+0000","last_became_active":"2024-02-26T10:42:42.188418+0000","last_became_peered":"2024-02-26T10:42:42.188418+0000","last_unstale":"2024-02-26T10:42:48.799926+0000","last_undegraded":"2024-02-26T10:42:48.799926+0000","last_fullsized":"2024-02-26T10:42:48.799926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:54:15.697746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.213288+0000","last_change":"2024-02-26T10:42:42.174221+0000","last_active":"2024-02-26T10:42:48.213288+0000","last_peered":"2024-02-26T10:42:48.213288+0000","last_clean":"2024-02-26T10:42:48.213288+0000","last_became_active":"2024-02-26T10:42:42.171308+0000","last_became_peered":"2024-02-26T10:42:42.171308+0000","last_unstale":"2024-02-26T10:42:48.213288+0000","last_undegraded":"2024-02-26T10:42:48.213288+0000","last_fullsized":"2024-02-26T10:42:48.213288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:46.765856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798848+0000","last_change":"2024-02-26T10:42:42.170749+0000","last_active":"2024-02-26T10:42:48.798848+0000","last_peered":"2024-02-26T10:42:48.798848+0000","last_clean":"2024-02-26T10:42:48.798848+0000","last_became_active":"2024-02-26T10:42:42.169966+0000","last_became_peered":"2024-02-26T10:42:42.169966+0000","last_unstale":"2024-02-26T10:42:48.798848+0000","last_undegraded":"2024-02-26T10:42:48.798848+0000","last_fullsized":"2024-02-26T10:42:48.798848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:02.633195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.209608+0000","last_change":"2024-02-26T10:42:42.166529+0000","last_active":"2024-02-26T10:42:48.209608+0000","last_peered":"2024-02-26T10:42:48.209608+0000","last_clean":"2024-02-26T10:42:48.209608+0000","last_became_active":"2024-02-26T10:42:42.166235+0000","last_became_peered":"2024-02-26T10:42:42.166235+0000","last_unstale":"2024-02-26T10:42:48.209608+0000","last_undegraded":"2024-02-26T10:42:48.209608+0000","last_fullsized":"2024-02-26T10:42:48.209608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:03.068328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.801713+0000","last_change":"2024-02-26T10:42:42.191312+0000","last_active":"2024-02-26T10:42:48.801713+0000","last_peered":"2024-02-26T10:42:48.801713+0000","last_clean":"2024-02-26T10:42:48.801713+0000","last_became_active":"2024-02-26T10:42:42.190964+0000","last_became_peered":"2024-02-26T10:42:42.190964+0000","last_unstale":"2024-02-26T10:42:48.801713+0000","last_undegraded":"2024-02-26T10:42:48.801713+0000","last_fullsized":"2024-02-26T10:42:48.801713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:24:04.697129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.797798+0000","last_change":"2024-02-26T10:42:42.182882+0000","last_active":"2024-02-26T10:42:48.797798+0000","last_peered":"2024-02-26T10:42:48.797798+0000","last_clean":"2024-02-26T10:42:48.797798+0000","last_became_active":"2024-02-26T10:42:42.182551+0000","last_became_peered":"2024-02-26T10:42:42.182551+0000","last_unstale":"2024-02-26T10:42:48.797798+0000","last_undegraded":"2024-02-26T10:42:48.797798+0000","last_fullsized":"2024-02-26T10:42:48.797798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:11:58.578832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.800871+0000","last_change":"2024-02-26T10:42:42.193353+0000","last_active":"2024-02-26T10:42:48.800871+0000","last_peered":"2024-02-26T10:42:48.800871+0000","last_clean":"2024-02-26T10:42:48.800871+0000","last_became_active":"2024-02-26T10:42:42.190935+0000","last_became_peered":"2024-02-26T10:42:42.190935+0000","last_unstale":"2024-02-26T10:42:48.800871+0000","last_undegraded":"2024-02-26T10:42:48.800871+0000","last_fullsized":"2024-02-26T10:42:48.800871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:03:14.319769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.799191+0000","last_change":"2024-02-26T10:42:42.194529+0000","last_active":"2024-02-26T10:42:48.799191+0000","last_peered":"2024-02-26T10:42:48.799191+0000","last_clean":"2024-02-26T10:42:48.799191+0000","last_became_active":"2024-02-26T10:42:42.194212+0000","last_became_peered":"2024-02-26T10:42:42.194212+0000","last_unstale":"2024-02-26T10:42:48.799191+0000","last_undegraded":"2024-02-26T10:42:48.799191+0000","last_fullsized":"2024-02-26T10:42:48.799191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:36.876672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.212786+0000","last_change":"2024-02-26T10:42:42.178119+0000","last_active":"2024-02-26T10:42:48.212786+0000","last_peered":"2024-02-26T10:42:48.212786+0000","last_clean":"2024-02-26T10:42:48.212786+0000","last_became_active":"2024-02-26T10:42:42.177798+0000","last_became_peered":"2024-02-26T10:42:42.177798+0000","last_unstale":"2024-02-26T10:42:48.212786+0000","last_undegraded":"2024-02-26T10:42:48.212786+0000","last_fullsized":"2024-02-26T10:42:48.212786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:52:29.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T10:42:48.798471+0000","last_change":"2024-02-26T10:42:42.167380+0000","last_active":"2024-02-26T10:42:48.798471+0000","last_peered":"2024-02-26T10:42:48.798471+0000","last_clean":"2024-02-26T10:42:48.798471+0000","last_became_active":"2024-02-26T10:42:42.167079+0000","last_became_peered":"2024-02-26T10:42:42.167079+0000","last_unstale":"2024-02-26T10:42:48.798471+0000","last_undegraded":"2024-02-26T10:42:48.798471+0000","last_fullsized":"2024-02-26T10:42:48.798471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:14:25.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250986,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41696,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":33,"num_read_kb":47,"num_write":121,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":93067,"data_compressed":3664,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":489888,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612634,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1105366,"data_compressed":51052,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":490672,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037798,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":490514,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1080838,"data_compressed":51052,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462961,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":489978,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":493730,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":489978,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":533144,"data_compressed":16560,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":490172,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":490923,"data_compressed":14728,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1085828,"data_compressed":51052,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":43256,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":39776,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:43:20.682 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T10:43:20.682 INFO:tasks.cephadm:Setting up client nodes... 2024-02-26T10:43:20.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-26T10:43:20.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:20 smithi113 ceph-mon[36126]: pgmap v342: 129 pgs: 129 active+clean; 617 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-02-26T10:43:20.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:20 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1636377300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:21.208 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:21.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:21 smithi060 ceph-mon[31539]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:43:21.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:21 smithi060 ceph-mon[31539]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:43:21.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:21 smithi100 ceph-mon[36051]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:43:21.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:21 smithi100 ceph-mon[36051]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:43:21.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:21 smithi113 ceph-mon[36126]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:43:21.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:21 smithi113 ceph-mon[36126]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:43:22.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:22 smithi060 ceph-mon[31539]: pgmap v343: 129 pgs: 129 active+clean; 617 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-02-26T10:43:22.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:22 smithi100 ceph-mon[36051]: pgmap v343: 129 pgs: 129 active+clean; 617 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-02-26T10:43:22.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:22 smithi113 ceph-mon[36126]: pgmap v343: 129 pgs: 129 active+clean; 617 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-02-26T10:43:23.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.610+0000 7f67c1800700 1 -- 172.21.15.60:0/4185143261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc100c70 msgr2=0x7f67bc101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:23.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.610+0000 7f67c1800700 1 --2- 172.21.15.60:0/4185143261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc100c70 0x7f67bc101090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0099c0 tx=0x7f67ac009cd0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.610+0000 7f67c1800700 1 -- 172.21.15.60:0/4185143261 shutdown_connections 2024-02-26T10:43:23.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.610+0000 7f67c1800700 1 --2- 172.21.15.60:0/4185143261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc102810 0x7f67bc1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.610+0000 7f67c1800700 1 --2- 172.21.15.60:0/4185143261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc100c70 0x7f67bc101090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.610+0000 7f67c1800700 1 --2- 172.21.15.60:0/4185143261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67bc101e70 0x7f67bc1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.610+0000 7f67c1800700 1 -- 172.21.15.60:0/4185143261 >> 172.21.15.60:0/4185143261 conn(0x7f67bc0fc210 msgr2=0x7f67bc0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:23.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.610+0000 7f67c1800700 1 -- 172.21.15.60:0/4185143261 shutdown_connections 2024-02-26T10:43:23.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.610+0000 7f67c1800700 1 -- 172.21.15.60:0/4185143261 wait complete. 2024-02-26T10:43:23.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.612+0000 7f67c1800700 1 Processor -- start 2024-02-26T10:43:23.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.612+0000 7f67c1800700 1 -- start start 2024-02-26T10:43:23.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.612+0000 7f67c1800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc100c70 0x7f67bc104c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:23.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.612+0000 7f67c1800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67bc101e70 0x7f67bc103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:23.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.612+0000 7f67c1800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc102810 0x7f67bc103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:23.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.612+0000 7f67c1800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc10c880 con 0x7f67bc102810 2024-02-26T10:43:23.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.612+0000 7f67c1800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc10c700 con 0x7f67bc100c70 2024-02-26T10:43:23.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.612+0000 7f67c1800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc10ca00 con 0x7f67bc101e70 2024-02-26T10:43:23.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67baa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc100c70 0x7f67bc104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:23.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67bb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc102810 0x7f67bc103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:23.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67baa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc100c70 0x7f67bc104c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37984/0 (socket says 172.21.15.60:37984) 2024-02-26T10:43:23.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67baa00700 1 -- 172.21.15.60:0/2347227923 learned_addr learned my addr 172.21.15.60:0/2347227923 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:23.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67ba000700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67bc101e70 0x7f67bc103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:23.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67baa00700 1 -- 172.21.15.60:0/2347227923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67bc101e70 msgr2=0x7f67bc103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67baa00700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67bc101e70 0x7f67bc103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67baa00700 1 -- 172.21.15.60:0/2347227923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc102810 msgr2=0x7f67bc103790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67baa00700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc102810 0x7f67bc103790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67baa00700 1 -- 172.21.15.60:0/2347227923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc104040 con 0x7f67bc100c70 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.613+0000 7f67baa00700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc100c70 0x7f67bc104c00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b580 tx=0x7f67ac00bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.614+0000 7f67b3400700 1 -- 172.21.15.60:0/2347227923 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac009e60 con 0x7f67bc100c70 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.614+0000 7f67b3400700 1 -- 172.21.15.60:0/2347227923 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67ac003e30 con 0x7f67bc100c70 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.614+0000 7f67b3400700 1 -- 172.21.15.60:0/2347227923 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0173f0 con 0x7f67bc100c70 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.614+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc1a2020 con 0x7f67bc100c70 2024-02-26T10:43:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.614+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc1a2420 con 0x7f67bc100c70 2024-02-26T10:43:23.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.615+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc050260 con 0x7f67bc100c70 2024-02-26T10:43:23.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.617+0000 7f67b3400700 1 -- 172.21.15.60:0/2347227923 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f67ac00f680 con 0x7f67bc100c70 2024-02-26T10:43:23.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.618+0000 7f67b3400700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f67a00a1fd0 0x7f67a00a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:23.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.619+0000 7f67b3400700 1 -- 172.21.15.60:0/2347227923 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f67ac0ceef0 con 0x7f67bc100c70 2024-02-26T10:43:23.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.619+0000 7f67ba000700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f67a00a1fd0 0x7f67a00a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:23.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.619+0000 7f67ba000700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f67a00a1fd0 0x7f67a00a4490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009c60 tx=0x7f67a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:23.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.623+0000 7f67b3400700 1 -- 172.21.15.60:0/2347227923 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67ac013070 con 0x7f67bc100c70 2024-02-26T10:43:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.940+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f67bc1a26a0 con 0x7f67bc100c70 2024-02-26T10:43:23.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.947+0000 7f67b3400700 1 -- 172.21.15.60:0/2347227923 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f67ac09bfb0 con 0x7f67bc100c70 2024-02-26T10:43:23.944 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-02-26T10:43:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQBLa9xlbWErOBAADqc9xSMyclgl7DAGdsI1OQ== 2024-02-26T10:43:23.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f67a00a1fd0 msgr2=0x7f67a00a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:23.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f67a00a1fd0 0x7f67a00a4490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009c60 tx=0x7f67a8009260 comp rx=0 tx=0).stop 2024-02-26T10:43:23.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc100c70 msgr2=0x7f67bc104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:23.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc100c70 0x7f67bc104c00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b580 tx=0x7f67ac00bee0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 shutdown_connections 2024-02-26T10:43:23.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f67a00a1fd0 0x7f67a00a4490 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc102810 0x7f67bc103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc100c70 0x7f67bc104c00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 --2- 172.21.15.60:0/2347227923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67bc101e70 0x7f67bc103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:23.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 >> 172.21.15.60:0/2347227923 conn(0x7f67bc0fc210 msgr2=0x7f67bc0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:23.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 shutdown_connections 2024-02-26T10:43:23.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:23.951+0000 7f67c1800700 1 -- 172.21.15.60:0/2347227923 wait complete. 2024-02-26T10:43:24.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:24 smithi060 ceph-mon[31539]: pgmap v344: 129 pgs: 129 active+clean; 617 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-02-26T10:43:24.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:24 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2347227923' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T10:43:24.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:24 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T10:43:24.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:24 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T10:43:24.789 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:43:24.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-26T10:43:24.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-26T10:43:24.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:24 smithi113 ceph-mon[36126]: pgmap v344: 129 pgs: 129 active+clean; 617 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-02-26T10:43:24.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:24 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2347227923' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T10:43:24.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:24 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T10:43:24.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:24 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T10:43:24.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:24 smithi100 ceph-mon[36051]: pgmap v344: 129 pgs: 129 active+clean; 617 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-02-26T10:43:24.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:24 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2347227923' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T10:43:24.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:24 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T10:43:24.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:24 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T10:43:24.947 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-26T10:43:24.947 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-26T10:43:24.947 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph mgr dump --format=json 2024-02-26T10:43:25.469 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:26.696 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:26 smithi060 ceph-mon[31539]: pgmap v345: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-02-26T10:43:26.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:26 smithi113 ceph-mon[36126]: pgmap v345: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-02-26T10:43:26.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:26 smithi100 ceph-mon[36051]: pgmap v345: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-02-26T10:43:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.788+0000 7f0934a00700 1 -- 172.21.15.60:0/1128797921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 msgr2=0x7f0930108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.788+0000 7f0934a00700 1 --2- 172.21.15.60:0/1128797921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 0x7f0930108b80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f091c009990 tx=0x7f091c009ca0 comp rx=0 tx=0).stop 2024-02-26T10:43:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.788+0000 7f0934a00700 1 -- 172.21.15.60:0/1128797921 shutdown_connections 2024-02-26T10:43:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.789+0000 7f0934a00700 1 --2- 172.21.15.60:0/1128797921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930109260 0x7f0930110e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.789+0000 7f0934a00700 1 --2- 172.21.15.60:0/1128797921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0930104180 0x7f0930104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.789+0000 7f0934a00700 1 --2- 172.21.15.60:0/1128797921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 0x7f0930108b80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.789+0000 7f0934a00700 1 -- 172.21.15.60:0/1128797921 >> 172.21.15.60:0/1128797921 conn(0x7f09300ffa30 msgr2=0x7f0930101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:27.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.789+0000 7f0934a00700 1 -- 172.21.15.60:0/1128797921 shutdown_connections 2024-02-26T10:43:27.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.789+0000 7f0934a00700 1 -- 172.21.15.60:0/1128797921 wait complete. 2024-02-26T10:43:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.790+0000 7f0934a00700 1 Processor -- start 2024-02-26T10:43:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.790+0000 7f0934a00700 1 -- start start 2024-02-26T10:43:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.790+0000 7f0934a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930104180 0x7f09301a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.790+0000 7f0934a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 0x7f09301a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.790+0000 7f0934a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0930109260 0x7f093019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.790+0000 7f0934a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0930114290 con 0x7f0930109260 2024-02-26T10:43:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.790+0000 7f0934a00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0930114110 con 0x7f0930104b30 2024-02-26T10:43:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.790+0000 7f0934a00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0930114410 con 0x7f0930104180 2024-02-26T10:43:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092e800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0930109260 0x7f093019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092de00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930104180 0x7f09301a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092d400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 0x7f09301a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092e800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0930109260 0x7f093019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45506/0 (socket says 172.21.15.60:45506) 2024-02-26T10:43:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092d400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 0x7f09301a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:43694/0 (socket says 172.21.15.60:43694) 2024-02-26T10:43:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092e800700 1 -- 172.21.15.60:0/599315491 learned_addr learned my addr 172.21.15.60:0/599315491 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092de00700 1 -- 172.21.15.60:0/599315491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 msgr2=0x7f09301a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092de00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 0x7f09301a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092de00700 1 -- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0930109260 msgr2=0x7f093019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092de00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0930109260 0x7f093019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092de00700 1 -- 172.21.15.60:0/599315491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093019ad50 con 0x7f0930104180 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092d400700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 0x7f09301a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.791+0000 7f092e800700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0930109260 0x7f093019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.792+0000 7f092de00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930104180 0x7f09301a03e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f092000e9f0 tx=0x7f092000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.792+0000 7f0926a00700 1 -- 172.21.15.60:0/599315491 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092000c9c0 con 0x7f0930104180 2024-02-26T10:43:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.792+0000 7f0926a00700 1 -- 172.21.15.60:0/599315491 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f092000cb60 con 0x7f0930104180 2024-02-26T10:43:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.792+0000 7f0926a00700 1 -- 172.21.15.60:0/599315491 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920018680 con 0x7f0930104180 2024-02-26T10:43:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.792+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093019b040 con 0x7f0930104180 2024-02-26T10:43:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.792+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09301a72b0 con 0x7f0930104180 2024-02-26T10:43:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.794+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0930068a10 con 0x7f0930104180 2024-02-26T10:43:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.795+0000 7f0926a00700 1 -- 172.21.15.60:0/599315491 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f0920010430 con 0x7f0930104180 2024-02-26T10:43:27.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.797+0000 7f0926a00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09140a1ea0 0x7f09140a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:27.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.797+0000 7f0926a00700 1 -- 172.21.15.60:0/599315491 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f09200ce2e0 con 0x7f0930104180 2024-02-26T10:43:27.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.797+0000 7f092d400700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09140a1ea0 0x7f09140a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:27.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.798+0000 7f092d400700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09140a1ea0 0x7f09140a4360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f093010a860 tx=0x7f091c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:27.801+0000 7f0926a00700 1 -- 172.21.15.60:0/599315491 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f092009b310 con 0x7f0930104180 2024-02-26T10:43:28.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.088+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f093002d7f0 con 0x7f0930104180 2024-02-26T10:43:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.094+0000 7f0926a00700 1 -- 172.21.15.60:0/599315491 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261466 (secure 0 0 0) 0x7f092009ac10 con 0x7f0930104180 2024-02-26T10:43:28.098 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:28.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09140a1ea0 msgr2=0x7f09140a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:28.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09140a1ea0 0x7f09140a4360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f093010a860 tx=0x7f091c01a040 comp rx=0 tx=0).stop 2024-02-26T10:43:28.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930104180 msgr2=0x7f09301a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:28.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930104180 0x7f09301a03e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f092000e9f0 tx=0x7f092000ed00 comp rx=0 tx=0).stop 2024-02-26T10:43:28.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 shutdown_connections 2024-02-26T10:43:28.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09140a1ea0 0x7f09140a4360 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:28.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0930109260 0x7f093019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:28.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0930104180 0x7f09301a03e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:28.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 --2- 172.21.15.60:0/599315491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0930104b30 0x7f09301a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:28.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 >> 172.21.15.60:0/599315491 conn(0x7f09300ffa30 msgr2=0x7f0930101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:28.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.099+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 shutdown_connections 2024-02-26T10:43:28.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:28.100+0000 7f0934a00700 1 -- 172.21.15.60:0/599315491 wait complete. 2024-02-26T10:43:28.680 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:28 smithi100 ceph-mon[36051]: pgmap v346: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 13 op/s 2024-02-26T10:43:28.680 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:28 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/599315491' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T10:43:28.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:28 smithi060 ceph-mon[31539]: pgmap v346: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 13 op/s 2024-02-26T10:43:28.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:28 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/599315491' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T10:43:28.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:28 smithi113 ceph-mon[36126]: pgmap v346: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 13 op/s 2024-02-26T10:43:28.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:28 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/599315491' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T10:43:28.970 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3493132094},{"type":"v1","addr":"172.21.15.60:6801","nonce":3493132094}]},"active_addr":"172.21.15.60:6801/3493132094","active_change":"2024-02-26T10:34:36.734063+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.60:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":4152931325}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":3500886609}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":988911145}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":806578503}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":91130208}]}]} 2024-02-26T10:43:28.974 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-26T10:43:28.974 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-26T10:43:28.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T10:43:29.428 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:30.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:30 smithi060 ceph-mon[31539]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-02-26T10:43:30.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:30 smithi113 ceph-mon[36126]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-02-26T10:43:30.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:30 smithi100 ceph-mon[36051]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-02-26T10:43:31.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.586+0000 7f8a70200700 1 -- 172.21.15.60:0/55126936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a681029b0 msgr2=0x7f8a681069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:31.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.586+0000 7f8a70200700 1 --2- 172.21.15.60:0/55126936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a681029b0 0x7f8a681069f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a540099c0 tx=0x7f8a54009cd0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.586+0000 7f8a70200700 1 -- 172.21.15.60:0/55126936 shutdown_connections 2024-02-26T10:43:31.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.586+0000 7f8a70200700 1 --2- 172.21.15.60:0/55126936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a681070d0 0x7f8a6810eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.586+0000 7f8a70200700 1 --2- 172.21.15.60:0/55126936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a68102000 0x7f8a681023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.586+0000 7f8a70200700 1 --2- 172.21.15.60:0/55126936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a681029b0 0x7f8a681069f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.586+0000 7f8a70200700 1 -- 172.21.15.60:0/55126936 >> 172.21.15.60:0/55126936 conn(0x7f8a68078eb0 msgr2=0x7f8a680792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:31.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.586+0000 7f8a70200700 1 -- 172.21.15.60:0/55126936 shutdown_connections 2024-02-26T10:43:31.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.586+0000 7f8a70200700 1 -- 172.21.15.60:0/55126936 wait complete. 2024-02-26T10:43:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.587+0000 7f8a70200700 1 Processor -- start 2024-02-26T10:43:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.587+0000 7f8a70200700 1 -- start start 2024-02-26T10:43:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a70200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a68102000 0x7f8a68074050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a70200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a681029b0 0x7f8a68074590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a70200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a681070d0 0x7f8a68074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a70200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a681120e0 con 0x7f8a681070d0 2024-02-26T10:43:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a70200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68111f60 con 0x7f8a681029b0 2024-02-26T10:43:31.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a70200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68112260 con 0x7f8a68102000 2024-02-26T10:43:31.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a6d800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a68102000 0x7f8a68074050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:31.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a6e200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a681070d0 0x7f8a68074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:31.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a6d800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a68102000 0x7f8a68074050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:60310/0 (socket says 172.21.15.60:60310) 2024-02-26T10:43:31.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a6d800700 1 -- 172.21.15.60:0/2302199289 learned_addr learned my addr 172.21.15.60:0/2302199289 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:31.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.588+0000 7f8a6ce00700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a681029b0 0x7f8a68074590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a6d800700 1 -- 172.21.15.60:0/2302199289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a681029b0 msgr2=0x7f8a68074590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a6d800700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a681029b0 0x7f8a68074590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a6d800700 1 -- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a681070d0 msgr2=0x7f8a68074ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a6d800700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a681070d0 0x7f8a68074ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a6d800700 1 -- 172.21.15.60:0/2302199289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a681a6b00 con 0x7f8a68102000 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a6ce00700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a681029b0 0x7f8a68074590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a6e200700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a681070d0 0x7f8a68074ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a6d800700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a68102000 0x7f8a68074050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00c370 tx=0x7f8a5c00c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a66000700 1 -- 172.21.15.60:0/2302199289 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c0092e0 con 0x7f8a68102000 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a66000700 1 -- 172.21.15.60:0/2302199289 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c00cbe0 con 0x7f8a68102000 2024-02-26T10:43:31.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.589+0000 7f8a66000700 1 -- 172.21.15.60:0/2302199289 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c0047e0 con 0x7f8a68102000 2024-02-26T10:43:31.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.590+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a681a6d90 con 0x7f8a68102000 2024-02-26T10:43:31.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.590+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a681a71f0 con 0x7f8a68102000 2024-02-26T10:43:31.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.591+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68068a10 con 0x7f8a68102000 2024-02-26T10:43:31.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.593+0000 7f8a66000700 1 -- 172.21.15.60:0/2302199289 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f8a5c01a020 con 0x7f8a68102000 2024-02-26T10:43:31.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.594+0000 7f8a66000700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8a500a1f20 0x7f8a500a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:31.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.594+0000 7f8a66000700 1 -- 172.21.15.60:0/2302199289 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8a5c0cdac0 con 0x7f8a68102000 2024-02-26T10:43:31.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.595+0000 7f8a6ce00700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8a500a1f20 0x7f8a500a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:31.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.595+0000 7f8a6ce00700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8a500a1f20 0x7f8a500a43e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000c00 tx=0x7f8a54005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:31.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.598+0000 7f8a66000700 1 -- 172.21.15.60:0/2302199289 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a5c010040 con 0x7f8a68102000 2024-02-26T10:43:31.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.828+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a6802d9b0 con 0x7f8a68102000 2024-02-26T10:43:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.830+0000 7f8a66000700 1 -- 172.21.15.60:0/2302199289 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7f8a5c09aaf0 con 0x7f8a68102000 2024-02-26T10:43:31.828 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:31.828 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8a500a1f20 msgr2=0x7f8a500a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8a500a1f20 0x7f8a500a43e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000c00 tx=0x7f8a54005fb0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a68102000 msgr2=0x7f8a68074050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a68102000 0x7f8a68074050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00c370 tx=0x7f8a5c00c680 comp rx=0 tx=0).stop 2024-02-26T10:43:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 shutdown_connections 2024-02-26T10:43:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8a500a1f20 0x7f8a500a43e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a681070d0 0x7f8a68074ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a68102000 0x7f8a68074050 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 --2- 172.21.15.60:0/2302199289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a681029b0 0x7f8a68074590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 >> 172.21.15.60:0/2302199289 conn(0x7f8a68078eb0 msgr2=0x7f8a680ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.834+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 shutdown_connections 2024-02-26T10:43:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:31.835+0000 7f8a70200700 1 -- 172.21.15.60:0/2302199289 wait complete. 2024-02-26T10:43:32.597 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:32 smithi060 ceph-mon[31539]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-02-26T10:43:32.597 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:32 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2302199289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:32.653 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-26T10:43:32.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T10:43:32.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:32 smithi113 ceph-mon[36126]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-02-26T10:43:32.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:32 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2302199289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:32.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:32 smithi100 ceph-mon[36051]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-02-26T10:43:32.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:32 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2302199289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:34.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:34 smithi060 ceph-mon[31539]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-26T10:43:34.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:34 smithi113 ceph-mon[36126]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-26T10:43:34.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:34 smithi100 ceph-mon[36051]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.359+0000 7f75a9c00700 1 -- 172.21.15.60:0/3032855391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75a410e750 msgr2=0x7f75a4110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.359+0000 7f75a9c00700 1 --2- 172.21.15.60:0/3032855391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75a410e750 0x7f75a4110b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f759c00b0a0 tx=0x7f759c00b3b0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.359+0000 7f75a9c00700 1 -- 172.21.15.60:0/3032855391 shutdown_connections 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.359+0000 7f75a9c00700 1 --2- 172.21.15.60:0/3032855391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75a410e750 0x7f75a4110b40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.359+0000 7f75a9c00700 1 --2- 172.21.15.60:0/3032855391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a4073f80 0x7f75a4074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.359+0000 7f75a9c00700 1 --2- 172.21.15.60:0/3032855391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a40748a0 0x7f75a410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.359+0000 7f75a9c00700 1 -- 172.21.15.60:0/3032855391 >> 172.21.15.60:0/3032855391 conn(0x7f75a4078eb0 msgr2=0x7f75a40792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.359+0000 7f75a9c00700 1 -- 172.21.15.60:0/3032855391 shutdown_connections 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.359+0000 7f75a9c00700 1 -- 172.21.15.60:0/3032855391 wait complete. 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.360+0000 7f75a9c00700 1 Processor -- start 2024-02-26T10:43:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.360+0000 7f75a9c00700 1 -- start start 2024-02-26T10:43:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f75a9c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75a4073f80 0x7f75a419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f75a9c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a40748a0 0x7f75a419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f75a9c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a410e750 0x7f75a419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f75a9c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4114360 con 0x7f75a4073f80 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f75a9c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a41141e0 con 0x7f75a40748a0 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f75a9c00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a41144e0 con 0x7f75a410e750 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f75a2a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75a4073f80 0x7f75a419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f759a000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a40748a0 0x7f75a419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f759a000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a40748a0 0x7f75a419acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:44860/0 (socket says 172.21.15.60:44860) 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f759a000700 1 -- 172.21.15.60:0/1308673343 learned_addr learned my addr 172.21.15.60:0/1308673343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.361+0000 7f75a3400700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a410e750 0x7f75a419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f759a000700 1 -- 172.21.15.60:0/1308673343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a410e750 msgr2=0x7f75a419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f759a000700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a410e750 0x7f75a419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f759a000700 1 -- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75a4073f80 msgr2=0x7f75a419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f759a000700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75a4073f80 0x7f75a419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f759a000700 1 -- 172.21.15.60:0/1308673343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a419f730 con 0x7f75a40748a0 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f75a2a00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75a4073f80 0x7f75a419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:35.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f759a000700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a40748a0 0x7f75a419acb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f758c00e9c0 tx=0x7f758c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f759be00700 1 -- 172.21.15.60:0/1308673343 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c00cbd0 con 0x7f75a40748a0 2024-02-26T10:43:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a419fa20 con 0x7f75a40748a0 2024-02-26T10:43:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f759be00700 1 -- 172.21.15.60:0/1308673343 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f758c00ee60 con 0x7f75a40748a0 2024-02-26T10:43:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f759be00700 1 -- 172.21.15.60:0/1308673343 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c005630 con 0x7f75a40748a0 2024-02-26T10:43:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.362+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a41a72a0 con 0x7f75a40748a0 2024-02-26T10:43:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.363+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a4068a10 con 0x7f75a40748a0 2024-02-26T10:43:35.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.366+0000 7f759be00700 1 -- 172.21.15.60:0/1308673343 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f758c004750 con 0x7f75a40748a0 2024-02-26T10:43:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.367+0000 7f759be00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75880a1ea0 0x7f75880a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.367+0000 7f759be00700 1 -- 172.21.15.60:0/1308673343 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f758c0ce650 con 0x7f75a40748a0 2024-02-26T10:43:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.367+0000 7f75a2a00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75880a1ea0 0x7f75880a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.368+0000 7f75a2a00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75880a1ea0 0x7f75880a4360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7594009680 tx=0x7f7594009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:35.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.369+0000 7f759be00700 1 -- 172.21.15.60:0/1308673343 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f758c052d20 con 0x7f75a40748a0 2024-02-26T10:43:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.594+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75a41a7470 con 0x7f75a40748a0 2024-02-26T10:43:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.596+0000 7f759be00700 1 -- 172.21.15.60:0/1308673343 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24137 (secure 0 0 0) 0x7f758c09b680 con 0x7f75a40748a0 2024-02-26T10:43:35.593 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:43:35.594 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:42:47.704859+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75880a1ea0 msgr2=0x7f75880a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75880a1ea0 0x7f75880a4360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7594009680 tx=0x7f7594009260 comp rx=0 tx=0).stop 2024-02-26T10:43:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a40748a0 msgr2=0x7f75a419acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a40748a0 0x7f75a419acb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f758c00e9c0 tx=0x7f758c00ecd0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 shutdown_connections 2024-02-26T10:43:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f75880a1ea0 0x7f75880a4360 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a410e750 0x7f75a419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75a4073f80 0x7f75a419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 --2- 172.21.15.60:0/1308673343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a40748a0 0x7f75a419acb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.599+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 >> 172.21.15.60:0/1308673343 conn(0x7f75a4078eb0 msgr2=0x7f75a40feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.600+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 shutdown_connections 2024-02-26T10:43:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:35.600+0000 7f75a9c00700 1 -- 172.21.15.60:0/1308673343 wait complete. 2024-02-26T10:43:36.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-26T10:43:36.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-26T10:43:36.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-26T10:43:36.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-26T10:43:36.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-26T10:43:36.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-26T10:43:36.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-26T10:43:36.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-26T10:43:36.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-26T10:43:36.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-26T10:43:36.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-26T10:43:36.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-26T10:43:36.598 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:36 smithi060 ceph-mon[31539]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-02-26T10:43:36.598 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:36 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1308673343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:36.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:36 smithi113 ceph-mon[36126]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-02-26T10:43:36.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:36 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1308673343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:36.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:36 smithi100 ceph-mon[36051]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-02-26T10:43:36.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:36 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1308673343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:43:37.532 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:37.550 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:37.696 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-02-26T10:43:37.696 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-02-26T10:43:37.696 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-02-26T10:43:37.696 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-02-26T10:43:37.696 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-02-26T10:43:37.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-02-26T10:43:37.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-02-26T10:43:37.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-02-26T10:43:37.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-02-26T10:43:37.697 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:37 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-02-26T10:43:37.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-02-26T10:43:37.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-02-26T10:43:37.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-02-26T10:43:37.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-02-26T10:43:37.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-02-26T10:43:37.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-02-26T10:43:37.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-02-26T10:43:37.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-02-26T10:43:37.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-02-26T10:43:37.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:37 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-02-26T10:43:37.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-02-26T10:43:37.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-02-26T10:43:37.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-02-26T10:43:37.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-02-26T10:43:37.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-02-26T10:43:37.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-02-26T10:43:37.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-02-26T10:43:37.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-02-26T10:43:37.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-02-26T10:43:37.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:37 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-02-26T10:43:37.968 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:38.394 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:38.400 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-02-26T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-02-26T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-02-26T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-02-26T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-02-26T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-02-26T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-02-26T10:43:38.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-02-26T10:43:38.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-02-26T10:43:38.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-02-26T10:43:38.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-02-26T10:43:38.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:38 smithi060 ceph-mon[31539]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T10:43:38.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-02-26T10:43:38.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:38 smithi113 ceph-mon[36126]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-02-26T10:43:38.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-02-26T10:43:38.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-02-26T10:43:38.879 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:38 smithi100 ceph-mon[36051]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T10:43:39.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:39 smithi060 ceph-mon[31539]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T10:43:39.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:39 smithi113 ceph-mon[36126]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T10:43:39.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:39 smithi100 ceph-mon[36051]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T10:43:40.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:40 smithi060 ceph-mon[31539]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 7 op/s 2024-02-26T10:43:40.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:40 smithi113 ceph-mon[36126]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 7 op/s 2024-02-26T10:43:40.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:40 smithi100 ceph-mon[36051]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 7 op/s 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.285+0000 7f0bc2000700 1 -- 172.21.15.60:0/2730215563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bbc06a3c0 msgr2=0x7f0bbc10eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.285+0000 7f0bc2000700 1 --2- 172.21.15.60:0/2730215563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bbc06a3c0 0x7f0bbc10eba0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400b0d0 tx=0x7f0bb400b3e0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.285+0000 7f0bc2000700 1 -- 172.21.15.60:0/2730215563 shutdown_connections 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.285+0000 7f0bc2000700 1 --2- 172.21.15.60:0/2730215563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bbc06a3c0 0x7f0bbc10eba0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.285+0000 7f0bc2000700 1 --2- 172.21.15.60:0/2730215563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bbc107d80 0x7f0bbc108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.285+0000 7f0bc2000700 1 --2- 172.21.15.60:0/2730215563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc069a00 0x7f0bbc069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.285+0000 7f0bc2000700 1 -- 172.21.15.60:0/2730215563 >> 172.21.15.60:0/2730215563 conn(0x7f0bbc077a00 msgr2=0x7f0bbc077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.285+0000 7f0bc2000700 1 -- 172.21.15.60:0/2730215563 shutdown_connections 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.285+0000 7f0bc2000700 1 -- 172.21.15.60:0/2730215563 wait complete. 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.286+0000 7f0bc2000700 1 Processor -- start 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.286+0000 7f0bc2000700 1 -- start start 2024-02-26T10:43:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bc2000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bbc069a00 0x7f0bbc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bc2000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bbc06a3c0 0x7f0bbc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bc2000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc107d80 0x7f0bbc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bc2000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc112050 con 0x7f0bbc06a3c0 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bc2000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc111ed0 con 0x7f0bbc069a00 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bc2000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc1121d0 con 0x7f0bbc107d80 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bbaa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bbc06a3c0 0x7f0bbc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bbbe00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc107d80 0x7f0bbc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bbbe00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc107d80 0x7f0bbc19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:38142/0 (socket says 172.21.15.60:38142) 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bbbe00700 1 -- 172.21.15.60:0/1259404277 learned_addr learned my addr 172.21.15.60:0/1259404277 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.287+0000 7f0bbb400700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bbc069a00 0x7f0bbc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bbb400700 1 -- 172.21.15.60:0/1259404277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc107d80 msgr2=0x7f0bbc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bbb400700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bbc107d80 0x7f0bbc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bbb400700 1 -- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bbc06a3c0 msgr2=0x7f0bbc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bbb400700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bbc06a3c0 0x7f0bbc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bbb400700 1 -- 172.21.15.60:0/1259404277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc19f670 con 0x7f0bbc069a00 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bbaa00700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bbc06a3c0 0x7f0bbc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bbb400700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bbc069a00 0x7f0bbc19a6b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00e9f0 tx=0x7f0bac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bb3e00700 1 -- 172.21.15.60:0/1259404277 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00cb70 con 0x7f0bbc069a00 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bb3e00700 1 -- 172.21.15.60:0/1259404277 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bac00ee90 con 0x7f0bbc069a00 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bb3e00700 1 -- 172.21.15.60:0/1259404277 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac003dc0 con 0x7f0bbc069a00 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc19f960 con 0x7f0bbc069a00 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.288+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bbc1a71e0 con 0x7f0bbc069a00 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.290+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0bbc100340 con 0x7f0bbc069a00 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.291+0000 7f0bb3e00700 1 -- 172.21.15.60:0/1259404277 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f0bac003fa0 con 0x7f0bbc069a00 2024-02-26T10:43:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.293+0000 7f0bb3e00700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ba00a1ea0 0x7f0ba00a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.294+0000 7f0bb3e00700 1 -- 172.21.15.60:0/1259404277 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f0bac0ce390 con 0x7f0bbc069a00 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.294+0000 7f0bbaa00700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ba00a1ea0 0x7f0ba00a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.294+0000 7f0bb3e00700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] conn(0x7f0ba00ad000 0x7f0ba00af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.294+0000 7f0bb3e00700 1 -- 172.21.15.60:0/1259404277 --> [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bac01ee50 con 0x7f0ba00ad000 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.294+0000 7f0bb3e00700 1 -- 172.21.15.60:0/1259404277 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bac0d5050 con 0x7f0bbc069a00 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.294+0000 7f0bbbe00700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] conn(0x7f0ba00ad000 0x7f0ba00af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.295+0000 7f0bbaa00700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ba00a1ea0 0x7f0ba00a4360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc103090 tx=0x7f0ba8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.295+0000 7f0bbbe00700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] conn(0x7f0ba00ad000 0x7f0ba00af4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.296+0000 7f0bb3e00700 1 -- 172.21.15.60:0/1259404277 <== osd.0 v2:172.21.15.60:6802/810878839 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0bac01ee50 con 0x7f0ba00ad000 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.315+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 --> [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0bbc103410 con 0x7f0ba00ad000 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.315+0000 7f0bb3e00700 1 -- 172.21.15.60:0/1259404277 <== osd.0 v2:172.21.15.60:6802/810878839 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0bbc103410 con 0x7f0ba00ad000 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] conn(0x7f0ba00ad000 msgr2=0x7f0ba00af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] conn(0x7f0ba00ad000 0x7f0ba00af4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ba00a1ea0 msgr2=0x7f0ba00a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ba00a1ea0 0x7f0ba00a4360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc103090 tx=0x7f0ba8009260 comp rx=0 tx=0).stop 2024-02-26T10:43:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bbc069a00 msgr2=0x7f0bbc19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 --2- 172.21.15.60:0/1259404277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bbc069a00 0x7f0bbc19a6b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00e9f0 tx=0x7f0bac00ed00 comp rx=0 tx=0).stop 2024-02-26T10:43:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bbbe00700 1 -- 172.21.15.60:0/1259404277 reap_dead start 2024-02-26T10:43:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 shutdown_connections 2024-02-26T10:43:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 >> 172.21.15.60:0/1259404277 conn(0x7f0bbc077a00 msgr2=0x7f0bbc0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 shutdown_connections 2024-02-26T10:43:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:40.316+0000 7f0bc2000700 1 -- 172.21.15.60:0/1259404277 wait complete. 2024-02-26T10:43:42.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:42 smithi113 ceph-mon[36126]: pgmap v355: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s; 16 B/s, 1 objects/s recovering 2024-02-26T10:43:42.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:42 smithi060 ceph-mon[31539]: pgmap v355: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s; 16 B/s, 1 objects/s recovering 2024-02-26T10:43:42.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:42 smithi100 ceph-mon[36051]: pgmap v355: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s; 16 B/s, 1 objects/s recovering 2024-02-26T10:43:42.972 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:42.978 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:42.979 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:42.979 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:42.979 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:43:43.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:43:43.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:43:43.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:43:44.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:44 smithi113 ceph-mon[36126]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 383 B/s wr, 3 op/s; 16 B/s, 1 objects/s recovering 2024-02-26T10:43:44.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:44 smithi100 ceph-mon[36051]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 383 B/s wr, 3 op/s; 16 B/s, 1 objects/s recovering 2024-02-26T10:43:44.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:44 smithi060 ceph-mon[31539]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 383 B/s wr, 3 op/s; 16 B/s, 1 objects/s recovering 2024-02-26T10:43:46.334 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:46 smithi113 ceph-mon[36126]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s; 7.7 KiB/s, 1 objects/s recovering 2024-02-26T10:43:46.334 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.334 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.334 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.334 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:46 smithi060 ceph-mon[31539]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s; 7.7 KiB/s, 1 objects/s recovering 2024-02-26T10:43:46.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:46 smithi100 ceph-mon[36051]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s; 7.7 KiB/s, 1 objects/s recovering 2024-02-26T10:43:46.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:46.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:43:48.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:48 smithi060 ceph-mon[31539]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 324 B/s wr, 5 op/s; 6.5 KiB/s, 1 objects/s recovering 2024-02-26T10:43:48.324 INFO:teuthology.orchestra.run.smithi060.stdout:34359738456 2024-02-26T10:43:48.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-26T10:43:48.334 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:48 smithi113 ceph-mon[36126]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 324 B/s wr, 5 op/s; 6.5 KiB/s, 1 objects/s recovering 2024-02-26T10:43:48.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:48 smithi100 ceph-mon[36051]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 324 B/s wr, 5 op/s; 6.5 KiB/s, 1 objects/s recovering 2024-02-26T10:43:50.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.294+0000 7fec75200700 1 -- 172.21.15.60:0/63670310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec70069a00 msgr2=0x7fec70069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.294+0000 7fec75200700 1 --2- 172.21.15.60:0/63670310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec70069a00 0x7fec70069e80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0099c0 tx=0x7fec5c009cd0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.294+0000 7fec75200700 1 -- 172.21.15.60:0/63670310 shutdown_connections 2024-02-26T10:43:50.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.294+0000 7fec75200700 1 --2- 172.21.15.60:0/63670310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec7006a3c0 0x7fec7010eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.294+0000 7fec75200700 1 --2- 172.21.15.60:0/63670310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec70107d80 0x7fec70108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.294+0000 7fec75200700 1 --2- 172.21.15.60:0/63670310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec70069a00 0x7fec70069e80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.294+0000 7fec75200700 1 -- 172.21.15.60:0/63670310 >> 172.21.15.60:0/63670310 conn(0x7fec70077a00 msgr2=0x7fec70077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:50.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.295+0000 7fec75200700 1 -- 172.21.15.60:0/63670310 shutdown_connections 2024-02-26T10:43:50.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.295+0000 7fec75200700 1 -- 172.21.15.60:0/63670310 wait complete. 2024-02-26T10:43:50.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.296+0000 7fec75200700 1 Processor -- start 2024-02-26T10:43:50.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.296+0000 7fec75200700 1 -- start start 2024-02-26T10:43:50.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.296+0000 7fec75200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec70069a00 0x7fec7019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.296+0000 7fec75200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7006a3c0 0x7fec7019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.296+0000 7fec75200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec70107d80 0x7fec7019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.296+0000 7fec75200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70112050 con 0x7fec70107d80 2024-02-26T10:43:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.296+0000 7fec75200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70111ed0 con 0x7fec70069a00 2024-02-26T10:43:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.296+0000 7fec75200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701121d0 con 0x7fec7006a3c0 2024-02-26T10:43:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6de00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec70069a00 0x7fec7019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6de00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec70069a00 0x7fec7019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:59292/0 (socket says 172.21.15.60:59292) 2024-02-26T10:43:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6e800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec70107d80 0x7fec7019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6de00700 1 -- 172.21.15.60:0/3415781066 learned_addr learned my addr 172.21.15.60:0/3415781066 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6d400700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7006a3c0 0x7fec7019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6d400700 1 -- 172.21.15.60:0/3415781066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec70069a00 msgr2=0x7fec7019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6d400700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec70069a00 0x7fec7019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6d400700 1 -- 172.21.15.60:0/3415781066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec70107d80 msgr2=0x7fec7019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6d400700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec70107d80 0x7fec7019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6d400700 1 -- 172.21.15.60:0/3415781066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7019f6e0 con 0x7fec7006a3c0 2024-02-26T10:43:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.297+0000 7fec6d400700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7006a3c0 0x7fec7019ac60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0098e0 tx=0x7fec5c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:50.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.298+0000 7fec66a00700 1 -- 172.21.15.60:0/3415781066 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c027070 con 0x7fec7006a3c0 2024-02-26T10:43:50.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.298+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7019f970 con 0x7fec7006a3c0 2024-02-26T10:43:50.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.298+0000 7fec66a00700 1 -- 172.21.15.60:0/3415781066 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec5c003680 con 0x7fec7006a3c0 2024-02-26T10:43:50.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.298+0000 7fec66a00700 1 -- 172.21.15.60:0/3415781066 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0173f0 con 0x7fec7006a3c0 2024-02-26T10:43:50.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.298+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec701a7250 con 0x7fec7006a3c0 2024-02-26T10:43:50.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.299+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec70100470 con 0x7fec7006a3c0 2024-02-26T10:43:50.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.301+0000 7fec66a00700 1 -- 172.21.15.60:0/3415781066 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fec5c02a070 con 0x7fec7006a3c0 2024-02-26T10:43:50.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.302+0000 7fec66a00700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec540a1ea0 0x7fec540a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.303+0000 7fec66a00700 1 -- 172.21.15.60:0/3415781066 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fec5c0cf1d0 con 0x7fec7006a3c0 2024-02-26T10:43:50.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.303+0000 7fec6de00700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec540a1ea0 0x7fec540a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.303+0000 7fec66a00700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] conn(0x7fec540ad000 0x7fec540af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.303+0000 7fec66a00700 1 -- 172.21.15.60:0/3415781066 --> [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec540afb70 con 0x7fec540ad000 2024-02-26T10:43:50.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.303+0000 7fec66a00700 1 -- 172.21.15.60:0/3415781066 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec5c0cf580 con 0x7fec7006a3c0 2024-02-26T10:43:50.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.303+0000 7fec6e800700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] conn(0x7fec540ad000 0x7fec540af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.304+0000 7fec6de00700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec540a1ea0 0x7fec540a4360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec60009ca0 tx=0x7fec60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:50.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.304+0000 7fec6e800700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] conn(0x7fec540ad000 0x7fec540af4a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:50.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.305+0000 7fec66a00700 1 -- 172.21.15.60:0/3415781066 <== osd.2 v2:172.21.15.113:6800/1221563222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fec540afb70 con 0x7fec540ad000 2024-02-26T10:43:50.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.323+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 --> [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec70103410 con 0x7fec540ad000 2024-02-26T10:43:50.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.324+0000 7fec66a00700 1 -- 172.21.15.60:0/3415781066 <== osd.2 v2:172.21.15.113:6800/1221563222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fec70103410 con 0x7fec540ad000 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 >> [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] conn(0x7fec540ad000 msgr2=0x7fec540af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] conn(0x7fec540ad000 0x7fec540af4a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec540a1ea0 msgr2=0x7fec540a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fec540a1ea0 0x7fec540a4360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec60009ca0 tx=0x7fec60009260 comp rx=0 tx=0).stop 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7006a3c0 msgr2=0x7fec7019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 --2- 172.21.15.60:0/3415781066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7006a3c0 0x7fec7019ac60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0098e0 tx=0x7fec5c00fa00 comp rx=0 tx=0).stop 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec6e800700 1 -- 172.21.15.60:0/3415781066 reap_dead start 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 shutdown_connections 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 >> 172.21.15.60:0/3415781066 conn(0x7fec70077a00 msgr2=0x7fec700ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 shutdown_connections 2024-02-26T10:43:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.325+0000 7fec75200700 1 -- 172.21.15.60:0/3415781066 wait complete. 2024-02-26T10:43:50.334 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:50 smithi113 ceph-mon[36126]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 5.9 KiB/s, 1 objects/s recovering 2024-02-26T10:43:50.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:50 smithi060 ceph-mon[31539]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 5.9 KiB/s, 1 objects/s recovering 2024-02-26T10:43:50.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:50 smithi100 ceph-mon[36051]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 5.9 KiB/s, 1 objects/s recovering 2024-02-26T10:43:50.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.533+0000 7fd6fe000700 1 -- 172.21.15.60:0/4184004253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6f8108f50 msgr2=0x7fd6f810cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.533+0000 7fd6fe000700 1 --2- 172.21.15.60:0/4184004253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6f8108f50 0x7fd6f810cfa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009990 tx=0x7fd6e4009ca0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.533+0000 7fd6fe000700 1 -- 172.21.15.60:0/4184004253 shutdown_connections 2024-02-26T10:43:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.533+0000 7fd6fe000700 1 --2- 172.21.15.60:0/4184004253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6f810d680 0x7fd6f8115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.533+0000 7fd6fe000700 1 --2- 172.21.15.60:0/4184004253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f81085a0 0x7fd6f8108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.533+0000 7fd6fe000700 1 --2- 172.21.15.60:0/4184004253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6f8108f50 0x7fd6f810cfa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.534+0000 7fd6fe000700 1 -- 172.21.15.60:0/4184004253 >> 172.21.15.60:0/4184004253 conn(0x7fd6f8075520 msgr2=0x7fd6f8077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.534+0000 7fd6fe000700 1 -- 172.21.15.60:0/4184004253 shutdown_connections 2024-02-26T10:43:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.534+0000 7fd6fe000700 1 -- 172.21.15.60:0/4184004253 wait complete. 2024-02-26T10:43:50.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.535+0000 7fd6fe000700 1 Processor -- start 2024-02-26T10:43:50.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.535+0000 7fd6fe000700 1 -- start start 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6fe000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6f81085a0 0x7fd6f81a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6fe000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f8108f50 0x7fd6f81a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6fe000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6f810d680 0x7fd6f819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6fe000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8118690 con 0x7fd6f8108f50 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6fe000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8118510 con 0x7fd6f81085a0 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6fe000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8118810 con 0x7fd6f810d680 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6f7e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6f810d680 0x7fd6f819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6f6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f8108f50 0x7fd6f81a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6f7400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6f81085a0 0x7fd6f81a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6f7e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6f810d680 0x7fd6f819e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:33814/0 (socket says 172.21.15.60:33814) 2024-02-26T10:43:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.536+0000 7fd6f7e00700 1 -- 172.21.15.60:0/1274048412 learned_addr learned my addr 172.21.15.60:0/1274048412 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6f7e00700 1 -- 172.21.15.60:0/1274048412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6f81085a0 msgr2=0x7fd6f81a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6f7e00700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6f81085a0 0x7fd6f81a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6f7e00700 1 -- 172.21.15.60:0/1274048412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f8108f50 msgr2=0x7fd6f81a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6f7e00700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f8108f50 0x7fd6f81a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6f7e00700 1 -- 172.21.15.60:0/1274048412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f819f0f0 con 0x7fd6f810d680 2024-02-26T10:43:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6f7400700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6f81085a0 0x7fd6f81a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6f7e00700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6f810d680 0x7fd6f819e990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0007ee0 tx=0x7fd6f000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6efe00700 1 -- 172.21.15.60:0/1274048412 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000dd20 con 0x7fd6f810d680 2024-02-26T10:43:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6efe00700 1 -- 172.21.15.60:0/1274048412 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6f0004020 con 0x7fd6f810d680 2024-02-26T10:43:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6efe00700 1 -- 172.21.15.60:0/1274048412 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0015660 con 0x7fd6f810d680 2024-02-26T10:43:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f819f3e0 con 0x7fd6f810d680 2024-02-26T10:43:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.537+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f81ab6a0 con 0x7fd6f810d680 2024-02-26T10:43:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.538+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd6f80624a0 con 0x7fd6f810d680 2024-02-26T10:43:50.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.541+0000 7fd6efe00700 1 -- 172.21.15.60:0/1274048412 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fd6f00041d0 con 0x7fd6f810d680 2024-02-26T10:43:50.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.542+0000 7fd6efe00700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6dc0a1f20 0x7fd6dc0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.542+0000 7fd6efe00700 1 -- 172.21.15.60:0/1274048412 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd6f00ce800 con 0x7fd6f810d680 2024-02-26T10:43:50.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.542+0000 7fd6f7400700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6dc0a1f20 0x7fd6dc0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.542+0000 7fd6efe00700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] conn(0x7fd6dc0ad080 0x7fd6dc0af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:50.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.543+0000 7fd6efe00700 1 -- 172.21.15.60:0/1274048412 --> [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6f0020e50 con 0x7fd6dc0ad080 2024-02-26T10:43:50.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.543+0000 7fd6efe00700 1 -- 172.21.15.60:0/1274048412 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6f00cebb0 con 0x7fd6f810d680 2024-02-26T10:43:50.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.543+0000 7fd6f6a00700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] conn(0x7fd6dc0ad080 0x7fd6dc0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:50.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.543+0000 7fd6f7400700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6dc0a1f20 0x7fd6dc0a43e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6f819fad0 tx=0x7fd6e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:50.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.544+0000 7fd6f6a00700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] conn(0x7fd6dc0ad080 0x7fd6dc0af520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:50.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.545+0000 7fd6efe00700 1 -- 172.21.15.60:0/1274048412 <== osd.4 v2:172.21.15.100:6808/2467117088 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd6f0020e50 con 0x7fd6dc0ad080 2024-02-26T10:43:50.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.563+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 --> [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd6f8062920 con 0x7fd6dc0ad080 2024-02-26T10:43:50.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.564+0000 7fd6efe00700 1 -- 172.21.15.60:0/1274048412 <== osd.4 v2:172.21.15.100:6808/2467117088 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd6f8062920 con 0x7fd6dc0ad080 2024-02-26T10:43:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.564+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 >> [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] conn(0x7fd6dc0ad080 msgr2=0x7fd6dc0af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.564+0000 7fd6fe000700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] conn(0x7fd6dc0ad080 0x7fd6dc0af520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.565+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6dc0a1f20 msgr2=0x7fd6dc0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.565+0000 7fd6fe000700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6dc0a1f20 0x7fd6dc0a43e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6f819fad0 tx=0x7fd6e8009260 comp rx=0 tx=0).stop 2024-02-26T10:43:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.565+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6f810d680 msgr2=0x7fd6f819e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.565+0000 7fd6fe000700 1 --2- 172.21.15.60:0/1274048412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6f810d680 0x7fd6f819e990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0007ee0 tx=0x7fd6f000d3b0 comp rx=0 tx=0).stop 2024-02-26T10:43:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.565+0000 7fd6f7e00700 1 -- 172.21.15.60:0/1274048412 reap_dead start 2024-02-26T10:43:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.565+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 shutdown_connections 2024-02-26T10:43:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.565+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 >> 172.21.15.60:0/1274048412 conn(0x7fd6f8075520 msgr2=0x7fd6f80777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.565+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 shutdown_connections 2024-02-26T10:43:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:50.565+0000 7fd6fe000700 1 -- 172.21.15.60:0/1274048412 wait complete. 2024-02-26T10:43:52.334 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:52 smithi113 ceph-mon[36126]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 5.2 KiB/s, 1 objects/s recovering 2024-02-26T10:43:52.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:52 smithi060 ceph-mon[31539]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 5.2 KiB/s, 1 objects/s recovering 2024-02-26T10:43:52.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:52 smithi100 ceph-mon[36051]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 5.2 KiB/s, 1 objects/s recovering 2024-02-26T10:43:53.122 INFO:teuthology.orchestra.run.smithi060.stdout:77309411408 2024-02-26T10:43:53.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-26T10:43:54.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:54 smithi060 ceph-mon[31539]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 5.1 KiB/s, 0 objects/s recovering 2024-02-26T10:43:54.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:54 smithi100 ceph-mon[36051]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 5.1 KiB/s, 0 objects/s recovering 2024-02-26T10:43:54.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:54 smithi113 ceph-mon[36126]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 5.1 KiB/s, 0 objects/s recovering 2024-02-26T10:43:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.130+0000 7f3fcde00700 1 -- 172.21.15.60:0/3467264170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc8109250 msgr2=0x7f3fc8110e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.130+0000 7f3fcde00700 1 --2- 172.21.15.60:0/3467264170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc8109250 0x7f3fc8110e20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000b0d0 tx=0x7f3fc000b3e0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.130+0000 7f3fcde00700 1 -- 172.21.15.60:0/3467264170 shutdown_connections 2024-02-26T10:43:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.130+0000 7f3fcde00700 1 --2- 172.21.15.60:0/3467264170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc8109250 0x7f3fc8110e20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.130+0000 7f3fcde00700 1 --2- 172.21.15.60:0/3467264170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8104170 0x7f3fc8104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.130+0000 7f3fcde00700 1 --2- 172.21.15.60:0/3467264170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8104b20 0x7f3fc8108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.130+0000 7f3fcde00700 1 -- 172.21.15.60:0/3467264170 >> 172.21.15.60:0/3467264170 conn(0x7f3fc80ffa20 msgr2=0x7f3fc8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.131+0000 7f3fcde00700 1 -- 172.21.15.60:0/3467264170 shutdown_connections 2024-02-26T10:43:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.131+0000 7f3fcde00700 1 -- 172.21.15.60:0/3467264170 wait complete. 2024-02-26T10:43:55.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.132+0000 7f3fcde00700 1 Processor -- start 2024-02-26T10:43:55.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.132+0000 7f3fcde00700 1 -- start start 2024-02-26T10:43:55.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fcde00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc8104170 0x7f3fc819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fcde00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8104b20 0x7f3fc819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fcde00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8109250 0x7f3fc819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fcde00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8114240 con 0x7f3fc8109250 2024-02-26T10:43:55.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fcde00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc81140c0 con 0x7f3fc8104b20 2024-02-26T10:43:55.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fcde00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc81143c0 con 0x7f3fc8104170 2024-02-26T10:43:55.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fc6a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc8104170 0x7f3fc819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fc6000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8104b20 0x7f3fc819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fc6a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc8104170 0x7f3fc819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49650/0 (socket says 172.21.15.60:49650) 2024-02-26T10:43:55.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fc6a00700 1 -- 172.21.15.60:0/1935796493 learned_addr learned my addr 172.21.15.60:0/1935796493 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.133+0000 7f3fc7400700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8109250 0x7f3fc819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.134+0000 7f3fc6a00700 1 -- 172.21.15.60:0/1935796493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8104b20 msgr2=0x7f3fc819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.134+0000 7f3fc6a00700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8104b20 0x7f3fc819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.134+0000 7f3fc6a00700 1 -- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8109250 msgr2=0x7f3fc819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.134+0000 7f3fc6a00700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8109250 0x7f3fc819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.134+0000 7f3fc6a00700 1 -- 172.21.15.60:0/1935796493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc819f760 con 0x7f3fc8104170 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.134+0000 7f3fc6a00700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc8104170 0x7f3fc819a750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800eb30 tx=0x7f3fb800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.135+0000 7f3fcca00700 1 -- 172.21.15.60:0/1935796493 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800cbd0 con 0x7f3fc8104170 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.135+0000 7f3fcca00700 1 -- 172.21.15.60:0/1935796493 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fb8004500 con 0x7f3fc8104170 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.135+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc819fa50 con 0x7f3fc8104170 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.135+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc81a72d0 con 0x7f3fc8104170 2024-02-26T10:43:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.135+0000 7f3fcca00700 1 -- 172.21.15.60:0/1935796493 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8010430 con 0x7f3fc8104170 2024-02-26T10:43:55.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.136+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3fc8062460 con 0x7f3fc8104170 2024-02-26T10:43:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.138+0000 7f3fcca00700 1 -- 172.21.15.60:0/1935796493 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f3fb8003f00 con 0x7f3fc8104170 2024-02-26T10:43:55.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.139+0000 7f3fcca00700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fac0a1ea0 0x7f3fac0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.139+0000 7f3fcca00700 1 -- 172.21.15.60:0/1935796493 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f3fb80ce660 con 0x7f3fc8104170 2024-02-26T10:43:55.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.139+0000 7f3fc6000700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fac0a1ea0 0x7f3fac0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.140+0000 7f3fcca00700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] conn(0x7f3fac0ad000 0x7f3fac0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.140+0000 7f3fcca00700 1 -- 172.21.15.60:0/1935796493 --> [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fb801ee50 con 0x7f3fac0ad000 2024-02-26T10:43:55.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.140+0000 7f3fcca00700 1 -- 172.21.15.60:0/1935796493 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb80d5050 con 0x7f3fc8104170 2024-02-26T10:43:55.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.140+0000 7f3fc7400700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] conn(0x7f3fac0ad000 0x7f3fac0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.140+0000 7f3fc6000700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fac0a1ea0 0x7f3fac0a4360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8100f90 tx=0x7f3fb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:55.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.141+0000 7f3fc7400700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] conn(0x7f3fac0ad000 0x7f3fac0af4a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.142+0000 7f3fcca00700 1 -- 172.21.15.60:0/1935796493 <== osd.6 v2:172.21.15.60:6818/1002313688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3fb801ee50 con 0x7f3fac0ad000 2024-02-26T10:43:55.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.160+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 --> [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3fc8062910 con 0x7f3fac0ad000 2024-02-26T10:43:55.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.161+0000 7f3fcca00700 1 -- 172.21.15.60:0/1935796493 <== osd.6 v2:172.21.15.60:6818/1002313688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3fc8062910 con 0x7f3fac0ad000 2024-02-26T10:43:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.161+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] conn(0x7f3fac0ad000 msgr2=0x7f3fac0af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.161+0000 7f3fcde00700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] conn(0x7f3fac0ad000 0x7f3fac0af4a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.162+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fac0a1ea0 msgr2=0x7f3fac0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.162+0000 7f3fcde00700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fac0a1ea0 0x7f3fac0a4360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8100f90 tx=0x7f3fb4009260 comp rx=0 tx=0).stop 2024-02-26T10:43:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.162+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc8104170 msgr2=0x7f3fc819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.162+0000 7f3fcde00700 1 --2- 172.21.15.60:0/1935796493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc8104170 0x7f3fc819a750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800eb30 tx=0x7f3fb800ee40 comp rx=0 tx=0).stop 2024-02-26T10:43:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.162+0000 7f3fc7400700 1 -- 172.21.15.60:0/1935796493 reap_dead start 2024-02-26T10:43:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.162+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 shutdown_connections 2024-02-26T10:43:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.162+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 >> 172.21.15.60:0/1935796493 conn(0x7f3fc80ffa20 msgr2=0x7f3fc8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.162+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 shutdown_connections 2024-02-26T10:43:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.162+0000 7f3fcde00700 1 -- 172.21.15.60:0/1935796493 wait complete. 2024-02-26T10:43:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.793+0000 7f9770400700 1 -- 172.21.15.60:0/883724127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9768104b50 msgr2=0x7f9768108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.793+0000 7f9770400700 1 --2- 172.21.15.60:0/883724127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9768104b50 0x7f9768108ba0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9754009990 tx=0x7f9754009ca0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.793+0000 7f9770400700 1 -- 172.21.15.60:0/883724127 shutdown_connections 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.793+0000 7f9770400700 1 --2- 172.21.15.60:0/883724127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9768109280 0x7f9768110e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.793+0000 7f9770400700 1 --2- 172.21.15.60:0/883724127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97681041a0 0x7f9768104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.793+0000 7f9770400700 1 --2- 172.21.15.60:0/883724127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9768104b50 0x7f9768108ba0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.793+0000 7f9770400700 1 -- 172.21.15.60:0/883724127 >> 172.21.15.60:0/883724127 conn(0x7f97680ffa30 msgr2=0x7f9768101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.793+0000 7f9770400700 1 -- 172.21.15.60:0/883724127 shutdown_connections 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.793+0000 7f9770400700 1 -- 172.21.15.60:0/883724127 wait complete. 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.794+0000 7f9770400700 1 Processor -- start 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.794+0000 7f9770400700 1 -- start start 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.794+0000 7f9770400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681041a0 0x7f976819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f9770400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9768104b50 0x7f976819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f9770400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9768109280 0x7f976819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f9770400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97681142b0 con 0x7f9768109280 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f9770400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768114130 con 0x7f97681041a0 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f9770400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768114430 con 0x7f9768104b50 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976e400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9768109280 0x7f976819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976e400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9768109280 0x7f976819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42038/0 (socket says 172.21.15.60:42038) 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976e400700 1 -- 172.21.15.60:0/4107946311 learned_addr learned my addr 172.21.15.60:0/4107946311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976da00700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681041a0 0x7f976819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976d000700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9768104b50 0x7f976819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976da00700 1 -- 172.21.15.60:0/4107946311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9768104b50 msgr2=0x7f976819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976da00700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9768104b50 0x7f976819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976da00700 1 -- 172.21.15.60:0/4107946311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9768109280 msgr2=0x7f976819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976da00700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9768109280 0x7f976819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976da00700 1 -- 172.21.15.60:0/4107946311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976819f700 con 0x7f97681041a0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.795+0000 7f976e400700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9768109280 0x7f976819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.796+0000 7f976da00700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681041a0 0x7f976819a740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e9f0 tx=0x7f975c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.796+0000 7f9766000700 1 -- 172.21.15.60:0/4107946311 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c004220 con 0x7f97681041a0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.796+0000 7f9766000700 1 -- 172.21.15.60:0/4107946311 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f975c004d10 con 0x7f97681041a0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.796+0000 7f9766000700 1 -- 172.21.15.60:0/4107946311 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c005020 con 0x7f97681041a0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.796+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976819f9f0 con 0x7f97681041a0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.796+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97681a7270 con 0x7f97681041a0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.797+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97680624e0 con 0x7f97681041a0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.800+0000 7f9766000700 1 -- 172.21.15.60:0/4107946311 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f975c004750 con 0x7f97681041a0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.801+0000 7f9766000700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97500a1f20 0x7f97500a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.801+0000 7f9766000700 1 -- 172.21.15.60:0/4107946311 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f975c0ce3d0 con 0x7f97681041a0 2024-02-26T10:43:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.801+0000 7f976d000700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97500a1f20 0x7f97500a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.801+0000 7f9766000700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] conn(0x7f97500ad080 0x7f97500af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.801+0000 7f9766000700 1 -- 172.21.15.60:0/4107946311 --> [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97500afbf0 con 0x7f97500ad080 2024-02-26T10:43:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.801+0000 7f9766000700 1 -- 172.21.15.60:0/4107946311 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f975c0ce780 con 0x7f97681041a0 2024-02-26T10:43:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.802+0000 7f976e400700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] conn(0x7f97500ad080 0x7f97500af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.802+0000 7f976d000700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97500a1f20 0x7f97500a43e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9768103150 tx=0x7f975400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.802+0000 7f976e400700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] conn(0x7f97500ad080 0x7f97500af520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.803+0000 7f9766000700 1 -- 172.21.15.60:0/4107946311 <== osd.5 v2:172.21.15.113:6808/3892917193 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f97500afbf0 con 0x7f97500ad080 2024-02-26T10:43:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.822+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 --> [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9768062960 con 0x7f97500ad080 2024-02-26T10:43:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.823+0000 7f9766000700 1 -- 172.21.15.60:0/4107946311 <== osd.5 v2:172.21.15.113:6808/3892917193 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9768062960 con 0x7f97500ad080 2024-02-26T10:43:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.824+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 >> [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] conn(0x7f97500ad080 msgr2=0x7f97500af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.824+0000 7f9770400700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] conn(0x7f97500ad080 0x7f97500af520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.824+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97500a1f20 msgr2=0x7f97500a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.824+0000 7f9770400700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97500a1f20 0x7f97500a43e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9768103150 tx=0x7f975400b260 comp rx=0 tx=0).stop 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.824+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681041a0 msgr2=0x7f976819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.824+0000 7f9770400700 1 --2- 172.21.15.60:0/4107946311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97681041a0 0x7f976819a740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e9f0 tx=0x7f975c00ed00 comp rx=0 tx=0).stop 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.824+0000 7f976e400700 1 -- 172.21.15.60:0/4107946311 reap_dead start 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.824+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 shutdown_connections 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.824+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 >> 172.21.15.60:0/4107946311 conn(0x7f97680ffa30 msgr2=0x7f97681005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.825+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 shutdown_connections 2024-02-26T10:43:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:55.825+0000 7f9770400700 1 -- 172.21.15.60:0/4107946311 wait complete. 2024-02-26T10:43:56.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:56 smithi060 ceph-mon[31539]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 5.1 KiB/s, 0 objects/s recovering 2024-02-26T10:43:56.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:56 smithi100 ceph-mon[36051]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 5.1 KiB/s, 0 objects/s recovering 2024-02-26T10:43:56.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:56 smithi113 ceph-mon[36126]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 5.1 KiB/s, 0 objects/s recovering 2024-02-26T10:43:56.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.623+0000 7fd5c3c00700 1 -- 172.21.15.60:0/4125122732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc109260 msgr2=0x7fd5bc110e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:56.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.623+0000 7fd5c3c00700 1 --2- 172.21.15.60:0/4125122732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc109260 0x7fd5bc110e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400b0a0 tx=0x7fd5b400b3b0 comp rx=0 tx=0).stop 2024-02-26T10:43:56.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.623+0000 7fd5c3c00700 1 -- 172.21.15.60:0/4125122732 shutdown_connections 2024-02-26T10:43:56.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.623+0000 7fd5c3c00700 1 --2- 172.21.15.60:0/4125122732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc109260 0x7fd5bc110e30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:56.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.623+0000 7fd5c3c00700 1 --2- 172.21.15.60:0/4125122732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc104180 0x7fd5bc104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:56.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.623+0000 7fd5c3c00700 1 --2- 172.21.15.60:0/4125122732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc104b30 0x7fd5bc108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:56.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.623+0000 7fd5c3c00700 1 -- 172.21.15.60:0/4125122732 >> 172.21.15.60:0/4125122732 conn(0x7fd5bc0ffa30 msgr2=0x7fd5bc101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:56.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.623+0000 7fd5c3c00700 1 -- 172.21.15.60:0/4125122732 shutdown_connections 2024-02-26T10:43:56.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.623+0000 7fd5c3c00700 1 -- 172.21.15.60:0/4125122732 wait complete. 2024-02-26T10:43:56.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.625+0000 7fd5c3c00700 1 Processor -- start 2024-02-26T10:43:56.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.625+0000 7fd5c3c00700 1 -- start start 2024-02-26T10:43:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.625+0000 7fd5c3c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc104180 0x7fd5bc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.625+0000 7fd5c3c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc104b30 0x7fd5bc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.625+0000 7fd5c3c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc109260 0x7fd5bc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.625+0000 7fd5c3c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc1142c0 con 0x7fd5bc104b30 2024-02-26T10:43:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.625+0000 7fd5c3c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc114140 con 0x7fd5bc109260 2024-02-26T10:43:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.625+0000 7fd5c3c00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc114440 con 0x7fd5bc104180 2024-02-26T10:43:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5bbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc104b30 0x7fd5bc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc109260 0x7fd5bc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc104180 0x7fd5bc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc109260 0x7fd5bc19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38160/0 (socket says 172.21.15.60:38160) 2024-02-26T10:43:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5bbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc104b30 0x7fd5bc19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42046/0 (socket says 172.21.15.60:42046) 2024-02-26T10:43:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1c00700 1 -- 172.21.15.60:0/1541202732 learned_addr learned my addr 172.21.15.60:0/1541202732 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc104180 0x7fd5bc19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49686/0 (socket says 172.21.15.60:49686) 2024-02-26T10:43:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1c00700 1 -- 172.21.15.60:0/1541202732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc104180 msgr2=0x7fd5bc19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1c00700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc104180 0x7fd5bc19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1c00700 1 -- 172.21.15.60:0/1541202732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc104b30 msgr2=0x7fd5bc19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1c00700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc104b30 0x7fd5bc19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.626+0000 7fd5c1c00700 1 -- 172.21.15.60:0/1541202732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc19f6a0 con 0x7fd5bc109260 2024-02-26T10:43:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.627+0000 7fd5c1c00700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc109260 0x7fd5bc19ef60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4006070 tx=0x7fd5b4004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.627+0000 7fd5b9600700 1 -- 172.21.15.60:0/1541202732 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4021070 con 0x7fd5bc109260 2024-02-26T10:43:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.627+0000 7fd5b9600700 1 -- 172.21.15.60:0/1541202732 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5b4003c00 con 0x7fd5bc109260 2024-02-26T10:43:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.627+0000 7fd5b9600700 1 -- 172.21.15.60:0/1541202732 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b40047d0 con 0x7fd5bc109260 2024-02-26T10:43:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.627+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc19f930 con 0x7fd5bc109260 2024-02-26T10:43:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.627+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc1a7210 con 0x7fd5bc109260 2024-02-26T10:43:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.628+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5bc100ec0 con 0x7fd5bc109260 2024-02-26T10:43:56.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.631+0000 7fd5b9600700 1 -- 172.21.15.60:0/1541202732 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fd5b4007980 con 0x7fd5bc109260 2024-02-26T10:43:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.632+0000 7fd5b9600700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd5a40a1f20 0x7fd5a40a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.633+0000 7fd5b9600700 1 -- 172.21.15.60:0/1541202732 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd5b400f020 con 0x7fd5bc109260 2024-02-26T10:43:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.633+0000 7fd5c1200700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd5a40a1f20 0x7fd5a40a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.633+0000 7fd5b9600700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] conn(0x7fd5a40ad080 0x7fd5a40af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.633+0000 7fd5b9600700 1 -- 172.21.15.60:0/1541202732 --> [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5a40afbf0 con 0x7fd5a40ad080 2024-02-26T10:43:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.633+0000 7fd5b9600700 1 -- 172.21.15.60:0/1541202732 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b40cfc90 con 0x7fd5bc109260 2024-02-26T10:43:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.633+0000 7fd5bbe00700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] conn(0x7fd5a40ad080 0x7fd5a40af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.634+0000 7fd5c1200700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd5a40a1f20 0x7fd5a40a43e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009bf0 tx=0x7fd5b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.634+0000 7fd5bbe00700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] conn(0x7fd5a40ad080 0x7fd5a40af520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:56.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.635+0000 7fd5b9600700 1 -- 172.21.15.60:0/1541202732 <== osd.8 v2:172.21.15.113:6816/3414484490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd5a40afbf0 con 0x7fd5a40ad080 2024-02-26T10:43:56.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.653+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 --> [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5bc105660 con 0x7fd5a40ad080 2024-02-26T10:43:56.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.654+0000 7fd5b9600700 1 -- 172.21.15.60:0/1541202732 <== osd.8 v2:172.21.15.113:6816/3414484490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd5bc105660 con 0x7fd5a40ad080 2024-02-26T10:43:56.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 >> [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] conn(0x7fd5a40ad080 msgr2=0x7fd5a40af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:56.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] conn(0x7fd5a40ad080 0x7fd5a40af520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:56.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd5a40a1f20 msgr2=0x7fd5a40a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:56.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd5a40a1f20 0x7fd5a40a43e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009bf0 tx=0x7fd5b0009260 comp rx=0 tx=0).stop 2024-02-26T10:43:56.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc109260 msgr2=0x7fd5bc19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:56.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 --2- 172.21.15.60:0/1541202732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc109260 0x7fd5bc19ef60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4006070 tx=0x7fd5b4004c80 comp rx=0 tx=0).stop 2024-02-26T10:43:56.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c1c00700 1 -- 172.21.15.60:0/1541202732 reap_dead start 2024-02-26T10:43:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 shutdown_connections 2024-02-26T10:43:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 >> 172.21.15.60:0/1541202732 conn(0x7fd5bc0ffa30 msgr2=0x7fd5bc1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 shutdown_connections 2024-02-26T10:43:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:56.655+0000 7fd5c3c00700 1 -- 172.21.15.60:0/1541202732 wait complete. 2024-02-26T10:43:57.732 INFO:teuthology.orchestra.run.smithi060.stdout:257698037807 2024-02-26T10:43:57.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-26T10:43:57.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.420+0000 7f5de8400700 1 -- 172.21.15.60:0/1534125056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de0069a00 msgr2=0x7f5de0069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:57.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.420+0000 7f5de8400700 1 --2- 172.21.15.60:0/1534125056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de0069a00 0x7f5de0069e80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0099c0 tx=0x7f5dcc009cd0 comp rx=0 tx=0).stop 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.420+0000 7f5de8400700 1 -- 172.21.15.60:0/1534125056 shutdown_connections 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.420+0000 7f5de8400700 1 --2- 172.21.15.60:0/1534125056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de006a3c0 0x7f5de010eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.420+0000 7f5de8400700 1 --2- 172.21.15.60:0/1534125056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0107d80 0x7f5de0108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.420+0000 7f5de8400700 1 --2- 172.21.15.60:0/1534125056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de0069a00 0x7f5de0069e80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.420+0000 7f5de8400700 1 -- 172.21.15.60:0/1534125056 >> 172.21.15.60:0/1534125056 conn(0x7f5de0077a00 msgr2=0x7f5de0077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.420+0000 7f5de8400700 1 -- 172.21.15.60:0/1534125056 shutdown_connections 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.420+0000 7f5de8400700 1 -- 172.21.15.60:0/1534125056 wait complete. 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.422+0000 7f5de8400700 1 Processor -- start 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.422+0000 7f5de8400700 1 -- start start 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.422+0000 7f5de8400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de0069a00 0x7f5de019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.422+0000 7f5de8400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de006a3c0 0x7f5de019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.422+0000 7f5de8400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de0107d80 0x7f5de019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.422+0000 7f5de8400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0112000 con 0x7f5de0107d80 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.422+0000 7f5de8400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0111e80 con 0x7f5de0069a00 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.422+0000 7f5de8400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0112180 con 0x7f5de006a3c0 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.422+0000 7f5de5a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de0069a00 0x7f5de019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de006a3c0 0x7f5de019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de0069a00 0x7f5de019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38190/0 (socket says 172.21.15.60:38190) 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de006a3c0 0x7f5de019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49710/0 (socket says 172.21.15.60:49710) 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5a00700 1 -- 172.21.15.60:0/3440777817 learned_addr learned my addr 172.21.15.60:0/3440777817 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de6400700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de0107d80 0x7f5de019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5000700 1 -- 172.21.15.60:0/3440777817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de0069a00 msgr2=0x7f5de019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5000700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de0069a00 0x7f5de019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5000700 1 -- 172.21.15.60:0/3440777817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de0107d80 msgr2=0x7f5de019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5000700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de0107d80 0x7f5de019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5000700 1 -- 172.21.15.60:0/3440777817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de019f6e0 con 0x7f5de006a3c0 2024-02-26T10:43:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de6400700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de0107d80 0x7f5de019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:43:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5a00700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de0069a00 0x7f5de019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:43:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.423+0000 7f5de5000700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de006a3c0 0x7f5de019ac10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0098e0 tx=0x7f5dcc005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.424+0000 7f5dde000700 1 -- 172.21.15.60:0/3440777817 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc027070 con 0x7f5de006a3c0 2024-02-26T10:43:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.424+0000 7f5dde000700 1 -- 172.21.15.60:0/3440777817 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5dcc003680 con 0x7f5de006a3c0 2024-02-26T10:43:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.424+0000 7f5dde000700 1 -- 172.21.15.60:0/3440777817 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0173f0 con 0x7f5de006a3c0 2024-02-26T10:43:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.424+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de019f970 con 0x7f5de006a3c0 2024-02-26T10:43:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.424+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de01a7250 con 0x7f5de006a3c0 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.425+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5de019fe00 con 0x7f5de006a3c0 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.427+0000 7f5dde000700 1 -- 172.21.15.60:0/3440777817 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f5dcc02a070 con 0x7f5de006a3c0 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.428+0000 7f5dde000700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5dc80a1ea0 0x7f5dc80a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.429+0000 7f5dde000700 1 -- 172.21.15.60:0/3440777817 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f5dcc0cf9a0 con 0x7f5de006a3c0 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.429+0000 7f5de5a00700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5dc80a1ea0 0x7f5dc80a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.429+0000 7f5dde000700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] conn(0x7f5dc80ad000 0x7f5dc80af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.429+0000 7f5dde000700 1 -- 172.21.15.60:0/3440777817 --> [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5dcc00eea0 con 0x7f5dc80ad000 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.429+0000 7f5dde000700 1 -- 172.21.15.60:0/3440777817 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dcc0cfd50 con 0x7f5de006a3c0 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.429+0000 7f5de6400700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] conn(0x7f5dc80ad000 0x7f5dc80af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.430+0000 7f5de5a00700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5dc80a1ea0 0x7f5dc80a4360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009480 tx=0x7f5dd400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.430+0000 7f5de6400700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] conn(0x7f5dc80ad000 0x7f5dc80af4a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.431+0000 7f5dde000700 1 -- 172.21.15.60:0/3440777817 <== osd.10 v2:172.21.15.100:6824/304375829 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5dcc00eea0 con 0x7f5dc80ad000 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.450+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 --> [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5de010db40 con 0x7f5dc80ad000 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.451+0000 7f5dde000700 1 -- 172.21.15.60:0/3440777817 <== osd.10 v2:172.21.15.100:6824/304375829 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5de010db40 con 0x7f5dc80ad000 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.451+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 >> [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] conn(0x7f5dc80ad000 msgr2=0x7f5dc80af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.451+0000 7f5de8400700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] conn(0x7f5dc80ad000 0x7f5dc80af4a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.451+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5dc80a1ea0 msgr2=0x7f5dc80a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.451+0000 7f5de8400700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5dc80a1ea0 0x7f5dc80a4360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009480 tx=0x7f5dd400b040 comp rx=0 tx=0).stop 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.451+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de006a3c0 msgr2=0x7f5de019ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.451+0000 7f5de8400700 1 --2- 172.21.15.60:0/3440777817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de006a3c0 0x7f5de019ac10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0098e0 tx=0x7f5dcc005f10 comp rx=0 tx=0).stop 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.452+0000 7f5de6400700 1 -- 172.21.15.60:0/3440777817 reap_dead start 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.452+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 shutdown_connections 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.452+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 >> 172.21.15.60:0/3440777817 conn(0x7f5de0077a00 msgr2=0x7f5de010ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.452+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 shutdown_connections 2024-02-26T10:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:43:57.452+0000 7f5de8400700 1 -- 172.21.15.60:0/3440777817 wait complete. 2024-02-26T10:43:58.362 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:43:58 smithi060 ceph-mon[31539]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-26T10:43:58.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:43:58 smithi100 ceph-mon[36051]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-26T10:43:58.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:43:58 smithi113 ceph-mon[36126]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-02-26T10:43:58.899 INFO:teuthology.orchestra.run.smithi060.stdout:201863462969 2024-02-26T10:43:58.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-26T10:44:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.114+0000 7fe72c000700 1 -- 172.21.15.60:0/1952752683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724073f80 msgr2=0x7fe724074360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.114+0000 7fe72c000700 1 --2- 172.21.15.60:0/1952752683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724073f80 0x7fe724074360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe718009990 tx=0x7fe718009ca0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.114+0000 7fe72c000700 1 -- 172.21.15.60:0/1952752683 shutdown_connections 2024-02-26T10:44:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.114+0000 7fe72c000700 1 --2- 172.21.15.60:0/1952752683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72410e780 0x7fe724110b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.114+0000 7fe72c000700 1 --2- 172.21.15.60:0/1952752683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe724073f80 0x7fe724074360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.114+0000 7fe72c000700 1 --2- 172.21.15.60:0/1952752683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7240748a0 0x7fe72410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.114+0000 7fe72c000700 1 -- 172.21.15.60:0/1952752683 >> 172.21.15.60:0/1952752683 conn(0x7fe724078ea0 msgr2=0x7fe7240792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.114+0000 7fe72c000700 1 -- 172.21.15.60:0/1952752683 shutdown_connections 2024-02-26T10:44:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.114+0000 7fe72c000700 1 -- 172.21.15.60:0/1952752683 wait complete. 2024-02-26T10:44:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.115+0000 7fe72c000700 1 Processor -- start 2024-02-26T10:44:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.115+0000 7fe72c000700 1 -- start start 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe72c000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe724073f80 0x7fe7241a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe72c000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7240748a0 0x7fe7241a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe72c000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72410e780 0x7fe72419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe72c000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe724114330 con 0x7fe7240748a0 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe72c000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7241141b0 con 0x7fe724073f80 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe72c000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7241144b0 con 0x7fe72410e780 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe728c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7240748a0 0x7fe7241a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe728c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7240748a0 0x7fe7241a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42098/0 (socket says 172.21.15.60:42098) 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe728c00700 1 -- 172.21.15.60:0/3590160270 learned_addr learned my addr 172.21.15.60:0/3590160270 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.116+0000 7fe729600700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe724073f80 0x7fe7241a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe728c00700 1 -- 172.21.15.60:0/3590160270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72410e780 msgr2=0x7fe72419a5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe728c00700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe72410e780 0x7fe72419a5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe728c00700 1 -- 172.21.15.60:0/3590160270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe724073f80 msgr2=0x7fe7241a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe728c00700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe724073f80 0x7fe7241a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe728c00700 1 -- 172.21.15.60:0/3590160270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72419ad50 con 0x7fe7240748a0 2024-02-26T10:44:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe728c00700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7240748a0 0x7fe7241a0920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9a0 tx=0x7fe71000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe722000700 1 -- 172.21.15.60:0/3590160270 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000cbd0 con 0x7fe7240748a0 2024-02-26T10:44:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72419b040 con 0x7fe7240748a0 2024-02-26T10:44:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe722000700 1 -- 172.21.15.60:0/3590160270 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe71000cd70 con 0x7fe7240748a0 2024-02-26T10:44:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.117+0000 7fe722000700 1 -- 172.21.15.60:0/3590160270 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710018750 con 0x7fe7240748a0 2024-02-26T10:44:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.118+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7241a72b0 con 0x7fe7240748a0 2024-02-26T10:44:00.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.118+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7240ff970 con 0x7fe7240748a0 2024-02-26T10:44:00.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.121+0000 7fe722000700 1 -- 172.21.15.60:0/3590160270 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fe710010430 con 0x7fe7240748a0 2024-02-26T10:44:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.122+0000 7fe722000700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe70c0a1ea0 0x7fe70c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.122+0000 7fe722000700 1 -- 172.21.15.60:0/3590160270 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe7100cee00 con 0x7fe7240748a0 2024-02-26T10:44:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.123+0000 7fe729600700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe70c0a1ea0 0x7fe70c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.123+0000 7fe722000700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] conn(0x7fe70c0ad000 0x7fe70c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.123+0000 7fe722000700 1 -- 172.21.15.60:0/3590160270 --> [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe71000ae50 con 0x7fe70c0ad000 2024-02-26T10:44:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.123+0000 7fe722000700 1 -- 172.21.15.60:0/3590160270 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7100cf1b0 con 0x7fe7240748a0 2024-02-26T10:44:00.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.123+0000 7fe72a000700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] conn(0x7fe70c0ad000 0x7fe70c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:00.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.123+0000 7fe729600700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe70c0a1ea0 0x7fe70c0a4360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe718006010 tx=0x7fe71800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:00.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.124+0000 7fe72a000700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] conn(0x7fe70c0ad000 0x7fe70c0af4a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:00.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.125+0000 7fe722000700 1 -- 172.21.15.60:0/3590160270 <== osd.7 v2:172.21.15.100:6816/1677982377 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe71000ae50 con 0x7fe70c0ad000 2024-02-26T10:44:00.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.145+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 --> [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe724075640 con 0x7fe70c0ad000 2024-02-26T10:44:00.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.146+0000 7fe722000700 1 -- 172.21.15.60:0/3590160270 <== osd.7 v2:172.21.15.100:6816/1677982377 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe724075640 con 0x7fe70c0ad000 2024-02-26T10:44:00.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.146+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 >> [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] conn(0x7fe70c0ad000 msgr2=0x7fe70c0af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.146+0000 7fe72c000700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] conn(0x7fe70c0ad000 0x7fe70c0af4a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.147+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe70c0a1ea0 msgr2=0x7fe70c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.147+0000 7fe72c000700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe70c0a1ea0 0x7fe70c0a4360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe718006010 tx=0x7fe71800b260 comp rx=0 tx=0).stop 2024-02-26T10:44:00.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.147+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7240748a0 msgr2=0x7fe7241a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.147+0000 7fe72c000700 1 --2- 172.21.15.60:0/3590160270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7240748a0 0x7fe7241a0920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9a0 tx=0x7fe71000ecb0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.147+0000 7fe72a000700 1 -- 172.21.15.60:0/3590160270 reap_dead start 2024-02-26T10:44:00.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.147+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 shutdown_connections 2024-02-26T10:44:00.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.147+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 >> 172.21.15.60:0/3590160270 conn(0x7fe724078ea0 msgr2=0x7fe7240fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:00.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.147+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 shutdown_connections 2024-02-26T10:44:00.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.147+0000 7fe72c000700 1 -- 172.21.15.60:0/3590160270 wait complete. 2024-02-26T10:44:00.358 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:00 smithi060 ceph-mon[31539]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:00.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:00 smithi100 ceph-mon[36051]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:00.571 INFO:teuthology.orchestra.run.smithi060.stdout:141733920837 2024-02-26T10:44:00.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-26T10:44:00.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:00 smithi113 ceph-mon[36126]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:00.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.592+0000 7f69fdc00700 1 -- 172.21.15.60:0/450776921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f80748a0 msgr2=0x7f69f81129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.592+0000 7f69fdc00700 1 --2- 172.21.15.60:0/450776921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f80748a0 0x7f69f81129e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f69e00099c0 tx=0x7f69e0009cd0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.593+0000 7f69fdc00700 1 -- 172.21.15.60:0/450776921 shutdown_connections 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.593+0000 7f69fdc00700 1 --2- 172.21.15.60:0/450776921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69f8112f20 0x7f69f8115310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.593+0000 7f69fdc00700 1 --2- 172.21.15.60:0/450776921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69f8073f80 0x7f69f8074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.593+0000 7f69fdc00700 1 --2- 172.21.15.60:0/450776921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f80748a0 0x7f69f81129e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.593+0000 7f69fdc00700 1 -- 172.21.15.60:0/450776921 >> 172.21.15.60:0/450776921 conn(0x7f69f8078f40 msgr2=0x7f69f8079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.593+0000 7f69fdc00700 1 -- 172.21.15.60:0/450776921 shutdown_connections 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.593+0000 7f69fdc00700 1 -- 172.21.15.60:0/450776921 wait complete. 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.594+0000 7f69fdc00700 1 Processor -- start 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.594+0000 7f69fdc00700 1 -- start start 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.594+0000 7f69fdc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69f8073f80 0x7f69f81a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.594+0000 7f69fdc00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69f80748a0 0x7f69f81a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.594+0000 7f69fdc00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f8112f20 0x7f69f819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.594+0000 7f69fdc00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f81186c0 con 0x7f69f8073f80 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.594+0000 7f69fdc00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8118540 con 0x7f69f8112f20 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.594+0000 7f69fdc00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8118840 con 0x7f69f80748a0 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69f8073f80 0x7f69f81a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69f8073f80 0x7f69f81a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42120/0 (socket says 172.21.15.60:42120) 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f6a00700 1 -- 172.21.15.60:0/207571717 learned_addr learned my addr 172.21.15.60:0/207571717 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f7400700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f8112f20 0x7f69f819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69efe00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69f80748a0 0x7f69f81a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f7400700 1 -- 172.21.15.60:0/207571717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69f80748a0 msgr2=0x7f69f81a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f7400700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69f80748a0 0x7f69f81a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f7400700 1 -- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69f8073f80 msgr2=0x7f69f81a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f7400700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69f8073f80 0x7f69f81a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f7400700 1 -- 172.21.15.60:0/207571717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f819f060 con 0x7f69f8112f20 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69f6a00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69f8073f80 0x7f69f81a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.595+0000 7f69efe00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69f80748a0 0x7f69f81a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:00.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.596+0000 7f69f7400700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f8112f20 0x7f69f819e900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69f000efc0 tx=0x7f69f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.596+0000 7f69ef400700 1 -- 172.21.15.60:0/207571717 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f00092e0 con 0x7f69f8112f20 2024-02-26T10:44:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.596+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f819f2f0 con 0x7f69f8112f20 2024-02-26T10:44:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.596+0000 7f69ef400700 1 -- 172.21.15.60:0/207571717 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69f0004bb0 con 0x7f69f8112f20 2024-02-26T10:44:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.596+0000 7f69ef400700 1 -- 172.21.15.60:0/207571717 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0004280 con 0x7f69f8112f20 2024-02-26T10:44:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.596+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f81ab610 con 0x7f69f8112f20 2024-02-26T10:44:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.597+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f69f810f3a0 con 0x7f69f8112f20 2024-02-26T10:44:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.600+0000 7f69ef400700 1 -- 172.21.15.60:0/207571717 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f69f001b030 con 0x7f69f8112f20 2024-02-26T10:44:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.602+0000 7f69ef400700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f69dc0a1f20 0x7f69dc0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.602+0000 7f69ef400700 1 -- 172.21.15.60:0/207571717 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f69f00ce5f0 con 0x7f69f8112f20 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.602+0000 7f69f6a00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f69dc0a1f20 0x7f69dc0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.602+0000 7f69ef400700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] conn(0x7f69dc0ad080 0x7f69dc0af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.602+0000 7f69ef400700 1 -- 172.21.15.60:0/207571717 --> [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69f001fe50 con 0x7f69dc0ad080 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.602+0000 7f69ef400700 1 -- 172.21.15.60:0/207571717 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69f00ce9a0 con 0x7f69f8112f20 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.603+0000 7f69efe00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] conn(0x7f69dc0ad080 0x7f69dc0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.603+0000 7f69f6a00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f69dc0a1f20 0x7f69dc0a43e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f69f819fa40 tx=0x7f69e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.603+0000 7f69efe00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] conn(0x7f69dc0ad080 0x7f69dc0af520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.604+0000 7f69ef400700 1 -- 172.21.15.60:0/207571717 <== osd.3 v2:172.21.15.60:6810/3095490526 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f69f001fe50 con 0x7f69dc0ad080 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.623+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 --> [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f69f8075690 con 0x7f69dc0ad080 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69ef400700 1 -- 172.21.15.60:0/207571717 <== osd.3 v2:172.21.15.60:6810/3095490526 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f69f8075690 con 0x7f69dc0ad080 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] conn(0x7f69dc0ad080 msgr2=0x7f69dc0af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69fdc00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] conn(0x7f69dc0ad080 0x7f69dc0af520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f69dc0a1f20 msgr2=0x7f69dc0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69fdc00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f69dc0a1f20 0x7f69dc0a43e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f69f819fa40 tx=0x7f69e8009260 comp rx=0 tx=0).stop 2024-02-26T10:44:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f8112f20 msgr2=0x7f69f819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:00.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69fdc00700 1 --2- 172.21.15.60:0/207571717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69f8112f20 0x7f69f819e900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69f000efc0 tx=0x7f69f000c3a0 comp rx=0 tx=0).stop 2024-02-26T10:44:00.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69f7400700 1 -- 172.21.15.60:0/207571717 reap_dead start 2024-02-26T10:44:00.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 shutdown_connections 2024-02-26T10:44:00.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 >> 172.21.15.60:0/207571717 conn(0x7f69f8078f40 msgr2=0x7f69f810e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:00.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.624+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 shutdown_connections 2024-02-26T10:44:00.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.625+0000 7f69fdc00700 1 -- 172.21.15.60:0/207571717 wait complete. 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.826+0000 7ff57c200700 1 -- 172.21.15.60:0/76078297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff57410e610 msgr2=0x7ff574110a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.826+0000 7ff57c200700 1 --2- 172.21.15.60:0/76078297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff57410e610 0x7ff574110a20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00b0a0 tx=0x7ff56c00b3b0 comp rx=0 tx=0).stop 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.826+0000 7ff57c200700 1 -- 172.21.15.60:0/76078297 shutdown_connections 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.826+0000 7ff57c200700 1 --2- 172.21.15.60:0/76078297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff57410e610 0x7ff574110a20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.826+0000 7ff57c200700 1 --2- 172.21.15.60:0/76078297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff574074120 0x7ff574074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.826+0000 7ff57c200700 1 --2- 172.21.15.60:0/76078297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff574074a40 0x7ff57410e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.826+0000 7ff57c200700 1 -- 172.21.15.60:0/76078297 >> 172.21.15.60:0/76078297 conn(0x7ff5740fd820 msgr2=0x7ff5740ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.826+0000 7ff57c200700 1 -- 172.21.15.60:0/76078297 shutdown_connections 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.826+0000 7ff57c200700 1 -- 172.21.15.60:0/76078297 wait complete. 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.828+0000 7ff57c200700 1 Processor -- start 2024-02-26T10:44:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.828+0000 7ff57c200700 1 -- start start 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff57c200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff574074120 0x7ff57419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff57c200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff574074a40 0x7ff57419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff57c200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff57410e610 0x7ff57419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff57c200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741142f0 con 0x7ff57410e610 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff57c200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574114170 con 0x7ff574074120 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff57c200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574114470 con 0x7ff574074a40 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff579800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff574074120 0x7ff57419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff57a200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff57410e610 0x7ff57419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff579800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff574074120 0x7ff57419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38254/0 (socket says 172.21.15.60:38254) 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff57a200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff57410e610 0x7ff57419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42138/0 (socket says 172.21.15.60:42138) 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff579800700 1 -- 172.21.15.60:0/116346475 learned_addr learned my addr 172.21.15.60:0/116346475 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.829+0000 7ff578e00700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff574074a40 0x7ff57419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff578e00700 1 -- 172.21.15.60:0/116346475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff574074120 msgr2=0x7ff57419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff578e00700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff574074120 0x7ff57419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff578e00700 1 -- 172.21.15.60:0/116346475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff57410e610 msgr2=0x7ff57419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff578e00700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff57410e610 0x7ff57419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff578e00700 1 -- 172.21.15.60:0/116346475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57419f710 con 0x7ff574074a40 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff579800700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff574074120 0x7ff57419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff57a200700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff57410e610 0x7ff57419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff578e00700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff574074a40 0x7ff57419ac40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9f0 tx=0x7ff56000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff572000700 1 -- 172.21.15.60:0/116346475 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56000c9c0 con 0x7ff574074a40 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff572000700 1 -- 172.21.15.60:0/116346475 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff56000cb60 con 0x7ff574074a40 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57419fa00 con 0x7ff574074a40 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff572000700 1 -- 172.21.15.60:0/116346475 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560018680 con 0x7ff574074a40 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.830+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5741a7280 con 0x7ff574074a40 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.831+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5740624e0 con 0x7ff574074a40 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.834+0000 7ff572000700 1 -- 172.21.15.60:0/116346475 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff560010430 con 0x7ff574074a40 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.835+0000 7ff572000700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff55c0a1f20 0x7ff55c0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.835+0000 7ff572000700 1 -- 172.21.15.60:0/116346475 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff5600ce780 con 0x7ff574074a40 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.835+0000 7ff579800700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff55c0a1f20 0x7ff55c0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.835+0000 7ff572000700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] conn(0x7ff55c0ad080 0x7ff55c0af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.835+0000 7ff572000700 1 -- 172.21.15.60:0/116346475 --> [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff56001bec0 con 0x7ff55c0ad080 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.835+0000 7ff572000700 1 -- 172.21.15.60:0/116346475 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5600ceb30 con 0x7ff574074a40 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.836+0000 7ff57a200700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] conn(0x7ff55c0ad080 0x7ff55c0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.836+0000 7ff579800700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff55c0a1f20 0x7ff55c0a43e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff568011590 tx=0x7ff568009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.837+0000 7ff57a200700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] conn(0x7ff55c0ad080 0x7ff55c0af520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.838+0000 7ff572000700 1 -- 172.21.15.60:0/116346475 <== osd.1 v2:172.21.15.100:6800/3893830913 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff56001bec0 con 0x7ff55c0ad080 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.856+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 --> [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff574062910 con 0x7ff55c0ad080 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.857+0000 7ff572000700 1 -- 172.21.15.60:0/116346475 <== osd.1 v2:172.21.15.100:6800/3893830913 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff574062910 con 0x7ff55c0ad080 2024-02-26T10:44:01.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.857+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 >> [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] conn(0x7ff55c0ad080 msgr2=0x7ff55c0af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.857+0000 7ff57c200700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] conn(0x7ff55c0ad080 0x7ff55c0af520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.857+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff55c0a1f20 msgr2=0x7ff55c0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.857+0000 7ff57c200700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff55c0a1f20 0x7ff55c0a43e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff568011590 tx=0x7ff568009260 comp rx=0 tx=0).stop 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.857+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff574074a40 msgr2=0x7ff57419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.857+0000 7ff57c200700 1 --2- 172.21.15.60:0/116346475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff574074a40 0x7ff57419ac40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9f0 tx=0x7ff56000ed00 comp rx=0 tx=0).stop 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.858+0000 7ff57a200700 1 -- 172.21.15.60:0/116346475 reap_dead start 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.858+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 shutdown_connections 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.858+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 >> 172.21.15.60:0/116346475 conn(0x7ff5740fd820 msgr2=0x7ff5740fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.858+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 shutdown_connections 2024-02-26T10:44:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:00.858+0000 7ff57c200700 1 -- 172.21.15.60:0/116346475 wait complete. 2024-02-26T10:44:02.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:02 smithi060 ceph-mon[31539]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:02.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:02 smithi100 ceph-mon[36051]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:02.508 INFO:teuthology.orchestra.run.smithi060.stdout:107374182474 2024-02-26T10:44:02.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-26T10:44:02.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:02 smithi113 ceph-mon[36126]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:03.036 INFO:teuthology.orchestra.run.smithi060.stdout:167503724608 2024-02-26T10:44:03.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-26T10:44:03.577 INFO:teuthology.orchestra.run.smithi060.stdout:55834574935 2024-02-26T10:44:03.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-26T10:44:03.948 INFO:teuthology.orchestra.run.smithi060.stdout:313532612642 2024-02-26T10:44:03.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-26T10:44:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.358+0000 7f7164600700 1 -- 172.21.15.60:0/2508629860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f715c069a00 msgr2=0x7f715c069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.358+0000 7f7164600700 1 --2- 172.21.15.60:0/2508629860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f715c069a00 0x7f715c069e80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f71480099c0 tx=0x7f7148009cd0 comp rx=0 tx=0).stop 2024-02-26T10:44:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.359+0000 7f7164600700 1 -- 172.21.15.60:0/2508629860 shutdown_connections 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.359+0000 7f7164600700 1 --2- 172.21.15.60:0/2508629860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06a3c0 0x7f715c10eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.359+0000 7f7164600700 1 --2- 172.21.15.60:0/2508629860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c107d80 0x7f715c108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.359+0000 7f7164600700 1 --2- 172.21.15.60:0/2508629860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f715c069a00 0x7f715c069e80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.359+0000 7f7164600700 1 -- 172.21.15.60:0/2508629860 >> 172.21.15.60:0/2508629860 conn(0x7f715c077a00 msgr2=0x7f715c077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.359+0000 7f7164600700 1 -- 172.21.15.60:0/2508629860 shutdown_connections 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.360+0000 7f7164600700 1 -- 172.21.15.60:0/2508629860 wait complete. 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7164600700 1 Processor -- start 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7164600700 1 -- start start 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7164600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f715c069a00 0x7f715c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7164600700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06a3c0 0x7f715c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7164600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c107d80 0x7f715c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7164600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c112080 con 0x7f715c107d80 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7164600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c111f00 con 0x7f715c069a00 2024-02-26T10:44:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7164600700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c112200 con 0x7f715c06a3c0 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7161c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f715c069a00 0x7f715c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7162600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c107d80 0x7f715c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7161c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f715c069a00 0x7f715c19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38284/0 (socket says 172.21.15.60:38284) 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.361+0000 7f7161c00700 1 -- 172.21.15.60:0/166060358 learned_addr learned my addr 172.21.15.60:0/166060358 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f7161200700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06a3c0 0x7f715c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f7161c00700 1 -- 172.21.15.60:0/166060358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06a3c0 msgr2=0x7f715c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f7161c00700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06a3c0 0x7f715c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f7161c00700 1 -- 172.21.15.60:0/166060358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c107d80 msgr2=0x7f715c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f7161c00700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c107d80 0x7f715c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f7161c00700 1 -- 172.21.15.60:0/166060358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c19f750 con 0x7f715c069a00 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f7162600700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c107d80 0x7f715c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f7161c00700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f715c069a00 0x7f715c19a740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f715000e9a0 tx=0x7f715000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f715a000700 1 -- 172.21.15.60:0/166060358 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715000ca10 con 0x7f715c069a00 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f715a000700 1 -- 172.21.15.60:0/166060358 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f715000cbb0 con 0x7f715c069a00 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f715a000700 1 -- 172.21.15.60:0/166060358 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150018680 con 0x7f715c069a00 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.362+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c19fa40 con 0x7f715c069a00 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.363+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c1a72c0 con 0x7f715c069a00 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.364+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f715c10d870 con 0x7f715c069a00 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.366+0000 7f715a000700 1 -- 172.21.15.60:0/166060358 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f7150010430 con 0x7f715c069a00 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.367+0000 7f715a000700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71440a1ea0 0x7f71440a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.367+0000 7f715a000700 1 -- 172.21.15.60:0/166060358 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f71500cece0 con 0x7f715c069a00 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.367+0000 7f7161200700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71440a1ea0 0x7f71440a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.367+0000 7f715a000700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] conn(0x7f71440ad000 0x7f71440af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.367+0000 7f715a000700 1 -- 172.21.15.60:0/166060358 --> [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7150019ec0 con 0x7f71440ad000 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.367+0000 7f715a000700 1 -- 172.21.15.60:0/166060358 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71500cf090 con 0x7f715c069a00 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.368+0000 7f7162600700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] conn(0x7f71440ad000 0x7f71440af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.368+0000 7f7161200700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71440a1ea0 0x7f71440a4360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f715c1030b0 tx=0x7f7148005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.368+0000 7f7162600700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] conn(0x7f71440ad000 0x7f71440af4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:04.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.369+0000 7f715a000700 1 -- 172.21.15.60:0/166060358 <== osd.11 v2:172.21.15.113:6824/2587204649 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7150019ec0 con 0x7f71440ad000 2024-02-26T10:44:04.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.388+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 --> [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f715c103410 con 0x7f71440ad000 2024-02-26T10:44:04.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.388+0000 7f715a000700 1 -- 172.21.15.60:0/166060358 <== osd.11 v2:172.21.15.113:6824/2587204649 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f715c103410 con 0x7f71440ad000 2024-02-26T10:44:04.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 >> [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] conn(0x7f71440ad000 msgr2=0x7f71440af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:04.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] conn(0x7f71440ad000 0x7f71440af4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:04.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71440a1ea0 msgr2=0x7f71440a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:04.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71440a1ea0 0x7f71440a4360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f715c1030b0 tx=0x7f7148005fb0 comp rx=0 tx=0).stop 2024-02-26T10:44:04.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f715c069a00 msgr2=0x7f715c19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 --2- 172.21.15.60:0/166060358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f715c069a00 0x7f715c19a740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f715000e9a0 tx=0x7f715000ecb0 comp rx=0 tx=0).stop 2024-02-26T10:44:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7162600700 1 -- 172.21.15.60:0/166060358 reap_dead start 2024-02-26T10:44:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 shutdown_connections 2024-02-26T10:44:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 >> 172.21.15.60:0/166060358 conn(0x7f715c077a00 msgr2=0x7f715c10ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 shutdown_connections 2024-02-26T10:44:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:03.389+0000 7f7164600700 1 -- 172.21.15.60:0/166060358 wait complete. 2024-02-26T10:44:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:04 smithi060 ceph-mon[31539]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:04.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:04 smithi100 ceph-mon[36051]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:04.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:04 smithi113 ceph-mon[36126]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:05.190 INFO:teuthology.orchestra.run.smithi060.stdout:223338299444 2024-02-26T10:44:05.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-26T10:44:05.718 INFO:teuthology.orchestra.run.smithi060.stdout:347892351006 2024-02-26T10:44:05.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-26T10:44:06.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:06 smithi060 ceph-mon[31539]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T10:44:06.326 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:06.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.330+0000 7f1134800700 1 -- 172.21.15.60:0/732756217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112c101f70 msgr2=0x7f112c102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:06.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.330+0000 7f1134800700 1 --2- 172.21.15.60:0/732756217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112c101f70 0x7f112c102350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f11200098f0 tx=0x7f1120009c00 comp rx=0 tx=0).stop 2024-02-26T10:44:06.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.330+0000 7f1134800700 1 -- 172.21.15.60:0/732756217 shutdown_connections 2024-02-26T10:44:06.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.330+0000 7f1134800700 1 --2- 172.21.15.60:0/732756217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c107050 0x7f112c10ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:06.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.330+0000 7f1134800700 1 --2- 172.21.15.60:0/732756217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112c101f70 0x7f112c102350 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:06.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.330+0000 7f1134800700 1 --2- 172.21.15.60:0/732756217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c102920 0x7f112c106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:06.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.330+0000 7f1134800700 1 -- 172.21.15.60:0/732756217 >> 172.21.15.60:0/732756217 conn(0x7f112c0fd7c0 msgr2=0x7f112c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:06.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.331+0000 7f1134800700 1 -- 172.21.15.60:0/732756217 shutdown_connections 2024-02-26T10:44:06.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.331+0000 7f1134800700 1 -- 172.21.15.60:0/732756217 wait complete. 2024-02-26T10:44:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.333+0000 7f1134800700 1 Processor -- start 2024-02-26T10:44:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.333+0000 7f1134800700 1 -- start start 2024-02-26T10:44:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.333+0000 7f1134800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112c101f70 0x7f112c074050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.333+0000 7f1134800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c102920 0x7f112c074590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.333+0000 7f1134800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c107050 0x7f112c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.333+0000 7f1134800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c112080 con 0x7f112c107050 2024-02-26T10:44:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.333+0000 7f1134800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c111f00 con 0x7f112c101f70 2024-02-26T10:44:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.333+0000 7f1134800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c112200 con 0x7f112c102920 2024-02-26T10:44:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.334+0000 7f1132800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c107050 0x7f112c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.334+0000 7f1132800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c107050 0x7f112c074ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58156/0 (socket says 172.21.15.60:58156) 2024-02-26T10:44:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.334+0000 7f1132800700 1 -- 172.21.15.60:0/2512304010 learned_addr learned my addr 172.21.15.60:0/2512304010 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.334+0000 7f1131e00700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112c101f70 0x7f112c074050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.334+0000 7f1131400700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c102920 0x7f112c074590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.335+0000 7f1132800700 1 -- 172.21.15.60:0/2512304010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c102920 msgr2=0x7f112c074590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.335+0000 7f1132800700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c102920 0x7f112c074590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.335+0000 7f1132800700 1 -- 172.21.15.60:0/2512304010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112c101f70 msgr2=0x7f112c074050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.335+0000 7f1132800700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112c101f70 0x7f112c074050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.335+0000 7f1132800700 1 -- 172.21.15.60:0/2512304010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c1aaf40 con 0x7f112c107050 2024-02-26T10:44:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.335+0000 7f1131e00700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f112c101f70 0x7f112c074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:44:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.335+0000 7f1132800700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c107050 0x7f112c074ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f112800ef30 tx=0x7f112800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.335+0000 7f1126a00700 1 -- 172.21.15.60:0/2512304010 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11280092e0 con 0x7f112c107050 2024-02-26T10:44:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.336+0000 7f1126a00700 1 -- 172.21.15.60:0/2512304010 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1128004bb0 con 0x7f112c107050 2024-02-26T10:44:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.336+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c1ab170 con 0x7f112c107050 2024-02-26T10:44:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.336+0000 7f1126a00700 1 -- 172.21.15.60:0/2512304010 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128004280 con 0x7f112c107050 2024-02-26T10:44:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.336+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c1ab630 con 0x7f112c107050 2024-02-26T10:44:06.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.340+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1110001030 con 0x7f112c107050 2024-02-26T10:44:06.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.342+0000 7f1126a00700 1 -- 172.21.15.60:0/2512304010 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f112801b030 con 0x7f112c107050 2024-02-26T10:44:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.345+0000 7f1126a00700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11140a2120 0x7f11140a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.345+0000 7f1126a00700 1 -- 172.21.15.60:0/2512304010 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f1128004020 con 0x7f112c107050 2024-02-26T10:44:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.346+0000 7f1126a00700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] conn(0x7f11140ad280 0x7f11140af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.346+0000 7f1126a00700 1 -- 172.21.15.60:0/2512304010 --> [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11140afdf0 con 0x7f11140ad280 2024-02-26T10:44:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.346+0000 7f1126a00700 1 -- 172.21.15.60:0/2512304010 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11280d22a0 con 0x7f112c107050 2024-02-26T10:44:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.347+0000 7f1131400700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] conn(0x7f11140ad280 0x7f11140af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.347+0000 7f1131e00700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11140a2120 0x7f11140a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.349+0000 7f1131400700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] conn(0x7f11140ad280 0x7f11140af720 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.349+0000 7f1131e00700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11140a2120 0x7f11140a45e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1120009810 tx=0x7f1120019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:06.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.351+0000 7f1126a00700 1 -- 172.21.15.60:0/2512304010 <== osd.9 v2:172.21.15.60:6826/3674402617 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f11140afdf0 con 0x7f11140ad280 2024-02-26T10:44:06.355 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:06.361 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:06.367 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:06.370 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:06.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:06 smithi100 ceph-mon[36051]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T10:44:06.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.394+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 --> [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1110002e60 con 0x7f11140ad280 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.398+0000 7f1126a00700 1 -- 172.21.15.60:0/2512304010 <== osd.9 v2:172.21.15.60:6826/3674402617 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1110002e60 con 0x7f11140ad280 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.399+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] conn(0x7f11140ad280 msgr2=0x7f11140af720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.399+0000 7f1134800700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] conn(0x7f11140ad280 0x7f11140af720 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.399+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11140a2120 msgr2=0x7f11140a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.399+0000 7f1134800700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11140a2120 0x7f11140a45e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1120009810 tx=0x7f1120019040 comp rx=0 tx=0).stop 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.399+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c107050 msgr2=0x7f112c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.399+0000 7f1134800700 1 --2- 172.21.15.60:0/2512304010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c107050 0x7f112c074ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f112800ef30 tx=0x7f112800c450 comp rx=0 tx=0).stop 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.399+0000 7f1132800700 1 -- 172.21.15.60:0/2512304010 reap_dead start 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.400+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 shutdown_connections 2024-02-26T10:44:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.400+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 >> 172.21.15.60:0/2512304010 conn(0x7f112c0fd7c0 msgr2=0x7f112c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.405+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 shutdown_connections 2024-02-26T10:44:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:06.405+0000 7f1134800700 1 -- 172.21.15.60:0/2512304010 wait complete. 2024-02-26T10:44:06.411 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:06.416 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:06.441 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:06.443 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:06.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:06 smithi113 ceph-mon[36126]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T10:44:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:08.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:08 smithi060 ceph-mon[31539]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:08.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:08 smithi100 ceph-mon[36051]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:08.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:08 smithi113 ceph-mon[36126]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:10.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:10 smithi060 ceph-mon[31539]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-02-26T10:44:10.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:10 smithi100 ceph-mon[36051]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-02-26T10:44:10.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:10 smithi113 ceph-mon[36126]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-02-26T10:44:12.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:12 smithi060 ceph-mon[31539]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:12.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:12 smithi100 ceph-mon[36051]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:12.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:12 smithi113 ceph-mon[36126]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:14.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:14 smithi060 ceph-mon[31539]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:14.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:14 smithi113 ceph-mon[36126]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:14.616 INFO:teuthology.orchestra.run.smithi060.stdout:292057776170 2024-02-26T10:44:14.617 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-26T10:44:14.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:14 smithi100 ceph-mon[36051]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.861+0000 7f7d51800700 1 -- 172.21.15.60:0/3131849999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4c10d680 msgr2=0x7f7d4c115250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.861+0000 7f7d51800700 1 --2- 172.21.15.60:0/3131849999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4c10d680 0x7f7d4c115250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400b0d0 tx=0x7f7d4400b3e0 comp rx=0 tx=0).stop 2024-02-26T10:44:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.861+0000 7f7d51800700 1 -- 172.21.15.60:0/3131849999 shutdown_connections 2024-02-26T10:44:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.861+0000 7f7d51800700 1 --2- 172.21.15.60:0/3131849999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4c10d680 0x7f7d4c115250 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.861+0000 7f7d51800700 1 --2- 172.21.15.60:0/3131849999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c1085a0 0x7f7d4c108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.861+0000 7f7d51800700 1 --2- 172.21.15.60:0/3131849999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c108f50 0x7f7d4c10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.861+0000 7f7d51800700 1 -- 172.21.15.60:0/3131849999 >> 172.21.15.60:0/3131849999 conn(0x7f7d4c075520 msgr2=0x7f7d4c077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.861+0000 7f7d51800700 1 -- 172.21.15.60:0/3131849999 shutdown_connections 2024-02-26T10:44:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.861+0000 7f7d51800700 1 -- 172.21.15.60:0/3131849999 wait complete. 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.862+0000 7f7d51800700 1 Processor -- start 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d51800700 1 -- start start 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d51800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c1085a0 0x7f7d4c1a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d51800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c108f50 0x7f7d4c1a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d51800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4c10d680 0x7f7d4c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d51800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c1186e0 con 0x7f7d4c10d680 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d51800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c118560 con 0x7f7d4c1085a0 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d51800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c118860 con 0x7f7d4c108f50 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d4aa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c1085a0 0x7f7d4c1a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d4b400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4c10d680 0x7f7d4c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d4aa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c1085a0 0x7f7d4c1a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51432/0 (socket says 172.21.15.60:51432) 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d4aa00700 1 -- 172.21.15.60:0/3303878266 learned_addr learned my addr 172.21.15.60:0/3303878266 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.863+0000 7f7d4a000700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c108f50 0x7f7d4c1a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.864+0000 7f7d4b400700 1 -- 172.21.15.60:0/3303878266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c108f50 msgr2=0x7f7d4c1a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.864+0000 7f7d4b400700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c108f50 0x7f7d4c1a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.864+0000 7f7d4b400700 1 -- 172.21.15.60:0/3303878266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c1085a0 msgr2=0x7f7d4c1a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.864+0000 7f7d4b400700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c1085a0 0x7f7d4c1a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.864+0000 7f7d4b400700 1 -- 172.21.15.60:0/3303878266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c19f0c0 con 0x7f7d4c10d680 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.864+0000 7f7d4a000700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c108f50 0x7f7d4c1a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.864+0000 7f7d4aa00700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c1085a0 0x7f7d4c1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.864+0000 7f7d4b400700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4c10d680 0x7f7d4c19e960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400d010 tx=0x7f7d44009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.865+0000 7f7d43400700 1 -- 172.21.15.60:0/3303878266 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44003a00 con 0x7f7d4c10d680 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.865+0000 7f7d43400700 1 -- 172.21.15.60:0/3303878266 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d44022070 con 0x7f7d4c10d680 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.865+0000 7f7d43400700 1 -- 172.21.15.60:0/3303878266 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4401b6c0 con 0x7f7d4c10d680 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.865+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4c19f350 con 0x7f7d4c10d680 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.865+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4c1ab670 con 0x7f7d4c10d680 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.866+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4c109a80 con 0x7f7d4c10d680 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.868+0000 7f7d43400700 1 -- 172.21.15.60:0/3303878266 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f7d44003ba0 con 0x7f7d4c10d680 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.869+0000 7f7d43400700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7d300a1f20 0x7f7d300a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.869+0000 7f7d43400700 1 -- 172.21.15.60:0/3303878266 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f7d4400f020 con 0x7f7d4c10d680 2024-02-26T10:44:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.869+0000 7f7d4aa00700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7d300a1f20 0x7f7d300a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.870+0000 7f7d4aa00700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7d300a1f20 0x7f7d300a43e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c19faa0 tx=0x7f7d3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:15.874+0000 7f7d43400700 1 -- 172.21.15.60:0/3303878266 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d4409bd70 con 0x7f7d4c10d680 2024-02-26T10:44:16.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.103+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7d4c1ab840 con 0x7f7d4c10d680 2024-02-26T10:44:16.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.104+0000 7f7d43400700 1 -- 172.21.15.60:0/3303878266 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d4409b670 con 0x7f7d4c10d680 2024-02-26T10:44:16.100 INFO:teuthology.orchestra.run.smithi060.stdout:141733920842 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7d300a1f20 msgr2=0x7f7d300a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7d300a1f20 0x7f7d300a43e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c19faa0 tx=0x7f7d3c009260 comp rx=0 tx=0).stop 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4c10d680 msgr2=0x7f7d4c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4c10d680 0x7f7d4c19e960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400d010 tx=0x7f7d44009810 comp rx=0 tx=0).stop 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 shutdown_connections 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7d300a1f20 0x7f7d300a43e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4c10d680 0x7f7d4c19e960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d4c1085a0 0x7f7d4c1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 --2- 172.21.15.60:0/3303878266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c108f50 0x7f7d4c1a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.107+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 >> 172.21.15.60:0/3303878266 conn(0x7f7d4c075520 msgr2=0x7f7d4c075e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.108+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 shutdown_connections 2024-02-26T10:44:16.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.108+0000 7f7d51800700 1 -- 172.21.15.60:0/3303878266 wait complete. 2024-02-26T10:44:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.179+0000 7f8d70800700 1 -- 172.21.15.60:0/81899082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d681029b0 msgr2=0x7f8d681069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.179+0000 7f8d70800700 1 --2- 172.21.15.60:0/81899082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d681029b0 0x7f8d681069f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8d580099c0 tx=0x7f8d58009cd0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.179+0000 7f8d70800700 1 -- 172.21.15.60:0/81899082 shutdown_connections 2024-02-26T10:44:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.179+0000 7f8d70800700 1 --2- 172.21.15.60:0/81899082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d681070d0 0x7f8d6810eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.179+0000 7f8d70800700 1 --2- 172.21.15.60:0/81899082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102000 0x7f8d681023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.179+0000 7f8d70800700 1 --2- 172.21.15.60:0/81899082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d681029b0 0x7f8d681069f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.179+0000 7f8d70800700 1 -- 172.21.15.60:0/81899082 >> 172.21.15.60:0/81899082 conn(0x7f8d68078eb0 msgr2=0x7f8d680792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.179+0000 7f8d70800700 1 -- 172.21.15.60:0/81899082 shutdown_connections 2024-02-26T10:44:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.179+0000 7f8d70800700 1 -- 172.21.15.60:0/81899082 wait complete. 2024-02-26T10:44:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.180+0000 7f8d70800700 1 Processor -- start 2024-02-26T10:44:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.181+0000 7f8d70800700 1 -- start start 2024-02-26T10:44:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.181+0000 7f8d70800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102000 0x7f8d681a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.181+0000 7f8d70800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d681029b0 0x7f8d681a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.181+0000 7f8d70800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d681070d0 0x7f8d6819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.181+0000 7f8d70800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68112110 con 0x7f8d681070d0 2024-02-26T10:44:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.181+0000 7f8d70800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68111f90 con 0x7f8d68102000 2024-02-26T10:44:16.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.181+0000 7f8d70800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68112290 con 0x7f8d681029b0 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.181+0000 7f8d6de00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102000 0x7f8d681a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6e800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d681070d0 0x7f8d6819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6de00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102000 0x7f8d681a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51454/0 (socket says 172.21.15.60:51454) 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6de00700 1 -- 172.21.15.60:0/3047748003 learned_addr learned my addr 172.21.15.60:0/3047748003 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6d400700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d681029b0 0x7f8d681a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6e800700 1 -- 172.21.15.60:0/3047748003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d681029b0 msgr2=0x7f8d681a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6e800700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d681029b0 0x7f8d681a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6e800700 1 -- 172.21.15.60:0/3047748003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102000 msgr2=0x7f8d681a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6e800700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102000 0x7f8d681a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6e800700 1 -- 172.21.15.60:0/3047748003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6819af60 con 0x7f8d681070d0 2024-02-26T10:44:16.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.182+0000 7f8d6d400700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d681029b0 0x7f8d681a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:16.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.183+0000 7f8d6e800700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d681070d0 0x7f8d6819a680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400efc0 tx=0x7f8d6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:16.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.183+0000 7f8d62a00700 1 -- 172.21.15.60:0/3047748003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640092e0 con 0x7f8d681070d0 2024-02-26T10:44:16.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.183+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6819b1f0 con 0x7f8d681070d0 2024-02-26T10:44:16.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.183+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d681a7530 con 0x7f8d681070d0 2024-02-26T10:44:16.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.183+0000 7f8d62a00700 1 -- 172.21.15.60:0/3047748003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d64004bb0 con 0x7f8d681070d0 2024-02-26T10:44:16.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.183+0000 7f8d62a00700 1 -- 172.21.15.60:0/3047748003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64004280 con 0x7f8d681070d0 2024-02-26T10:44:16.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.184+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d681034d0 con 0x7f8d681070d0 2024-02-26T10:44:16.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.187+0000 7f8d62a00700 1 -- 172.21.15.60:0/3047748003 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f8d6401b030 con 0x7f8d681070d0 2024-02-26T10:44:16.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.188+0000 7f8d62a00700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d500a1f20 0x7f8d500a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:16.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.188+0000 7f8d62a00700 1 -- 172.21.15.60:0/3047748003 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8d640cdbd0 con 0x7f8d681070d0 2024-02-26T10:44:16.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.188+0000 7f8d6de00700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d500a1f20 0x7f8d500a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:16.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.189+0000 7f8d6de00700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d500a1f20 0x7f8d500a43e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009d20 tx=0x7f8d5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:16.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.194+0000 7f8d62a00700 1 -- 172.21.15.60:0/3047748003 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d64022020 con 0x7f8d681070d0 2024-02-26T10:44:16.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:16 smithi060 ceph-mon[31539]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:16.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:16 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3303878266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T10:44:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.433+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8d6819bbb0 con 0x7f8d681070d0 2024-02-26T10:44:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.434+0000 7f8d62a00700 1 -- 172.21.15.60:0/3047748003 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8d64010040 con 0x7f8d681070d0 2024-02-26T10:44:16.430 INFO:teuthology.orchestra.run.smithi060.stdout:55834574938 2024-02-26T10:44:16.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d500a1f20 msgr2=0x7f8d500a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:16.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d500a1f20 0x7f8d500a43e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009d20 tx=0x7f8d5c009260 comp rx=0 tx=0).stop 2024-02-26T10:44:16.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d681070d0 msgr2=0x7f8d6819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:16.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d681070d0 0x7f8d6819a680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400efc0 tx=0x7f8d6400c3a0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 shutdown_connections 2024-02-26T10:44:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8d500a1f20 0x7f8d500a43e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d681070d0 0x7f8d6819a680 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d68102000 0x7f8d681a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 --2- 172.21.15.60:0/3047748003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d681029b0 0x7f8d681a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 >> 172.21.15.60:0/3047748003 conn(0x7f8d68078eb0 msgr2=0x7f8d680ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 shutdown_connections 2024-02-26T10:44:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:16.437+0000 7f8d70800700 1 -- 172.21.15.60:0/3047748003 wait complete. 2024-02-26T10:44:16.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:16 smithi113 ceph-mon[36126]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:16.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:16 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3303878266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T10:44:16.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:16 smithi100 ceph-mon[36051]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:16.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:16 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3303878266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T10:44:17.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:17 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3047748003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T10:44:17.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:17 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3047748003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T10:44:17.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:17 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3047748003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T10:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.214+0000 7f99f8a00700 1 -- 172.21.15.60:0/4035187931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f41070d0 msgr2=0x7f99f410eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.214+0000 7f99f8a00700 1 --2- 172.21.15.60:0/4035187931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f41070d0 0x7f99f410eca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009920 tx=0x7f99ec009c30 comp rx=0 tx=0).stop 2024-02-26T10:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.214+0000 7f99f8a00700 1 -- 172.21.15.60:0/4035187931 shutdown_connections 2024-02-26T10:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.214+0000 7f99f8a00700 1 --2- 172.21.15.60:0/4035187931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f41070d0 0x7f99f410eca0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.214+0000 7f99f8a00700 1 --2- 172.21.15.60:0/4035187931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f4102000 0x7f99f41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.214+0000 7f99f8a00700 1 --2- 172.21.15.60:0/4035187931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f41029b0 0x7f99f41069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.214+0000 7f99f8a00700 1 -- 172.21.15.60:0/4035187931 >> 172.21.15.60:0/4035187931 conn(0x7f99f4078eb0 msgr2=0x7f99f40792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.214+0000 7f99f8a00700 1 -- 172.21.15.60:0/4035187931 shutdown_connections 2024-02-26T10:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.214+0000 7f99f8a00700 1 -- 172.21.15.60:0/4035187931 wait complete. 2024-02-26T10:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.215+0000 7f99f8a00700 1 Processor -- start 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f8a00700 1 -- start start 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f8a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f4102000 0x7f99f41a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f41029b0 0x7f99f41a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f8a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f41070d0 0x7f99f419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f8a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f4112120 con 0x7f99f41029b0 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f8a00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f4111fa0 con 0x7f99f41070d0 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f8a00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f41122a0 con 0x7f99f4102000 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f2800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f41070d0 0x7f99f419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f1e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f4102000 0x7f99f41a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f2800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f41070d0 0x7f99f419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51472/0 (socket says 172.21.15.60:51472) 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f2800700 1 -- 172.21.15.60:0/1239865366 learned_addr learned my addr 172.21.15.60:0/1239865366 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.216+0000 7f99f1400700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f41029b0 0x7f99f41a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.217+0000 7f99f2800700 1 -- 172.21.15.60:0/1239865366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f4102000 msgr2=0x7f99f41a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.217+0000 7f99f2800700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f4102000 0x7f99f41a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.217+0000 7f99f2800700 1 -- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f41029b0 msgr2=0x7f99f41a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.217+0000 7f99f2800700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f41029b0 0x7f99f41a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.217+0000 7f99f2800700 1 -- 172.21.15.60:0/1239865366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f419ad50 con 0x7f99f41070d0 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.217+0000 7f99f1400700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f41029b0 0x7f99f41a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.217+0000 7f99f2800700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f41070d0 0x7f99f419a5f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009230 tx=0x7f99ec00fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.218+0000 7f99eaa00700 1 -- 172.21.15.60:0/1239865366 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec005040 con 0x7f99f41070d0 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.218+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f419afe0 con 0x7f99f41070d0 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.218+0000 7f99eaa00700 1 -- 172.21.15.60:0/1239865366 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99ec0056c0 con 0x7f99f41070d0 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.218+0000 7f99eaa00700 1 -- 172.21.15.60:0/1239865366 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0039b0 con 0x7f99f41070d0 2024-02-26T10:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.218+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f41a7300 con 0x7f99f41070d0 2024-02-26T10:44:18.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.219+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f41034d0 con 0x7f99f41070d0 2024-02-26T10:44:18.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.221+0000 7f99eaa00700 1 -- 172.21.15.60:0/1239865366 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f99ec00b7f0 con 0x7f99f41070d0 2024-02-26T10:44:18.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.223+0000 7f99eaa00700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f99d80a1f20 0x7f99d80a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:18.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.223+0000 7f99eaa00700 1 -- 172.21.15.60:0/1239865366 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f99ec0cf100 con 0x7f99f41070d0 2024-02-26T10:44:18.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.223+0000 7f99f1e00700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f99d80a1f20 0x7f99d80a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:18.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.223+0000 7f99f1e00700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f99d80a1f20 0x7f99d80a43e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009da0 tx=0x7f99e400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:18.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.225+0000 7f99eaa00700 1 -- 172.21.15.60:0/1239865366 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99ec013030 con 0x7f99f41070d0 2024-02-26T10:44:18.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:18 smithi060 ceph-mon[31539]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:18.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.452+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f99f419bb20 con 0x7f99f41070d0 2024-02-26T10:44:18.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.453+0000 7f99eaa00700 1 -- 172.21.15.60:0/1239865366 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f99ec09c0a0 con 0x7f99f41070d0 2024-02-26T10:44:18.449 INFO:teuthology.orchestra.run.smithi060.stdout:313532612646 2024-02-26T10:44:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f99d80a1f20 msgr2=0x7f99d80a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f99d80a1f20 0x7f99d80a43e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009da0 tx=0x7f99e400b040 comp rx=0 tx=0).stop 2024-02-26T10:44:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f41070d0 msgr2=0x7f99f419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f41070d0 0x7f99f419a5f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009230 tx=0x7f99ec00fa70 comp rx=0 tx=0).stop 2024-02-26T10:44:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 shutdown_connections 2024-02-26T10:44:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f99d80a1f20 0x7f99d80a43e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f41070d0 0x7f99f419a5f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f4102000 0x7f99f41a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 --2- 172.21.15.60:0/1239865366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f41029b0 0x7f99f41a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.456+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 >> 172.21.15.60:0/1239865366 conn(0x7f99f4078eb0 msgr2=0x7f99f40ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.457+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 shutdown_connections 2024-02-26T10:44:18.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:18.457+0000 7f99f8a00700 1 -- 172.21.15.60:0/1239865366 wait complete. 2024-02-26T10:44:18.484 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:18 smithi113 ceph-mon[36126]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:18.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:18 smithi100 ceph-mon[36051]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:19.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:19 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1239865366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T10:44:19.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:19 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1239865366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T10:44:19.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:19 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1239865366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T10:44:19.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920837 got 141733920842 for osd.4 2024-02-26T10:44:19.843 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:20.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:20 smithi060 ceph-mon[31539]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:20.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:20 smithi113 ceph-mon[36126]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:20.585 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574935 got 55834574938 for osd.1 2024-02-26T10:44:20.585 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:20.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:20 smithi100 ceph-mon[36051]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:22.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:22 smithi060 ceph-mon[31539]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:22.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:22 smithi113 ceph-mon[36126]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:22.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:22 smithi100 ceph-mon[36051]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:22.739 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612642 got 313532612646 for osd.10 2024-02-26T10:44:22.739 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:24.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:24 smithi060 ceph-mon[31539]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:24.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:24 smithi113 ceph-mon[36126]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:24.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:24 smithi100 ceph-mon[36051]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.879+0000 7ff430600700 1 -- 172.21.15.60:0/1922727131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 msgr2=0x7ff4281069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.879+0000 7ff430600700 1 --2- 172.21.15.60:0/1922727131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 0x7ff4281069f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff414009990 tx=0x7ff414009ca0 comp rx=0 tx=0).stop 2024-02-26T10:44:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.880+0000 7ff430600700 1 -- 172.21.15.60:0/1922727131 shutdown_connections 2024-02-26T10:44:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.880+0000 7ff430600700 1 --2- 172.21.15.60:0/1922727131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4281070d0 0x7ff42810eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.880+0000 7ff430600700 1 --2- 172.21.15.60:0/1922727131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff428102000 0x7ff4281023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.880+0000 7ff430600700 1 --2- 172.21.15.60:0/1922727131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 0x7ff4281069f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.880+0000 7ff430600700 1 -- 172.21.15.60:0/1922727131 >> 172.21.15.60:0/1922727131 conn(0x7ff428078eb0 msgr2=0x7ff4280792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.880+0000 7ff430600700 1 -- 172.21.15.60:0/1922727131 shutdown_connections 2024-02-26T10:44:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.880+0000 7ff430600700 1 -- 172.21.15.60:0/1922727131 wait complete. 2024-02-26T10:44:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.881+0000 7ff430600700 1 Processor -- start 2024-02-26T10:44:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.881+0000 7ff430600700 1 -- start start 2024-02-26T10:44:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff430600700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff428102000 0x7ff428074050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff430600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 0x7ff428074590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff430600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4281070d0 0x7ff428074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff430600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4281120e0 con 0x7ff4281070d0 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff430600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff428111f60 con 0x7ff4281029b0 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff430600700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff428112260 con 0x7ff428102000 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff42d200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 0x7ff428074590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff42dc00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff428102000 0x7ff428074050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff42dc00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff428102000 0x7ff428074050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:50536/0 (socket says 172.21.15.60:50536) 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff42d200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 0x7ff428074590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52070/0 (socket says 172.21.15.60:52070) 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff42dc00700 1 -- 172.21.15.60:0/2100898347 learned_addr learned my addr 172.21.15.60:0/2100898347 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.882+0000 7ff42e600700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4281070d0 0x7ff428074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff42dc00700 1 -- 172.21.15.60:0/2100898347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 msgr2=0x7ff428074590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff42dc00700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 0x7ff428074590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff42dc00700 1 -- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4281070d0 msgr2=0x7ff428074ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff42dc00700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4281070d0 0x7ff428074ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff42dc00700 1 -- 172.21.15.60:0/2100898347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4281a6b30 con 0x7ff428102000 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff42d200700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 0x7ff428074590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff42dc00700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff428102000 0x7ff428074050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9a0 tx=0x7ff41c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff426000700 1 -- 172.21.15.60:0/2100898347 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00cbd0 con 0x7ff428102000 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff426000700 1 -- 172.21.15.60:0/2100898347 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff41c00cd70 con 0x7ff428102000 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4281a6e20 con 0x7ff428102000 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff426000700 1 -- 172.21.15.60:0/2100898347 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c018750 con 0x7ff428102000 2024-02-26T10:44:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.883+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4281a7270 con 0x7ff428102000 2024-02-26T10:44:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.885+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff428068a10 con 0x7ff428102000 2024-02-26T10:44:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.888+0000 7ff426000700 1 -- 172.21.15.60:0/2100898347 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff41c010430 con 0x7ff428102000 2024-02-26T10:44:24.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.889+0000 7ff426000700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4100a1ea0 0x7ff4100a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:24.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.890+0000 7ff426000700 1 -- 172.21.15.60:0/2100898347 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff41c0ce500 con 0x7ff428102000 2024-02-26T10:44:24.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.890+0000 7ff42d200700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4100a1ea0 0x7ff4100a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:24.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.890+0000 7ff42d200700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4100a1ea0 0x7ff4100a4360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff428075a90 tx=0x7ff41400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:24.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:24.894+0000 7ff426000700 1 -- 172.21.15.60:0/2100898347 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff41c09b470 con 0x7ff428102000 2024-02-26T10:44:25.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.144+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff42802d910 con 0x7ff428102000 2024-02-26T10:44:25.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.145+0000 7ff426000700 1 -- 172.21.15.60:0/2100898347 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff41c09ad70 con 0x7ff428102000 2024-02-26T10:44:25.141 INFO:teuthology.orchestra.run.smithi060.stdout:34359738464 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4100a1ea0 msgr2=0x7ff4100a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4100a1ea0 0x7ff4100a4360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff428075a90 tx=0x7ff41400b260 comp rx=0 tx=0).stop 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff428102000 msgr2=0x7ff428074050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff428102000 0x7ff428074050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9a0 tx=0x7ff41c00ecb0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 shutdown_connections 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4100a1ea0 0x7ff4100a4360 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4281070d0 0x7ff428074ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff428102000 0x7ff428074050 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 --2- 172.21.15.60:0/2100898347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4281029b0 0x7ff428074590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 >> 172.21.15.60:0/2100898347 conn(0x7ff428078eb0 msgr2=0x7ff428100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 shutdown_connections 2024-02-26T10:44:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.148+0000 7ff430600700 1 -- 172.21.15.60:0/2100898347 wait complete. 2024-02-26T10:44:25.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:25 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2100898347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T10:44:25.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.546+0000 7f328ae00700 1 -- 172.21.15.60:0/2805475195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284101f90 msgr2=0x7f3284102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.546+0000 7f328ae00700 1 --2- 172.21.15.60:0/2805475195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284101f90 0x7f3284102370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32740099c0 tx=0x7f3274009cd0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.546+0000 7f328ae00700 1 -- 172.21.15.60:0/2805475195 shutdown_connections 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.546+0000 7f328ae00700 1 --2- 172.21.15.60:0/2805475195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284107070 0x7f328410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.546+0000 7f328ae00700 1 --2- 172.21.15.60:0/2805475195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284101f90 0x7f3284102370 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.546+0000 7f328ae00700 1 --2- 172.21.15.60:0/2805475195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3284102940 0x7f3284106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.546+0000 7f328ae00700 1 -- 172.21.15.60:0/2805475195 >> 172.21.15.60:0/2805475195 conn(0x7f32840fd800 msgr2=0x7f32840ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.546+0000 7f328ae00700 1 -- 172.21.15.60:0/2805475195 shutdown_connections 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.546+0000 7f328ae00700 1 -- 172.21.15.60:0/2805475195 wait complete. 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.548+0000 7f328ae00700 1 Processor -- start 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.548+0000 7f328ae00700 1 -- start start 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.548+0000 7f328ae00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284101f90 0x7f32841a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.548+0000 7f328ae00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3284102940 0x7f32841a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:25.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.548+0000 7f328ae00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284107070 0x7f328419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.548+0000 7f328ae00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32841120d0 con 0x7f3284102940 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.548+0000 7f328ae00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3284111f50 con 0x7f3284101f90 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.548+0000 7f328ae00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3284112250 con 0x7f3284107070 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3288e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284107070 0x7f328419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3283e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284101f90 0x7f32841a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3283400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3284102940 0x7f32841a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3288e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284107070 0x7f328419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:50544/0 (socket says 172.21.15.60:50544) 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3288e00700 1 -- 172.21.15.60:0/679657381 learned_addr learned my addr 172.21.15.60:0/679657381 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3288e00700 1 -- 172.21.15.60:0/679657381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284101f90 msgr2=0x7f32841a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3288e00700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284101f90 0x7f32841a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3288e00700 1 -- 172.21.15.60:0/679657381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3284102940 msgr2=0x7f32841a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:25.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3288e00700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3284102940 0x7f32841a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.549+0000 7f3288e00700 1 -- 172.21.15.60:0/679657381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328419acf0 con 0x7f3284107070 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.550+0000 7f3288e00700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284107070 0x7f328419a590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f327c00ef30 tx=0x7f327c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.550+0000 7f3280c00700 1 -- 172.21.15.60:0/679657381 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0092e0 con 0x7f3284107070 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.550+0000 7f3280c00700 1 -- 172.21.15.60:0/679657381 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f327c004bb0 con 0x7f3284107070 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.550+0000 7f3280c00700 1 -- 172.21.15.60:0/679657381 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c004280 con 0x7f3284107070 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.550+0000 7f328ae00700 1 -- 172.21.15.60:0/679657381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328419af80 con 0x7f3284107070 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.550+0000 7f328ae00700 1 -- 172.21.15.60:0/679657381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32841a72a0 con 0x7f3284107070 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.553+0000 7f327a000700 1 -- 172.21.15.60:0/679657381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3264005330 con 0x7f3284107070 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.554+0000 7f3280c00700 1 -- 172.21.15.60:0/679657381 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f327c01b030 con 0x7f3284107070 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.555+0000 7f3280c00700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f32680a1ea0 0x7f32680a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.555+0000 7f3280c00700 1 -- 172.21.15.60:0/679657381 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f327c0cdc60 con 0x7f3284107070 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.555+0000 7f3283e00700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f32680a1ea0 0x7f32680a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:25.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.556+0000 7f3283e00700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f32680a1ea0 0x7f32680a4360 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3274000c00 tx=0x7f3274005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:25.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.559+0000 7f3280c00700 1 -- 172.21.15.60:0/679657381 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f327c022020 con 0x7f3284107070 2024-02-26T10:44:25.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:25 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2100898347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T10:44:25.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:25 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2100898347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T10:44:25.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.798+0000 7f327a000700 1 -- 172.21.15.60:0/679657381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f32640058f0 con 0x7f3284107070 2024-02-26T10:44:25.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.799+0000 7f3280c00700 1 -- 172.21.15.60:0/679657381 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f327c010040 con 0x7f3284107070 2024-02-26T10:44:25.796 INFO:teuthology.orchestra.run.smithi060.stdout:107374182479 2024-02-26T10:44:25.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.802+0000 7f327a000700 1 -- 172.21.15.60:0/679657381 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f32680a1ea0 msgr2=0x7f32680a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:25.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.802+0000 7f327a000700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f32680a1ea0 0x7f32680a4360 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3274000c00 tx=0x7f3274005fb0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.802+0000 7f327a000700 1 -- 172.21.15.60:0/679657381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284107070 msgr2=0x7f328419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:25.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.802+0000 7f327a000700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284107070 0x7f328419a590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f327c00ef30 tx=0x7f327c00c3a0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.803+0000 7f327a000700 1 -- 172.21.15.60:0/679657381 shutdown_connections 2024-02-26T10:44:25.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.803+0000 7f327a000700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f32680a1ea0 0x7f32680a4360 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.803+0000 7f327a000700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3284107070 0x7f328419a590 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.803+0000 7f327a000700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284101f90 0x7f32841a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.803+0000 7f327a000700 1 --2- 172.21.15.60:0/679657381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3284102940 0x7f32841a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:25.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.803+0000 7f327a000700 1 -- 172.21.15.60:0/679657381 >> 172.21.15.60:0/679657381 conn(0x7f32840fd800 msgr2=0x7f328410df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:25.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.803+0000 7f327a000700 1 -- 172.21.15.60:0/679657381 shutdown_connections 2024-02-26T10:44:25.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:25.803+0000 7f327a000700 1 -- 172.21.15.60:0/679657381 wait complete. 2024-02-26T10:44:26.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:26 smithi060 ceph-mon[31539]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:26.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:26 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/679657381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T10:44:26.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:26 smithi113 ceph-mon[36126]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:26.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:26 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/679657381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T10:44:26.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:26 smithi100 ceph-mon[36051]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:26.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:26 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/679657381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T10:44:27.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.496+0000 7f1efa400700 1 -- 172.21.15.60:0/3078990180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 msgr2=0x7f1ef4102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.496+0000 7f1efa400700 1 --2- 172.21.15.60:0/3078990180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 0x7f1ef4102270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009990 tx=0x7f1ed8009ca0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.496+0000 7f1efa400700 1 -- 172.21.15.60:0/3078990180 shutdown_connections 2024-02-26T10:44:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.496+0000 7f1efa400700 1 --2- 172.21.15.60:0/3078990180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef4106f70 0x7f1ef410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.496+0000 7f1efa400700 1 --2- 172.21.15.60:0/3078990180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 0x7f1ef4102270 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.496+0000 7f1efa400700 1 --2- 172.21.15.60:0/3078990180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef4102840 0x7f1ef4106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.496+0000 7f1efa400700 1 -- 172.21.15.60:0/3078990180 >> 172.21.15.60:0/3078990180 conn(0x7f1ef40fd720 msgr2=0x7f1ef40ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.496+0000 7f1efa400700 1 -- 172.21.15.60:0/3078990180 shutdown_connections 2024-02-26T10:44:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.496+0000 7f1efa400700 1 -- 172.21.15.60:0/3078990180 wait complete. 2024-02-26T10:44:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.498+0000 7f1efa400700 1 Processor -- start 2024-02-26T10:44:27.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.498+0000 7f1efa400700 1 -- start start 2024-02-26T10:44:27.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1efa400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 0x7f1ef4074050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:27.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1efa400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef4102840 0x7f1ef4074590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1efa400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef4106f70 0x7f1ef4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1efa400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4111f50 con 0x7f1ef4106f70 2024-02-26T10:44:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1efa400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4111dd0 con 0x7f1ef4101e90 2024-02-26T10:44:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1efa400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef41120d0 con 0x7f1ef4102840 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1ef9000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 0x7f1ef4074050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1eebe00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef4102840 0x7f1ef4074590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1ef9000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 0x7f1ef4074050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52114/0 (socket says 172.21.15.60:52114) 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1ef9000700 1 -- 172.21.15.60:0/3088489795 learned_addr learned my addr 172.21.15.60:0/3088489795 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1ef9a00700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef4106f70 0x7f1ef4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1eebe00700 1 -- 172.21.15.60:0/3088489795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 msgr2=0x7f1ef4074050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1eebe00700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 0x7f1ef4074050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1eebe00700 1 -- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef4106f70 msgr2=0x7f1ef4074ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.499+0000 7f1eebe00700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef4106f70 0x7f1ef4074ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.500+0000 7f1eebe00700 1 -- 172.21.15.60:0/3088489795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef41a6920 con 0x7f1ef4102840 2024-02-26T10:44:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.500+0000 7f1ef9000700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 0x7f1ef4074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:27.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.500+0000 7f1eebe00700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef4102840 0x7f1ef4074590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e9c0 tx=0x7f1ee000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:27.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.500+0000 7f1ef2000700 1 -- 172.21.15.60:0/3088489795 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000cbd0 con 0x7f1ef4102840 2024-02-26T10:44:27.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.500+0000 7f1ef2000700 1 -- 172.21.15.60:0/3088489795 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee000cd70 con 0x7f1ef4102840 2024-02-26T10:44:27.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.500+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef41a6c10 con 0x7f1ef4102840 2024-02-26T10:44:27.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.500+0000 7f1ef2000700 1 -- 172.21.15.60:0/3088489795 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0018680 con 0x7f1ef4102840 2024-02-26T10:44:27.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.500+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef41a71a0 con 0x7f1ef4102840 2024-02-26T10:44:27.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.502+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef4068a10 con 0x7f1ef4102840 2024-02-26T10:44:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.503+0000 7f1ef2000700 1 -- 172.21.15.60:0/3088489795 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f1ee0010430 con 0x7f1ef4102840 2024-02-26T10:44:27.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.505+0000 7f1ef2000700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ed00a1f20 0x7f1ed00a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:27.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.505+0000 7f1ef2000700 1 -- 172.21.15.60:0/3088489795 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f1ee00ce490 con 0x7f1ef4102840 2024-02-26T10:44:27.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.505+0000 7f1ef9000700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ed00a1f20 0x7f1ed00a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:27.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.506+0000 7f1ef9000700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ed00a1f20 0x7f1ed00a43e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800b2e0 tx=0x7f1ed801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.509+0000 7f1ef2000700 1 -- 172.21.15.60:0/3088489795 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ee009b3a0 con 0x7f1ef4102840 2024-02-26T10:44:27.615 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182474 got 107374182479 for osd.3 2024-02-26T10:44:27.615 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:27.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.745+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1ef4075530 con 0x7f1ef4102840 2024-02-26T10:44:27.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.745+0000 7f1ef2000700 1 -- 172.21.15.60:0/3088489795 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ee009aca0 con 0x7f1ef4102840 2024-02-26T10:44:27.741 INFO:teuthology.orchestra.run.smithi060.stdout:167503724614 2024-02-26T10:44:27.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ed00a1f20 msgr2=0x7f1ed00a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:27.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ed00a1f20 0x7f1ed00a43e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800b2e0 tx=0x7f1ed801a040 comp rx=0 tx=0).stop 2024-02-26T10:44:27.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef4102840 msgr2=0x7f1ef4074590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:27.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef4102840 0x7f1ef4074590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e9c0 tx=0x7f1ee000ecd0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 shutdown_connections 2024-02-26T10:44:27.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1ed00a1f20 0x7f1ed00a43e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ef4106f70 0x7f1ef4074ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef4101e90 0x7f1ef4074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 --2- 172.21.15.60:0/3088489795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ef4102840 0x7f1ef4074590 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:27.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 >> 172.21.15.60:0/3088489795 conn(0x7f1ef40fd720 msgr2=0x7f1ef4106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:27.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 shutdown_connections 2024-02-26T10:44:27.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:27.749+0000 7f1efa400700 1 -- 172.21.15.60:0/3088489795 wait complete. 2024-02-26T10:44:28.429 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:28 smithi100 ceph-mon[36051]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:28.429 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:28 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3088489795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T10:44:28.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:28 smithi113 ceph-mon[36126]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:28.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:28 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3088489795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T10:44:28.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:28 smithi060 ceph-mon[31539]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:28.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:28 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3088489795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T10:44:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.397+0000 7f86fee00700 1 -- 172.21.15.60:0/2065353243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f810e750 msgr2=0x7f86f8110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.397+0000 7f86fee00700 1 --2- 172.21.15.60:0/2065353243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f810e750 0x7f86f8110b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009f90 tx=0x7f86f000b330 comp rx=0 tx=0).stop 2024-02-26T10:44:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.398+0000 7f86fee00700 1 -- 172.21.15.60:0/2065353243 shutdown_connections 2024-02-26T10:44:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.398+0000 7f86fee00700 1 --2- 172.21.15.60:0/2065353243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f810e750 0x7f86f8110b40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.398+0000 7f86fee00700 1 --2- 172.21.15.60:0/2065353243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f8073f80 0x7f86f8074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.398+0000 7f86fee00700 1 --2- 172.21.15.60:0/2065353243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f80748a0 0x7f86f810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.398+0000 7f86fee00700 1 -- 172.21.15.60:0/2065353243 >> 172.21.15.60:0/2065353243 conn(0x7f86f8078f10 msgr2=0x7f86f8079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.398+0000 7f86fee00700 1 -- 172.21.15.60:0/2065353243 shutdown_connections 2024-02-26T10:44:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.398+0000 7f86fee00700 1 -- 172.21.15.60:0/2065353243 wait complete. 2024-02-26T10:44:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.399+0000 7f86fee00700 1 Processor -- start 2024-02-26T10:44:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.399+0000 7f86fee00700 1 -- start start 2024-02-26T10:44:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86fee00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f8073f80 0x7f86f819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86fee00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f80748a0 0x7f86f819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86fee00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f810e750 0x7f86f819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86fee00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f81142e0 con 0x7f86f80748a0 2024-02-26T10:44:29.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86fee00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8114160 con 0x7f86f8073f80 2024-02-26T10:44:29.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86fee00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8114460 con 0x7f86f810e750 2024-02-26T10:44:29.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86f7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f80748a0 0x7f86f819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:29.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86f7e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f8073f80 0x7f86f819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:29.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86fce00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f810e750 0x7f86f819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:29.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86f7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f80748a0 0x7f86f819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43346/0 (socket says 172.21.15.60:43346) 2024-02-26T10:44:29.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86f7e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f8073f80 0x7f86f819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52144/0 (socket says 172.21.15.60:52144) 2024-02-26T10:44:29.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.400+0000 7f86f7400700 1 -- 172.21.15.60:0/96845598 learned_addr learned my addr 172.21.15.60:0/96845598 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:29.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86fce00700 1 -- 172.21.15.60:0/96845598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f8073f80 msgr2=0x7f86f819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:29.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86fce00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f8073f80 0x7f86f819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:29.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86fce00700 1 -- 172.21.15.60:0/96845598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f80748a0 msgr2=0x7f86f819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:29.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86fce00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f80748a0 0x7f86f819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86fce00700 1 -- 172.21.15.60:0/96845598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f819f700 con 0x7f86f810e750 2024-02-26T10:44:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86f7e00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f8073f80 0x7f86f819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86fce00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f810e750 0x7f86f819efc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f86f806b510 tx=0x7f86f0004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86f4c00700 1 -- 172.21.15.60:0/96845598 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0021070 con 0x7f86f810e750 2024-02-26T10:44:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86f4c00700 1 -- 172.21.15.60:0/96845598 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86f0006020 con 0x7f86f810e750 2024-02-26T10:44:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f819f990 con 0x7f86f810e750 2024-02-26T10:44:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86f4c00700 1 -- 172.21.15.60:0/96845598 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0005040 con 0x7f86f810e750 2024-02-26T10:44:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.401+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f81a7270 con 0x7f86f810e750 2024-02-26T10:44:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.402+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f8075610 con 0x7f86f810e750 2024-02-26T10:44:29.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.404+0000 7f86f4c00700 1 -- 172.21.15.60:0/96845598 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f86f0024070 con 0x7f86f810e750 2024-02-26T10:44:29.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.405+0000 7f86f4c00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f86dc0a1ea0 0x7f86dc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:29.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.406+0000 7f86f4c00700 1 -- 172.21.15.60:0/96845598 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f86f000f020 con 0x7f86f810e750 2024-02-26T10:44:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.406+0000 7f86f7e00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f86dc0a1ea0 0x7f86dc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.406+0000 7f86f7e00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f86dc0a1ea0 0x7f86dc0a4360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86e8011490 tx=0x7f86e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:29.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.408+0000 7f86f4c00700 1 -- 172.21.15.60:0/96845598 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f86f002b020 con 0x7f86f810e750 2024-02-26T10:44:29.458 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:29 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4252783374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T10:44:29.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:29 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/4252783374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T10:44:29.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:29 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/4252783374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T10:44:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.643+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f86f819ba50 con 0x7f86f810e750 2024-02-26T10:44:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.644+0000 7f86f4c00700 1 -- 172.21.15.60:0/96845598 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f86f009c660 con 0x7f86f810e750 2024-02-26T10:44:29.640 INFO:teuthology.orchestra.run.smithi060.stdout:77309411416 2024-02-26T10:44:29.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f86dc0a1ea0 msgr2=0x7f86dc0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:29.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f86dc0a1ea0 0x7f86dc0a4360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86e8011490 tx=0x7f86e8009260 comp rx=0 tx=0).stop 2024-02-26T10:44:29.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f810e750 msgr2=0x7f86f819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:29.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f810e750 0x7f86f819efc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f86f806b510 tx=0x7f86f0004f00 comp rx=0 tx=0).stop 2024-02-26T10:44:29.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 shutdown_connections 2024-02-26T10:44:29.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f86dc0a1ea0 0x7f86dc0a4360 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:29.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f810e750 0x7f86f819efc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:29.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f8073f80 0x7f86f819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:29.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 --2- 172.21.15.60:0/96845598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f80748a0 0x7f86f819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:29.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 >> 172.21.15.60:0/96845598 conn(0x7f86f8078f10 msgr2=0x7f86f80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:29.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 shutdown_connections 2024-02-26T10:44:29.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.647+0000 7f86fee00700 1 -- 172.21.15.60:0/96845598 wait complete. 2024-02-26T10:44:30.289 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738456 got 34359738464 for osd.0 2024-02-26T10:44:30.289 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:30.324 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:30 smithi060 ceph-mon[31539]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:30.324 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:30 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/96845598' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T10:44:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.066+0000 7f7cace00700 1 -- 172.21.15.60:0/3835960362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ca8101ea0 msgr2=0x7f7ca8102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.066+0000 7f7cace00700 1 --2- 172.21.15.60:0/3835960362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ca8101ea0 0x7f7ca8102280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7c980099c0 tx=0x7f7c98009cd0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.066+0000 7f7cace00700 1 -- 172.21.15.60:0/3835960362 shutdown_connections 2024-02-26T10:44:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.066+0000 7f7cace00700 1 --2- 172.21.15.60:0/3835960362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca8106f80 0x7f7ca810eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.066+0000 7f7cace00700 1 --2- 172.21.15.60:0/3835960362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ca8101ea0 0x7f7ca8102280 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.066+0000 7f7cace00700 1 --2- 172.21.15.60:0/3835960362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ca8102850 0x7f7ca81068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.066+0000 7f7cace00700 1 -- 172.21.15.60:0/3835960362 >> 172.21.15.60:0/3835960362 conn(0x7f7ca80fd750 msgr2=0x7f7ca80ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.066+0000 7f7cace00700 1 -- 172.21.15.60:0/3835960362 shutdown_connections 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.066+0000 7f7cace00700 1 -- 172.21.15.60:0/3835960362 wait complete. 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.067+0000 7f7cace00700 1 Processor -- start 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.068+0000 7f7cace00700 1 -- start start 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.068+0000 7f7cace00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ca8101ea0 0x7f7ca819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.068+0000 7f7cace00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca8102850 0x7f7ca819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.068+0000 7f7cace00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ca8106f80 0x7f7ca819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.068+0000 7f7cace00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8111f70 con 0x7f7ca8101ea0 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.068+0000 7f7cace00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8111df0 con 0x7f7ca8102850 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.068+0000 7f7cace00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca81120f0 con 0x7f7ca8106f80 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ca8106f80 0x7f7ca819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ca8101ea0 0x7f7ca819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ca8106f80 0x7f7ca819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:50598/0 (socket says 172.21.15.60:50598) 2024-02-26T10:44:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7e00700 1 -- 172.21.15.60:0/2505471617 learned_addr learned my addr 172.21.15.60:0/2505471617 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca6a00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca8102850 0x7f7ca819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7e00700 1 -- 172.21.15.60:0/2505471617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca8102850 msgr2=0x7f7ca819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7e00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca8102850 0x7f7ca819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7e00700 1 -- 172.21.15.60:0/2505471617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ca8101ea0 msgr2=0x7f7ca819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7e00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ca8101ea0 0x7f7ca819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7e00700 1 -- 172.21.15.60:0/2505471617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca819f5e0 con 0x7f7ca8106f80 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.069+0000 7f7ca7e00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ca8106f80 0x7f7ca819eea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000ef30 tx=0x7f7ca000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.070+0000 7f7c9fe00700 1 -- 172.21.15.60:0/2505471617 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00092e0 con 0x7f7ca8106f80 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.070+0000 7f7c9fe00700 1 -- 172.21.15.60:0/2505471617 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ca0004bb0 con 0x7f7ca8106f80 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.070+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca819f870 con 0x7f7ca8106f80 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.070+0000 7f7c9fe00700 1 -- 172.21.15.60:0/2505471617 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0004280 con 0x7f7ca8106f80 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.070+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca81a7150 con 0x7f7ca8106f80 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.071+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c88005330 con 0x7f7ca8106f80 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.073+0000 7f7c9fe00700 1 -- 172.21.15.60:0/2505471617 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f7ca001b030 con 0x7f7ca8106f80 2024-02-26T10:44:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.074+0000 7f7c9fe00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7c8c0a1ea0 0x7f7c8c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.074+0000 7f7c9fe00700 1 -- 172.21.15.60:0/2505471617 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f7ca00ce5a0 con 0x7f7ca8106f80 2024-02-26T10:44:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.074+0000 7f7ca7400700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7c8c0a1ea0 0x7f7c8c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.075+0000 7f7ca7400700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7c8c0a1ea0 0x7f7c8c0a4360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7c98000c00 tx=0x7f7c98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.076+0000 7f7c9fe00700 1 -- 172.21.15.60:0/2505471617 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7ca0022020 con 0x7f7ca8106f80 2024-02-26T10:44:30.347 INFO:teuthology.orchestra.run.smithi060.stdout:347892351011 2024-02-26T10:44:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.305+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f7c880058f0 con 0x7f7ca8106f80 2024-02-26T10:44:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.306+0000 7f7c9fe00700 1 -- 172.21.15.60:0/2505471617 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f7ca0010040 con 0x7f7ca8106f80 2024-02-26T10:44:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7c8c0a1ea0 msgr2=0x7f7c8c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7c8c0a1ea0 0x7f7c8c0a4360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7c98000c00 tx=0x7f7c98005fb0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ca8106f80 msgr2=0x7f7ca819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ca8106f80 0x7f7ca819eea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000ef30 tx=0x7f7ca000c3a0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 shutdown_connections 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7c8c0a1ea0 0x7f7c8c0a4360 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ca8106f80 0x7f7ca819eea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ca8101ea0 0x7f7ca819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 --2- 172.21.15.60:0/2505471617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca8102850 0x7f7ca819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 >> 172.21.15.60:0/2505471617 conn(0x7f7ca80fd750 msgr2=0x7f7ca80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 shutdown_connections 2024-02-26T10:44:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:30.309+0000 7f7cace00700 1 -- 172.21.15.60:0/2505471617 wait complete. 2024-02-26T10:44:30.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:30 smithi113 ceph-mon[36126]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:30.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:30 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/96845598' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T10:44:30.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:30 smithi100 ceph-mon[36051]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:30.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:30 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/96845598' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T10:44:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.113+0000 7f20cf200700 1 -- 172.21.15.60:0/1706696882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c8108f70 msgr2=0x7f20c810cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.113+0000 7f20cf200700 1 --2- 172.21.15.60:0/1706696882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c8108f70 0x7f20c810cfc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20b40099c0 tx=0x7f20b4009cd0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.113+0000 7f20cf200700 1 -- 172.21.15.60:0/1706696882 shutdown_connections 2024-02-26T10:44:31.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.113+0000 7f20cf200700 1 --2- 172.21.15.60:0/1706696882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c810d6a0 0x7f20c8115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.113+0000 7f20cf200700 1 --2- 172.21.15.60:0/1706696882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c81085c0 0x7f20c81089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.113+0000 7f20cf200700 1 --2- 172.21.15.60:0/1706696882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c8108f70 0x7f20c810cfc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.113+0000 7f20cf200700 1 -- 172.21.15.60:0/1706696882 >> 172.21.15.60:0/1706696882 conn(0x7f20c8075520 msgr2=0x7f20c8077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:31.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.113+0000 7f20cf200700 1 -- 172.21.15.60:0/1706696882 shutdown_connections 2024-02-26T10:44:31.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.113+0000 7f20cf200700 1 -- 172.21.15.60:0/1706696882 wait complete. 2024-02-26T10:44:31.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.115+0000 7f20cf200700 1 Processor -- start 2024-02-26T10:44:31.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.115+0000 7f20cf200700 1 -- start start 2024-02-26T10:44:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20cf200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c81085c0 0x7f20c81a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20cf200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c8108f70 0x7f20c81a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20cf200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c810d6a0 0x7f20c819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20cf200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8118680 con 0x7f20c8108f70 2024-02-26T10:44:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20cf200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8118500 con 0x7f20c81085c0 2024-02-26T10:44:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20cf200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8118800 con 0x7f20c810d6a0 2024-02-26T10:44:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20c7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c8108f70 0x7f20c81a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20c7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c8108f70 0x7f20c81a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43402/0 (socket says 172.21.15.60:43402) 2024-02-26T10:44:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20c7400700 1 -- 172.21.15.60:0/2563520595 learned_addr learned my addr 172.21.15.60:0/2563520595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20cd200700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c810d6a0 0x7f20c819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.116+0000 7f20c7e00700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c81085c0 0x7f20c81a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20c7e00700 1 -- 172.21.15.60:0/2563520595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c810d6a0 msgr2=0x7f20c819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20c7e00700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c810d6a0 0x7f20c819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20c7e00700 1 -- 172.21.15.60:0/2563520595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c8108f70 msgr2=0x7f20c81a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20c7e00700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c8108f70 0x7f20c81a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20c7e00700 1 -- 172.21.15.60:0/2563520595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c819f080 con 0x7f20c81085c0 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20c7e00700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c81085c0 0x7f20c81a4760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e9f0 tx=0x7f20b800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20c4c00700 1 -- 172.21.15.60:0/2563520595 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800c9c0 con 0x7f20c81085c0 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20c4c00700 1 -- 172.21.15.60:0/2563520595 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20b800cb60 con 0x7f20c81085c0 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20c4c00700 1 -- 172.21.15.60:0/2563520595 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8018680 con 0x7f20c81085c0 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c819f370 con 0x7f20c81085c0 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.117+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c81ab630 con 0x7f20c81085c0 2024-02-26T10:44:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.118+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c8068a10 con 0x7f20c81085c0 2024-02-26T10:44:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.120+0000 7f20c4c00700 1 -- 172.21.15.60:0/2563520595 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f20b8010430 con 0x7f20c81085c0 2024-02-26T10:44:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.122+0000 7f20c4c00700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f20ac0a1ea0 0x7f20ac0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.122+0000 7f20c4c00700 1 -- 172.21.15.60:0/2563520595 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f20b80ce400 con 0x7f20c81085c0 2024-02-26T10:44:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.122+0000 7f20c7400700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f20ac0a1ea0 0x7f20ac0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.123+0000 7f20c7400700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f20ac0a1ea0 0x7f20ac0a4360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20b4005850 tx=0x7f20b4005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:31.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.126+0000 7f20c4c00700 1 -- 172.21.15.60:0/2563520595 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20b809b310 con 0x7f20c81085c0 2024-02-26T10:44:31.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.364+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f20c802d7f0 con 0x7f20c81085c0 2024-02-26T10:44:31.360 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:31 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2505471617' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T10:44:31.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.364+0000 7f20c4c00700 1 -- 172.21.15.60:0/2563520595 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20b809ac10 con 0x7f20c81085c0 2024-02-26T10:44:31.360 INFO:teuthology.orchestra.run.smithi060.stdout:257698037814 2024-02-26T10:44:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f20ac0a1ea0 msgr2=0x7f20ac0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f20ac0a1ea0 0x7f20ac0a4360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20b4005850 tx=0x7f20b4005a90 comp rx=0 tx=0).stop 2024-02-26T10:44:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c81085c0 msgr2=0x7f20c81a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c81085c0 0x7f20c81a4760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e9f0 tx=0x7f20b800ed00 comp rx=0 tx=0).stop 2024-02-26T10:44:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 shutdown_connections 2024-02-26T10:44:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f20ac0a1ea0 0x7f20ac0a4360 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c810d6a0 0x7f20c819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c81085c0 0x7f20c81a4760 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 --2- 172.21.15.60:0/2563520595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c8108f70 0x7f20c81a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 >> 172.21.15.60:0/2563520595 conn(0x7f20c8075520 msgr2=0x7f20c8075fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:31.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 shutdown_connections 2024-02-26T10:44:31.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:31.368+0000 7f20cf200700 1 -- 172.21.15.60:0/2563520595 wait complete. 2024-02-26T10:44:31.366 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724608 got 167503724614 for osd.5 2024-02-26T10:44:31.366 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.826+0000 7fcab3400700 1 -- 172.21.15.60:0/3114243333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaac06aea0 msgr2=0x7fcaac106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.826+0000 7fcab3400700 1 --2- 172.21.15.60:0/3114243333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaac06aea0 0x7fcaac106950 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fca98009990 tx=0x7fca98009ca0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.826+0000 7fcab3400700 1 -- 172.21.15.60:0/3114243333 shutdown_connections 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.826+0000 7fcab3400700 1 --2- 172.21.15.60:0/3114243333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcaac107030 0x7fcaac10ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.826+0000 7fcab3400700 1 --2- 172.21.15.60:0/3114243333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaac06a4f0 0x7fcaac06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.826+0000 7fcab3400700 1 --2- 172.21.15.60:0/3114243333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaac06aea0 0x7fcaac106950 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.826+0000 7fcab3400700 1 -- 172.21.15.60:0/3114243333 >> 172.21.15.60:0/3114243333 conn(0x7fcaac076d30 msgr2=0x7fcaac077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.826+0000 7fcab3400700 1 -- 172.21.15.60:0/3114243333 shutdown_connections 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.827+0000 7fcab3400700 1 -- 172.21.15.60:0/3114243333 wait complete. 2024-02-26T10:44:31.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.828+0000 7fcab3400700 1 Processor -- start 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.828+0000 7fcab3400700 1 -- start start 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcab3400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcaac06a4f0 0x7fcaac100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcab3400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaac06aea0 0x7fcaac101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcab3400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaac107030 0x7fcaac1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcab3400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac111ff0 con 0x7fcaac06a4f0 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcab3400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac111e70 con 0x7fcaac06aea0 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcab3400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac112170 con 0x7fcaac107030 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcaabe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaac06aea0 0x7fcaac101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcaabe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaac06aea0 0x7fcaac101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52136/0 (socket says 172.21.15.60:52136) 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcaabe00700 1 -- 172.21.15.60:0/4252783374 learned_addr learned my addr 172.21.15.60:0/4252783374 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcab0a00700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcaac06a4f0 0x7fcaac100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcab1400700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaac107030 0x7fcaac1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcaabe00700 1 -- 172.21.15.60:0/4252783374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaac107030 msgr2=0x7fcaac1017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcaabe00700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaac107030 0x7fcaac1017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcaabe00700 1 -- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcaac06a4f0 msgr2=0x7fcaac100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.829+0000 7fcaabe00700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcaac06a4f0 0x7fcaac100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.830+0000 7fcaabe00700 1 -- 172.21.15.60:0/4252783374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac1a69c0 con 0x7fcaac06aea0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.830+0000 7fcaabe00700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaac06aea0 0x7fcaac101290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fca98000c00 tx=0x7fca9800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.830+0000 7fcaa9600700 1 -- 172.21.15.60:0/4252783374 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98027070 con 0x7fcaac06aea0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.830+0000 7fcaa9600700 1 -- 172.21.15.60:0/4252783374 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca98003d70 con 0x7fcaac06aea0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.830+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac1a6c50 con 0x7fcaac06aea0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.830+0000 7fcaa9600700 1 -- 172.21.15.60:0/4252783374 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800f460 con 0x7fcaac06aea0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.830+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac1a70a0 con 0x7fcaac06aea0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.831+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaac068a10 con 0x7fcaac06aea0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.833+0000 7fcaa9600700 1 -- 172.21.15.60:0/4252783374 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fca9802a070 con 0x7fcaac06aea0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.836+0000 7fcaa9600700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca940a1ea0 0x7fca940a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.836+0000 7fcaa9600700 1 -- 172.21.15.60:0/4252783374 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fca980cf3d0 con 0x7fcaac06aea0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.836+0000 7fcab0a00700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca940a1ea0 0x7fca940a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.837+0000 7fcab0a00700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca940a1ea0 0x7fca940a4360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009c60 tx=0x7fcaa0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:31.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:28.837+0000 7fcaa9600700 1 -- 172.21.15.60:0/4252783374 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca98013070 con 0x7fcaac06aea0 2024-02-26T10:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout:201863462975 2024-02-26T10:44:31.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.071+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fcaac02da50 con 0x7fcaac06aea0 2024-02-26T10:44:31.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.071+0000 7fcaa9600700 1 -- 172.21.15.60:0/4252783374 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca9809c370 con 0x7fcaac06aea0 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.074+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca940a1ea0 msgr2=0x7fca940a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.074+0000 7fcab3400700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca940a1ea0 0x7fca940a4360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009c60 tx=0x7fcaa0009260 comp rx=0 tx=0).stop 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.074+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaac06aea0 msgr2=0x7fcaac101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.075+0000 7fcab3400700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaac06aea0 0x7fcaac101290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fca98000c00 tx=0x7fca9800bab0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.075+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 shutdown_connections 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.075+0000 7fcab3400700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fca940a1ea0 0x7fca940a4360 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.075+0000 7fcab3400700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaac107030 0x7fcaac1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.075+0000 7fcab3400700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcaac06a4f0 0x7fcaac100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.075+0000 7fcab3400700 1 --2- 172.21.15.60:0/4252783374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcaac06aea0 0x7fcaac101290 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.075+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 >> 172.21.15.60:0/4252783374 conn(0x7fcaac076d30 msgr2=0x7fcaac100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.075+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 shutdown_connections 2024-02-26T10:44:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:29.075+0000 7fcab3400700 1 -- 172.21.15.60:0/4252783374 wait complete. 2024-02-26T10:44:31.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:31 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2505471617' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T10:44:31.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:31 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2505471617' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T10:44:32.061 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:32.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:32 smithi113 ceph-mon[36126]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T10:44:32.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:32 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2563520595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T10:44:32.607 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:32 smithi060 ceph-mon[31539]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T10:44:32.607 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:32 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2563520595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T10:44:32.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:32 smithi100 ceph-mon[36051]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T10:44:32.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:32 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2563520595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T10:44:32.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037807 got 257698037814 for osd.8 2024-02-26T10:44:32.683 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:33.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.507+0000 7fd466800700 1 -- 172.21.15.60:0/1536777340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460107d80 msgr2=0x7fd460108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:33.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.507+0000 7fd466800700 1 --2- 172.21.15.60:0/1536777340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460107d80 0x7fd460108160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd450009920 tx=0x7fd450009c30 comp rx=0 tx=0).stop 2024-02-26T10:44:33.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.507+0000 7fd466800700 1 -- 172.21.15.60:0/1536777340 shutdown_connections 2024-02-26T10:44:33.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.507+0000 7fd466800700 1 --2- 172.21.15.60:0/1536777340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46006a3c0 0x7fd46010eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.507+0000 7fd466800700 1 --2- 172.21.15.60:0/1536777340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460107d80 0x7fd460108160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.507+0000 7fd466800700 1 --2- 172.21.15.60:0/1536777340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd460069a00 0x7fd460069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.507+0000 7fd466800700 1 -- 172.21.15.60:0/1536777340 >> 172.21.15.60:0/1536777340 conn(0x7fd460077a00 msgr2=0x7fd460077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:33.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.507+0000 7fd466800700 1 -- 172.21.15.60:0/1536777340 shutdown_connections 2024-02-26T10:44:33.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.507+0000 7fd466800700 1 -- 172.21.15.60:0/1536777340 wait complete. 2024-02-26T10:44:33.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.508+0000 7fd466800700 1 Processor -- start 2024-02-26T10:44:33.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.508+0000 7fd466800700 1 -- start start 2024-02-26T10:44:33.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd466800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460069a00 0x7fd460198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:33.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd466800700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46006a3c0 0x7fd460198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:33.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd466800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460107d80 0x7fd46019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd466800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd460112080 con 0x7fd460107d80 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd466800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd460111f00 con 0x7fd460069a00 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd466800700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd460112200 con 0x7fd46006a3c0 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd45ea00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46006a3c0 0x7fd460198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd45ea00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46006a3c0 0x7fd460198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:50644/0 (socket says 172.21.15.60:50644) 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd45ea00700 1 -- 172.21.15.60:0/1238395628 learned_addr learned my addr 172.21.15.60:0/1238395628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd45f400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460069a00 0x7fd460198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.509+0000 7fd45fe00700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460107d80 0x7fd46019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd45ea00700 1 -- 172.21.15.60:0/1238395628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460069a00 msgr2=0x7fd460198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd45ea00700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460069a00 0x7fd460198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd45ea00700 1 -- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460107d80 msgr2=0x7fd46019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd45ea00700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460107d80 0x7fd46019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd45ea00700 1 -- 172.21.15.60:0/1238395628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46019d5a0 con 0x7fd46006a3c0 2024-02-26T10:44:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd45f400700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460069a00 0x7fd460198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:33.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd45fe00700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460107d80 0x7fd46019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:44:33.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd45ea00700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46006a3c0 0x7fd460198ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e9c0 tx=0x7fd44c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:33.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd457e00700 1 -- 172.21.15.60:0/1238395628 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00cbd0 con 0x7fd46006a3c0 2024-02-26T10:44:33.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd457e00700 1 -- 172.21.15.60:0/1238395628 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd44c00cd70 con 0x7fd46006a3c0 2024-02-26T10:44:33.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46019d890 con 0x7fd46006a3c0 2024-02-26T10:44:33.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd457e00700 1 -- 172.21.15.60:0/1238395628 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c010430 con 0x7fd46006a3c0 2024-02-26T10:44:33.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.510+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4601a5110 con 0x7fd46006a3c0 2024-02-26T10:44:33.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.511+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460068a10 con 0x7fd46006a3c0 2024-02-26T10:44:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.513+0000 7fd457e00700 1 -- 172.21.15.60:0/1238395628 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fd44c004750 con 0x7fd46006a3c0 2024-02-26T10:44:33.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.515+0000 7fd457e00700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4440a1ea0 0x7fd4440a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:33.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.515+0000 7fd457e00700 1 -- 172.21.15.60:0/1238395628 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd44c0ce580 con 0x7fd46006a3c0 2024-02-26T10:44:33.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.515+0000 7fd45f400700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4440a1ea0 0x7fd4440a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:33.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.516+0000 7fd45f400700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4440a1ea0 0x7fd4440a4360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd4500098f0 tx=0x7fd450019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:33.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.519+0000 7fd457e00700 1 -- 172.21.15.60:0/1238395628 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd44c09b490 con 0x7fd46006a3c0 2024-02-26T10:44:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.756+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd46002d7f0 con 0x7fd46006a3c0 2024-02-26T10:44:33.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.757+0000 7fd457e00700 1 -- 172.21.15.60:0/1238395628 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd44c09ad90 con 0x7fd46006a3c0 2024-02-26T10:44:33.753 INFO:teuthology.orchestra.run.smithi060.stdout:223338299450 2024-02-26T10:44:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4440a1ea0 msgr2=0x7fd4440a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4440a1ea0 0x7fd4440a4360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd4500098f0 tx=0x7fd450019040 comp rx=0 tx=0).stop 2024-02-26T10:44:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46006a3c0 msgr2=0x7fd460198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46006a3c0 0x7fd460198ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e9c0 tx=0x7fd44c00ecd0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 shutdown_connections 2024-02-26T10:44:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4440a1ea0 0x7fd4440a4360 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd460107d80 0x7fd46019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd460069a00 0x7fd460198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 --2- 172.21.15.60:0/1238395628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46006a3c0 0x7fd460198ad0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 >> 172.21.15.60:0/1238395628 conn(0x7fd460077a00 msgr2=0x7fd4600ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.760+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 shutdown_connections 2024-02-26T10:44:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:33.761+0000 7fd466800700 1 -- 172.21.15.60:0/1238395628 wait complete. 2024-02-26T10:44:33.861 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462969 got 201863462975 for osd.6 2024-02-26T10:44:33.861 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:33.982 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411416 for osd.2 2024-02-26T10:44:33.982 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:34.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:34 smithi060 ceph-mon[31539]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-26T10:44:34.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:34 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1238395628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T10:44:34.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:34 smithi113 ceph-mon[36126]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-26T10:44:34.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:34 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1238395628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T10:44:34.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:34 smithi100 ceph-mon[36051]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-26T10:44:34.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:34 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1238395628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T10:44:34.959 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351006 got 347892351011 for osd.11 2024-02-26T10:44:34.959 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:35.446 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299444 got 223338299450 for osd.7 2024-02-26T10:44:35.446 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:36.317 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:36 smithi060 ceph-mon[31539]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T10:44:36.317 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:44:36.317 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:36 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:44:36.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:36 smithi113 ceph-mon[36126]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T10:44:36.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:36 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:44:36.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:36 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:44:36.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:36 smithi100 ceph-mon[36051]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-02-26T10:44:36.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:36 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:44:36.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:36 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:44:37.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.116+0000 7f2c5fa00700 1 -- 172.21.15.60:0/2845126109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c58073f80 msgr2=0x7f2c58074360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:37.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.116+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/2845126109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c58073f80 0x7f2c58074360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0099c0 tx=0x7f2c4c009cd0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.116+0000 7f2c5fa00700 1 -- 172.21.15.60:0/2845126109 shutdown_connections 2024-02-26T10:44:37.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.116+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/2845126109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c58112f20 0x7f2c58115310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.116+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/2845126109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c58073f80 0x7f2c58074360 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.116+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/2845126109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c580748a0 0x7f2c581129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.116+0000 7f2c5fa00700 1 -- 172.21.15.60:0/2845126109 >> 172.21.15.60:0/2845126109 conn(0x7f2c58078f40 msgr2=0x7f2c58079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:37.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.116+0000 7f2c5fa00700 1 -- 172.21.15.60:0/2845126109 shutdown_connections 2024-02-26T10:44:37.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.116+0000 7f2c5fa00700 1 -- 172.21.15.60:0/2845126109 wait complete. 2024-02-26T10:44:37.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.118+0000 7f2c5fa00700 1 Processor -- start 2024-02-26T10:44:37.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.118+0000 7f2c5fa00700 1 -- start start 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.118+0000 7f2c5fa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c58073f80 0x7f2c581a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5fa00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c580748a0 0x7f2c581a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5fa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c58112f20 0x7f2c5819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5fa00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c58118670 con 0x7f2c58112f20 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5fa00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c581184f0 con 0x7f2c58073f80 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5fa00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c581187f0 con 0x7f2c580748a0 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5d000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c58073f80 0x7f2c581a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5d000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c58073f80 0x7f2c581a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52028/0 (socket says 172.21.15.60:52028) 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5d000700 1 -- 172.21.15.60:0/1251687761 learned_addr learned my addr 172.21.15.60:0/1251687761 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c57e00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c580748a0 0x7f2c581a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5da00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c58112f20 0x7f2c5819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5d000700 1 -- 172.21.15.60:0/1251687761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c580748a0 msgr2=0x7f2c581a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5d000700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c580748a0 0x7f2c581a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5d000700 1 -- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c58112f20 msgr2=0x7f2c5819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5d000700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c58112f20 0x7f2c5819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5d000700 1 -- 172.21.15.60:0/1251687761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5819f030 con 0x7f2c58073f80 2024-02-26T10:44:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.119+0000 7f2c5da00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c58112f20 0x7f2c5819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:44:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.120+0000 7f2c5d000700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c58073f80 0x7f2c581a4710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c006010 tx=0x7f2c4c003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.120+0000 7f2c55600700 1 -- 172.21.15.60:0/1251687761 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c003680 con 0x7f2c58073f80 2024-02-26T10:44:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.120+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5819f2c0 con 0x7f2c58073f80 2024-02-26T10:44:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.120+0000 7f2c55600700 1 -- 172.21.15.60:0/1251687761 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c4c0055a0 con 0x7f2c58073f80 2024-02-26T10:44:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.120+0000 7f2c55600700 1 -- 172.21.15.60:0/1251687761 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0173f0 con 0x7f2c58073f80 2024-02-26T10:44:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.120+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c581ab5e0 con 0x7f2c58073f80 2024-02-26T10:44:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.122+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c58068a10 con 0x7f2c58073f80 2024-02-26T10:44:37.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.123+0000 7f2c55600700 1 -- 172.21.15.60:0/1251687761 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f2c4c00f5d0 con 0x7f2c58073f80 2024-02-26T10:44:37.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.125+0000 7f2c55600700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c400a2040 0x7f2c400a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:37.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.125+0000 7f2c55600700 1 -- 172.21.15.60:0/1251687761 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2c4c0cefe0 con 0x7f2c58073f80 2024-02-26T10:44:37.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.125+0000 7f2c57e00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c400a2040 0x7f2c400a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:37.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.126+0000 7f2c57e00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c400a2040 0x7f2c400a4500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c5810f7e0 tx=0x7f2c44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:37.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.129+0000 7f2c55600700 1 -- 172.21.15.60:0/1251687761 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c4c013070 con 0x7f2c58073f80 2024-02-26T10:44:37.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.237+0000 7f2c55600700 1 -- 172.21.15.60:0/1251687761 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2c4c097f50 con 0x7f2c58073f80 2024-02-26T10:44:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.378+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f2c581ab7b0 con 0x7f2c58073f80 2024-02-26T10:44:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.379+0000 7f2c55600700 1 -- 172.21.15.60:0/1251687761 <== mon.2 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c4c09bfe0 con 0x7f2c58073f80 2024-02-26T10:44:37.375 INFO:teuthology.orchestra.run.smithi060.stdout:292057776176 2024-02-26T10:44:37.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c400a2040 msgr2=0x7f2c400a4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:37.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c400a2040 0x7f2c400a4500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c5810f7e0 tx=0x7f2c44009260 comp rx=0 tx=0).stop 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c58073f80 msgr2=0x7f2c581a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c58073f80 0x7f2c581a4710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c006010 tx=0x7f2c4c003a90 comp rx=0 tx=0).stop 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 shutdown_connections 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c400a2040 0x7f2c400a4500 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c58112f20 0x7f2c5819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c58073f80 0x7f2c581a4710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 --2- 172.21.15.60:0/1251687761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c580748a0 0x7f2c581a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.383+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 >> 172.21.15.60:0/1251687761 conn(0x7f2c58078f40 msgr2=0x7f2c58111d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.384+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 shutdown_connections 2024-02-26T10:44:37.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:37.384+0000 7f2c5fa00700 1 -- 172.21.15.60:0/1251687761 wait complete. 2024-02-26T10:44:38.189 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776170 got 292057776176 for osd.9 2024-02-26T10:44:38.189 DEBUG:teuthology.parallel:result is None 2024-02-26T10:44:38.190 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-26T10:44:38.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-26T10:44:38.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:38 smithi113 ceph-mon[36126]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T10:44:38.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:38 smithi113 ceph-mon[36126]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-02-26T10:44:38.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:38 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1251687761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T10:44:38.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-02-26T10:44:38.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-02-26T10:44:38.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2024-02-26T10:44:38.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:44:38.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:44:38.584 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:44:38.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:38 smithi060 ceph-mon[31539]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T10:44:38.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:38 smithi060 ceph-mon[31539]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-02-26T10:44:38.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:38 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1251687761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T10:44:38.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-02-26T10:44:38.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-02-26T10:44:38.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2024-02-26T10:44:38.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:44:38.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:44:38.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:44:38.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:38 smithi100 ceph-mon[36051]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-26T10:44:38.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:38 smithi100 ceph-mon[36051]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-02-26T10:44:38.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:38 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1251687761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T10:44:38.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-02-26T10:44:38.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-02-26T10:44:38.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2024-02-26T10:44:38.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:44:38.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:44:38.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:44:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:39.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:39 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-02-26T10:44:39.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:39 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-02-26T10:44:39.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:39 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2024-02-26T10:44:39.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:39 smithi060 ceph-mon[31539]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T10:44:39.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:39 smithi060 ceph-mon[31539]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T10:44:39.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:39 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-02-26T10:44:39.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:39 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-02-26T10:44:39.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:39 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2024-02-26T10:44:39.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:39 smithi113 ceph-mon[36126]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T10:44:39.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:39 smithi113 ceph-mon[36126]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T10:44:39.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:39 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-02-26T10:44:39.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:39 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-02-26T10:44:39.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:39 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2024-02-26T10:44:39.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:39 smithi100 ceph-mon[36051]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T10:44:39.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:39 smithi100 ceph-mon[36051]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T10:44:40.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:40 smithi060 ceph-mon[31539]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-02-26T10:44:40.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:40 smithi060 ceph-mon[31539]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-26T10:44:40.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:40 smithi113 ceph-mon[36126]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-02-26T10:44:40.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:40 smithi113 ceph-mon[36126]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-26T10:44:40.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:40 smithi100 ceph-mon[36051]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-02-26T10:44:40.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:40 smithi100 ceph-mon[36051]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-26T10:44:41.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.038+0000 7eff06c00700 1 -- 172.21.15.60:0/2175180600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff00108f50 msgr2=0x7eff0010cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:41.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.038+0000 7eff06c00700 1 --2- 172.21.15.60:0/2175180600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff00108f50 0x7eff0010cfa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efeec0099c0 tx=0x7efeec009cd0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.038+0000 7eff06c00700 1 -- 172.21.15.60:0/2175180600 shutdown_connections 2024-02-26T10:44:41.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.038+0000 7eff06c00700 1 --2- 172.21.15.60:0/2175180600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff0010d680 0x7eff00115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.038+0000 7eff06c00700 1 --2- 172.21.15.60:0/2175180600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff001085a0 0x7eff00108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.038+0000 7eff06c00700 1 --2- 172.21.15.60:0/2175180600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff00108f50 0x7eff0010cfa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.038+0000 7eff06c00700 1 -- 172.21.15.60:0/2175180600 >> 172.21.15.60:0/2175180600 conn(0x7eff00075520 msgr2=0x7eff00077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:41.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.038+0000 7eff06c00700 1 -- 172.21.15.60:0/2175180600 shutdown_connections 2024-02-26T10:44:41.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.038+0000 7eff06c00700 1 -- 172.21.15.60:0/2175180600 wait complete. 2024-02-26T10:44:41.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.039+0000 7eff06c00700 1 Processor -- start 2024-02-26T10:44:41.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.039+0000 7eff06c00700 1 -- start start 2024-02-26T10:44:41.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7eff06c00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff001085a0 0x7eff001a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:41.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7eff06c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff00108f50 0x7eff001a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:41.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7eff06c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff0010d680 0x7eff0019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:41.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7eff06c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff00118660 con 0x7eff0010d680 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7eff06c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff001184e0 con 0x7eff00108f50 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7eff06c00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff001187e0 con 0x7eff001085a0 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7efeff400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff00108f50 0x7eff001a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7efeffe00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff001085a0 0x7eff001a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7efeff400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff00108f50 0x7eff001a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52038/0 (socket says 172.21.15.60:52038) 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7efeff400700 1 -- 172.21.15.60:0/1779339939 learned_addr learned my addr 172.21.15.60:0/1779339939 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.040+0000 7eff04c00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff0010d680 0x7eff0019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efeff400700 1 -- 172.21.15.60:0/1779339939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff001085a0 msgr2=0x7eff001a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efeff400700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff001085a0 0x7eff001a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efeff400700 1 -- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff0010d680 msgr2=0x7eff0019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efeff400700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff0010d680 0x7eff0019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efeff400700 1 -- 172.21.15.60:0/1779339939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0019f020 con 0x7eff00108f50 2024-02-26T10:44:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efeffe00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff001085a0 0x7eff001a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efeff400700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff00108f50 0x7eff001a4c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efeec006010 tx=0x7efeec003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efefcc00700 1 -- 172.21.15.60:0/1779339939 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec009e60 con 0x7eff00108f50 2024-02-26T10:44:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0019f2b0 con 0x7eff00108f50 2024-02-26T10:44:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efefcc00700 1 -- 172.21.15.60:0/1779339939 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efeec003680 con 0x7eff00108f50 2024-02-26T10:44:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.041+0000 7efefcc00700 1 -- 172.21.15.60:0/1779339939 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec0173f0 con 0x7eff00108f50 2024-02-26T10:44:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.042+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff001ab5d0 con 0x7eff00108f50 2024-02-26T10:44:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.043+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff00109a80 con 0x7eff00108f50 2024-02-26T10:44:41.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.045+0000 7efefcc00700 1 -- 172.21.15.60:0/1779339939 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7efeec00f5d0 con 0x7eff00108f50 2024-02-26T10:44:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.046+0000 7efefcc00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efee40a1e10 0x7efee40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.046+0000 7efefcc00700 1 -- 172.21.15.60:0/1779339939 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7efeec0ceee0 con 0x7eff00108f50 2024-02-26T10:44:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.046+0000 7efeffe00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efee40a1e10 0x7efee40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.047+0000 7efeffe00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efee40a1e10 0x7efee40a42d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff0019fa00 tx=0x7efef00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:41.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.050+0000 7efefcc00700 1 -- 172.21.15.60:0/1779339939 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efeec013070 con 0x7eff00108f50 2024-02-26T10:44:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.272+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff0019fdf0 con 0x7efee40a1e10 2024-02-26T10:44:41.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.290+0000 7efefcc00700 1 -- 172.21.15.60:0/1779339939 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340846 (secure 0 0 0) 0x7eff0019fdf0 con 0x7efee40a1e10 2024-02-26T10:44:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:44:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.297+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efee40a1e10 msgr2=0x7efee40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.297+0000 7eff06c00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efee40a1e10 0x7efee40a42d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff0019fa00 tx=0x7efef00073d0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff00108f50 msgr2=0x7eff001a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff00108f50 0x7eff001a4c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efeec006010 tx=0x7efeec003e00 comp rx=0 tx=0).stop 2024-02-26T10:44:41.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 shutdown_connections 2024-02-26T10:44:41.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7efee40a1e10 0x7efee40a42d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff0010d680 0x7eff0019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff001085a0 0x7eff001a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 --2- 172.21.15.60:0/1779339939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff00108f50 0x7eff001a4c40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:41.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 >> 172.21.15.60:0/1779339939 conn(0x7eff00075520 msgr2=0x7eff00076040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:41.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 shutdown_connections 2024-02-26T10:44:41.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:41.298+0000 7eff06c00700 1 -- 172.21.15.60:0/1779339939 wait complete. 2024-02-26T10:44:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T10:44:42.187 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-02-26T10:44:40.933830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657618,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":642,"num_read_kb":437,"num_write":354,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1244077,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370012,"kb_used_data":12504,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571812,"statfs":{"total":1151940427776,"available":1151561535488,"internally_reserved":0,"allocated":12804096,"data_stored":8195804,"data_compressed":398642,"data_compressed_allocated":536576,"data_compressed_original":2609152,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003858"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308540+0000","last_change":"2024-02-26T10:42:42.176634+0000","last_active":"2024-02-26T10:44:39.308540+0000","last_peered":"2024-02-26T10:44:39.308540+0000","last_clean":"2024-02-26T10:44:39.308540+0000","last_became_active":"2024-02-26T10:42:42.176386+0000","last_became_peered":"2024-02-26T10:42:42.176386+0000","last_unstale":"2024-02-26T10:44:39.308540+0000","last_undegraded":"2024-02-26T10:44:39.308540+0000","last_fullsized":"2024-02-26T10:44:39.308540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:45.005506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528307+0000","last_change":"2024-02-26T10:42:42.176769+0000","last_active":"2024-02-26T10:44:39.528307+0000","last_peered":"2024-02-26T10:44:39.528307+0000","last_clean":"2024-02-26T10:44:39.528307+0000","last_became_active":"2024-02-26T10:42:42.174868+0000","last_became_peered":"2024-02-26T10:42:42.174868+0000","last_unstale":"2024-02-26T10:44:39.528307+0000","last_undegraded":"2024-02-26T10:44:39.528307+0000","last_fullsized":"2024-02-26T10:44:39.528307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:16.202371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528036+0000","last_change":"2024-02-26T10:42:42.184410+0000","last_active":"2024-02-26T10:44:39.528036+0000","last_peered":"2024-02-26T10:44:39.528036+0000","last_clean":"2024-02-26T10:44:39.528036+0000","last_became_active":"2024-02-26T10:42:42.182986+0000","last_became_peered":"2024-02-26T10:42:42.182986+0000","last_unstale":"2024-02-26T10:44:39.528036+0000","last_undegraded":"2024-02-26T10:44:39.528036+0000","last_fullsized":"2024-02-26T10:44:39.528036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:28:45.445325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408778+0000","last_change":"2024-02-26T10:43:38.462828+0000","last_active":"2024-02-26T10:44:39.408778+0000","last_peered":"2024-02-26T10:44:39.408778+0000","last_clean":"2024-02-26T10:44:39.408778+0000","last_became_active":"2024-02-26T10:43:38.461014+0000","last_became_peered":"2024-02-26T10:43:38.461014+0000","last_unstale":"2024-02-26T10:44:39.408778+0000","last_undegraded":"2024-02-26T10:44:39.408778+0000","last_fullsized":"2024-02-26T10:44:39.408778+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:07.857849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234518+0000","last_change":"2024-02-26T10:42:42.192448+0000","last_active":"2024-02-26T10:44:39.234518+0000","last_peered":"2024-02-26T10:44:39.234518+0000","last_clean":"2024-02-26T10:44:39.234518+0000","last_became_active":"2024-02-26T10:42:42.192133+0000","last_became_peered":"2024-02-26T10:42:42.192133+0000","last_unstale":"2024-02-26T10:44:39.234518+0000","last_undegraded":"2024-02-26T10:44:39.234518+0000","last_fullsized":"2024-02-26T10:44:39.234518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:43:49.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.882580+0000","last_change":"2024-02-26T10:42:42.169999+0000","last_active":"2024-02-26T10:44:36.882580+0000","last_peered":"2024-02-26T10:44:36.882580+0000","last_clean":"2024-02-26T10:44:36.882580+0000","last_became_active":"2024-02-26T10:42:42.169678+0000","last_became_peered":"2024-02-26T10:42:42.169678+0000","last_unstale":"2024-02-26T10:44:36.882580+0000","last_undegraded":"2024-02-26T10:44:36.882580+0000","last_fullsized":"2024-02-26T10:44:36.882580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:54.435203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234911+0000","last_change":"2024-02-26T10:42:42.177538+0000","last_active":"2024-02-26T10:44:39.234911+0000","last_peered":"2024-02-26T10:44:39.234911+0000","last_clean":"2024-02-26T10:44:39.234911+0000","last_became_active":"2024-02-26T10:42:42.177204+0000","last_became_peered":"2024-02-26T10:42:42.177204+0000","last_unstale":"2024-02-26T10:44:39.234911+0000","last_undegraded":"2024-02-26T10:44:39.234911+0000","last_fullsized":"2024-02-26T10:44:39.234911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:09.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.447941+0000","last_change":"2024-02-26T10:42:42.191342+0000","last_active":"2024-02-26T10:43:38.447941+0000","last_peered":"2024-02-26T10:43:38.447941+0000","last_clean":"2024-02-26T10:43:38.447941+0000","last_became_active":"2024-02-26T10:42:42.190990+0000","last_became_peered":"2024-02-26T10:42:42.190990+0000","last_unstale":"2024-02-26T10:43:38.447941+0000","last_undegraded":"2024-02-26T10:43:38.447941+0000","last_fullsized":"2024-02-26T10:43:38.447941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:20:45.190300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464568+0000","last_change":"2024-02-26T10:42:42.173175+0000","last_active":"2024-02-26T10:44:39.464568+0000","last_peered":"2024-02-26T10:44:39.464568+0000","last_clean":"2024-02-26T10:44:39.464568+0000","last_became_active":"2024-02-26T10:42:42.172828+0000","last_became_peered":"2024-02-26T10:42:42.172828+0000","last_unstale":"2024-02-26T10:44:39.464568+0000","last_undegraded":"2024-02-26T10:44:39.464568+0000","last_fullsized":"2024-02-26T10:44:39.464568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:17:34.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.883970+0000","last_change":"2024-02-26T10:42:42.167267+0000","last_active":"2024-02-26T10:44:36.883970+0000","last_peered":"2024-02-26T10:44:36.883970+0000","last_clean":"2024-02-26T10:44:36.883970+0000","last_became_active":"2024-02-26T10:42:42.166948+0000","last_became_peered":"2024-02-26T10:42:42.166948+0000","last_unstale":"2024-02-26T10:44:36.883970+0000","last_undegraded":"2024-02-26T10:44:36.883970+0000","last_fullsized":"2024-02-26T10:44:36.883970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:52.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235556+0000","last_change":"2024-02-26T10:42:42.171299+0000","last_active":"2024-02-26T10:44:39.235556+0000","last_peered":"2024-02-26T10:44:39.235556+0000","last_clean":"2024-02-26T10:44:39.235556+0000","last_became_active":"2024-02-26T10:42:42.170616+0000","last_became_peered":"2024-02-26T10:42:42.170616+0000","last_unstale":"2024-02-26T10:44:39.235556+0000","last_undegraded":"2024-02-26T10:44:39.235556+0000","last_fullsized":"2024-02-26T10:44:39.235556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:50.320859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.885430+0000","last_change":"2024-02-26T10:42:42.168408+0000","last_active":"2024-02-26T10:44:36.885430+0000","last_peered":"2024-02-26T10:44:36.885430+0000","last_clean":"2024-02-26T10:44:36.885430+0000","last_became_active":"2024-02-26T10:42:42.167150+0000","last_became_peered":"2024-02-26T10:42:42.167150+0000","last_unstale":"2024-02-26T10:44:36.885430+0000","last_undegraded":"2024-02-26T10:44:36.885430+0000","last_fullsized":"2024-02-26T10:44:36.885430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:34.763331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.234727+0000","last_change":"2024-02-26T10:42:42.171720+0000","last_active":"2024-02-26T10:44:38.234727+0000","last_peered":"2024-02-26T10:44:38.234727+0000","last_clean":"2024-02-26T10:44:38.234727+0000","last_became_active":"2024-02-26T10:42:42.171309+0000","last_became_peered":"2024-02-26T10:42:42.171309+0000","last_unstale":"2024-02-26T10:44:38.234727+0000","last_undegraded":"2024-02-26T10:44:38.234727+0000","last_fullsized":"2024-02-26T10:44:38.234727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:20:52.151553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527224+0000","last_change":"2024-02-26T10:42:42.176857+0000","last_active":"2024-02-26T10:44:39.527224+0000","last_peered":"2024-02-26T10:44:39.527224+0000","last_clean":"2024-02-26T10:44:39.527224+0000","last_became_active":"2024-02-26T10:42:42.176385+0000","last_became_peered":"2024-02-26T10:42:42.176385+0000","last_unstale":"2024-02-26T10:44:39.527224+0000","last_undegraded":"2024-02-26T10:44:39.527224+0000","last_fullsized":"2024-02-26T10:44:39.527224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:45:51.852212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527244+0000","last_change":"2024-02-26T10:42:42.174682+0000","last_active":"2024-02-26T10:44:39.527244+0000","last_peered":"2024-02-26T10:44:39.527244+0000","last_clean":"2024-02-26T10:44:39.527244+0000","last_became_active":"2024-02-26T10:42:42.174328+0000","last_became_peered":"2024-02-26T10:42:42.174328+0000","last_unstale":"2024-02-26T10:44:39.527244+0000","last_undegraded":"2024-02-26T10:44:39.527244+0000","last_fullsized":"2024-02-26T10:44:39.527244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:47.161020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463881+0000","last_change":"2024-02-26T10:42:42.185289+0000","last_active":"2024-02-26T10:44:39.463881+0000","last_peered":"2024-02-26T10:44:39.463881+0000","last_clean":"2024-02-26T10:44:39.463881+0000","last_became_active":"2024-02-26T10:42:42.185000+0000","last_became_peered":"2024-02-26T10:42:42.185000+0000","last_unstale":"2024-02-26T10:44:39.463881+0000","last_undegraded":"2024-02-26T10:44:39.463881+0000","last_fullsized":"2024-02-26T10:44:39.463881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:48.552584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308437+0000","last_change":"2024-02-26T10:42:42.176558+0000","last_active":"2024-02-26T10:44:39.308437+0000","last_peered":"2024-02-26T10:44:39.308437+0000","last_clean":"2024-02-26T10:44:39.308437+0000","last_became_active":"2024-02-26T10:42:42.176334+0000","last_became_peered":"2024-02-26T10:42:42.176334+0000","last_unstale":"2024-02-26T10:44:39.308437+0000","last_undegraded":"2024-02-26T10:44:39.308437+0000","last_fullsized":"2024-02-26T10:44:39.308437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:59.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409569+0000","last_change":"2024-02-26T10:42:42.168534+0000","last_active":"2024-02-26T10:44:39.409569+0000","last_peered":"2024-02-26T10:44:39.409569+0000","last_clean":"2024-02-26T10:44:39.409569+0000","last_became_active":"2024-02-26T10:42:42.168179+0000","last_became_peered":"2024-02-26T10:42:42.168179+0000","last_unstale":"2024-02-26T10:44:39.409569+0000","last_undegraded":"2024-02-26T10:44:39.409569+0000","last_fullsized":"2024-02-26T10:44:39.409569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:09.625969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.232712+0000","last_change":"2024-02-26T10:42:42.192446+0000","last_active":"2024-02-26T10:44:38.232712+0000","last_peered":"2024-02-26T10:44:38.232712+0000","last_clean":"2024-02-26T10:44:38.232712+0000","last_became_active":"2024-02-26T10:42:42.191863+0000","last_became_peered":"2024-02-26T10:42:42.191863+0000","last_unstale":"2024-02-26T10:44:38.232712+0000","last_undegraded":"2024-02-26T10:44:38.232712+0000","last_fullsized":"2024-02-26T10:44:38.232712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:30:19.477529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.232464+0000","last_change":"2024-02-26T10:42:47.257605+0000","last_active":"2024-02-26T10:44:38.232464+0000","last_peered":"2024-02-26T10:44:38.232464+0000","last_clean":"2024-02-26T10:44:38.232464+0000","last_became_active":"2024-02-26T10:42:47.257206+0000","last_became_peered":"2024-02-26T10:42:47.257206+0000","last_unstale":"2024-02-26T10:44:38.232464+0000","last_undegraded":"2024-02-26T10:44:38.232464+0000","last_fullsized":"2024-02-26T10:44:38.232464+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:26:17.681862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409781+0000","last_change":"2024-02-26T10:42:41.613087+0000","last_active":"2024-02-26T10:44:39.409781+0000","last_peered":"2024-02-26T10:44:39.409781+0000","last_clean":"2024-02-26T10:44:39.409781+0000","last_became_active":"2024-02-26T10:42:41.612664+0000","last_became_peered":"2024-02-26T10:42:41.612664+0000","last_unstale":"2024-02-26T10:44:39.409781+0000","last_undegraded":"2024-02-26T10:44:39.409781+0000","last_fullsized":"2024-02-26T10:44:39.409781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:00.352166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.454293+0000","last_change":"2024-02-26T10:42:42.191438+0000","last_active":"2024-02-26T10:43:38.454293+0000","last_peered":"2024-02-26T10:43:38.454293+0000","last_clean":"2024-02-26T10:43:38.454293+0000","last_became_active":"2024-02-26T10:42:42.191095+0000","last_became_peered":"2024-02-26T10:42:42.191095+0000","last_unstale":"2024-02-26T10:43:38.454293+0000","last_undegraded":"2024-02-26T10:43:38.454293+0000","last_fullsized":"2024-02-26T10:43:38.454293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:20.918715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.448095+0000","last_change":"2024-02-26T10:42:47.247503+0000","last_active":"2024-02-26T10:43:38.448095+0000","last_peered":"2024-02-26T10:43:38.448095+0000","last_clean":"2024-02-26T10:43:38.448095+0000","last_became_active":"2024-02-26T10:42:47.247121+0000","last_became_peered":"2024-02-26T10:42:47.247121+0000","last_unstale":"2024-02-26T10:43:38.448095+0000","last_undegraded":"2024-02-26T10:43:38.448095+0000","last_fullsized":"2024-02-26T10:43:38.448095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:01:38.733131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.452104+0000","last_change":"2024-02-26T10:42:41.626227+0000","last_active":"2024-02-26T10:43:38.452104+0000","last_peered":"2024-02-26T10:43:38.452104+0000","last_clean":"2024-02-26T10:43:38.452104+0000","last_became_active":"2024-02-26T10:42:41.625760+0000","last_became_peered":"2024-02-26T10:42:41.625760+0000","last_unstale":"2024-02-26T10:43:38.452104+0000","last_undegraded":"2024-02-26T10:43:38.452104+0000","last_fullsized":"2024-02-26T10:43:38.452104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:04.600151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527980+0000","last_change":"2024-02-26T10:42:42.182808+0000","last_active":"2024-02-26T10:44:39.527980+0000","last_peered":"2024-02-26T10:44:39.527980+0000","last_clean":"2024-02-26T10:44:39.527980+0000","last_became_active":"2024-02-26T10:42:42.182485+0000","last_became_peered":"2024-02-26T10:42:42.182485+0000","last_unstale":"2024-02-26T10:44:39.527980+0000","last_undegraded":"2024-02-26T10:44:39.527980+0000","last_fullsized":"2024-02-26T10:44:39.527980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:45.205969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.448363+0000","last_change":"2024-02-26T10:42:47.250694+0000","last_active":"2024-02-26T10:43:38.448363+0000","last_peered":"2024-02-26T10:43:38.448363+0000","last_clean":"2024-02-26T10:43:38.448363+0000","last_became_active":"2024-02-26T10:42:47.249566+0000","last_became_peered":"2024-02-26T10:42:47.249566+0000","last_unstale":"2024-02-26T10:43:38.448363+0000","last_undegraded":"2024-02-26T10:43:38.448363+0000","last_fullsized":"2024-02-26T10:43:38.448363+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:44.016413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235006+0000","last_change":"2024-02-26T10:42:41.606849+0000","last_active":"2024-02-26T10:44:39.235006+0000","last_peered":"2024-02-26T10:44:39.235006+0000","last_clean":"2024-02-26T10:44:39.235006+0000","last_became_active":"2024-02-26T10:42:41.606494+0000","last_became_peered":"2024-02-26T10:42:41.606494+0000","last_unstale":"2024-02-26T10:44:39.235006+0000","last_undegraded":"2024-02-26T10:44:39.235006+0000","last_fullsized":"2024-02-26T10:44:39.235006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:36.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464226+0000","last_change":"2024-02-26T10:42:42.175926+0000","last_active":"2024-02-26T10:44:39.464226+0000","last_peered":"2024-02-26T10:44:39.464226+0000","last_clean":"2024-02-26T10:44:39.464226+0000","last_became_active":"2024-02-26T10:42:42.175614+0000","last_became_peered":"2024-02-26T10:42:42.175614+0000","last_unstale":"2024-02-26T10:44:39.464226+0000","last_undegraded":"2024-02-26T10:44:39.464226+0000","last_fullsized":"2024-02-26T10:44:39.464226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:20:20.829849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.448235+0000","last_change":"2024-02-26T10:42:47.227215+0000","last_active":"2024-02-26T10:43:38.448235+0000","last_peered":"2024-02-26T10:43:38.448235+0000","last_clean":"2024-02-26T10:43:38.448235+0000","last_became_active":"2024-02-26T10:42:47.226771+0000","last_became_peered":"2024-02-26T10:42:47.226771+0000","last_unstale":"2024-02-26T10:43:38.448235+0000","last_undegraded":"2024-02-26T10:43:38.448235+0000","last_fullsized":"2024-02-26T10:43:38.448235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:12:58.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234833+0000","last_change":"2024-02-26T10:42:41.606407+0000","last_active":"2024-02-26T10:44:39.234833+0000","last_peered":"2024-02-26T10:44:39.234833+0000","last_clean":"2024-02-26T10:44:39.234833+0000","last_became_active":"2024-02-26T10:42:41.606094+0000","last_became_peered":"2024-02-26T10:42:41.606094+0000","last_unstale":"2024-02-26T10:44:39.234833+0000","last_undegraded":"2024-02-26T10:44:39.234833+0000","last_fullsized":"2024-02-26T10:44:39.234833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:53.673017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.895602+0000","last_change":"2024-02-26T10:42:42.172967+0000","last_active":"2024-02-26T10:44:36.895602+0000","last_peered":"2024-02-26T10:44:36.895602+0000","last_clean":"2024-02-26T10:44:36.895602+0000","last_became_active":"2024-02-26T10:42:42.172675+0000","last_became_peered":"2024-02-26T10:42:42.172675+0000","last_unstale":"2024-02-26T10:44:36.895602+0000","last_undegraded":"2024-02-26T10:44:36.895602+0000","last_fullsized":"2024-02-26T10:44:36.895602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.232585+0000","last_change":"2024-02-26T10:42:47.229375+0000","last_active":"2024-02-26T10:44:38.232585+0000","last_peered":"2024-02-26T10:44:38.232585+0000","last_clean":"2024-02-26T10:44:38.232585+0000","last_became_active":"2024-02-26T10:42:47.228874+0000","last_became_peered":"2024-02-26T10:42:47.228874+0000","last_unstale":"2024-02-26T10:44:38.232585+0000","last_undegraded":"2024-02-26T10:44:38.232585+0000","last_fullsized":"2024-02-26T10:44:38.232585+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:02.188519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.165413+0000","last_change":"2024-02-26T10:42:41.602783+0000","last_active":"2024-02-26T10:44:36.165413+0000","last_peered":"2024-02-26T10:44:36.165413+0000","last_clean":"2024-02-26T10:44:36.165413+0000","last_became_active":"2024-02-26T10:42:41.602491+0000","last_became_peered":"2024-02-26T10:42:41.602491+0000","last_unstale":"2024-02-26T10:44:36.165413+0000","last_undegraded":"2024-02-26T10:44:36.165413+0000","last_fullsized":"2024-02-26T10:44:36.165413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:23:33.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234438+0000","last_change":"2024-02-26T10:43:38.461429+0000","last_active":"2024-02-26T10:44:39.234438+0000","last_peered":"2024-02-26T10:44:39.234438+0000","last_clean":"2024-02-26T10:44:39.234438+0000","last_became_active":"2024-02-26T10:43:38.461019+0000","last_became_peered":"2024-02-26T10:43:38.461019+0000","last_unstale":"2024-02-26T10:44:39.234438+0000","last_undegraded":"2024-02-26T10:44:39.234438+0000","last_fullsized":"2024-02-26T10:44:39.234438+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:20.911179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":93,"state":"peering","last_fresh":"2024-02-26T10:43:38.447669+0000","last_change":"2024-02-26T10:44:38.233098+0000","last_active":"2024-02-26T10:44:38.233098+0000","last_peered":"2024-02-26T10:43:38.447669+0000","last_clean":"2024-02-26T10:43:38.447669+0000","last_became_active":"2024-02-26T10:42:47.249560+0000","last_became_peered":"2024-02-26T10:42:47.249560+0000","last_unstale":"2024-02-26T10:43:38.447669+0000","last_undegraded":"2024-02-26T10:43:38.447669+0000","last_fullsized":"2024-02-26T10:43:38.447669+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,9,10],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.445736+0000","last_change":"2024-02-26T10:42:41.602887+0000","last_active":"2024-02-26T10:43:38.445736+0000","last_peered":"2024-02-26T10:43:38.445736+0000","last_clean":"2024-02-26T10:43:38.445736+0000","last_became_active":"2024-02-26T10:42:41.602564+0000","last_became_peered":"2024-02-26T10:42:41.602564+0000","last_unstale":"2024-02-26T10:43:38.445736+0000","last_undegraded":"2024-02-26T10:43:38.445736+0000","last_fullsized":"2024-02-26T10:43:38.445736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:22:41.711283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.892425+0000","last_change":"2024-02-26T10:42:42.192700+0000","last_active":"2024-02-26T10:44:36.892425+0000","last_peered":"2024-02-26T10:44:36.892425+0000","last_clean":"2024-02-26T10:44:36.892425+0000","last_became_active":"2024-02-26T10:42:42.192387+0000","last_became_peered":"2024-02-26T10:42:42.192387+0000","last_unstale":"2024-02-26T10:44:36.892425+0000","last_undegraded":"2024-02-26T10:44:36.892425+0000","last_fullsized":"2024-02-26T10:44:36.892425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:21:15.104872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307788+0000","last_change":"2024-02-26T10:42:47.247000+0000","last_active":"2024-02-26T10:44:39.307788+0000","last_peered":"2024-02-26T10:44:39.307788+0000","last_clean":"2024-02-26T10:44:39.307788+0000","last_became_active":"2024-02-26T10:42:47.246520+0000","last_became_peered":"2024-02-26T10:42:47.246520+0000","last_unstale":"2024-02-26T10:44:39.307788+0000","last_undegraded":"2024-02-26T10:44:39.307788+0000","last_fullsized":"2024-02-26T10:44:39.307788+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:29.977858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.234288+0000","last_change":"2024-02-26T10:42:41.609460+0000","last_active":"2024-02-26T10:44:38.234288+0000","last_peered":"2024-02-26T10:44:38.234288+0000","last_clean":"2024-02-26T10:44:38.234288+0000","last_became_active":"2024-02-26T10:42:41.608981+0000","last_became_peered":"2024-02-26T10:42:41.608981+0000","last_unstale":"2024-02-26T10:44:38.234288+0000","last_undegraded":"2024-02-26T10:44:38.234288+0000","last_fullsized":"2024-02-26T10:44:38.234288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:04.599910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.447744+0000","last_change":"2024-02-26T10:42:42.185359+0000","last_active":"2024-02-26T10:43:38.447744+0000","last_peered":"2024-02-26T10:43:38.447744+0000","last_clean":"2024-02-26T10:43:38.447744+0000","last_became_active":"2024-02-26T10:42:42.184221+0000","last_became_peered":"2024-02-26T10:42:42.184221+0000","last_unstale":"2024-02-26T10:43:38.447744+0000","last_undegraded":"2024-02-26T10:43:38.447744+0000","last_fullsized":"2024-02-26T10:43:38.447744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:53.907290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526520+0000","last_change":"2024-02-26T10:42:47.241700+0000","last_active":"2024-02-26T10:44:39.526520+0000","last_peered":"2024-02-26T10:44:39.526520+0000","last_clean":"2024-02-26T10:44:39.526520+0000","last_became_active":"2024-02-26T10:42:47.240667+0000","last_became_peered":"2024-02-26T10:42:47.240667+0000","last_unstale":"2024-02-26T10:44:39.526520+0000","last_undegraded":"2024-02-26T10:44:39.526520+0000","last_fullsized":"2024-02-26T10:44:39.526520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:19.071244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.454695+0000","last_change":"2024-02-26T10:42:41.604519+0000","last_active":"2024-02-26T10:43:38.454695+0000","last_peered":"2024-02-26T10:43:38.454695+0000","last_clean":"2024-02-26T10:43:38.454695+0000","last_became_active":"2024-02-26T10:42:41.603596+0000","last_became_peered":"2024-02-26T10:42:41.603596+0000","last_unstale":"2024-02-26T10:43:38.454695+0000","last_undegraded":"2024-02-26T10:43:38.454695+0000","last_fullsized":"2024-02-26T10:43:38.454695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:39.764730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408032+0000","last_change":"2024-02-26T10:42:42.187532+0000","last_active":"2024-02-26T10:44:39.408032+0000","last_peered":"2024-02-26T10:44:39.408032+0000","last_clean":"2024-02-26T10:44:39.408032+0000","last_became_active":"2024-02-26T10:42:42.185635+0000","last_became_peered":"2024-02-26T10:42:42.185635+0000","last_unstale":"2024-02-26T10:44:39.408032+0000","last_undegraded":"2024-02-26T10:44:39.408032+0000","last_fullsized":"2024-02-26T10:44:39.408032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:16.894747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.453023+0000","last_change":"2024-02-26T10:42:47.233507+0000","last_active":"2024-02-26T10:43:38.453023+0000","last_peered":"2024-02-26T10:43:38.453023+0000","last_clean":"2024-02-26T10:43:38.453023+0000","last_became_active":"2024-02-26T10:42:47.229589+0000","last_became_peered":"2024-02-26T10:42:47.229589+0000","last_unstale":"2024-02-26T10:43:38.453023+0000","last_undegraded":"2024-02-26T10:43:38.453023+0000","last_fullsized":"2024-02-26T10:43:38.453023+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:02:42.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":114,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.164643+0000","last_change":"2024-02-26T10:42:41.613952+0000","last_active":"2024-02-26T10:44:36.164643+0000","last_peered":"2024-02-26T10:44:36.164643+0000","last_clean":"2024-02-26T10:44:36.164643+0000","last_became_active":"2024-02-26T10:42:41.613656+0000","last_became_peered":"2024-02-26T10:42:41.613656+0000","last_unstale":"2024-02-26T10:44:36.164643+0000","last_undegraded":"2024-02-26T10:44:36.164643+0000","last_fullsized":"2024-02-26T10:44:36.164643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:14.090821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464548+0000","last_change":"2024-02-26T10:42:42.178357+0000","last_active":"2024-02-26T10:44:39.464548+0000","last_peered":"2024-02-26T10:44:39.464548+0000","last_clean":"2024-02-26T10:44:39.464548+0000","last_became_active":"2024-02-26T10:42:42.178051+0000","last_became_peered":"2024-02-26T10:42:42.178051+0000","last_unstale":"2024-02-26T10:44:39.464548+0000","last_undegraded":"2024-02-26T10:44:39.464548+0000","last_fullsized":"2024-02-26T10:44:39.464548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:13:01.326122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406022+0000","last_change":"2024-02-26T10:42:47.246231+0000","last_active":"2024-02-26T10:44:39.406022+0000","last_peered":"2024-02-26T10:44:39.406022+0000","last_clean":"2024-02-26T10:44:39.406022+0000","last_became_active":"2024-02-26T10:42:47.245739+0000","last_became_peered":"2024-02-26T10:42:47.245739+0000","last_unstale":"2024-02-26T10:44:39.406022+0000","last_undegraded":"2024-02-26T10:44:39.406022+0000","last_fullsized":"2024-02-26T10:44:39.406022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:30.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235472+0000","last_change":"2024-02-26T10:42:41.603126+0000","last_active":"2024-02-26T10:44:39.235472+0000","last_peered":"2024-02-26T10:44:39.235472+0000","last_clean":"2024-02-26T10:44:39.235472+0000","last_became_active":"2024-02-26T10:42:41.602790+0000","last_became_peered":"2024-02-26T10:42:41.602790+0000","last_unstale":"2024-02-26T10:44:39.235472+0000","last_undegraded":"2024-02-26T10:44:39.235472+0000","last_fullsized":"2024-02-26T10:44:39.235472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:31:58.908209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528369+0000","last_change":"2024-02-26T10:43:38.459527+0000","last_active":"2024-02-26T10:44:39.528369+0000","last_peered":"2024-02-26T10:44:39.528369+0000","last_clean":"2024-02-26T10:44:39.528369+0000","last_became_active":"2024-02-26T10:43:38.459150+0000","last_became_peered":"2024-02-26T10:43:38.459150+0000","last_unstale":"2024-02-26T10:44:39.528369+0000","last_undegraded":"2024-02-26T10:44:39.528369+0000","last_fullsized":"2024-02-26T10:44:39.528369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:21:54.028328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.448590+0000","last_change":"2024-02-26T10:42:47.250075+0000","last_active":"2024-02-26T10:43:38.448590+0000","last_peered":"2024-02-26T10:43:38.448590+0000","last_clean":"2024-02-26T10:43:38.448590+0000","last_became_active":"2024-02-26T10:42:47.249490+0000","last_became_peered":"2024-02-26T10:42:47.249490+0000","last_unstale":"2024-02-26T10:43:38.448590+0000","last_undegraded":"2024-02-26T10:43:38.448590+0000","last_fullsized":"2024-02-26T10:43:38.448590+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:42.659282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.454914+0000","last_change":"2024-02-26T10:42:41.615314+0000","last_active":"2024-02-26T10:43:38.454914+0000","last_peered":"2024-02-26T10:43:38.454914+0000","last_clean":"2024-02-26T10:43:38.454914+0000","last_became_active":"2024-02-26T10:42:41.615091+0000","last_became_peered":"2024-02-26T10:42:41.615091+0000","last_unstale":"2024-02-26T10:43:38.454914+0000","last_undegraded":"2024-02-26T10:43:38.454914+0000","last_fullsized":"2024-02-26T10:43:38.454914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:41:52.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464386+0000","last_change":"2024-02-26T10:42:42.176609+0000","last_active":"2024-02-26T10:44:39.464386+0000","last_peered":"2024-02-26T10:44:39.464386+0000","last_clean":"2024-02-26T10:44:39.464386+0000","last_became_active":"2024-02-26T10:42:42.176311+0000","last_became_peered":"2024-02-26T10:42:42.176311+0000","last_unstale":"2024-02-26T10:44:39.464386+0000","last_undegraded":"2024-02-26T10:44:39.464386+0000","last_fullsized":"2024-02-26T10:44:39.464386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:25:38.415311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406108+0000","last_change":"2024-02-26T10:42:47.250255+0000","last_active":"2024-02-26T10:44:39.406108+0000","last_peered":"2024-02-26T10:44:39.406108+0000","last_clean":"2024-02-26T10:44:39.406108+0000","last_became_active":"2024-02-26T10:42:47.249465+0000","last_became_peered":"2024-02-26T10:42:47.249465+0000","last_unstale":"2024-02-26T10:44:39.406108+0000","last_undegraded":"2024-02-26T10:44:39.406108+0000","last_fullsized":"2024-02-26T10:44:39.406108+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:23.173863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":255,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:37.550179+0000","last_change":"2024-02-26T10:43:38.460725+0000","last_active":"2024-02-26T10:44:37.550179+0000","last_peered":"2024-02-26T10:44:37.550179+0000","last_clean":"2024-02-26T10:44:37.550179+0000","last_became_active":"2024-02-26T10:43:38.451900+0000","last_became_peered":"2024-02-26T10:43:38.451900+0000","last_unstale":"2024-02-26T10:44:37.550179+0000","last_undegraded":"2024-02-26T10:44:37.550179+0000","last_fullsized":"2024-02-26T10:44:37.550179+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:35:09.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":170,"num_read_kb":146,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":62625,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234883+0000","last_change":"2024-02-26T10:42:42.168174+0000","last_active":"2024-02-26T10:44:39.234883+0000","last_peered":"2024-02-26T10:44:39.234883+0000","last_clean":"2024-02-26T10:44:39.234883+0000","last_became_active":"2024-02-26T10:42:42.167840+0000","last_became_peered":"2024-02-26T10:42:42.167840+0000","last_unstale":"2024-02-26T10:44:39.234883+0000","last_undegraded":"2024-02-26T10:44:39.234883+0000","last_fullsized":"2024-02-26T10:44:39.234883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:43.079139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.455518+0000","last_change":"2024-02-26T10:42:47.230048+0000","last_active":"2024-02-26T10:43:38.455518+0000","last_peered":"2024-02-26T10:43:38.455518+0000","last_clean":"2024-02-26T10:43:38.455518+0000","last_became_active":"2024-02-26T10:42:47.229396+0000","last_became_peered":"2024-02-26T10:42:47.229396+0000","last_unstale":"2024-02-26T10:43:38.455518+0000","last_undegraded":"2024-02-26T10:43:38.455518+0000","last_fullsized":"2024-02-26T10:43:38.455518+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:57.804297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.232360+0000","last_change":"2024-02-26T10:42:41.614413+0000","last_active":"2024-02-26T10:44:38.232360+0000","last_peered":"2024-02-26T10:44:38.232360+0000","last_clean":"2024-02-26T10:44:38.232360+0000","last_became_active":"2024-02-26T10:42:41.614065+0000","last_became_peered":"2024-02-26T10:42:41.614065+0000","last_unstale":"2024-02-26T10:44:38.232360+0000","last_undegraded":"2024-02-26T10:44:38.232360+0000","last_fullsized":"2024-02-26T10:44:38.232360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:50:20.552340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.234950+0000","last_change":"2024-02-26T10:42:42.188140+0000","last_active":"2024-02-26T10:44:38.234950+0000","last_peered":"2024-02-26T10:44:38.234950+0000","last_clean":"2024-02-26T10:44:38.234950+0000","last_became_active":"2024-02-26T10:42:42.187055+0000","last_became_peered":"2024-02-26T10:42:42.187055+0000","last_unstale":"2024-02-26T10:44:38.234950+0000","last_undegraded":"2024-02-26T10:44:38.234950+0000","last_fullsized":"2024-02-26T10:44:38.234950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:49:43.568562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308052+0000","last_change":"2024-02-26T10:43:38.450681+0000","last_active":"2024-02-26T10:44:39.308052+0000","last_peered":"2024-02-26T10:44:39.308052+0000","last_clean":"2024-02-26T10:44:39.308052+0000","last_became_active":"2024-02-26T10:43:38.450120+0000","last_became_peered":"2024-02-26T10:43:38.450120+0000","last_unstale":"2024-02-26T10:44:39.308052+0000","last_undegraded":"2024-02-26T10:44:39.308052+0000","last_fullsized":"2024-02-26T10:44:39.308052+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:10.983700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.455246+0000","last_change":"2024-02-26T10:42:41.612337+0000","last_active":"2024-02-26T10:43:38.455246+0000","last_peered":"2024-02-26T10:43:38.455246+0000","last_clean":"2024-02-26T10:43:38.455246+0000","last_became_active":"2024-02-26T10:42:41.611920+0000","last_became_peered":"2024-02-26T10:42:41.611920+0000","last_unstale":"2024-02-26T10:43:38.455246+0000","last_undegraded":"2024-02-26T10:43:38.455246+0000","last_fullsized":"2024-02-26T10:43:38.455246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:29:13.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235264+0000","last_change":"2024-02-26T10:43:38.456919+0000","last_active":"2024-02-26T10:44:39.235264+0000","last_peered":"2024-02-26T10:44:39.235264+0000","last_clean":"2024-02-26T10:44:39.235264+0000","last_became_active":"2024-02-26T10:43:38.456461+0000","last_became_peered":"2024-02-26T10:43:38.456461+0000","last_unstale":"2024-02-26T10:44:39.235264+0000","last_undegraded":"2024-02-26T10:44:39.235264+0000","last_fullsized":"2024-02-26T10:44:39.235264+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:44.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406978+0000","last_change":"2024-02-26T10:42:47.254585+0000","last_active":"2024-02-26T10:44:39.406978+0000","last_peered":"2024-02-26T10:44:39.406978+0000","last_clean":"2024-02-26T10:44:39.406978+0000","last_became_active":"2024-02-26T10:42:47.253870+0000","last_became_peered":"2024-02-26T10:42:47.253870+0000","last_unstale":"2024-02-26T10:44:39.406978+0000","last_undegraded":"2024-02-26T10:44:39.406978+0000","last_fullsized":"2024-02-26T10:44:39.406978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:49.276906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":120,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:37.193530+0000","last_change":"2024-02-26T10:42:41.600901+0000","last_active":"2024-02-26T10:44:37.193530+0000","last_peered":"2024-02-26T10:44:37.193530+0000","last_clean":"2024-02-26T10:44:37.193530+0000","last_became_active":"2024-02-26T10:42:41.600018+0000","last_became_peered":"2024-02-26T10:42:41.600018+0000","last_unstale":"2024-02-26T10:44:37.193530+0000","last_undegraded":"2024-02-26T10:44:37.193530+0000","last_fullsized":"2024-02-26T10:44:37.193530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:07.876694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234957+0000","last_change":"2024-02-26T10:42:42.168702+0000","last_active":"2024-02-26T10:44:39.234957+0000","last_peered":"2024-02-26T10:44:39.234957+0000","last_clean":"2024-02-26T10:44:39.234957+0000","last_became_active":"2024-02-26T10:42:42.168420+0000","last_became_peered":"2024-02-26T10:42:42.168420+0000","last_unstale":"2024-02-26T10:44:39.234957+0000","last_undegraded":"2024-02-26T10:44:39.234957+0000","last_fullsized":"2024-02-26T10:44:39.234957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:35.111699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.236213+0000","last_change":"2024-02-26T10:42:47.250714+0000","last_active":"2024-02-26T10:44:38.236213+0000","last_peered":"2024-02-26T10:44:38.236213+0000","last_clean":"2024-02-26T10:44:38.236213+0000","last_became_active":"2024-02-26T10:42:47.249506+0000","last_became_peered":"2024-02-26T10:42:47.249506+0000","last_unstale":"2024-02-26T10:44:38.236213+0000","last_undegraded":"2024-02-26T10:44:38.236213+0000","last_fullsized":"2024-02-26T10:44:38.236213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:05.058052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307933+0000","last_change":"2024-02-26T10:43:38.472808+0000","last_active":"2024-02-26T10:44:39.307933+0000","last_peered":"2024-02-26T10:44:39.307933+0000","last_clean":"2024-02-26T10:44:39.307933+0000","last_became_active":"2024-02-26T10:43:38.456509+0000","last_became_peered":"2024-02-26T10:43:38.456509+0000","last_unstale":"2024-02-26T10:44:39.307933+0000","last_undegraded":"2024-02-26T10:44:39.307933+0000","last_fullsized":"2024-02-26T10:44:39.307933+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:03:06.127386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.888225+0000","last_change":"2024-02-26T10:42:42.172845+0000","last_active":"2024-02-26T10:44:36.888225+0000","last_peered":"2024-02-26T10:44:36.888225+0000","last_clean":"2024-02-26T10:44:36.888225+0000","last_became_active":"2024-02-26T10:42:42.171263+0000","last_became_peered":"2024-02-26T10:42:42.171263+0000","last_unstale":"2024-02-26T10:44:36.888225+0000","last_undegraded":"2024-02-26T10:44:36.888225+0000","last_fullsized":"2024-02-26T10:44:36.888225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:21.439474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.454713+0000","last_change":"2024-02-26T10:42:47.236434+0000","last_active":"2024-02-26T10:43:38.454713+0000","last_peered":"2024-02-26T10:43:38.454713+0000","last_clean":"2024-02-26T10:43:38.454713+0000","last_became_active":"2024-02-26T10:42:47.236040+0000","last_became_peered":"2024-02-26T10:42:47.236040+0000","last_unstale":"2024-02-26T10:43:38.454713+0000","last_undegraded":"2024-02-26T10:43:38.454713+0000","last_fullsized":"2024-02-26T10:43:38.454713+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:20:23.230348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235189+0000","last_change":"2024-02-26T10:42:41.604208+0000","last_active":"2024-02-26T10:44:39.235189+0000","last_peered":"2024-02-26T10:44:39.235189+0000","last_clean":"2024-02-26T10:44:39.235189+0000","last_became_active":"2024-02-26T10:42:41.603894+0000","last_became_peered":"2024-02-26T10:42:41.603894+0000","last_unstale":"2024-02-26T10:44:39.235189+0000","last_undegraded":"2024-02-26T10:44:39.235189+0000","last_fullsized":"2024-02-26T10:44:39.235189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:57:13.775868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308356+0000","last_change":"2024-02-26T10:42:42.169395+0000","last_active":"2024-02-26T10:44:39.308356+0000","last_peered":"2024-02-26T10:44:39.308356+0000","last_clean":"2024-02-26T10:44:39.308356+0000","last_became_active":"2024-02-26T10:42:42.168686+0000","last_became_peered":"2024-02-26T10:42:42.168686+0000","last_unstale":"2024-02-26T10:44:39.308356+0000","last_undegraded":"2024-02-26T10:44:39.308356+0000","last_fullsized":"2024-02-26T10:44:39.308356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:21:35.085188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234616+0000","last_change":"2024-02-26T10:42:47.252119+0000","last_active":"2024-02-26T10:44:39.234616+0000","last_peered":"2024-02-26T10:44:39.234616+0000","last_clean":"2024-02-26T10:44:39.234616+0000","last_became_active":"2024-02-26T10:42:47.251723+0000","last_became_peered":"2024-02-26T10:42:47.251723+0000","last_unstale":"2024-02-26T10:44:39.234616+0000","last_undegraded":"2024-02-26T10:44:39.234616+0000","last_fullsized":"2024-02-26T10:44:39.234616+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:17.130806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408505+0000","last_change":"2024-02-26T10:42:41.621393+0000","last_active":"2024-02-26T10:44:39.408505+0000","last_peered":"2024-02-26T10:44:39.408505+0000","last_clean":"2024-02-26T10:44:39.408505+0000","last_became_active":"2024-02-26T10:42:41.621008+0000","last_became_peered":"2024-02-26T10:42:41.621008+0000","last_unstale":"2024-02-26T10:44:39.408505+0000","last_undegraded":"2024-02-26T10:44:39.408505+0000","last_fullsized":"2024-02-26T10:44:39.408505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:46.979854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.891076+0000","last_change":"2024-02-26T10:43:38.458926+0000","last_active":"2024-02-26T10:44:36.891076+0000","last_peered":"2024-02-26T10:44:36.891076+0000","last_clean":"2024-02-26T10:44:36.891076+0000","last_became_active":"2024-02-26T10:43:38.458537+0000","last_became_peered":"2024-02-26T10:43:38.458537+0000","last_unstale":"2024-02-26T10:44:36.891076+0000","last_undegraded":"2024-02-26T10:44:36.891076+0000","last_fullsized":"2024-02-26T10:44:36.891076+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:08.204646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.234671+0000","last_change":"2024-02-26T10:42:47.238520+0000","last_active":"2024-02-26T10:44:38.234671+0000","last_peered":"2024-02-26T10:44:38.234671+0000","last_clean":"2024-02-26T10:44:38.234671+0000","last_became_active":"2024-02-26T10:42:47.237213+0000","last_became_peered":"2024-02-26T10:42:47.237213+0000","last_unstale":"2024-02-26T10:44:38.234671+0000","last_undegraded":"2024-02-26T10:44:38.234671+0000","last_fullsized":"2024-02-26T10:44:38.234671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:29.283850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235026+0000","last_change":"2024-02-26T10:42:41.609479+0000","last_active":"2024-02-26T10:44:39.235026+0000","last_peered":"2024-02-26T10:44:39.235026+0000","last_clean":"2024-02-26T10:44:39.235026+0000","last_became_active":"2024-02-26T10:42:41.609084+0000","last_became_peered":"2024-02-26T10:42:41.609084+0000","last_unstale":"2024-02-26T10:44:39.235026+0000","last_undegraded":"2024-02-26T10:44:39.235026+0000","last_fullsized":"2024-02-26T10:44:39.235026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:32.412507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.891979+0000","last_change":"2024-02-26T10:42:42.174985+0000","last_active":"2024-02-26T10:44:36.891979+0000","last_peered":"2024-02-26T10:44:36.891979+0000","last_clean":"2024-02-26T10:44:36.891979+0000","last_became_active":"2024-02-26T10:42:42.174694+0000","last_became_peered":"2024-02-26T10:42:42.174694+0000","last_unstale":"2024-02-26T10:44:36.891979+0000","last_undegraded":"2024-02-26T10:44:36.891979+0000","last_fullsized":"2024-02-26T10:44:36.891979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:10.000154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.237848+0000","last_change":"2024-02-26T10:42:47.242587+0000","last_active":"2024-02-26T10:44:38.237848+0000","last_peered":"2024-02-26T10:44:38.237848+0000","last_clean":"2024-02-26T10:44:38.237848+0000","last_became_active":"2024-02-26T10:42:47.242099+0000","last_became_peered":"2024-02-26T10:42:47.242099+0000","last_unstale":"2024-02-26T10:44:38.237848+0000","last_undegraded":"2024-02-26T10:44:38.237848+0000","last_fullsized":"2024-02-26T10:44:38.237848+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:38:42.852080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.874819+0000","last_change":"2024-02-26T10:42:42.170613+0000","last_active":"2024-02-26T10:44:36.874819+0000","last_peered":"2024-02-26T10:44:36.874819+0000","last_clean":"2024-02-26T10:44:36.874819+0000","last_became_active":"2024-02-26T10:42:42.170317+0000","last_became_peered":"2024-02-26T10:42:42.170317+0000","last_unstale":"2024-02-26T10:44:36.874819+0000","last_undegraded":"2024-02-26T10:44:36.874819+0000","last_fullsized":"2024-02-26T10:44:36.874819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:29:44.276718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":119,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.233476+0000","last_change":"2024-02-26T10:42:41.617447+0000","last_active":"2024-02-26T10:44:38.233476+0000","last_peered":"2024-02-26T10:44:38.233476+0000","last_clean":"2024-02-26T10:44:38.233476+0000","last_became_active":"2024-02-26T10:42:41.617107+0000","last_became_peered":"2024-02-26T10:42:41.617107+0000","last_unstale":"2024-02-26T10:44:38.233476+0000","last_undegraded":"2024-02-26T10:44:38.233476+0000","last_fullsized":"2024-02-26T10:44:38.233476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:06.052364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235377+0000","last_change":"2024-02-26T10:42:47.251971+0000","last_active":"2024-02-26T10:44:39.235377+0000","last_peered":"2024-02-26T10:44:39.235377+0000","last_clean":"2024-02-26T10:44:39.235377+0000","last_became_active":"2024-02-26T10:42:47.251506+0000","last_became_peered":"2024-02-26T10:42:47.251506+0000","last_unstale":"2024-02-26T10:44:39.235377+0000","last_undegraded":"2024-02-26T10:44:39.235377+0000","last_fullsized":"2024-02-26T10:44:39.235377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:37.029348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.870680+0000","last_change":"2024-02-26T10:42:42.194414+0000","last_active":"2024-02-26T10:44:36.870680+0000","last_peered":"2024-02-26T10:44:36.870680+0000","last_clean":"2024-02-26T10:44:36.870680+0000","last_became_active":"2024-02-26T10:42:42.193959+0000","last_became_peered":"2024-02-26T10:42:42.193959+0000","last_unstale":"2024-02-26T10:44:36.870680+0000","last_undegraded":"2024-02-26T10:44:36.870680+0000","last_fullsized":"2024-02-26T10:44:36.870680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:50.398674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.448419+0000","last_change":"2024-02-26T10:42:41.607947+0000","last_active":"2024-02-26T10:43:38.448419+0000","last_peered":"2024-02-26T10:43:38.448419+0000","last_clean":"2024-02-26T10:43:38.448419+0000","last_became_active":"2024-02-26T10:42:41.607522+0000","last_became_peered":"2024-02-26T10:42:41.607522+0000","last_unstale":"2024-02-26T10:43:38.448419+0000","last_undegraded":"2024-02-26T10:43:38.448419+0000","last_fullsized":"2024-02-26T10:43:38.448419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:32:36.647621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234832+0000","last_change":"2024-02-26T10:41:21.177202+0000","last_active":"2024-02-26T10:44:39.234832+0000","last_peered":"2024-02-26T10:44:39.234832+0000","last_clean":"2024-02-26T10:44:39.234832+0000","last_became_active":"2024-02-26T10:41:21.176762+0000","last_became_peered":"2024-02-26T10:41:21.176762+0000","last_unstale":"2024-02-26T10:44:39.234832+0000","last_undegraded":"2024-02-26T10:44:39.234832+0000","last_fullsized":"2024-02-26T10:44:39.234832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_clean_scrub_stamp":"2024-02-26T10:37:25.424790+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:19.619104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.455116+0000","last_change":"2024-02-26T10:42:47.244346+0000","last_active":"2024-02-26T10:43:38.455116+0000","last_peered":"2024-02-26T10:43:38.455116+0000","last_clean":"2024-02-26T10:43:38.455116+0000","last_became_active":"2024-02-26T10:42:47.243872+0000","last_became_peered":"2024-02-26T10:42:47.243872+0000","last_unstale":"2024-02-26T10:43:38.455116+0000","last_undegraded":"2024-02-26T10:43:38.455116+0000","last_fullsized":"2024-02-26T10:43:38.455116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:36.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.233499+0000","last_change":"2024-02-26T10:42:42.166266+0000","last_active":"2024-02-26T10:44:38.233499+0000","last_peered":"2024-02-26T10:44:38.233499+0000","last_clean":"2024-02-26T10:44:38.233499+0000","last_became_active":"2024-02-26T10:42:42.165721+0000","last_became_peered":"2024-02-26T10:42:42.165721+0000","last_unstale":"2024-02-26T10:44:38.233499+0000","last_undegraded":"2024-02-26T10:44:38.233499+0000","last_fullsized":"2024-02-26T10:44:38.233499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:32.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234923+0000","last_change":"2024-02-26T10:42:41.606268+0000","last_active":"2024-02-26T10:44:39.234923+0000","last_peered":"2024-02-26T10:44:39.234923+0000","last_clean":"2024-02-26T10:44:39.234923+0000","last_became_active":"2024-02-26T10:42:41.605950+0000","last_became_peered":"2024-02-26T10:42:41.605950+0000","last_unstale":"2024-02-26T10:44:39.234923+0000","last_undegraded":"2024-02-26T10:44:39.234923+0000","last_fullsized":"2024-02-26T10:44:39.234923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:40:03.616987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307750+0000","last_change":"2024-02-26T10:42:47.233203+0000","last_active":"2024-02-26T10:44:39.307750+0000","last_peered":"2024-02-26T10:44:39.307750+0000","last_clean":"2024-02-26T10:44:39.307750+0000","last_became_active":"2024-02-26T10:42:47.232705+0000","last_became_peered":"2024-02-26T10:42:47.232705+0000","last_unstale":"2024-02-26T10:44:39.307750+0000","last_undegraded":"2024-02-26T10:44:39.307750+0000","last_fullsized":"2024-02-26T10:44:39.307750+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:53.818901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":19,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308190+0000","last_change":"2024-02-26T10:43:38.479911+0000","last_active":"2024-02-26T10:44:39.308190+0000","last_peered":"2024-02-26T10:44:39.308190+0000","last_clean":"2024-02-26T10:44:39.308190+0000","last_became_active":"2024-02-26T10:43:38.458874+0000","last_became_peered":"2024-02-26T10:43:38.458874+0000","last_unstale":"2024-02-26T10:44:39.308190+0000","last_undegraded":"2024-02-26T10:44:39.308190+0000","last_fullsized":"2024-02-26T10:44:39.308190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:56.093775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.235600+0000","last_change":"2024-02-26T10:42:42.167674+0000","last_active":"2024-02-26T10:44:38.235600+0000","last_peered":"2024-02-26T10:44:38.235600+0000","last_clean":"2024-02-26T10:44:38.235600+0000","last_became_active":"2024-02-26T10:42:42.167224+0000","last_became_peered":"2024-02-26T10:42:42.167224+0000","last_unstale":"2024-02-26T10:44:38.235600+0000","last_undegraded":"2024-02-26T10:44:38.235600+0000","last_fullsized":"2024-02-26T10:44:38.235600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:49:20.433913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.446523+0000","last_change":"2024-02-26T10:42:47.244076+0000","last_active":"2024-02-26T10:43:38.446523+0000","last_peered":"2024-02-26T10:43:38.446523+0000","last_clean":"2024-02-26T10:43:38.446523+0000","last_became_active":"2024-02-26T10:42:47.243569+0000","last_became_peered":"2024-02-26T10:42:47.243569+0000","last_unstale":"2024-02-26T10:43:38.446523+0000","last_undegraded":"2024-02-26T10:43:38.446523+0000","last_fullsized":"2024-02-26T10:43:38.446523+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:46:19.855621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409415+0000","last_change":"2024-02-26T10:42:41.623604+0000","last_active":"2024-02-26T10:44:39.409415+0000","last_peered":"2024-02-26T10:44:39.409415+0000","last_clean":"2024-02-26T10:44:39.409415+0000","last_became_active":"2024-02-26T10:42:41.623272+0000","last_became_peered":"2024-02-26T10:42:41.623272+0000","last_unstale":"2024-02-26T10:44:39.409415+0000","last_undegraded":"2024-02-26T10:44:39.409415+0000","last_fullsized":"2024-02-26T10:44:39.409415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:13:57.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.454959+0000","last_change":"2024-02-26T10:42:42.186300+0000","last_active":"2024-02-26T10:43:38.454959+0000","last_peered":"2024-02-26T10:43:38.454959+0000","last_clean":"2024-02-26T10:43:38.454959+0000","last_became_active":"2024-02-26T10:42:42.185965+0000","last_became_peered":"2024-02-26T10:42:42.185965+0000","last_unstale":"2024-02-26T10:43:38.454959+0000","last_undegraded":"2024-02-26T10:43:38.454959+0000","last_fullsized":"2024-02-26T10:43:38.454959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:06:50.352308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.232290+0000","last_change":"2024-02-26T10:42:47.227753+0000","last_active":"2024-02-26T10:44:38.232290+0000","last_peered":"2024-02-26T10:44:38.232290+0000","last_clean":"2024-02-26T10:44:38.232290+0000","last_became_active":"2024-02-26T10:42:47.227300+0000","last_became_peered":"2024-02-26T10:42:47.227300+0000","last_unstale":"2024-02-26T10:44:38.232290+0000","last_undegraded":"2024-02-26T10:44:38.232290+0000","last_fullsized":"2024-02-26T10:44:38.232290+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:08.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.448447+0000","last_change":"2024-02-26T10:42:41.602960+0000","last_active":"2024-02-26T10:43:38.448447+0000","last_peered":"2024-02-26T10:43:38.448447+0000","last_clean":"2024-02-26T10:43:38.448447+0000","last_became_active":"2024-02-26T10:42:41.602676+0000","last_became_peered":"2024-02-26T10:42:41.602676+0000","last_unstale":"2024-02-26T10:43:38.448447+0000","last_undegraded":"2024-02-26T10:43:38.448447+0000","last_fullsized":"2024-02-26T10:43:38.448447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:57.997612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528654+0000","last_change":"2024-02-26T10:42:42.185451+0000","last_active":"2024-02-26T10:44:39.528654+0000","last_peered":"2024-02-26T10:44:39.528654+0000","last_clean":"2024-02-26T10:44:39.528654+0000","last_became_active":"2024-02-26T10:42:42.185116+0000","last_became_peered":"2024-02-26T10:42:42.185116+0000","last_unstale":"2024-02-26T10:44:39.528654+0000","last_undegraded":"2024-02-26T10:44:39.528654+0000","last_fullsized":"2024-02-26T10:44:39.528654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:43:39.823370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.448235+0000","last_change":"2024-02-26T10:42:47.250682+0000","last_active":"2024-02-26T10:43:38.448235+0000","last_peered":"2024-02-26T10:43:38.448235+0000","last_clean":"2024-02-26T10:43:38.448235+0000","last_became_active":"2024-02-26T10:42:47.249524+0000","last_became_peered":"2024-02-26T10:42:47.249524+0000","last_unstale":"2024-02-26T10:43:38.448235+0000","last_undegraded":"2024-02-26T10:43:38.448235+0000","last_fullsized":"2024-02-26T10:43:38.448235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:07:01.307206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463895+0000","last_change":"2024-02-26T10:42:41.620764+0000","last_active":"2024-02-26T10:44:39.463895+0000","last_peered":"2024-02-26T10:44:39.463895+0000","last_clean":"2024-02-26T10:44:39.463895+0000","last_became_active":"2024-02-26T10:42:41.620500+0000","last_became_peered":"2024-02-26T10:42:41.620500+0000","last_unstale":"2024-02-26T10:44:39.463895+0000","last_undegraded":"2024-02-26T10:44:39.463895+0000","last_fullsized":"2024-02-26T10:44:39.463895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:15.542396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234388+0000","last_change":"2024-02-26T10:42:42.165060+0000","last_active":"2024-02-26T10:44:39.234388+0000","last_peered":"2024-02-26T10:44:39.234388+0000","last_clean":"2024-02-26T10:44:39.234388+0000","last_became_active":"2024-02-26T10:42:42.164733+0000","last_became_peered":"2024-02-26T10:42:42.164733+0000","last_unstale":"2024-02-26T10:44:39.234388+0000","last_undegraded":"2024-02-26T10:44:39.234388+0000","last_fullsized":"2024-02-26T10:44:39.234388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:34.190747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":6,"reported_epoch":95,"state":"activating","last_fresh":"2024-02-26T10:44:39.557143+0000","last_change":"2024-02-26T10:44:39.235343+0000","last_active":"2024-02-26T10:42:46.177569+0000","last_peered":"2024-02-26T10:42:46.177569+0000","last_clean":"2024-02-26T10:42:46.177569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:44:39.557143+0000","last_undegraded":"2024-02-26T10:44:39.557143+0000","last_fullsized":"2024-02-26T10:44:39.557143+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":["8(0)","2(2)","3(1)","5(3)","9(0)"],"object_location_counts":[],"blocked_by":[1],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.451334+0000","last_change":"2024-02-26T10:42:41.624940+0000","last_active":"2024-02-26T10:43:38.451334+0000","last_peered":"2024-02-26T10:43:38.451334+0000","last_clean":"2024-02-26T10:43:38.451334+0000","last_became_active":"2024-02-26T10:42:41.624538+0000","last_became_peered":"2024-02-26T10:42:41.624538+0000","last_unstale":"2024-02-26T10:43:38.451334+0000","last_undegraded":"2024-02-26T10:43:38.451334+0000","last_fullsized":"2024-02-26T10:43:38.451334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:10.052806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307967+0000","last_change":"2024-02-26T10:42:42.174932+0000","last_active":"2024-02-26T10:44:39.307967+0000","last_peered":"2024-02-26T10:44:39.307967+0000","last_clean":"2024-02-26T10:44:39.307967+0000","last_became_active":"2024-02-26T10:42:42.174566+0000","last_became_peered":"2024-02-26T10:42:42.174566+0000","last_unstale":"2024-02-26T10:44:39.307967+0000","last_undegraded":"2024-02-26T10:44:39.307967+0000","last_fullsized":"2024-02-26T10:44:39.307967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:03.242487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.449804+0000","last_change":"2024-02-26T10:42:47.250023+0000","last_active":"2024-02-26T10:43:38.449804+0000","last_peered":"2024-02-26T10:43:38.449804+0000","last_clean":"2024-02-26T10:43:38.449804+0000","last_became_active":"2024-02-26T10:42:47.249571+0000","last_became_peered":"2024-02-26T10:42:47.249571+0000","last_unstale":"2024-02-26T10:43:38.449804+0000","last_undegraded":"2024-02-26T10:43:38.449804+0000","last_fullsized":"2024-02-26T10:43:38.449804+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:53.666184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.232236+0000","last_change":"2024-02-26T10:42:41.614403+0000","last_active":"2024-02-26T10:44:38.232236+0000","last_peered":"2024-02-26T10:44:38.232236+0000","last_clean":"2024-02-26T10:44:38.232236+0000","last_became_active":"2024-02-26T10:42:41.614057+0000","last_became_peered":"2024-02-26T10:42:41.614057+0000","last_unstale":"2024-02-26T10:44:38.232236+0000","last_undegraded":"2024-02-26T10:44:38.232236+0000","last_fullsized":"2024-02-26T10:44:38.232236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:16:54.739090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.451183+0000","last_change":"2024-02-26T10:42:42.187465+0000","last_active":"2024-02-26T10:43:38.451183+0000","last_peered":"2024-02-26T10:43:38.451183+0000","last_clean":"2024-02-26T10:43:38.451183+0000","last_became_active":"2024-02-26T10:42:42.182310+0000","last_became_peered":"2024-02-26T10:42:42.182310+0000","last_unstale":"2024-02-26T10:43:38.451183+0000","last_undegraded":"2024-02-26T10:43:38.451183+0000","last_fullsized":"2024-02-26T10:43:38.451183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:01:56.645330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.447866+0000","last_change":"2024-02-26T10:42:47.241751+0000","last_active":"2024-02-26T10:43:38.447866+0000","last_peered":"2024-02-26T10:43:38.447866+0000","last_clean":"2024-02-26T10:43:38.447866+0000","last_became_active":"2024-02-26T10:42:47.241271+0000","last_became_peered":"2024-02-26T10:42:47.241271+0000","last_unstale":"2024-02-26T10:43:38.447866+0000","last_undegraded":"2024-02-26T10:43:38.447866+0000","last_fullsized":"2024-02-26T10:43:38.447866+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:40:39.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234575+0000","last_change":"2024-02-26T10:42:41.607243+0000","last_active":"2024-02-26T10:44:39.234575+0000","last_peered":"2024-02-26T10:44:39.234575+0000","last_clean":"2024-02-26T10:44:39.234575+0000","last_became_active":"2024-02-26T10:42:41.606921+0000","last_became_peered":"2024-02-26T10:42:41.606921+0000","last_unstale":"2024-02-26T10:44:39.234575+0000","last_undegraded":"2024-02-26T10:44:39.234575+0000","last_fullsized":"2024-02-26T10:44:39.234575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:04.626391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235160+0000","last_change":"2024-02-26T10:42:42.178240+0000","last_active":"2024-02-26T10:44:39.235160+0000","last_peered":"2024-02-26T10:44:39.235160+0000","last_clean":"2024-02-26T10:44:39.235160+0000","last_became_active":"2024-02-26T10:42:42.177790+0000","last_became_peered":"2024-02-26T10:42:42.177790+0000","last_unstale":"2024-02-26T10:44:39.235160+0000","last_undegraded":"2024-02-26T10:44:39.235160+0000","last_fullsized":"2024-02-26T10:44:39.235160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:24.110406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463687+0000","last_change":"2024-02-26T10:42:47.233120+0000","last_active":"2024-02-26T10:44:39.463687+0000","last_peered":"2024-02-26T10:44:39.463687+0000","last_clean":"2024-02-26T10:44:39.463687+0000","last_became_active":"2024-02-26T10:42:47.232630+0000","last_became_peered":"2024-02-26T10:42:47.232630+0000","last_unstale":"2024-02-26T10:44:39.463687+0000","last_undegraded":"2024-02-26T10:44:39.463687+0000","last_fullsized":"2024-02-26T10:44:39.463687+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:05:03.023227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.477127+0000","last_change":"2024-02-26T10:43:38.477127+0000","last_active":"2024-02-26T10:43:38.477127+0000","last_peered":"2024-02-26T10:43:38.477127+0000","last_clean":"2024-02-26T10:43:38.477127+0000","last_became_active":"2024-02-26T10:43:38.460135+0000","last_became_peered":"2024-02-26T10:43:38.460135+0000","last_unstale":"2024-02-26T10:43:38.477127+0000","last_undegraded":"2024-02-26T10:43:38.477127+0000","last_fullsized":"2024-02-26T10:43:38.477127+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:13:16.521943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235554+0000","last_change":"2024-02-26T10:42:42.171821+0000","last_active":"2024-02-26T10:44:39.235554+0000","last_peered":"2024-02-26T10:44:39.235554+0000","last_clean":"2024-02-26T10:44:39.235554+0000","last_became_active":"2024-02-26T10:42:42.171523+0000","last_became_peered":"2024-02-26T10:42:42.171523+0000","last_unstale":"2024-02-26T10:44:39.235554+0000","last_undegraded":"2024-02-26T10:44:39.235554+0000","last_fullsized":"2024-02-26T10:44:39.235554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:00.877998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526525+0000","last_change":"2024-02-26T10:42:47.250072+0000","last_active":"2024-02-26T10:44:39.526525+0000","last_peered":"2024-02-26T10:44:39.526525+0000","last_clean":"2024-02-26T10:44:39.526525+0000","last_became_active":"2024-02-26T10:42:47.249570+0000","last_became_peered":"2024-02-26T10:42:47.249570+0000","last_unstale":"2024-02-26T10:44:39.526525+0000","last_undegraded":"2024-02-26T10:44:39.526525+0000","last_fullsized":"2024-02-26T10:44:39.526525+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:43:03.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.455033+0000","last_change":"2024-02-26T10:42:41.614686+0000","last_active":"2024-02-26T10:43:38.455033+0000","last_peered":"2024-02-26T10:43:38.455033+0000","last_clean":"2024-02-26T10:43:38.455033+0000","last_became_active":"2024-02-26T10:42:41.614019+0000","last_became_peered":"2024-02-26T10:42:41.614019+0000","last_unstale":"2024-02-26T10:43:38.455033+0000","last_undegraded":"2024-02-26T10:43:38.455033+0000","last_fullsized":"2024-02-26T10:43:38.455033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:13:28.127288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235338+0000","last_change":"2024-02-26T10:42:42.181774+0000","last_active":"2024-02-26T10:44:39.235338+0000","last_peered":"2024-02-26T10:44:39.235338+0000","last_clean":"2024-02-26T10:44:39.235338+0000","last_became_active":"2024-02-26T10:42:42.181412+0000","last_became_peered":"2024-02-26T10:42:42.181412+0000","last_unstale":"2024-02-26T10:44:39.235338+0000","last_undegraded":"2024-02-26T10:44:39.235338+0000","last_fullsized":"2024-02-26T10:44:39.235338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:09.159964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.454413+0000","last_change":"2024-02-26T10:42:47.228247+0000","last_active":"2024-02-26T10:43:38.454413+0000","last_peered":"2024-02-26T10:43:38.454413+0000","last_clean":"2024-02-26T10:43:38.454413+0000","last_became_active":"2024-02-26T10:42:47.227569+0000","last_became_peered":"2024-02-26T10:42:47.227569+0000","last_unstale":"2024-02-26T10:43:38.454413+0000","last_undegraded":"2024-02-26T10:43:38.454413+0000","last_fullsized":"2024-02-26T10:43:38.454413+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:15.001264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.236382+0000","last_change":"2024-02-26T10:42:41.626573+0000","last_active":"2024-02-26T10:44:38.236382+0000","last_peered":"2024-02-26T10:44:38.236382+0000","last_clean":"2024-02-26T10:44:38.236382+0000","last_became_active":"2024-02-26T10:42:41.624091+0000","last_became_peered":"2024-02-26T10:42:41.624091+0000","last_unstale":"2024-02-26T10:44:38.236382+0000","last_undegraded":"2024-02-26T10:44:38.236382+0000","last_fullsized":"2024-02-26T10:44:38.236382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:24:14.077234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235619+0000","last_change":"2024-02-26T10:42:42.166426+0000","last_active":"2024-02-26T10:44:39.235619+0000","last_peered":"2024-02-26T10:44:39.235619+0000","last_clean":"2024-02-26T10:44:39.235619+0000","last_became_active":"2024-02-26T10:42:42.166082+0000","last_became_peered":"2024-02-26T10:42:42.166082+0000","last_unstale":"2024-02-26T10:44:39.235619+0000","last_undegraded":"2024-02-26T10:44:39.235619+0000","last_fullsized":"2024-02-26T10:44:39.235619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:17.136568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308118+0000","last_change":"2024-02-26T10:42:42.168227+0000","last_active":"2024-02-26T10:44:39.308118+0000","last_peered":"2024-02-26T10:44:39.308118+0000","last_clean":"2024-02-26T10:44:39.308118+0000","last_became_active":"2024-02-26T10:42:42.167897+0000","last_became_peered":"2024-02-26T10:42:42.167897+0000","last_unstale":"2024-02-26T10:44:39.308118+0000","last_undegraded":"2024-02-26T10:44:39.308118+0000","last_fullsized":"2024-02-26T10:44:39.308118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:52:37.391022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.232564+0000","last_change":"2024-02-26T10:42:42.168640+0000","last_active":"2024-02-26T10:44:38.232564+0000","last_peered":"2024-02-26T10:44:38.232564+0000","last_clean":"2024-02-26T10:44:38.232564+0000","last_became_active":"2024-02-26T10:42:42.168237+0000","last_became_peered":"2024-02-26T10:42:42.168237+0000","last_unstale":"2024-02-26T10:44:38.232564+0000","last_undegraded":"2024-02-26T10:44:38.232564+0000","last_fullsized":"2024-02-26T10:44:38.232564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:14.853730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.232227+0000","last_change":"2024-02-26T10:42:42.192703+0000","last_active":"2024-02-26T10:44:38.232227+0000","last_peered":"2024-02-26T10:44:38.232227+0000","last_clean":"2024-02-26T10:44:38.232227+0000","last_became_active":"2024-02-26T10:42:42.191899+0000","last_became_peered":"2024-02-26T10:42:42.191899+0000","last_unstale":"2024-02-26T10:44:38.232227+0000","last_undegraded":"2024-02-26T10:44:38.232227+0000","last_fullsized":"2024-02-26T10:44:38.232227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:40.479523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527850+0000","last_change":"2024-02-26T10:42:42.188709+0000","last_active":"2024-02-26T10:44:39.527850+0000","last_peered":"2024-02-26T10:44:39.527850+0000","last_clean":"2024-02-26T10:44:39.527850+0000","last_became_active":"2024-02-26T10:42:42.188418+0000","last_became_peered":"2024-02-26T10:42:42.188418+0000","last_unstale":"2024-02-26T10:44:39.527850+0000","last_undegraded":"2024-02-26T10:44:39.527850+0000","last_fullsized":"2024-02-26T10:44:39.527850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:54:15.697746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.235348+0000","last_change":"2024-02-26T10:42:42.174221+0000","last_active":"2024-02-26T10:44:38.235348+0000","last_peered":"2024-02-26T10:44:38.235348+0000","last_clean":"2024-02-26T10:44:38.235348+0000","last_became_active":"2024-02-26T10:42:42.171308+0000","last_became_peered":"2024-02-26T10:42:42.171308+0000","last_unstale":"2024-02-26T10:44:38.235348+0000","last_undegraded":"2024-02-26T10:44:38.235348+0000","last_fullsized":"2024-02-26T10:44:38.235348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:46.765856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308405+0000","last_change":"2024-02-26T10:42:42.170749+0000","last_active":"2024-02-26T10:44:39.308405+0000","last_peered":"2024-02-26T10:44:39.308405+0000","last_clean":"2024-02-26T10:44:39.308405+0000","last_became_active":"2024-02-26T10:42:42.169966+0000","last_became_peered":"2024-02-26T10:42:42.169966+0000","last_unstale":"2024-02-26T10:44:39.308405+0000","last_undegraded":"2024-02-26T10:44:39.308405+0000","last_fullsized":"2024-02-26T10:44:39.308405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:02.633195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.232914+0000","last_change":"2024-02-26T10:42:42.166529+0000","last_active":"2024-02-26T10:44:38.232914+0000","last_peered":"2024-02-26T10:44:38.232914+0000","last_clean":"2024-02-26T10:44:38.232914+0000","last_became_active":"2024-02-26T10:42:42.166235+0000","last_became_peered":"2024-02-26T10:42:42.166235+0000","last_unstale":"2024-02-26T10:44:38.232914+0000","last_undegraded":"2024-02-26T10:44:38.232914+0000","last_fullsized":"2024-02-26T10:44:38.232914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:03.068328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:43:38.454198+0000","last_change":"2024-02-26T10:42:42.191312+0000","last_active":"2024-02-26T10:43:38.454198+0000","last_peered":"2024-02-26T10:43:38.454198+0000","last_clean":"2024-02-26T10:43:38.454198+0000","last_became_active":"2024-02-26T10:42:42.190964+0000","last_became_peered":"2024-02-26T10:42:42.190964+0000","last_unstale":"2024-02-26T10:43:38.454198+0000","last_undegraded":"2024-02-26T10:43:38.454198+0000","last_fullsized":"2024-02-26T10:43:38.454198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:24:04.697129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.886881+0000","last_change":"2024-02-26T10:42:42.182882+0000","last_active":"2024-02-26T10:44:36.886881+0000","last_peered":"2024-02-26T10:44:36.886881+0000","last_clean":"2024-02-26T10:44:36.886881+0000","last_became_active":"2024-02-26T10:42:42.182551+0000","last_became_peered":"2024-02-26T10:42:42.182551+0000","last_unstale":"2024-02-26T10:44:36.886881+0000","last_undegraded":"2024-02-26T10:44:36.886881+0000","last_fullsized":"2024-02-26T10:44:36.886881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:11:58.578832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T10:44:36.854348+0000","last_change":"2024-02-26T10:42:42.193353+0000","last_active":"2024-02-26T10:44:36.854348+0000","last_peered":"2024-02-26T10:44:36.854348+0000","last_clean":"2024-02-26T10:44:36.854348+0000","last_became_active":"2024-02-26T10:42:42.190935+0000","last_became_peered":"2024-02-26T10:42:42.190935+0000","last_unstale":"2024-02-26T10:44:36.854348+0000","last_undegraded":"2024-02-26T10:44:36.854348+0000","last_fullsized":"2024-02-26T10:44:36.854348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:03:14.319769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464135+0000","last_change":"2024-02-26T10:42:42.194529+0000","last_active":"2024-02-26T10:44:39.464135+0000","last_peered":"2024-02-26T10:44:39.464135+0000","last_clean":"2024-02-26T10:44:39.464135+0000","last_became_active":"2024-02-26T10:42:42.194212+0000","last_became_peered":"2024-02-26T10:42:42.194212+0000","last_unstale":"2024-02-26T10:44:39.464135+0000","last_undegraded":"2024-02-26T10:44:39.464135+0000","last_fullsized":"2024-02-26T10:44:39.464135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:36.876672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-26T10:44:38.234761+0000","last_change":"2024-02-26T10:42:42.178119+0000","last_active":"2024-02-26T10:44:38.234761+0000","last_peered":"2024-02-26T10:44:38.234761+0000","last_clean":"2024-02-26T10:44:38.234761+0000","last_became_active":"2024-02-26T10:42:42.177798+0000","last_became_peered":"2024-02-26T10:42:42.177798+0000","last_unstale":"2024-02-26T10:44:38.234761+0000","last_undegraded":"2024-02-26T10:44:38.234761+0000","last_fullsized":"2024-02-26T10:44:38.234761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:52:29.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308077+0000","last_change":"2024-02-26T10:42:42.167380+0000","last_active":"2024-02-26T10:44:39.308077+0000","last_peered":"2024-02-26T10:44:39.308077+0000","last_clean":"2024-02-26T10:44:39.308077+0000","last_became_active":"2024-02-26T10:42:42.167079+0000","last_became_peered":"2024-02-26T10:42:42.167079+0000","last_unstale":"2024-02-26T10:44:39.308077+0000","last_undegraded":"2024-02-26T10:44:39.308077+0000","last_fullsized":"2024-02-26T10:44:39.308077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:14:25.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":464,"ondisk_log_size":464,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250986,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67016,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":516,"num_read_kb":328,"num_write":171,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":63341,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":201048,"data_compressed":14031,"data_compressed_allocated":20480,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":523096,"data_compressed":23715,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1163264,"data_stored":1164170,"data_compressed":58886,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":527294,"data_compressed":23715,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037816,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":532387,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":1119168,"data_compressed":60039,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":508908,"data_compressed":20730,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":581158,"data_compressed":25547,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":529165,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":519052,"data_compressed":23715,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":509102,"data_compressed":20730,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":577546,"data_compressed":31097,"data_compressed_allocated":40960,"data_compressed_original":118784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1104758,"data_compressed":57054,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62874,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":64632,"data_compressed":10367,"data_compressed_allocated":12288,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64984,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:44:42.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-26T10:44:42.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:42 smithi060 ceph-mon[31539]: pgmap v387: 129 pgs: 1 activating, 1 peering, 127 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-02-26T10:44:42.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:42 smithi060 ceph-mon[31539]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:44:42.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:42 smithi113 ceph-mon[36126]: pgmap v387: 129 pgs: 1 activating, 1 peering, 127 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-02-26T10:44:42.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:42 smithi113 ceph-mon[36126]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:44:42.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:42 smithi100 ceph-mon[36051]: pgmap v387: 129 pgs: 1 activating, 1 peering, 127 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-02-26T10:44:42.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:42 smithi100 ceph-mon[36051]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:44:42.711 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:44.467 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:44 smithi060 ceph-mon[31539]: pgmap v388: 129 pgs: 1 activating, 1 peering, 127 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s 2024-02-26T10:44:44.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:44 smithi113 ceph-mon[36126]: pgmap v388: 129 pgs: 1 activating, 1 peering, 127 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s 2024-02-26T10:44:44.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:44 smithi100 ceph-mon[36051]: pgmap v388: 129 pgs: 1 activating, 1 peering, 127 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s 2024-02-26T10:44:45.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.010+0000 7f9644000700 1 -- 172.21.15.60:0/3206613525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 msgr2=0x7f963c108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:45.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.010+0000 7f9644000700 1 --2- 172.21.15.60:0/3206613525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 0x7f963c108980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f96300099c0 tx=0x7f9630009cd0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.010+0000 7f9644000700 1 -- 172.21.15.60:0/3206613525 shutdown_connections 2024-02-26T10:44:45.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.010+0000 7f9644000700 1 --2- 172.21.15.60:0/3206613525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f963c10d680 0x7f963c115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.010+0000 7f9644000700 1 --2- 172.21.15.60:0/3206613525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 0x7f963c108980 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.010+0000 7f9644000700 1 --2- 172.21.15.60:0/3206613525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963c108f50 0x7f963c10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.010+0000 7f9644000700 1 -- 172.21.15.60:0/3206613525 >> 172.21.15.60:0/3206613525 conn(0x7f963c075520 msgr2=0x7f963c077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:45.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.010+0000 7f9644000700 1 -- 172.21.15.60:0/3206613525 shutdown_connections 2024-02-26T10:44:45.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.010+0000 7f9644000700 1 -- 172.21.15.60:0/3206613525 wait complete. 2024-02-26T10:44:45.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.011+0000 7f9644000700 1 Processor -- start 2024-02-26T10:44:45.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.011+0000 7f9644000700 1 -- start start 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9644000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 0x7f963c1a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9644000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963c108f50 0x7f963c1a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9644000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f963c10d680 0x7f963c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9644000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c1186e0 con 0x7f963c1085a0 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9644000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c118560 con 0x7f963c108f50 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9644000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c118860 con 0x7f963c10d680 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9641600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 0x7f963c1a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9641600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 0x7f963c1a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33948/0 (socket says 172.21.15.60:33948) 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9641600700 1 -- 172.21.15.60:0/3533281419 learned_addr learned my addr 172.21.15.60:0/3533281419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:45.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9642000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f963c10d680 0x7f963c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:45.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.012+0000 7f9640c00700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963c108f50 0x7f963c1a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:45.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f9642000700 1 -- 172.21.15.60:0/3533281419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963c108f50 msgr2=0x7f963c1a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:45.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f9642000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963c108f50 0x7f963c1a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f9642000700 1 -- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 msgr2=0x7f963c1a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:45.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f9642000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 0x7f963c1a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f9642000700 1 -- 172.21.15.60:0/3533281419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c19f0c0 con 0x7f963c10d680 2024-02-26T10:44:45.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f9641600700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 0x7f963c1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:45.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f9642000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f963c10d680 0x7f963c19e960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f963400efc0 tx=0x7f963400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:45.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f963a000700 1 -- 172.21.15.60:0/3533281419 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96340092e0 con 0x7f963c10d680 2024-02-26T10:44:45.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c19f350 con 0x7f963c10d680 2024-02-26T10:44:45.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c1ab670 con 0x7f963c10d680 2024-02-26T10:44:45.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f963a000700 1 -- 172.21.15.60:0/3533281419 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9634004bb0 con 0x7f963c10d680 2024-02-26T10:44:45.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.013+0000 7f963a000700 1 -- 172.21.15.60:0/3533281419 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634004020 con 0x7f963c10d680 2024-02-26T10:44:45.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.014+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c068a10 con 0x7f963c10d680 2024-02-26T10:44:45.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.017+0000 7f963a000700 1 -- 172.21.15.60:0/3533281419 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f963401b030 con 0x7f963c10d680 2024-02-26T10:44:45.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.018+0000 7f963a000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f96240a1e10 0x7f96240a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:45.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.018+0000 7f963a000700 1 -- 172.21.15.60:0/3533281419 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f96340ce5c0 con 0x7f963c10d680 2024-02-26T10:44:45.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.018+0000 7f9641600700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f96240a1e10 0x7f96240a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:45.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.019+0000 7f9641600700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f96240a1e10 0x7f96240a42d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9630000c00 tx=0x7f9630005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:45.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.020+0000 7f963a000700 1 -- 172.21.15.60:0/3533281419 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9634022020 con 0x7f963c10d680 2024-02-26T10:44:45.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.241+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f963c062190 con 0x7f96240a1e10 2024-02-26T10:44:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.270+0000 7f963a000700 1 -- 172.21.15.60:0/3533281419 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340831 (secure 0 0 0) 0x7f963c062190 con 0x7f96240a1e10 2024-02-26T10:44:45.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:44:45.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.281+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f96240a1e10 msgr2=0x7f96240a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:45.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.281+0000 7f9644000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f96240a1e10 0x7f96240a42d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9630000c00 tx=0x7f9630005fb0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.281+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f963c10d680 msgr2=0x7f963c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.282+0000 7f9644000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f963c10d680 0x7f963c19e960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f963400efc0 tx=0x7f963400c3a0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.282+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 shutdown_connections 2024-02-26T10:44:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.282+0000 7f9644000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f96240a1e10 0x7f96240a42d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.282+0000 7f9644000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f963c10d680 0x7f963c19e960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.282+0000 7f9644000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f963c1085a0 0x7f963c1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.282+0000 7f9644000700 1 --2- 172.21.15.60:0/3533281419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963c108f50 0x7f963c1a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.282+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 >> 172.21.15.60:0/3533281419 conn(0x7f963c075520 msgr2=0x7f963c077800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.282+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 shutdown_connections 2024-02-26T10:44:45.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:45.282+0000 7f9644000700 1 -- 172.21.15.60:0/3533281419 wait complete. 2024-02-26T10:44:45.280 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T10:44:45.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:45 smithi113 ceph-mon[36126]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-26T10:44:45.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:45 smithi113 ceph-mon[36126]: Cluster is now healthy 2024-02-26T10:44:45.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:45 smithi060 ceph-mon[31539]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-26T10:44:45.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:45 smithi060 ceph-mon[31539]: Cluster is now healthy 2024-02-26T10:44:45.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:45 smithi100 ceph-mon[36051]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-26T10:44:45.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:45 smithi100 ceph-mon[36051]: Cluster is now healthy 2024-02-26T10:44:46.110 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-02-26T10:44:44.935722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657618,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":672,"num_read_kb":452,"num_write":354,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1244077,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370204,"kb_used_data":12672,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571620,"statfs":{"total":1151940427776,"available":1151561338880,"internally_reserved":0,"allocated":12976128,"data_stored":8327761,"data_compressed":440495,"data_compressed_allocated":593920,"data_compressed_original":2723840,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003345"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308540+0000","last_change":"2024-02-26T10:42:42.176634+0000","last_active":"2024-02-26T10:44:39.308540+0000","last_peered":"2024-02-26T10:44:39.308540+0000","last_clean":"2024-02-26T10:44:39.308540+0000","last_became_active":"2024-02-26T10:42:42.176386+0000","last_became_peered":"2024-02-26T10:42:42.176386+0000","last_unstale":"2024-02-26T10:44:39.308540+0000","last_undegraded":"2024-02-26T10:44:39.308540+0000","last_fullsized":"2024-02-26T10:44:39.308540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:45.005506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528307+0000","last_change":"2024-02-26T10:42:42.176769+0000","last_active":"2024-02-26T10:44:39.528307+0000","last_peered":"2024-02-26T10:44:39.528307+0000","last_clean":"2024-02-26T10:44:39.528307+0000","last_became_active":"2024-02-26T10:42:42.174868+0000","last_became_peered":"2024-02-26T10:42:42.174868+0000","last_unstale":"2024-02-26T10:44:39.528307+0000","last_undegraded":"2024-02-26T10:44:39.528307+0000","last_fullsized":"2024-02-26T10:44:39.528307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:16.202371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528036+0000","last_change":"2024-02-26T10:42:42.184410+0000","last_active":"2024-02-26T10:44:39.528036+0000","last_peered":"2024-02-26T10:44:39.528036+0000","last_clean":"2024-02-26T10:44:39.528036+0000","last_became_active":"2024-02-26T10:42:42.182986+0000","last_became_peered":"2024-02-26T10:42:42.182986+0000","last_unstale":"2024-02-26T10:44:39.528036+0000","last_undegraded":"2024-02-26T10:44:39.528036+0000","last_fullsized":"2024-02-26T10:44:39.528036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:28:45.445325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408778+0000","last_change":"2024-02-26T10:43:38.462828+0000","last_active":"2024-02-26T10:44:39.408778+0000","last_peered":"2024-02-26T10:44:39.408778+0000","last_clean":"2024-02-26T10:44:39.408778+0000","last_became_active":"2024-02-26T10:43:38.461014+0000","last_became_peered":"2024-02-26T10:43:38.461014+0000","last_unstale":"2024-02-26T10:44:39.408778+0000","last_undegraded":"2024-02-26T10:44:39.408778+0000","last_fullsized":"2024-02-26T10:44:39.408778+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:07.857849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234518+0000","last_change":"2024-02-26T10:42:42.192448+0000","last_active":"2024-02-26T10:44:39.234518+0000","last_peered":"2024-02-26T10:44:39.234518+0000","last_clean":"2024-02-26T10:44:39.234518+0000","last_became_active":"2024-02-26T10:42:42.192133+0000","last_became_peered":"2024-02-26T10:42:42.192133+0000","last_unstale":"2024-02-26T10:44:39.234518+0000","last_undegraded":"2024-02-26T10:44:39.234518+0000","last_fullsized":"2024-02-26T10:44:39.234518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:43:49.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.256925+0000","last_change":"2024-02-26T10:42:42.169999+0000","last_active":"2024-02-26T10:44:39.256925+0000","last_peered":"2024-02-26T10:44:39.256925+0000","last_clean":"2024-02-26T10:44:39.256925+0000","last_became_active":"2024-02-26T10:42:42.169678+0000","last_became_peered":"2024-02-26T10:42:42.169678+0000","last_unstale":"2024-02-26T10:44:39.256925+0000","last_undegraded":"2024-02-26T10:44:39.256925+0000","last_fullsized":"2024-02-26T10:44:39.256925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:54.435203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234911+0000","last_change":"2024-02-26T10:42:42.177538+0000","last_active":"2024-02-26T10:44:39.234911+0000","last_peered":"2024-02-26T10:44:39.234911+0000","last_clean":"2024-02-26T10:44:39.234911+0000","last_became_active":"2024-02-26T10:42:42.177204+0000","last_became_peered":"2024-02-26T10:42:42.177204+0000","last_unstale":"2024-02-26T10:44:39.234911+0000","last_undegraded":"2024-02-26T10:44:39.234911+0000","last_fullsized":"2024-02-26T10:44:39.234911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:09.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040311+0000","last_change":"2024-02-26T10:42:42.191342+0000","last_active":"2024-02-26T10:44:40.040311+0000","last_peered":"2024-02-26T10:44:40.040311+0000","last_clean":"2024-02-26T10:44:40.040311+0000","last_became_active":"2024-02-26T10:42:42.190990+0000","last_became_peered":"2024-02-26T10:42:42.190990+0000","last_unstale":"2024-02-26T10:44:40.040311+0000","last_undegraded":"2024-02-26T10:44:40.040311+0000","last_fullsized":"2024-02-26T10:44:40.040311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:20:45.190300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464568+0000","last_change":"2024-02-26T10:42:42.173175+0000","last_active":"2024-02-26T10:44:39.464568+0000","last_peered":"2024-02-26T10:44:39.464568+0000","last_clean":"2024-02-26T10:44:39.464568+0000","last_became_active":"2024-02-26T10:42:42.172828+0000","last_became_peered":"2024-02-26T10:42:42.172828+0000","last_unstale":"2024-02-26T10:44:39.464568+0000","last_undegraded":"2024-02-26T10:44:39.464568+0000","last_fullsized":"2024-02-26T10:44:39.464568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:17:34.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257447+0000","last_change":"2024-02-26T10:42:42.167267+0000","last_active":"2024-02-26T10:44:39.257447+0000","last_peered":"2024-02-26T10:44:39.257447+0000","last_clean":"2024-02-26T10:44:39.257447+0000","last_became_active":"2024-02-26T10:42:42.166948+0000","last_became_peered":"2024-02-26T10:42:42.166948+0000","last_unstale":"2024-02-26T10:44:39.257447+0000","last_undegraded":"2024-02-26T10:44:39.257447+0000","last_fullsized":"2024-02-26T10:44:39.257447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:52.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235556+0000","last_change":"2024-02-26T10:42:42.171299+0000","last_active":"2024-02-26T10:44:39.235556+0000","last_peered":"2024-02-26T10:44:39.235556+0000","last_clean":"2024-02-26T10:44:39.235556+0000","last_became_active":"2024-02-26T10:42:42.170616+0000","last_became_peered":"2024-02-26T10:42:42.170616+0000","last_unstale":"2024-02-26T10:44:39.235556+0000","last_undegraded":"2024-02-26T10:44:39.235556+0000","last_fullsized":"2024-02-26T10:44:39.235556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:50.320859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239290+0000","last_change":"2024-02-26T10:42:42.168408+0000","last_active":"2024-02-26T10:44:39.239290+0000","last_peered":"2024-02-26T10:44:39.239290+0000","last_clean":"2024-02-26T10:44:39.239290+0000","last_became_active":"2024-02-26T10:42:42.167150+0000","last_became_peered":"2024-02-26T10:42:42.167150+0000","last_unstale":"2024-02-26T10:44:39.239290+0000","last_undegraded":"2024-02-26T10:44:39.239290+0000","last_fullsized":"2024-02-26T10:44:39.239290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:34.763331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236310+0000","last_change":"2024-02-26T10:42:42.171720+0000","last_active":"2024-02-26T10:44:39.236310+0000","last_peered":"2024-02-26T10:44:39.236310+0000","last_clean":"2024-02-26T10:44:39.236310+0000","last_became_active":"2024-02-26T10:42:42.171309+0000","last_became_peered":"2024-02-26T10:42:42.171309+0000","last_unstale":"2024-02-26T10:44:39.236310+0000","last_undegraded":"2024-02-26T10:44:39.236310+0000","last_fullsized":"2024-02-26T10:44:39.236310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:20:52.151553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527224+0000","last_change":"2024-02-26T10:42:42.176857+0000","last_active":"2024-02-26T10:44:39.527224+0000","last_peered":"2024-02-26T10:44:39.527224+0000","last_clean":"2024-02-26T10:44:39.527224+0000","last_became_active":"2024-02-26T10:42:42.176385+0000","last_became_peered":"2024-02-26T10:42:42.176385+0000","last_unstale":"2024-02-26T10:44:39.527224+0000","last_undegraded":"2024-02-26T10:44:39.527224+0000","last_fullsized":"2024-02-26T10:44:39.527224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:45:51.852212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527244+0000","last_change":"2024-02-26T10:42:42.174682+0000","last_active":"2024-02-26T10:44:39.527244+0000","last_peered":"2024-02-26T10:44:39.527244+0000","last_clean":"2024-02-26T10:44:39.527244+0000","last_became_active":"2024-02-26T10:42:42.174328+0000","last_became_peered":"2024-02-26T10:42:42.174328+0000","last_unstale":"2024-02-26T10:44:39.527244+0000","last_undegraded":"2024-02-26T10:44:39.527244+0000","last_fullsized":"2024-02-26T10:44:39.527244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:47.161020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463881+0000","last_change":"2024-02-26T10:42:42.185289+0000","last_active":"2024-02-26T10:44:39.463881+0000","last_peered":"2024-02-26T10:44:39.463881+0000","last_clean":"2024-02-26T10:44:39.463881+0000","last_became_active":"2024-02-26T10:42:42.185000+0000","last_became_peered":"2024-02-26T10:42:42.185000+0000","last_unstale":"2024-02-26T10:44:39.463881+0000","last_undegraded":"2024-02-26T10:44:39.463881+0000","last_fullsized":"2024-02-26T10:44:39.463881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:48.552584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308437+0000","last_change":"2024-02-26T10:42:42.176558+0000","last_active":"2024-02-26T10:44:39.308437+0000","last_peered":"2024-02-26T10:44:39.308437+0000","last_clean":"2024-02-26T10:44:39.308437+0000","last_became_active":"2024-02-26T10:42:42.176334+0000","last_became_peered":"2024-02-26T10:42:42.176334+0000","last_unstale":"2024-02-26T10:44:39.308437+0000","last_undegraded":"2024-02-26T10:44:39.308437+0000","last_fullsized":"2024-02-26T10:44:39.308437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:59.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409569+0000","last_change":"2024-02-26T10:42:42.168534+0000","last_active":"2024-02-26T10:44:39.409569+0000","last_peered":"2024-02-26T10:44:39.409569+0000","last_clean":"2024-02-26T10:44:39.409569+0000","last_became_active":"2024-02-26T10:42:42.168179+0000","last_became_peered":"2024-02-26T10:42:42.168179+0000","last_unstale":"2024-02-26T10:44:39.409569+0000","last_undegraded":"2024-02-26T10:44:39.409569+0000","last_fullsized":"2024-02-26T10:44:39.409569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:09.625969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238815+0000","last_change":"2024-02-26T10:42:42.192446+0000","last_active":"2024-02-26T10:44:39.238815+0000","last_peered":"2024-02-26T10:44:39.238815+0000","last_clean":"2024-02-26T10:44:39.238815+0000","last_became_active":"2024-02-26T10:42:42.191863+0000","last_became_peered":"2024-02-26T10:42:42.191863+0000","last_unstale":"2024-02-26T10:44:39.238815+0000","last_undegraded":"2024-02-26T10:44:39.238815+0000","last_fullsized":"2024-02-26T10:44:39.238815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:30:19.477529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238434+0000","last_change":"2024-02-26T10:42:47.257605+0000","last_active":"2024-02-26T10:44:39.238434+0000","last_peered":"2024-02-26T10:44:39.238434+0000","last_clean":"2024-02-26T10:44:39.238434+0000","last_became_active":"2024-02-26T10:42:47.257206+0000","last_became_peered":"2024-02-26T10:42:47.257206+0000","last_unstale":"2024-02-26T10:44:39.238434+0000","last_undegraded":"2024-02-26T10:44:39.238434+0000","last_fullsized":"2024-02-26T10:44:39.238434+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:26:17.681862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409781+0000","last_change":"2024-02-26T10:42:41.613087+0000","last_active":"2024-02-26T10:44:39.409781+0000","last_peered":"2024-02-26T10:44:39.409781+0000","last_clean":"2024-02-26T10:44:39.409781+0000","last_became_active":"2024-02-26T10:42:41.612664+0000","last_became_peered":"2024-02-26T10:42:41.612664+0000","last_unstale":"2024-02-26T10:44:39.409781+0000","last_undegraded":"2024-02-26T10:44:39.409781+0000","last_fullsized":"2024-02-26T10:44:39.409781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:00.352166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042024+0000","last_change":"2024-02-26T10:42:42.191438+0000","last_active":"2024-02-26T10:44:40.042024+0000","last_peered":"2024-02-26T10:44:40.042024+0000","last_clean":"2024-02-26T10:44:40.042024+0000","last_became_active":"2024-02-26T10:42:42.191095+0000","last_became_peered":"2024-02-26T10:42:42.191095+0000","last_unstale":"2024-02-26T10:44:40.042024+0000","last_undegraded":"2024-02-26T10:44:40.042024+0000","last_fullsized":"2024-02-26T10:44:40.042024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:20.918715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238903+0000","last_change":"2024-02-26T10:42:47.247503+0000","last_active":"2024-02-26T10:44:39.238903+0000","last_peered":"2024-02-26T10:44:39.238903+0000","last_clean":"2024-02-26T10:44:39.238903+0000","last_became_active":"2024-02-26T10:42:47.247121+0000","last_became_peered":"2024-02-26T10:42:47.247121+0000","last_unstale":"2024-02-26T10:44:39.238903+0000","last_undegraded":"2024-02-26T10:44:39.238903+0000","last_fullsized":"2024-02-26T10:44:39.238903+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:01:38.733131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042711+0000","last_change":"2024-02-26T10:42:41.626227+0000","last_active":"2024-02-26T10:44:40.042711+0000","last_peered":"2024-02-26T10:44:40.042711+0000","last_clean":"2024-02-26T10:44:40.042711+0000","last_became_active":"2024-02-26T10:42:41.625760+0000","last_became_peered":"2024-02-26T10:42:41.625760+0000","last_unstale":"2024-02-26T10:44:40.042711+0000","last_undegraded":"2024-02-26T10:44:40.042711+0000","last_fullsized":"2024-02-26T10:44:40.042711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:04.600151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527980+0000","last_change":"2024-02-26T10:42:42.182808+0000","last_active":"2024-02-26T10:44:39.527980+0000","last_peered":"2024-02-26T10:44:39.527980+0000","last_clean":"2024-02-26T10:44:39.527980+0000","last_became_active":"2024-02-26T10:42:42.182485+0000","last_became_peered":"2024-02-26T10:42:42.182485+0000","last_unstale":"2024-02-26T10:44:39.527980+0000","last_undegraded":"2024-02-26T10:44:39.527980+0000","last_fullsized":"2024-02-26T10:44:39.527980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:45.205969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239097+0000","last_change":"2024-02-26T10:42:47.250694+0000","last_active":"2024-02-26T10:44:39.239097+0000","last_peered":"2024-02-26T10:44:39.239097+0000","last_clean":"2024-02-26T10:44:39.239097+0000","last_became_active":"2024-02-26T10:42:47.249566+0000","last_became_peered":"2024-02-26T10:42:47.249566+0000","last_unstale":"2024-02-26T10:44:39.239097+0000","last_undegraded":"2024-02-26T10:44:39.239097+0000","last_fullsized":"2024-02-26T10:44:39.239097+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:44.016413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235006+0000","last_change":"2024-02-26T10:42:41.606849+0000","last_active":"2024-02-26T10:44:39.235006+0000","last_peered":"2024-02-26T10:44:39.235006+0000","last_clean":"2024-02-26T10:44:39.235006+0000","last_became_active":"2024-02-26T10:42:41.606494+0000","last_became_peered":"2024-02-26T10:42:41.606494+0000","last_unstale":"2024-02-26T10:44:39.235006+0000","last_undegraded":"2024-02-26T10:44:39.235006+0000","last_fullsized":"2024-02-26T10:44:39.235006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:36.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464226+0000","last_change":"2024-02-26T10:42:42.175926+0000","last_active":"2024-02-26T10:44:39.464226+0000","last_peered":"2024-02-26T10:44:39.464226+0000","last_clean":"2024-02-26T10:44:39.464226+0000","last_became_active":"2024-02-26T10:42:42.175614+0000","last_became_peered":"2024-02-26T10:42:42.175614+0000","last_unstale":"2024-02-26T10:44:39.464226+0000","last_undegraded":"2024-02-26T10:44:39.464226+0000","last_fullsized":"2024-02-26T10:44:39.464226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:20:20.829849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.043633+0000","last_change":"2024-02-26T10:44:40.043633+0000","last_active":"2024-02-26T10:44:40.043633+0000","last_peered":"2024-02-26T10:44:40.043633+0000","last_clean":"2024-02-26T10:44:40.043633+0000","last_became_active":"2024-02-26T10:44:40.043209+0000","last_became_peered":"2024-02-26T10:44:40.043209+0000","last_unstale":"2024-02-26T10:44:40.043633+0000","last_undegraded":"2024-02-26T10:44:40.043633+0000","last_fullsized":"2024-02-26T10:44:40.043633+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:43.466990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234833+0000","last_change":"2024-02-26T10:42:41.606407+0000","last_active":"2024-02-26T10:44:39.234833+0000","last_peered":"2024-02-26T10:44:39.234833+0000","last_clean":"2024-02-26T10:44:39.234833+0000","last_became_active":"2024-02-26T10:42:41.606094+0000","last_became_peered":"2024-02-26T10:42:41.606094+0000","last_unstale":"2024-02-26T10:44:39.234833+0000","last_undegraded":"2024-02-26T10:44:39.234833+0000","last_fullsized":"2024-02-26T10:44:39.234833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:53.673017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557384+0000","last_change":"2024-02-26T10:42:42.172967+0000","last_active":"2024-02-26T10:44:39.557384+0000","last_peered":"2024-02-26T10:44:39.557384+0000","last_clean":"2024-02-26T10:44:39.557384+0000","last_became_active":"2024-02-26T10:42:42.172675+0000","last_became_peered":"2024-02-26T10:42:42.172675+0000","last_unstale":"2024-02-26T10:44:39.557384+0000","last_undegraded":"2024-02-26T10:44:39.557384+0000","last_fullsized":"2024-02-26T10:44:39.557384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238485+0000","last_change":"2024-02-26T10:42:47.229375+0000","last_active":"2024-02-26T10:44:39.238485+0000","last_peered":"2024-02-26T10:44:39.238485+0000","last_clean":"2024-02-26T10:44:39.238485+0000","last_became_active":"2024-02-26T10:42:47.228874+0000","last_became_peered":"2024-02-26T10:42:47.228874+0000","last_unstale":"2024-02-26T10:44:39.238485+0000","last_undegraded":"2024-02-26T10:44:39.238485+0000","last_fullsized":"2024-02-26T10:44:39.238485+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:02.188519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":127,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:41.173383+0000","last_change":"2024-02-26T10:42:41.602783+0000","last_active":"2024-02-26T10:44:41.173383+0000","last_peered":"2024-02-26T10:44:41.173383+0000","last_clean":"2024-02-26T10:44:41.173383+0000","last_became_active":"2024-02-26T10:42:41.602491+0000","last_became_peered":"2024-02-26T10:42:41.602491+0000","last_unstale":"2024-02-26T10:44:41.173383+0000","last_undegraded":"2024-02-26T10:44:41.173383+0000","last_fullsized":"2024-02-26T10:44:41.173383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:23:33.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234438+0000","last_change":"2024-02-26T10:43:38.461429+0000","last_active":"2024-02-26T10:44:39.234438+0000","last_peered":"2024-02-26T10:44:39.234438+0000","last_clean":"2024-02-26T10:44:39.234438+0000","last_became_active":"2024-02-26T10:43:38.461019+0000","last_became_peered":"2024-02-26T10:43:38.461019+0000","last_unstale":"2024-02-26T10:44:39.234438+0000","last_undegraded":"2024-02-26T10:44:39.234438+0000","last_fullsized":"2024-02-26T10:44:39.234438+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:20.911179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.558649+0000","last_change":"2024-02-26T10:44:39.558649+0000","last_active":"2024-02-26T10:44:39.558649+0000","last_peered":"2024-02-26T10:44:39.558649+0000","last_clean":"2024-02-26T10:44:39.558649+0000","last_became_active":"2024-02-26T10:44:39.558013+0000","last_became_peered":"2024-02-26T10:44:39.558013+0000","last_unstale":"2024-02-26T10:44:39.558649+0000","last_undegraded":"2024-02-26T10:44:39.558649+0000","last_fullsized":"2024-02-26T10:44:39.558649+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:45.879391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.256854+0000","last_change":"2024-02-26T10:42:41.602887+0000","last_active":"2024-02-26T10:44:39.256854+0000","last_peered":"2024-02-26T10:44:39.256854+0000","last_clean":"2024-02-26T10:44:39.256854+0000","last_became_active":"2024-02-26T10:42:41.602564+0000","last_became_peered":"2024-02-26T10:42:41.602564+0000","last_unstale":"2024-02-26T10:44:39.256854+0000","last_undegraded":"2024-02-26T10:44:39.256854+0000","last_fullsized":"2024-02-26T10:44:39.256854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:22:41.711283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556892+0000","last_change":"2024-02-26T10:42:42.192700+0000","last_active":"2024-02-26T10:44:39.556892+0000","last_peered":"2024-02-26T10:44:39.556892+0000","last_clean":"2024-02-26T10:44:39.556892+0000","last_became_active":"2024-02-26T10:42:42.192387+0000","last_became_peered":"2024-02-26T10:42:42.192387+0000","last_unstale":"2024-02-26T10:44:39.556892+0000","last_undegraded":"2024-02-26T10:44:39.556892+0000","last_fullsized":"2024-02-26T10:44:39.556892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:21:15.104872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307788+0000","last_change":"2024-02-26T10:42:47.247000+0000","last_active":"2024-02-26T10:44:39.307788+0000","last_peered":"2024-02-26T10:44:39.307788+0000","last_clean":"2024-02-26T10:44:39.307788+0000","last_became_active":"2024-02-26T10:42:47.246520+0000","last_became_peered":"2024-02-26T10:42:47.246520+0000","last_unstale":"2024-02-26T10:44:39.307788+0000","last_undegraded":"2024-02-26T10:44:39.307788+0000","last_fullsized":"2024-02-26T10:44:39.307788+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:29.977858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235731+0000","last_change":"2024-02-26T10:42:41.609460+0000","last_active":"2024-02-26T10:44:39.235731+0000","last_peered":"2024-02-26T10:44:39.235731+0000","last_clean":"2024-02-26T10:44:39.235731+0000","last_became_active":"2024-02-26T10:42:41.608981+0000","last_became_peered":"2024-02-26T10:42:41.608981+0000","last_unstale":"2024-02-26T10:44:39.235731+0000","last_undegraded":"2024-02-26T10:44:39.235731+0000","last_fullsized":"2024-02-26T10:44:39.235731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:04.599910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040305+0000","last_change":"2024-02-26T10:42:42.185359+0000","last_active":"2024-02-26T10:44:40.040305+0000","last_peered":"2024-02-26T10:44:40.040305+0000","last_clean":"2024-02-26T10:44:40.040305+0000","last_became_active":"2024-02-26T10:42:42.184221+0000","last_became_peered":"2024-02-26T10:42:42.184221+0000","last_unstale":"2024-02-26T10:44:40.040305+0000","last_undegraded":"2024-02-26T10:44:40.040305+0000","last_fullsized":"2024-02-26T10:44:40.040305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:53.907290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526520+0000","last_change":"2024-02-26T10:42:47.241700+0000","last_active":"2024-02-26T10:44:39.526520+0000","last_peered":"2024-02-26T10:44:39.526520+0000","last_clean":"2024-02-26T10:44:39.526520+0000","last_became_active":"2024-02-26T10:42:47.240667+0000","last_became_peered":"2024-02-26T10:42:47.240667+0000","last_unstale":"2024-02-26T10:44:39.526520+0000","last_undegraded":"2024-02-26T10:44:39.526520+0000","last_fullsized":"2024-02-26T10:44:39.526520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:19.071244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556499+0000","last_change":"2024-02-26T10:42:41.604519+0000","last_active":"2024-02-26T10:44:39.556499+0000","last_peered":"2024-02-26T10:44:39.556499+0000","last_clean":"2024-02-26T10:44:39.556499+0000","last_became_active":"2024-02-26T10:42:41.603596+0000","last_became_peered":"2024-02-26T10:42:41.603596+0000","last_unstale":"2024-02-26T10:44:39.556499+0000","last_undegraded":"2024-02-26T10:44:39.556499+0000","last_fullsized":"2024-02-26T10:44:39.556499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:39.764730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408032+0000","last_change":"2024-02-26T10:42:42.187532+0000","last_active":"2024-02-26T10:44:39.408032+0000","last_peered":"2024-02-26T10:44:39.408032+0000","last_clean":"2024-02-26T10:44:39.408032+0000","last_became_active":"2024-02-26T10:42:42.185635+0000","last_became_peered":"2024-02-26T10:42:42.185635+0000","last_unstale":"2024-02-26T10:44:39.408032+0000","last_undegraded":"2024-02-26T10:44:39.408032+0000","last_fullsized":"2024-02-26T10:44:39.408032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:16.894747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041306+0000","last_change":"2024-02-26T10:42:47.233507+0000","last_active":"2024-02-26T10:44:40.041306+0000","last_peered":"2024-02-26T10:44:40.041306+0000","last_clean":"2024-02-26T10:44:40.041306+0000","last_became_active":"2024-02-26T10:42:47.229589+0000","last_became_peered":"2024-02-26T10:42:47.229589+0000","last_unstale":"2024-02-26T10:44:40.041306+0000","last_undegraded":"2024-02-26T10:44:40.041306+0000","last_fullsized":"2024-02-26T10:44:40.041306+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:02:42.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":123,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:41.172795+0000","last_change":"2024-02-26T10:42:41.613952+0000","last_active":"2024-02-26T10:44:41.172795+0000","last_peered":"2024-02-26T10:44:41.172795+0000","last_clean":"2024-02-26T10:44:41.172795+0000","last_became_active":"2024-02-26T10:42:41.613656+0000","last_became_peered":"2024-02-26T10:42:41.613656+0000","last_unstale":"2024-02-26T10:44:41.172795+0000","last_undegraded":"2024-02-26T10:44:41.172795+0000","last_fullsized":"2024-02-26T10:44:41.172795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:14.090821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464548+0000","last_change":"2024-02-26T10:42:42.178357+0000","last_active":"2024-02-26T10:44:39.464548+0000","last_peered":"2024-02-26T10:44:39.464548+0000","last_clean":"2024-02-26T10:44:39.464548+0000","last_became_active":"2024-02-26T10:42:42.178051+0000","last_became_peered":"2024-02-26T10:42:42.178051+0000","last_unstale":"2024-02-26T10:44:39.464548+0000","last_undegraded":"2024-02-26T10:44:39.464548+0000","last_fullsized":"2024-02-26T10:44:39.464548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:13:01.326122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406022+0000","last_change":"2024-02-26T10:42:47.246231+0000","last_active":"2024-02-26T10:44:39.406022+0000","last_peered":"2024-02-26T10:44:39.406022+0000","last_clean":"2024-02-26T10:44:39.406022+0000","last_became_active":"2024-02-26T10:42:47.245739+0000","last_became_peered":"2024-02-26T10:42:47.245739+0000","last_unstale":"2024-02-26T10:44:39.406022+0000","last_undegraded":"2024-02-26T10:44:39.406022+0000","last_fullsized":"2024-02-26T10:44:39.406022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:30.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235472+0000","last_change":"2024-02-26T10:42:41.603126+0000","last_active":"2024-02-26T10:44:39.235472+0000","last_peered":"2024-02-26T10:44:39.235472+0000","last_clean":"2024-02-26T10:44:39.235472+0000","last_became_active":"2024-02-26T10:42:41.602790+0000","last_became_peered":"2024-02-26T10:42:41.602790+0000","last_unstale":"2024-02-26T10:44:39.235472+0000","last_undegraded":"2024-02-26T10:44:39.235472+0000","last_fullsized":"2024-02-26T10:44:39.235472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:31:58.908209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528369+0000","last_change":"2024-02-26T10:43:38.459527+0000","last_active":"2024-02-26T10:44:39.528369+0000","last_peered":"2024-02-26T10:44:39.528369+0000","last_clean":"2024-02-26T10:44:39.528369+0000","last_became_active":"2024-02-26T10:43:38.459150+0000","last_became_peered":"2024-02-26T10:43:38.459150+0000","last_unstale":"2024-02-26T10:44:39.528369+0000","last_undegraded":"2024-02-26T10:44:39.528369+0000","last_fullsized":"2024-02-26T10:44:39.528369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:21:54.028328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239232+0000","last_change":"2024-02-26T10:42:47.250075+0000","last_active":"2024-02-26T10:44:39.239232+0000","last_peered":"2024-02-26T10:44:39.239232+0000","last_clean":"2024-02-26T10:44:39.239232+0000","last_became_active":"2024-02-26T10:42:47.249490+0000","last_became_peered":"2024-02-26T10:42:47.249490+0000","last_unstale":"2024-02-26T10:44:39.239232+0000","last_undegraded":"2024-02-26T10:44:39.239232+0000","last_fullsized":"2024-02-26T10:44:39.239232+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:42.659282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557176+0000","last_change":"2024-02-26T10:42:41.615314+0000","last_active":"2024-02-26T10:44:39.557176+0000","last_peered":"2024-02-26T10:44:39.557176+0000","last_clean":"2024-02-26T10:44:39.557176+0000","last_became_active":"2024-02-26T10:42:41.615091+0000","last_became_peered":"2024-02-26T10:42:41.615091+0000","last_unstale":"2024-02-26T10:44:39.557176+0000","last_undegraded":"2024-02-26T10:44:39.557176+0000","last_fullsized":"2024-02-26T10:44:39.557176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:41:52.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464386+0000","last_change":"2024-02-26T10:42:42.176609+0000","last_active":"2024-02-26T10:44:39.464386+0000","last_peered":"2024-02-26T10:44:39.464386+0000","last_clean":"2024-02-26T10:44:39.464386+0000","last_became_active":"2024-02-26T10:42:42.176311+0000","last_became_peered":"2024-02-26T10:42:42.176311+0000","last_unstale":"2024-02-26T10:44:39.464386+0000","last_undegraded":"2024-02-26T10:44:39.464386+0000","last_fullsized":"2024-02-26T10:44:39.464386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:25:38.415311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406108+0000","last_change":"2024-02-26T10:42:47.250255+0000","last_active":"2024-02-26T10:44:39.406108+0000","last_peered":"2024-02-26T10:44:39.406108+0000","last_clean":"2024-02-26T10:44:39.406108+0000","last_became_active":"2024-02-26T10:42:47.249465+0000","last_became_peered":"2024-02-26T10:42:47.249465+0000","last_unstale":"2024-02-26T10:44:39.406108+0000","last_undegraded":"2024-02-26T10:44:39.406108+0000","last_fullsized":"2024-02-26T10:44:39.406108+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:23.173863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":269,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:42.561354+0000","last_change":"2024-02-26T10:43:38.460725+0000","last_active":"2024-02-26T10:44:42.561354+0000","last_peered":"2024-02-26T10:44:42.561354+0000","last_clean":"2024-02-26T10:44:42.561354+0000","last_became_active":"2024-02-26T10:43:38.451900+0000","last_became_peered":"2024-02-26T10:43:38.451900+0000","last_unstale":"2024-02-26T10:44:42.561354+0000","last_undegraded":"2024-02-26T10:44:42.561354+0000","last_fullsized":"2024-02-26T10:44:42.561354+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:35:09.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":151,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":62625,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234883+0000","last_change":"2024-02-26T10:42:42.168174+0000","last_active":"2024-02-26T10:44:39.234883+0000","last_peered":"2024-02-26T10:44:39.234883+0000","last_clean":"2024-02-26T10:44:39.234883+0000","last_became_active":"2024-02-26T10:42:42.167840+0000","last_became_peered":"2024-02-26T10:42:42.167840+0000","last_unstale":"2024-02-26T10:44:39.234883+0000","last_undegraded":"2024-02-26T10:44:39.234883+0000","last_fullsized":"2024-02-26T10:44:39.234883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:43.079139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557623+0000","last_change":"2024-02-26T10:42:47.230048+0000","last_active":"2024-02-26T10:44:39.557623+0000","last_peered":"2024-02-26T10:44:39.557623+0000","last_clean":"2024-02-26T10:44:39.557623+0000","last_became_active":"2024-02-26T10:42:47.229396+0000","last_became_peered":"2024-02-26T10:42:47.229396+0000","last_unstale":"2024-02-26T10:44:39.557623+0000","last_undegraded":"2024-02-26T10:44:39.557623+0000","last_fullsized":"2024-02-26T10:44:39.557623+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:57.804297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238456+0000","last_change":"2024-02-26T10:42:41.614413+0000","last_active":"2024-02-26T10:44:39.238456+0000","last_peered":"2024-02-26T10:44:39.238456+0000","last_clean":"2024-02-26T10:44:39.238456+0000","last_became_active":"2024-02-26T10:42:41.614065+0000","last_became_peered":"2024-02-26T10:42:41.614065+0000","last_unstale":"2024-02-26T10:44:39.238456+0000","last_undegraded":"2024-02-26T10:44:39.238456+0000","last_fullsized":"2024-02-26T10:44:39.238456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:50:20.552340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235910+0000","last_change":"2024-02-26T10:42:42.188140+0000","last_active":"2024-02-26T10:44:39.235910+0000","last_peered":"2024-02-26T10:44:39.235910+0000","last_clean":"2024-02-26T10:44:39.235910+0000","last_became_active":"2024-02-26T10:42:42.187055+0000","last_became_peered":"2024-02-26T10:42:42.187055+0000","last_unstale":"2024-02-26T10:44:39.235910+0000","last_undegraded":"2024-02-26T10:44:39.235910+0000","last_fullsized":"2024-02-26T10:44:39.235910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:49:43.568562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308052+0000","last_change":"2024-02-26T10:43:38.450681+0000","last_active":"2024-02-26T10:44:39.308052+0000","last_peered":"2024-02-26T10:44:39.308052+0000","last_clean":"2024-02-26T10:44:39.308052+0000","last_became_active":"2024-02-26T10:43:38.450120+0000","last_became_peered":"2024-02-26T10:43:38.450120+0000","last_unstale":"2024-02-26T10:44:39.308052+0000","last_undegraded":"2024-02-26T10:44:39.308052+0000","last_fullsized":"2024-02-26T10:44:39.308052+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:10.983700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556839+0000","last_change":"2024-02-26T10:42:41.612337+0000","last_active":"2024-02-26T10:44:39.556839+0000","last_peered":"2024-02-26T10:44:39.556839+0000","last_clean":"2024-02-26T10:44:39.556839+0000","last_became_active":"2024-02-26T10:42:41.611920+0000","last_became_peered":"2024-02-26T10:42:41.611920+0000","last_unstale":"2024-02-26T10:44:39.556839+0000","last_undegraded":"2024-02-26T10:44:39.556839+0000","last_fullsized":"2024-02-26T10:44:39.556839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:29:13.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235264+0000","last_change":"2024-02-26T10:43:38.456919+0000","last_active":"2024-02-26T10:44:39.235264+0000","last_peered":"2024-02-26T10:44:39.235264+0000","last_clean":"2024-02-26T10:44:39.235264+0000","last_became_active":"2024-02-26T10:43:38.456461+0000","last_became_peered":"2024-02-26T10:43:38.456461+0000","last_unstale":"2024-02-26T10:44:39.235264+0000","last_undegraded":"2024-02-26T10:44:39.235264+0000","last_fullsized":"2024-02-26T10:44:39.235264+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:44.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406978+0000","last_change":"2024-02-26T10:42:47.254585+0000","last_active":"2024-02-26T10:44:39.406978+0000","last_peered":"2024-02-26T10:44:39.406978+0000","last_clean":"2024-02-26T10:44:39.406978+0000","last_became_active":"2024-02-26T10:42:47.253870+0000","last_became_peered":"2024-02-26T10:42:47.253870+0000","last_unstale":"2024-02-26T10:44:39.406978+0000","last_undegraded":"2024-02-26T10:44:39.406978+0000","last_fullsized":"2024-02-26T10:44:39.406978+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:49.276906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":129,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:41.205615+0000","last_change":"2024-02-26T10:42:41.600901+0000","last_active":"2024-02-26T10:44:41.205615+0000","last_peered":"2024-02-26T10:44:41.205615+0000","last_clean":"2024-02-26T10:44:41.205615+0000","last_became_active":"2024-02-26T10:42:41.600018+0000","last_became_peered":"2024-02-26T10:42:41.600018+0000","last_unstale":"2024-02-26T10:44:41.205615+0000","last_undegraded":"2024-02-26T10:44:41.205615+0000","last_fullsized":"2024-02-26T10:44:41.205615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:07.876694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234957+0000","last_change":"2024-02-26T10:42:42.168702+0000","last_active":"2024-02-26T10:44:39.234957+0000","last_peered":"2024-02-26T10:44:39.234957+0000","last_clean":"2024-02-26T10:44:39.234957+0000","last_became_active":"2024-02-26T10:42:42.168420+0000","last_became_peered":"2024-02-26T10:42:42.168420+0000","last_unstale":"2024-02-26T10:44:39.234957+0000","last_undegraded":"2024-02-26T10:44:39.234957+0000","last_fullsized":"2024-02-26T10:44:39.234957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:35.111699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236619+0000","last_change":"2024-02-26T10:42:47.250714+0000","last_active":"2024-02-26T10:44:39.236619+0000","last_peered":"2024-02-26T10:44:39.236619+0000","last_clean":"2024-02-26T10:44:39.236619+0000","last_became_active":"2024-02-26T10:42:47.249506+0000","last_became_peered":"2024-02-26T10:42:47.249506+0000","last_unstale":"2024-02-26T10:44:39.236619+0000","last_undegraded":"2024-02-26T10:44:39.236619+0000","last_fullsized":"2024-02-26T10:44:39.236619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:05.058052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307933+0000","last_change":"2024-02-26T10:43:38.472808+0000","last_active":"2024-02-26T10:44:39.307933+0000","last_peered":"2024-02-26T10:44:39.307933+0000","last_clean":"2024-02-26T10:44:39.307933+0000","last_became_active":"2024-02-26T10:43:38.456509+0000","last_became_peered":"2024-02-26T10:43:38.456509+0000","last_unstale":"2024-02-26T10:44:39.307933+0000","last_undegraded":"2024-02-26T10:44:39.307933+0000","last_fullsized":"2024-02-26T10:44:39.307933+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:03:06.127386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556356+0000","last_change":"2024-02-26T10:42:42.172845+0000","last_active":"2024-02-26T10:44:39.556356+0000","last_peered":"2024-02-26T10:44:39.556356+0000","last_clean":"2024-02-26T10:44:39.556356+0000","last_became_active":"2024-02-26T10:42:42.171263+0000","last_became_peered":"2024-02-26T10:42:42.171263+0000","last_unstale":"2024-02-26T10:44:39.556356+0000","last_undegraded":"2024-02-26T10:44:39.556356+0000","last_fullsized":"2024-02-26T10:44:39.556356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:21.439474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556346+0000","last_change":"2024-02-26T10:42:47.236434+0000","last_active":"2024-02-26T10:44:39.556346+0000","last_peered":"2024-02-26T10:44:39.556346+0000","last_clean":"2024-02-26T10:44:39.556346+0000","last_became_active":"2024-02-26T10:42:47.236040+0000","last_became_peered":"2024-02-26T10:42:47.236040+0000","last_unstale":"2024-02-26T10:44:39.556346+0000","last_undegraded":"2024-02-26T10:44:39.556346+0000","last_fullsized":"2024-02-26T10:44:39.556346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:20:23.230348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235189+0000","last_change":"2024-02-26T10:42:41.604208+0000","last_active":"2024-02-26T10:44:39.235189+0000","last_peered":"2024-02-26T10:44:39.235189+0000","last_clean":"2024-02-26T10:44:39.235189+0000","last_became_active":"2024-02-26T10:42:41.603894+0000","last_became_peered":"2024-02-26T10:42:41.603894+0000","last_unstale":"2024-02-26T10:44:39.235189+0000","last_undegraded":"2024-02-26T10:44:39.235189+0000","last_fullsized":"2024-02-26T10:44:39.235189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:57:13.775868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308356+0000","last_change":"2024-02-26T10:42:42.169395+0000","last_active":"2024-02-26T10:44:39.308356+0000","last_peered":"2024-02-26T10:44:39.308356+0000","last_clean":"2024-02-26T10:44:39.308356+0000","last_became_active":"2024-02-26T10:42:42.168686+0000","last_became_peered":"2024-02-26T10:42:42.168686+0000","last_unstale":"2024-02-26T10:44:39.308356+0000","last_undegraded":"2024-02-26T10:44:39.308356+0000","last_fullsized":"2024-02-26T10:44:39.308356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:21:35.085188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234616+0000","last_change":"2024-02-26T10:42:47.252119+0000","last_active":"2024-02-26T10:44:39.234616+0000","last_peered":"2024-02-26T10:44:39.234616+0000","last_clean":"2024-02-26T10:44:39.234616+0000","last_became_active":"2024-02-26T10:42:47.251723+0000","last_became_peered":"2024-02-26T10:42:47.251723+0000","last_unstale":"2024-02-26T10:44:39.234616+0000","last_undegraded":"2024-02-26T10:44:39.234616+0000","last_fullsized":"2024-02-26T10:44:39.234616+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:17.130806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408505+0000","last_change":"2024-02-26T10:42:41.621393+0000","last_active":"2024-02-26T10:44:39.408505+0000","last_peered":"2024-02-26T10:44:39.408505+0000","last_clean":"2024-02-26T10:44:39.408505+0000","last_became_active":"2024-02-26T10:42:41.621008+0000","last_became_peered":"2024-02-26T10:42:41.621008+0000","last_unstale":"2024-02-26T10:44:39.408505+0000","last_undegraded":"2024-02-26T10:44:39.408505+0000","last_fullsized":"2024-02-26T10:44:39.408505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:46.979854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257031+0000","last_change":"2024-02-26T10:43:38.458926+0000","last_active":"2024-02-26T10:44:39.257031+0000","last_peered":"2024-02-26T10:44:39.257031+0000","last_clean":"2024-02-26T10:44:39.257031+0000","last_became_active":"2024-02-26T10:43:38.458537+0000","last_became_peered":"2024-02-26T10:43:38.458537+0000","last_unstale":"2024-02-26T10:44:39.257031+0000","last_undegraded":"2024-02-26T10:44:39.257031+0000","last_fullsized":"2024-02-26T10:44:39.257031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:08.204646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236580+0000","last_change":"2024-02-26T10:42:47.238520+0000","last_active":"2024-02-26T10:44:39.236580+0000","last_peered":"2024-02-26T10:44:39.236580+0000","last_clean":"2024-02-26T10:44:39.236580+0000","last_became_active":"2024-02-26T10:42:47.237213+0000","last_became_peered":"2024-02-26T10:42:47.237213+0000","last_unstale":"2024-02-26T10:44:39.236580+0000","last_undegraded":"2024-02-26T10:44:39.236580+0000","last_fullsized":"2024-02-26T10:44:39.236580+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:29.283850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235026+0000","last_change":"2024-02-26T10:42:41.609479+0000","last_active":"2024-02-26T10:44:39.235026+0000","last_peered":"2024-02-26T10:44:39.235026+0000","last_clean":"2024-02-26T10:44:39.235026+0000","last_became_active":"2024-02-26T10:42:41.609084+0000","last_became_peered":"2024-02-26T10:42:41.609084+0000","last_unstale":"2024-02-26T10:44:39.235026+0000","last_undegraded":"2024-02-26T10:44:39.235026+0000","last_fullsized":"2024-02-26T10:44:39.235026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:32.412507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257015+0000","last_change":"2024-02-26T10:42:42.174985+0000","last_active":"2024-02-26T10:44:39.257015+0000","last_peered":"2024-02-26T10:44:39.257015+0000","last_clean":"2024-02-26T10:44:39.257015+0000","last_became_active":"2024-02-26T10:42:42.174694+0000","last_became_peered":"2024-02-26T10:42:42.174694+0000","last_unstale":"2024-02-26T10:44:39.257015+0000","last_undegraded":"2024-02-26T10:44:39.257015+0000","last_fullsized":"2024-02-26T10:44:39.257015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:10.000154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236989+0000","last_change":"2024-02-26T10:42:47.242587+0000","last_active":"2024-02-26T10:44:39.236989+0000","last_peered":"2024-02-26T10:44:39.236989+0000","last_clean":"2024-02-26T10:44:39.236989+0000","last_became_active":"2024-02-26T10:42:47.242099+0000","last_became_peered":"2024-02-26T10:42:47.242099+0000","last_unstale":"2024-02-26T10:44:39.236989+0000","last_undegraded":"2024-02-26T10:44:39.236989+0000","last_fullsized":"2024-02-26T10:44:39.236989+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:38:42.852080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556815+0000","last_change":"2024-02-26T10:42:42.170613+0000","last_active":"2024-02-26T10:44:39.556815+0000","last_peered":"2024-02-26T10:44:39.556815+0000","last_clean":"2024-02-26T10:44:39.556815+0000","last_became_active":"2024-02-26T10:42:42.170317+0000","last_became_peered":"2024-02-26T10:42:42.170317+0000","last_unstale":"2024-02-26T10:44:39.556815+0000","last_undegraded":"2024-02-26T10:44:39.556815+0000","last_fullsized":"2024-02-26T10:44:39.556815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:29:44.276718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":126,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:42.207289+0000","last_change":"2024-02-26T10:42:41.617447+0000","last_active":"2024-02-26T10:44:42.207289+0000","last_peered":"2024-02-26T10:44:42.207289+0000","last_clean":"2024-02-26T10:44:42.207289+0000","last_became_active":"2024-02-26T10:42:41.617107+0000","last_became_peered":"2024-02-26T10:42:41.617107+0000","last_unstale":"2024-02-26T10:44:42.207289+0000","last_undegraded":"2024-02-26T10:44:42.207289+0000","last_fullsized":"2024-02-26T10:44:42.207289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:06.052364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235377+0000","last_change":"2024-02-26T10:42:47.251971+0000","last_active":"2024-02-26T10:44:39.235377+0000","last_peered":"2024-02-26T10:44:39.235377+0000","last_clean":"2024-02-26T10:44:39.235377+0000","last_became_active":"2024-02-26T10:42:47.251506+0000","last_became_peered":"2024-02-26T10:42:47.251506+0000","last_unstale":"2024-02-26T10:44:39.235377+0000","last_undegraded":"2024-02-26T10:44:39.235377+0000","last_fullsized":"2024-02-26T10:44:39.235377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:37.029348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040937+0000","last_change":"2024-02-26T10:42:42.194414+0000","last_active":"2024-02-26T10:44:40.040937+0000","last_peered":"2024-02-26T10:44:40.040937+0000","last_clean":"2024-02-26T10:44:40.040937+0000","last_became_active":"2024-02-26T10:42:42.193959+0000","last_became_peered":"2024-02-26T10:42:42.193959+0000","last_unstale":"2024-02-26T10:44:40.040937+0000","last_undegraded":"2024-02-26T10:44:40.040937+0000","last_fullsized":"2024-02-26T10:44:40.040937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:50.398674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239182+0000","last_change":"2024-02-26T10:42:41.607947+0000","last_active":"2024-02-26T10:44:39.239182+0000","last_peered":"2024-02-26T10:44:39.239182+0000","last_clean":"2024-02-26T10:44:39.239182+0000","last_became_active":"2024-02-26T10:42:41.607522+0000","last_became_peered":"2024-02-26T10:42:41.607522+0000","last_unstale":"2024-02-26T10:44:39.239182+0000","last_undegraded":"2024-02-26T10:44:39.239182+0000","last_fullsized":"2024-02-26T10:44:39.239182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:32:36.647621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234832+0000","last_change":"2024-02-26T10:41:21.177202+0000","last_active":"2024-02-26T10:44:39.234832+0000","last_peered":"2024-02-26T10:44:39.234832+0000","last_clean":"2024-02-26T10:44:39.234832+0000","last_became_active":"2024-02-26T10:41:21.176762+0000","last_became_peered":"2024-02-26T10:41:21.176762+0000","last_unstale":"2024-02-26T10:44:39.234832+0000","last_undegraded":"2024-02-26T10:44:39.234832+0000","last_fullsized":"2024-02-26T10:44:39.234832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_clean_scrub_stamp":"2024-02-26T10:37:25.424790+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:19.619104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.555930+0000","last_change":"2024-02-26T10:42:47.244346+0000","last_active":"2024-02-26T10:44:39.555930+0000","last_peered":"2024-02-26T10:44:39.555930+0000","last_clean":"2024-02-26T10:44:39.555930+0000","last_became_active":"2024-02-26T10:42:47.243872+0000","last_became_peered":"2024-02-26T10:42:47.243872+0000","last_unstale":"2024-02-26T10:44:39.555930+0000","last_undegraded":"2024-02-26T10:44:39.555930+0000","last_fullsized":"2024-02-26T10:44:39.555930+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:36.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235284+0000","last_change":"2024-02-26T10:42:42.166266+0000","last_active":"2024-02-26T10:44:39.235284+0000","last_peered":"2024-02-26T10:44:39.235284+0000","last_clean":"2024-02-26T10:44:39.235284+0000","last_became_active":"2024-02-26T10:42:42.165721+0000","last_became_peered":"2024-02-26T10:42:42.165721+0000","last_unstale":"2024-02-26T10:44:39.235284+0000","last_undegraded":"2024-02-26T10:44:39.235284+0000","last_fullsized":"2024-02-26T10:44:39.235284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:32.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234981+0000","last_change":"2024-02-26T10:42:41.606268+0000","last_active":"2024-02-26T10:44:39.234981+0000","last_peered":"2024-02-26T10:44:39.234981+0000","last_clean":"2024-02-26T10:44:39.234981+0000","last_became_active":"2024-02-26T10:42:41.605950+0000","last_became_peered":"2024-02-26T10:42:41.605950+0000","last_unstale":"2024-02-26T10:44:39.234981+0000","last_undegraded":"2024-02-26T10:44:39.234981+0000","last_fullsized":"2024-02-26T10:44:39.234981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:40:03.616987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307750+0000","last_change":"2024-02-26T10:42:47.233203+0000","last_active":"2024-02-26T10:44:39.307750+0000","last_peered":"2024-02-26T10:44:39.307750+0000","last_clean":"2024-02-26T10:44:39.307750+0000","last_became_active":"2024-02-26T10:42:47.232705+0000","last_became_peered":"2024-02-26T10:42:47.232705+0000","last_unstale":"2024-02-26T10:44:39.307750+0000","last_undegraded":"2024-02-26T10:44:39.307750+0000","last_fullsized":"2024-02-26T10:44:39.307750+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:53.818901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":19,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308190+0000","last_change":"2024-02-26T10:43:38.479911+0000","last_active":"2024-02-26T10:44:39.308190+0000","last_peered":"2024-02-26T10:44:39.308190+0000","last_clean":"2024-02-26T10:44:39.308190+0000","last_became_active":"2024-02-26T10:43:38.458874+0000","last_became_peered":"2024-02-26T10:43:38.458874+0000","last_unstale":"2024-02-26T10:44:39.308190+0000","last_undegraded":"2024-02-26T10:44:39.308190+0000","last_fullsized":"2024-02-26T10:44:39.308190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:56.093775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237018+0000","last_change":"2024-02-26T10:42:42.167674+0000","last_active":"2024-02-26T10:44:39.237018+0000","last_peered":"2024-02-26T10:44:39.237018+0000","last_clean":"2024-02-26T10:44:39.237018+0000","last_became_active":"2024-02-26T10:42:42.167224+0000","last_became_peered":"2024-02-26T10:42:42.167224+0000","last_unstale":"2024-02-26T10:44:39.237018+0000","last_undegraded":"2024-02-26T10:44:39.237018+0000","last_fullsized":"2024-02-26T10:44:39.237018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:49:20.433913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.039490+0000","last_change":"2024-02-26T10:42:47.244076+0000","last_active":"2024-02-26T10:44:40.039490+0000","last_peered":"2024-02-26T10:44:40.039490+0000","last_clean":"2024-02-26T10:44:40.039490+0000","last_became_active":"2024-02-26T10:42:47.243569+0000","last_became_peered":"2024-02-26T10:42:47.243569+0000","last_unstale":"2024-02-26T10:44:40.039490+0000","last_undegraded":"2024-02-26T10:44:40.039490+0000","last_fullsized":"2024-02-26T10:44:40.039490+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:46:19.855621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409415+0000","last_change":"2024-02-26T10:42:41.623604+0000","last_active":"2024-02-26T10:44:39.409415+0000","last_peered":"2024-02-26T10:44:39.409415+0000","last_clean":"2024-02-26T10:44:39.409415+0000","last_became_active":"2024-02-26T10:42:41.623272+0000","last_became_peered":"2024-02-26T10:42:41.623272+0000","last_unstale":"2024-02-26T10:44:39.409415+0000","last_undegraded":"2024-02-26T10:44:39.409415+0000","last_fullsized":"2024-02-26T10:44:39.409415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:13:57.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042003+0000","last_change":"2024-02-26T10:42:42.186300+0000","last_active":"2024-02-26T10:44:40.042003+0000","last_peered":"2024-02-26T10:44:40.042003+0000","last_clean":"2024-02-26T10:44:40.042003+0000","last_became_active":"2024-02-26T10:42:42.185965+0000","last_became_peered":"2024-02-26T10:42:42.185965+0000","last_unstale":"2024-02-26T10:44:40.042003+0000","last_undegraded":"2024-02-26T10:44:40.042003+0000","last_fullsized":"2024-02-26T10:44:40.042003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:06:50.352308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238265+0000","last_change":"2024-02-26T10:42:47.227753+0000","last_active":"2024-02-26T10:44:39.238265+0000","last_peered":"2024-02-26T10:44:39.238265+0000","last_clean":"2024-02-26T10:44:39.238265+0000","last_became_active":"2024-02-26T10:42:47.227300+0000","last_became_peered":"2024-02-26T10:42:47.227300+0000","last_unstale":"2024-02-26T10:44:39.238265+0000","last_undegraded":"2024-02-26T10:44:39.238265+0000","last_fullsized":"2024-02-26T10:44:39.238265+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:08.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239752+0000","last_change":"2024-02-26T10:42:41.602960+0000","last_active":"2024-02-26T10:44:39.239752+0000","last_peered":"2024-02-26T10:44:39.239752+0000","last_clean":"2024-02-26T10:44:39.239752+0000","last_became_active":"2024-02-26T10:42:41.602676+0000","last_became_peered":"2024-02-26T10:42:41.602676+0000","last_unstale":"2024-02-26T10:44:39.239752+0000","last_undegraded":"2024-02-26T10:44:39.239752+0000","last_fullsized":"2024-02-26T10:44:39.239752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:57.997612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528654+0000","last_change":"2024-02-26T10:42:42.185451+0000","last_active":"2024-02-26T10:44:39.528654+0000","last_peered":"2024-02-26T10:44:39.528654+0000","last_clean":"2024-02-26T10:44:39.528654+0000","last_became_active":"2024-02-26T10:42:42.185116+0000","last_became_peered":"2024-02-26T10:42:42.185116+0000","last_unstale":"2024-02-26T10:44:39.528654+0000","last_undegraded":"2024-02-26T10:44:39.528654+0000","last_fullsized":"2024-02-26T10:44:39.528654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:43:39.823370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239560+0000","last_change":"2024-02-26T10:42:47.250682+0000","last_active":"2024-02-26T10:44:39.239560+0000","last_peered":"2024-02-26T10:44:39.239560+0000","last_clean":"2024-02-26T10:44:39.239560+0000","last_became_active":"2024-02-26T10:42:47.249524+0000","last_became_peered":"2024-02-26T10:42:47.249524+0000","last_unstale":"2024-02-26T10:44:39.239560+0000","last_undegraded":"2024-02-26T10:44:39.239560+0000","last_fullsized":"2024-02-26T10:44:39.239560+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:07:01.307206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463895+0000","last_change":"2024-02-26T10:42:41.620764+0000","last_active":"2024-02-26T10:44:39.463895+0000","last_peered":"2024-02-26T10:44:39.463895+0000","last_clean":"2024-02-26T10:44:39.463895+0000","last_became_active":"2024-02-26T10:42:41.620500+0000","last_became_peered":"2024-02-26T10:42:41.620500+0000","last_unstale":"2024-02-26T10:44:39.463895+0000","last_undegraded":"2024-02-26T10:44:39.463895+0000","last_fullsized":"2024-02-26T10:44:39.463895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:15.542396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234388+0000","last_change":"2024-02-26T10:42:42.165060+0000","last_active":"2024-02-26T10:44:39.234388+0000","last_peered":"2024-02-26T10:44:39.234388+0000","last_clean":"2024-02-26T10:44:39.234388+0000","last_became_active":"2024-02-26T10:42:42.164733+0000","last_became_peered":"2024-02-26T10:42:42.164733+0000","last_unstale":"2024-02-26T10:44:39.234388+0000","last_undegraded":"2024-02-26T10:44:39.234388+0000","last_fullsized":"2024-02-26T10:44:39.234388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:34.190747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.044327+0000","last_change":"2024-02-26T10:44:40.044327+0000","last_active":"2024-02-26T10:44:40.044327+0000","last_peered":"2024-02-26T10:44:40.044327+0000","last_clean":"2024-02-26T10:44:40.044327+0000","last_became_active":"2024-02-26T10:44:40.043776+0000","last_became_peered":"2024-02-26T10:44:40.043776+0000","last_unstale":"2024-02-26T10:44:40.044327+0000","last_undegraded":"2024-02-26T10:44:40.044327+0000","last_fullsized":"2024-02-26T10:44:40.044327+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:50:51.502411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040238+0000","last_change":"2024-02-26T10:42:41.624940+0000","last_active":"2024-02-26T10:44:40.040238+0000","last_peered":"2024-02-26T10:44:40.040238+0000","last_clean":"2024-02-26T10:44:40.040238+0000","last_became_active":"2024-02-26T10:42:41.624538+0000","last_became_peered":"2024-02-26T10:42:41.624538+0000","last_unstale":"2024-02-26T10:44:40.040238+0000","last_undegraded":"2024-02-26T10:44:40.040238+0000","last_fullsized":"2024-02-26T10:44:40.040238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:10.052806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307967+0000","last_change":"2024-02-26T10:42:42.174932+0000","last_active":"2024-02-26T10:44:39.307967+0000","last_peered":"2024-02-26T10:44:39.307967+0000","last_clean":"2024-02-26T10:44:39.307967+0000","last_became_active":"2024-02-26T10:42:42.174566+0000","last_became_peered":"2024-02-26T10:42:42.174566+0000","last_unstale":"2024-02-26T10:44:39.307967+0000","last_undegraded":"2024-02-26T10:44:39.307967+0000","last_fullsized":"2024-02-26T10:44:39.307967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:03.242487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239812+0000","last_change":"2024-02-26T10:42:47.250023+0000","last_active":"2024-02-26T10:44:39.239812+0000","last_peered":"2024-02-26T10:44:39.239812+0000","last_clean":"2024-02-26T10:44:39.239812+0000","last_became_active":"2024-02-26T10:42:47.249571+0000","last_became_peered":"2024-02-26T10:42:47.249571+0000","last_unstale":"2024-02-26T10:44:39.239812+0000","last_undegraded":"2024-02-26T10:44:39.239812+0000","last_fullsized":"2024-02-26T10:44:39.239812+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:53.666184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238474+0000","last_change":"2024-02-26T10:42:41.614403+0000","last_active":"2024-02-26T10:44:39.238474+0000","last_peered":"2024-02-26T10:44:39.238474+0000","last_clean":"2024-02-26T10:44:39.238474+0000","last_became_active":"2024-02-26T10:42:41.614057+0000","last_became_peered":"2024-02-26T10:42:41.614057+0000","last_unstale":"2024-02-26T10:44:39.238474+0000","last_undegraded":"2024-02-26T10:44:39.238474+0000","last_fullsized":"2024-02-26T10:44:39.238474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:16:54.739090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040861+0000","last_change":"2024-02-26T10:42:42.187465+0000","last_active":"2024-02-26T10:44:40.040861+0000","last_peered":"2024-02-26T10:44:40.040861+0000","last_clean":"2024-02-26T10:44:40.040861+0000","last_became_active":"2024-02-26T10:42:42.182310+0000","last_became_peered":"2024-02-26T10:42:42.182310+0000","last_unstale":"2024-02-26T10:44:40.040861+0000","last_undegraded":"2024-02-26T10:44:40.040861+0000","last_fullsized":"2024-02-26T10:44:40.040861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:01:56.645330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239342+0000","last_change":"2024-02-26T10:42:47.241751+0000","last_active":"2024-02-26T10:44:39.239342+0000","last_peered":"2024-02-26T10:44:39.239342+0000","last_clean":"2024-02-26T10:44:39.239342+0000","last_became_active":"2024-02-26T10:42:47.241271+0000","last_became_peered":"2024-02-26T10:42:47.241271+0000","last_unstale":"2024-02-26T10:44:39.239342+0000","last_undegraded":"2024-02-26T10:44:39.239342+0000","last_fullsized":"2024-02-26T10:44:39.239342+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:40:39.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234575+0000","last_change":"2024-02-26T10:42:41.607243+0000","last_active":"2024-02-26T10:44:39.234575+0000","last_peered":"2024-02-26T10:44:39.234575+0000","last_clean":"2024-02-26T10:44:39.234575+0000","last_became_active":"2024-02-26T10:42:41.606921+0000","last_became_peered":"2024-02-26T10:42:41.606921+0000","last_unstale":"2024-02-26T10:44:39.234575+0000","last_undegraded":"2024-02-26T10:44:39.234575+0000","last_fullsized":"2024-02-26T10:44:39.234575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:04.626391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235160+0000","last_change":"2024-02-26T10:42:42.178240+0000","last_active":"2024-02-26T10:44:39.235160+0000","last_peered":"2024-02-26T10:44:39.235160+0000","last_clean":"2024-02-26T10:44:39.235160+0000","last_became_active":"2024-02-26T10:42:42.177790+0000","last_became_peered":"2024-02-26T10:42:42.177790+0000","last_unstale":"2024-02-26T10:44:39.235160+0000","last_undegraded":"2024-02-26T10:44:39.235160+0000","last_fullsized":"2024-02-26T10:44:39.235160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:24.110406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463687+0000","last_change":"2024-02-26T10:42:47.233120+0000","last_active":"2024-02-26T10:44:39.463687+0000","last_peered":"2024-02-26T10:44:39.463687+0000","last_clean":"2024-02-26T10:44:39.463687+0000","last_became_active":"2024-02-26T10:42:47.232630+0000","last_became_peered":"2024-02-26T10:42:47.232630+0000","last_unstale":"2024-02-26T10:44:39.463687+0000","last_undegraded":"2024-02-26T10:44:39.463687+0000","last_fullsized":"2024-02-26T10:44:39.463687+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:05:03.023227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239351+0000","last_change":"2024-02-26T10:43:38.477127+0000","last_active":"2024-02-26T10:44:39.239351+0000","last_peered":"2024-02-26T10:44:39.239351+0000","last_clean":"2024-02-26T10:44:39.239351+0000","last_became_active":"2024-02-26T10:43:38.460135+0000","last_became_peered":"2024-02-26T10:43:38.460135+0000","last_unstale":"2024-02-26T10:44:39.239351+0000","last_undegraded":"2024-02-26T10:44:39.239351+0000","last_fullsized":"2024-02-26T10:44:39.239351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:13:16.521943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235554+0000","last_change":"2024-02-26T10:42:42.171821+0000","last_active":"2024-02-26T10:44:39.235554+0000","last_peered":"2024-02-26T10:44:39.235554+0000","last_clean":"2024-02-26T10:44:39.235554+0000","last_became_active":"2024-02-26T10:42:42.171523+0000","last_became_peered":"2024-02-26T10:42:42.171523+0000","last_unstale":"2024-02-26T10:44:39.235554+0000","last_undegraded":"2024-02-26T10:44:39.235554+0000","last_fullsized":"2024-02-26T10:44:39.235554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:00.877998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526525+0000","last_change":"2024-02-26T10:42:47.250072+0000","last_active":"2024-02-26T10:44:39.526525+0000","last_peered":"2024-02-26T10:44:39.526525+0000","last_clean":"2024-02-26T10:44:39.526525+0000","last_became_active":"2024-02-26T10:42:47.249570+0000","last_became_peered":"2024-02-26T10:42:47.249570+0000","last_unstale":"2024-02-26T10:44:39.526525+0000","last_undegraded":"2024-02-26T10:44:39.526525+0000","last_fullsized":"2024-02-26T10:44:39.526525+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:43:03.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557469+0000","last_change":"2024-02-26T10:42:41.614686+0000","last_active":"2024-02-26T10:44:39.557469+0000","last_peered":"2024-02-26T10:44:39.557469+0000","last_clean":"2024-02-26T10:44:39.557469+0000","last_became_active":"2024-02-26T10:42:41.614019+0000","last_became_peered":"2024-02-26T10:42:41.614019+0000","last_unstale":"2024-02-26T10:44:39.557469+0000","last_undegraded":"2024-02-26T10:44:39.557469+0000","last_fullsized":"2024-02-26T10:44:39.557469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:13:28.127288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235338+0000","last_change":"2024-02-26T10:42:42.181774+0000","last_active":"2024-02-26T10:44:39.235338+0000","last_peered":"2024-02-26T10:44:39.235338+0000","last_clean":"2024-02-26T10:44:39.235338+0000","last_became_active":"2024-02-26T10:42:42.181412+0000","last_became_peered":"2024-02-26T10:42:42.181412+0000","last_unstale":"2024-02-26T10:44:39.235338+0000","last_undegraded":"2024-02-26T10:44:39.235338+0000","last_fullsized":"2024-02-26T10:44:39.235338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:09.159964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556072+0000","last_change":"2024-02-26T10:42:47.228247+0000","last_active":"2024-02-26T10:44:39.556072+0000","last_peered":"2024-02-26T10:44:39.556072+0000","last_clean":"2024-02-26T10:44:39.556072+0000","last_became_active":"2024-02-26T10:42:47.227569+0000","last_became_peered":"2024-02-26T10:42:47.227569+0000","last_unstale":"2024-02-26T10:44:39.556072+0000","last_undegraded":"2024-02-26T10:44:39.556072+0000","last_fullsized":"2024-02-26T10:44:39.556072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:15.001264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236773+0000","last_change":"2024-02-26T10:42:41.626573+0000","last_active":"2024-02-26T10:44:39.236773+0000","last_peered":"2024-02-26T10:44:39.236773+0000","last_clean":"2024-02-26T10:44:39.236773+0000","last_became_active":"2024-02-26T10:42:41.624091+0000","last_became_peered":"2024-02-26T10:42:41.624091+0000","last_unstale":"2024-02-26T10:44:39.236773+0000","last_undegraded":"2024-02-26T10:44:39.236773+0000","last_fullsized":"2024-02-26T10:44:39.236773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:24:14.077234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235619+0000","last_change":"2024-02-26T10:42:42.166426+0000","last_active":"2024-02-26T10:44:39.235619+0000","last_peered":"2024-02-26T10:44:39.235619+0000","last_clean":"2024-02-26T10:44:39.235619+0000","last_became_active":"2024-02-26T10:42:42.166082+0000","last_became_peered":"2024-02-26T10:42:42.166082+0000","last_unstale":"2024-02-26T10:44:39.235619+0000","last_undegraded":"2024-02-26T10:44:39.235619+0000","last_fullsized":"2024-02-26T10:44:39.235619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:17.136568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308118+0000","last_change":"2024-02-26T10:42:42.168227+0000","last_active":"2024-02-26T10:44:39.308118+0000","last_peered":"2024-02-26T10:44:39.308118+0000","last_clean":"2024-02-26T10:44:39.308118+0000","last_became_active":"2024-02-26T10:42:42.167897+0000","last_became_peered":"2024-02-26T10:42:42.167897+0000","last_unstale":"2024-02-26T10:44:39.308118+0000","last_undegraded":"2024-02-26T10:44:39.308118+0000","last_fullsized":"2024-02-26T10:44:39.308118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:52:37.391022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238659+0000","last_change":"2024-02-26T10:42:42.168640+0000","last_active":"2024-02-26T10:44:39.238659+0000","last_peered":"2024-02-26T10:44:39.238659+0000","last_clean":"2024-02-26T10:44:39.238659+0000","last_became_active":"2024-02-26T10:42:42.168237+0000","last_became_peered":"2024-02-26T10:42:42.168237+0000","last_unstale":"2024-02-26T10:44:39.238659+0000","last_undegraded":"2024-02-26T10:44:39.238659+0000","last_fullsized":"2024-02-26T10:44:39.238659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:14.853730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238275+0000","last_change":"2024-02-26T10:42:42.192703+0000","last_active":"2024-02-26T10:44:39.238275+0000","last_peered":"2024-02-26T10:44:39.238275+0000","last_clean":"2024-02-26T10:44:39.238275+0000","last_became_active":"2024-02-26T10:42:42.191899+0000","last_became_peered":"2024-02-26T10:42:42.191899+0000","last_unstale":"2024-02-26T10:44:39.238275+0000","last_undegraded":"2024-02-26T10:44:39.238275+0000","last_fullsized":"2024-02-26T10:44:39.238275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:40.479523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527850+0000","last_change":"2024-02-26T10:42:42.188709+0000","last_active":"2024-02-26T10:44:39.527850+0000","last_peered":"2024-02-26T10:44:39.527850+0000","last_clean":"2024-02-26T10:44:39.527850+0000","last_became_active":"2024-02-26T10:42:42.188418+0000","last_became_peered":"2024-02-26T10:42:42.188418+0000","last_unstale":"2024-02-26T10:44:39.527850+0000","last_undegraded":"2024-02-26T10:44:39.527850+0000","last_fullsized":"2024-02-26T10:44:39.527850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:54:15.697746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236803+0000","last_change":"2024-02-26T10:42:42.174221+0000","last_active":"2024-02-26T10:44:39.236803+0000","last_peered":"2024-02-26T10:44:39.236803+0000","last_clean":"2024-02-26T10:44:39.236803+0000","last_became_active":"2024-02-26T10:42:42.171308+0000","last_became_peered":"2024-02-26T10:42:42.171308+0000","last_unstale":"2024-02-26T10:44:39.236803+0000","last_undegraded":"2024-02-26T10:44:39.236803+0000","last_fullsized":"2024-02-26T10:44:39.236803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:46.765856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308405+0000","last_change":"2024-02-26T10:42:42.170749+0000","last_active":"2024-02-26T10:44:39.308405+0000","last_peered":"2024-02-26T10:44:39.308405+0000","last_clean":"2024-02-26T10:44:39.308405+0000","last_became_active":"2024-02-26T10:42:42.169966+0000","last_became_peered":"2024-02-26T10:42:42.169966+0000","last_unstale":"2024-02-26T10:44:39.308405+0000","last_undegraded":"2024-02-26T10:44:39.308405+0000","last_fullsized":"2024-02-26T10:44:39.308405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:02.633195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238872+0000","last_change":"2024-02-26T10:42:42.166529+0000","last_active":"2024-02-26T10:44:39.238872+0000","last_peered":"2024-02-26T10:44:39.238872+0000","last_clean":"2024-02-26T10:44:39.238872+0000","last_became_active":"2024-02-26T10:42:42.166235+0000","last_became_peered":"2024-02-26T10:42:42.166235+0000","last_unstale":"2024-02-26T10:44:39.238872+0000","last_undegraded":"2024-02-26T10:44:39.238872+0000","last_fullsized":"2024-02-26T10:44:39.238872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:03.068328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042489+0000","last_change":"2024-02-26T10:42:42.191312+0000","last_active":"2024-02-26T10:44:40.042489+0000","last_peered":"2024-02-26T10:44:40.042489+0000","last_clean":"2024-02-26T10:44:40.042489+0000","last_became_active":"2024-02-26T10:42:42.190964+0000","last_became_peered":"2024-02-26T10:42:42.190964+0000","last_unstale":"2024-02-26T10:44:40.042489+0000","last_undegraded":"2024-02-26T10:44:40.042489+0000","last_fullsized":"2024-02-26T10:44:40.042489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:24:04.697129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556727+0000","last_change":"2024-02-26T10:42:42.182882+0000","last_active":"2024-02-26T10:44:39.556727+0000","last_peered":"2024-02-26T10:44:39.556727+0000","last_clean":"2024-02-26T10:44:39.556727+0000","last_became_active":"2024-02-26T10:42:42.182551+0000","last_became_peered":"2024-02-26T10:42:42.182551+0000","last_unstale":"2024-02-26T10:44:39.556727+0000","last_undegraded":"2024-02-26T10:44:39.556727+0000","last_fullsized":"2024-02-26T10:44:39.556727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:11:58.578832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040583+0000","last_change":"2024-02-26T10:42:42.193353+0000","last_active":"2024-02-26T10:44:40.040583+0000","last_peered":"2024-02-26T10:44:40.040583+0000","last_clean":"2024-02-26T10:44:40.040583+0000","last_became_active":"2024-02-26T10:42:42.190935+0000","last_became_peered":"2024-02-26T10:42:42.190935+0000","last_unstale":"2024-02-26T10:44:40.040583+0000","last_undegraded":"2024-02-26T10:44:40.040583+0000","last_fullsized":"2024-02-26T10:44:40.040583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:03:14.319769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464135+0000","last_change":"2024-02-26T10:42:42.194529+0000","last_active":"2024-02-26T10:44:39.464135+0000","last_peered":"2024-02-26T10:44:39.464135+0000","last_clean":"2024-02-26T10:44:39.464135+0000","last_became_active":"2024-02-26T10:42:42.194212+0000","last_became_peered":"2024-02-26T10:42:42.194212+0000","last_unstale":"2024-02-26T10:44:39.464135+0000","last_undegraded":"2024-02-26T10:44:39.464135+0000","last_fullsized":"2024-02-26T10:44:39.464135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:36.876672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237200+0000","last_change":"2024-02-26T10:42:42.178119+0000","last_active":"2024-02-26T10:44:39.237200+0000","last_peered":"2024-02-26T10:44:39.237200+0000","last_clean":"2024-02-26T10:44:39.237200+0000","last_became_active":"2024-02-26T10:42:42.177798+0000","last_became_peered":"2024-02-26T10:42:42.177798+0000","last_unstale":"2024-02-26T10:44:39.237200+0000","last_undegraded":"2024-02-26T10:44:39.237200+0000","last_fullsized":"2024-02-26T10:44:39.237200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:52:29.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308077+0000","last_change":"2024-02-26T10:42:42.167380+0000","last_active":"2024-02-26T10:44:39.308077+0000","last_peered":"2024-02-26T10:44:39.308077+0000","last_clean":"2024-02-26T10:44:39.308077+0000","last_became_active":"2024-02-26T10:42:42.167079+0000","last_became_peered":"2024-02-26T10:42:42.167079+0000","last_unstale":"2024-02-26T10:44:39.308077+0000","last_undegraded":"2024-02-26T10:44:39.308077+0000","last_fullsized":"2024-02-26T10:44:39.308077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:14:25.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250986,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67016,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":546,"num_read_kb":343,"num_write":171,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":63341,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":201048,"data_compressed":14031,"data_compressed_allocated":20480,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":532499,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612652,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1187840,"data_stored":1183039,"data_compressed":64863,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":527294,"data_compressed":23715,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":532387,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":1128571,"data_compressed":63031,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":527777,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1105920,"data_stored":590561,"data_compressed":28539,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":529165,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":528455,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":527971,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":596415,"data_compressed":37074,"data_compressed_allocated":49152,"data_compressed_original":135168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724618,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1118208,"data_stored":1123627,"data_compressed":63031,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62874,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":64632,"data_compressed":10367,"data_compressed_allocated":12288,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64984,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:44:46.115 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-26T10:44:46.115 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-26T10:44:46.115 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-26T10:44:46.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph health --format=json 2024-02-26T10:44:46.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:46 smithi113 ceph-mon[36126]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-02-26T10:44:46.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:46 smithi113 ceph-mon[36126]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:44:46.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:46 smithi060 ceph-mon[31539]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-02-26T10:44:46.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:46 smithi060 ceph-mon[31539]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:44:46.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:46 smithi100 ceph-mon[36051]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-02-26T10:44:46.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:46 smithi100 ceph-mon[36051]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:44:46.648 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:48.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:48 smithi060 ceph-mon[31539]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 117 B/s wr, 4 op/s 2024-02-26T10:44:48.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:48 smithi113 ceph-mon[36126]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 117 B/s wr, 4 op/s 2024-02-26T10:44:48.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:48 smithi100 ceph-mon[36051]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 117 B/s wr, 4 op/s 2024-02-26T10:44:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.818+0000 7ffaeec00700 1 -- 172.21.15.60:0/400094838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae810e750 msgr2=0x7ffae8110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.818+0000 7ffaeec00700 1 --2- 172.21.15.60:0/400094838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae810e750 0x7ffae8110b40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffae000b0a0 tx=0x7ffae000b3b0 comp rx=0 tx=0).stop 2024-02-26T10:44:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.818+0000 7ffaeec00700 1 -- 172.21.15.60:0/400094838 shutdown_connections 2024-02-26T10:44:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.818+0000 7ffaeec00700 1 --2- 172.21.15.60:0/400094838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae810e750 0x7ffae8110b40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.818+0000 7ffaeec00700 1 --2- 172.21.15.60:0/400094838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffae8073f80 0x7ffae8074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.818+0000 7ffaeec00700 1 --2- 172.21.15.60:0/400094838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffae80748a0 0x7ffae810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.818+0000 7ffaeec00700 1 -- 172.21.15.60:0/400094838 >> 172.21.15.60:0/400094838 conn(0x7ffae8078e80 msgr2=0x7ffae8079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.818+0000 7ffaeec00700 1 -- 172.21.15.60:0/400094838 shutdown_connections 2024-02-26T10:44:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.818+0000 7ffaeec00700 1 -- 172.21.15.60:0/400094838 wait complete. 2024-02-26T10:44:48.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.819+0000 7ffaeec00700 1 Processor -- start 2024-02-26T10:44:48.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.820+0000 7ffaeec00700 1 -- start start 2024-02-26T10:44:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.820+0000 7ffaeec00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffae8073f80 0x7ffae819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.820+0000 7ffaeec00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffae80748a0 0x7ffae819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.820+0000 7ffaeec00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae810e750 0x7ffae819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.820+0000 7ffaeec00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae81141f0 con 0x7ffae80748a0 2024-02-26T10:44:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.820+0000 7ffaeec00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8114070 con 0x7ffae8073f80 2024-02-26T10:44:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.820+0000 7ffaeec00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8114370 con 0x7ffae810e750 2024-02-26T10:44:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffae7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffae80748a0 0x7ffae819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffae7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffae80748a0 0x7ffae819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33958/0 (socket says 172.21.15.60:33958) 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffae7400700 1 -- 172.21.15.60:0/3096516138 learned_addr learned my addr 172.21.15.60:0/3096516138 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffae7e00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffae8073f80 0x7ffae819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffaecc00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae810e750 0x7ffae819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffae7400700 1 -- 172.21.15.60:0/3096516138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae810e750 msgr2=0x7ffae819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffae7400700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae810e750 0x7ffae819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffae7400700 1 -- 172.21.15.60:0/3096516138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffae8073f80 msgr2=0x7ffae819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffae7400700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffae8073f80 0x7ffae819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffae7400700 1 -- 172.21.15.60:0/3096516138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae819f650 con 0x7ffae80748a0 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.821+0000 7ffaecc00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae810e750 0x7ffae819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.822+0000 7ffae7400700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffae80748a0 0x7ffae819ab80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffad400d950 tx=0x7ffad400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.822+0000 7ffae4c00700 1 -- 172.21.15.60:0/3096516138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4016070 con 0x7ffae80748a0 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.822+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae819f940 con 0x7ffae80748a0 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.822+0000 7ffae4c00700 1 -- 172.21.15.60:0/3096516138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffad400fd70 con 0x7ffae80748a0 2024-02-26T10:44:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.822+0000 7ffae4c00700 1 -- 172.21.15.60:0/3096516138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad400e610 con 0x7ffae80748a0 2024-02-26T10:44:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.822+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae81a71c0 con 0x7ffae80748a0 2024-02-26T10:44:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.823+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae8068a10 con 0x7ffae80748a0 2024-02-26T10:44:48.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.826+0000 7ffae4c00700 1 -- 172.21.15.60:0/3096516138 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ffad4019070 con 0x7ffae80748a0 2024-02-26T10:44:48.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.827+0000 7ffae4c00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ffacc0a1e10 0x7ffacc0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:48.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.827+0000 7ffae4c00700 1 -- 172.21.15.60:0/3096516138 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7ffad40cdbf0 con 0x7ffae80748a0 2024-02-26T10:44:48.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.828+0000 7ffae7e00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ffacc0a1e10 0x7ffacc0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:48.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.828+0000 7ffae7e00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ffacc0a1e10 0x7ffacc0a42d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009c60 tx=0x7ffad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:48.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:48.830+0000 7ffae4c00700 1 -- 172.21.15.60:0/3096516138 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffad409ac10 con 0x7ffae80748a0 2024-02-26T10:44:49.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.099+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffae819b9a0 con 0x7ffae80748a0 2024-02-26T10:44:49.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.100+0000 7ffae4c00700 1 -- 172.21.15.60:0/3096516138 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ffae819b9a0 con 0x7ffae80748a0 2024-02-26T10:44:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:44:49.096 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-26T10:44:49.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ffacc0a1e10 msgr2=0x7ffacc0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:49.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ffacc0a1e10 0x7ffacc0a42d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009c60 tx=0x7ffad8009260 comp rx=0 tx=0).stop 2024-02-26T10:44:49.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffae80748a0 msgr2=0x7ffae819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:49.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffae80748a0 0x7ffae819ab80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffad400d950 tx=0x7ffad400dc60 comp rx=0 tx=0).stop 2024-02-26T10:44:49.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 shutdown_connections 2024-02-26T10:44:49.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ffacc0a1e10 0x7ffacc0a42d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae810e750 0x7ffae819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffae8073f80 0x7ffae819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 --2- 172.21.15.60:0/3096516138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffae80748a0 0x7ffae819ab80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.104+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 >> 172.21.15.60:0/3096516138 conn(0x7ffae8078e80 msgr2=0x7ffae80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.105+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 shutdown_connections 2024-02-26T10:44:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:49.105+0000 7ffaeec00700 1 -- 172.21.15.60:0/3096516138 wait complete. 2024-02-26T10:44:49.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:49 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3096516138' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T10:44:49.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:49 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3096516138' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T10:44:49.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:49 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3096516138' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T10:44:49.948 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-26T10:44:49.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:44:50.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.189+0000 7f2834000640 1 -- 172.21.15.60:0/3025369324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c109110 msgr2=0x7f282c10b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.189+0000 7f2834000640 1 --2- 172.21.15.60:0/3025369324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c109110 0x7f282c10b5b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2824009ca0 tx=0x7f282402c180 comp rx=0 tx=0).stop 2024-02-26T10:44:50.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.189+0000 7f2834000640 1 -- 172.21.15.60:0/3025369324 shutdown_connections 2024-02-26T10:44:50.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.189+0000 7f2834000640 1 --2- 172.21.15.60:0/3025369324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c109110 0x7f282c10b5b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.189+0000 7f2834000640 1 --2- 172.21.15.60:0/3025369324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f282c073c60 0x7f282c0740c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.189+0000 7f2834000640 1 --2- 172.21.15.60:0/3025369324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f282c075240 0x7f282c073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.189+0000 7f2834000640 1 -- 172.21.15.60:0/3025369324 >> 172.21.15.60:0/3025369324 conn(0x7f282c0fc810 msgr2=0x7f282c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:50.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.189+0000 7f2834000640 1 -- 172.21.15.60:0/3025369324 shutdown_connections 2024-02-26T10:44:50.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.189+0000 7f2834000640 1 -- 172.21.15.60:0/3025369324 wait complete. 2024-02-26T10:44:50.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.190+0000 7f2834000640 1 Processor -- start 2024-02-26T10:44:50.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2834000640 1 -- start start 2024-02-26T10:44:50.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2834000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f282c073c60 0x7f282c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2834000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f282c075240 0x7f282c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2834000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c109110 0x7f282c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2834000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c10f110 con 0x7f282c073c60 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2834000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c10ef90 con 0x7f282c109110 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2834000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c10f290 con 0x7f282c075240 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2831600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f282c073c60 0x7f282c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2831600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f282c073c60 0x7f282c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33976/0 (socket says 172.21.15.60:33976) 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2831600640 1 -- 172.21.15.60:0/3715801084 learned_addr learned my addr 172.21.15.60:0/3715801084 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.191+0000 7f2832000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c109110 0x7f282c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.192+0000 7f2830c00640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f282c075240 0x7f282c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.192+0000 7f2832000640 1 -- 172.21.15.60:0/3715801084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f282c075240 msgr2=0x7f282c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.192+0000 7f2832000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f282c075240 0x7f282c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.192+0000 7f2832000640 1 -- 172.21.15.60:0/3715801084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f282c073c60 msgr2=0x7f282c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.192+0000 7f2832000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f282c073c60 0x7f282c199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.192+0000 7f2832000640 1 -- 172.21.15.60:0/3715801084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c1a1320 con 0x7f282c109110 2024-02-26T10:44:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.192+0000 7f2832000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c109110 0x7f282c1a0cb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2824037040 tx=0x7f28240093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.193+0000 7f282a000640 1 -- 172.21.15.60:0/3715801084 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824039040 con 0x7f282c109110 2024-02-26T10:44:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.193+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282c1a15b0 con 0x7f282c109110 2024-02-26T10:44:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.193+0000 7f282a000640 1 -- 172.21.15.60:0/3715801084 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2824044070 con 0x7f282c109110 2024-02-26T10:44:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.193+0000 7f282a000640 1 -- 172.21.15.60:0/3715801084 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28240042a0 con 0x7f282c109110 2024-02-26T10:44:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.193+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c1a1ac0 con 0x7f282c109110 2024-02-26T10:44:50.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.194+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282c107ed0 con 0x7f282c109110 2024-02-26T10:44:50.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.197+0000 7f282a000640 1 -- 172.21.15.60:0/3715801084 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2824004500 con 0x7f282c109110 2024-02-26T10:44:50.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.198+0000 7f282a000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f28140a1e00 0x7f28140a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:50.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.198+0000 7f282a000640 1 -- 172.21.15.60:0/3715801084 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f28240f0760 con 0x7f282c109110 2024-02-26T10:44:50.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.198+0000 7f2831600640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f28140a1e00 0x7f28140a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:50.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.199+0000 7f2831600640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f28140a1e00 0x7f28140a42c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2820009c90 tx=0x7f2820005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:50.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.200+0000 7f282a000640 1 -- 172.21.15.60:0/3715801084 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28240bd780 con 0x7f282c109110 2024-02-26T10:44:50.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.403+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f282c061e60 con 0x7f282c109110 2024-02-26T10:44:50.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.406+0000 7f282a000640 1 -- 172.21.15.60:0/3715801084 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7f28240098b0 con 0x7f282c109110 2024-02-26T10:44:50.402 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:44:50.402 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.408+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f28140a1e00 msgr2=0x7f28140a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.408+0000 7f2834000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f28140a1e00 0x7f28140a42c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2820009c90 tx=0x7f2820005f90 comp rx=0 tx=0).stop 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c109110 msgr2=0x7f282c1a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c109110 0x7f282c1a0cb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2824037040 tx=0x7f28240093d0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 shutdown_connections 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f28140a1e00 0x7f28140a42c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c109110 0x7f282c1a0cb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f282c075240 0x7f282c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 --2- 172.21.15.60:0/3715801084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f282c073c60 0x7f282c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 >> 172.21.15.60:0/3715801084 conn(0x7f282c0fc810 msgr2=0x7f282c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 shutdown_connections 2024-02-26T10:44:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.409+0000 7f2834000640 1 -- 172.21.15.60:0/3715801084 wait complete. 2024-02-26T10:44:50.424 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:44:50.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:44:50.583 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:50 smithi113 ceph-mon[36126]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-26T10:44:50.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:50 smithi060 ceph-mon[31539]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-26T10:44:50.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:50 smithi100 ceph-mon[36051]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-26T10:44:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.649+0000 7f3941c00640 1 -- 172.21.15.60:0/1887483228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393c102350 msgr2=0x7f393c1027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.649+0000 7f3941c00640 1 --2- 172.21.15.60:0/1887483228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393c102350 0x7f393c1027d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3924009860 tx=0x7f392402bb70 comp rx=0 tx=0).stop 2024-02-26T10:44:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.649+0000 7f3941c00640 1 -- 172.21.15.60:0/1887483228 shutdown_connections 2024-02-26T10:44:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.649+0000 7f3941c00640 1 --2- 172.21.15.60:0/1887483228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393c102350 0x7f393c1027d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.649+0000 7f3941c00640 1 --2- 172.21.15.60:0/1887483228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f393c102d10 0x7f393c1095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.649+0000 7f3941c00640 1 --2- 172.21.15.60:0/1887483228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393c101160 0x7f393c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.649+0000 7f3941c00640 1 -- 172.21.15.60:0/1887483228 >> 172.21.15.60:0/1887483228 conn(0x7f393c0fc6d0 msgr2=0x7f393c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.649+0000 7f3941c00640 1 -- 172.21.15.60:0/1887483228 shutdown_connections 2024-02-26T10:44:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.649+0000 7f3941c00640 1 -- 172.21.15.60:0/1887483228 wait complete. 2024-02-26T10:44:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.650+0000 7f3941c00640 1 Processor -- start 2024-02-26T10:44:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.650+0000 7f3941c00640 1 -- start start 2024-02-26T10:44:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f3941c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393c101160 0x7f393c075010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f3941c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f393c102350 0x7f393c073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f3941c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393c102d10 0x7f393c073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f3941c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c10cdf0 con 0x7f393c102350 2024-02-26T10:44:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f3941c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c10cc70 con 0x7f393c102d10 2024-02-26T10:44:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f3941c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c10cf70 con 0x7f393c101160 2024-02-26T10:44:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f393b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f393c102350 0x7f393c073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f393b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f393c102350 0x7f393c073650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33990/0 (socket says 172.21.15.60:33990) 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f393b400640 1 -- 172.21.15.60:0/2928835688 learned_addr learned my addr 172.21.15.60:0/2928835688 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f393be00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393c101160 0x7f393c075010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f3941200640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393c102d10 0x7f393c073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f393b400640 1 -- 172.21.15.60:0/2928835688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393c101160 msgr2=0x7f393c075010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f393b400640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393c101160 0x7f393c075010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f393b400640 1 -- 172.21.15.60:0/2928835688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393c102d10 msgr2=0x7f393c073b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f393b400640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393c102d10 0x7f393c073b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.651+0000 7f393b400640 1 -- 172.21.15.60:0/2928835688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c074420 con 0x7f393c102350 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.652+0000 7f393b400640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f393c102350 0x7f393c073650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f39240349e0 tx=0x7f3924032450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.652+0000 7f3938c00640 1 -- 172.21.15.60:0/2928835688 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924041030 con 0x7f393c102350 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.652+0000 7f3938c00640 1 -- 172.21.15.60:0/2928835688 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f392402c510 con 0x7f393c102350 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.652+0000 7f3938c00640 1 -- 172.21.15.60:0/2928835688 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924039040 con 0x7f393c102350 2024-02-26T10:44:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.652+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c1a14c0 con 0x7f393c102350 2024-02-26T10:44:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.652+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c1a1a10 con 0x7f393c102350 2024-02-26T10:44:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.654+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c108480 con 0x7f393c102350 2024-02-26T10:44:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.656+0000 7f3938c00640 1 -- 172.21.15.60:0/2928835688 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3924044070 con 0x7f393c102350 2024-02-26T10:44:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.657+0000 7f3938c00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f39200a1d80 0x7f39200a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:50.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.657+0000 7f3938c00640 1 -- 172.21.15.60:0/2928835688 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f39240ef6c0 con 0x7f393c102350 2024-02-26T10:44:50.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.657+0000 7f393be00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f39200a1d80 0x7f39200a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:50.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.658+0000 7f393be00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f39200a1d80 0x7f39200a4240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f392c0099a0 tx=0x7f392c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:50.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.661+0000 7f3938c00640 1 -- 172.21.15.60:0/2928835688 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f39240bc6e0 con 0x7f393c102350 2024-02-26T10:44:50.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.862+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f393c061e60 con 0x7f393c102350 2024-02-26T10:44:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.863+0000 7f3938c00640 1 -- 172.21.15.60:0/2928835688 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39240bbfe0 con 0x7f393c102350 2024-02-26T10:44:50.859 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:44:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.865+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f39200a1d80 msgr2=0x7f39200a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.865+0000 7f3941c00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f39200a1d80 0x7f39200a4240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f392c0099a0 tx=0x7f392c009500 comp rx=0 tx=0).stop 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.865+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f393c102350 msgr2=0x7f393c073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.865+0000 7f3941c00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f393c102350 0x7f393c073650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f39240349e0 tx=0x7f3924032450 comp rx=0 tx=0).stop 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.866+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 shutdown_connections 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.866+0000 7f3941c00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f39200a1d80 0x7f39200a4240 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.866+0000 7f3941c00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393c102d10 0x7f393c073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.866+0000 7f3941c00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f393c102350 0x7f393c073650 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.866+0000 7f3941c00640 1 --2- 172.21.15.60:0/2928835688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393c101160 0x7f393c075010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.866+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 >> 172.21.15.60:0/2928835688 conn(0x7f393c0fc6d0 msgr2=0x7f393c105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.866+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 shutdown_connections 2024-02-26T10:44:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:50.866+0000 7f3941c00640 1 -- 172.21.15.60:0/2928835688 wait complete. 2024-02-26T10:44:50.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:44:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.112+0000 7fa8a3200640 1 -- 172.21.15.60:0/1126174071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8a4103380 msgr2=0x7fa8a41037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.112+0000 7fa8a3200640 1 --2- 172.21.15.60:0/1126174071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8a4103380 0x7fa8a41037a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa894009900 tx=0x7fa89402bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.112+0000 7fa8a3200640 1 -- 172.21.15.60:0/1126174071 shutdown_connections 2024-02-26T10:44:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.112+0000 7fa8a3200640 1 --2- 172.21.15.60:0/1126174071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8a4104f30 0x7fa8a410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.112+0000 7fa8a3200640 1 --2- 172.21.15.60:0/1126174071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4104570 0x7fa8a41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.112+0000 7fa8a3200640 1 --2- 172.21.15.60:0/1126174071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8a4103380 0x7fa8a41037a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.112+0000 7fa8a3200640 1 -- 172.21.15.60:0/1126174071 >> 172.21.15.60:0/1126174071 conn(0x7fa8a40fe910 msgr2=0x7fa8a4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:51.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.112+0000 7fa8a3200640 1 -- 172.21.15.60:0/1126174071 shutdown_connections 2024-02-26T10:44:51.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.112+0000 7fa8a3200640 1 -- 172.21.15.60:0/1126174071 wait complete. 2024-02-26T10:44:51.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.113+0000 7fa8a3200640 1 Processor -- start 2024-02-26T10:44:51.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a3200640 1 -- start start 2024-02-26T10:44:51.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a3200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4103380 0x7fa8a41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:51.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8a4104570 0x7fa8a4199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:51.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a3200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8a4104f30 0x7fa8a41a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:51.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a3200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a410f030 con 0x7fa8a4104f30 2024-02-26T10:44:51.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a410eeb0 con 0x7fa8a4104570 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a3200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a410f1b0 con 0x7fa8a4103380 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4103380 0x7fa8a41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa89be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8a4104570 0x7fa8a4199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa89be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8a4104570 0x7fa8a4199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49310/0 (socket says 172.21.15.60:49310) 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa89be00640 1 -- 172.21.15.60:0/450099701 learned_addr learned my addr 172.21.15.60:0/450099701 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4103380 0x7fa8a41996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:53622/0 (socket says 172.21.15.60:53622) 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.114+0000 7fa8a2800640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8a4104f30 0x7fa8a41a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa8a2800640 1 -- 172.21.15.60:0/450099701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4103380 msgr2=0x7fa8a41996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa8a2800640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4103380 0x7fa8a41996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa8a2800640 1 -- 172.21.15.60:0/450099701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8a4104570 msgr2=0x7fa8a4199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa8a2800640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8a4104570 0x7fa8a4199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa8a2800640 1 -- 172.21.15.60:0/450099701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a41a12d0 con 0x7fa8a4104f30 2024-02-26T10:44:51.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa8a2800640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8a4104f30 0x7fa8a41a0c60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009fd0 tx=0x7fa89c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa89aa00640 1 -- 172.21.15.60:0/450099701 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c00ce20 con 0x7fa8a4104f30 2024-02-26T10:44:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa89aa00640 1 -- 172.21.15.60:0/450099701 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa89c005020 con 0x7fa8a4104f30 2024-02-26T10:44:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa89aa00640 1 -- 172.21.15.60:0/450099701 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c010720 con 0x7fa8a4104f30 2024-02-26T10:44:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.115+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a41a15c0 con 0x7fa8a4104f30 2024-02-26T10:44:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.116+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a41a1b10 con 0x7fa8a4104f30 2024-02-26T10:44:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.117+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a410a6c0 con 0x7fa8a4104f30 2024-02-26T10:44:51.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.119+0000 7fa89aa00640 1 -- 172.21.15.60:0/450099701 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa89c005540 con 0x7fa8a4104f30 2024-02-26T10:44:51.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.120+0000 7fa89aa00640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8880a1e00 0x7fa8880a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:51.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.120+0000 7fa89aa00640 1 -- 172.21.15.60:0/450099701 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa89c0ce820 con 0x7fa8a4104f30 2024-02-26T10:44:51.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.120+0000 7fa8a1e00640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8880a1e00 0x7fa8880a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:51.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.121+0000 7fa8a1e00640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8880a1e00 0x7fa8880a42c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa89402c170 tx=0x7fa89402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:51.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.125+0000 7fa89aa00640 1 -- 172.21.15.60:0/450099701 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa89c09b840 con 0x7fa8a4104f30 2024-02-26T10:44:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.323+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8a4061e60 con 0x7fa8a4104f30 2024-02-26T10:44:51.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.324+0000 7fa89aa00640 1 -- 172.21.15.60:0/450099701 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa89c09b140 con 0x7fa8a4104f30 2024-02-26T10:44:51.321 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:44:51.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.327+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8880a1e00 msgr2=0x7fa8880a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.327+0000 7fa8a3200640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8880a1e00 0x7fa8880a42c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa89402c170 tx=0x7fa89402df90 comp rx=0 tx=0).stop 2024-02-26T10:44:51.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.327+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8a4104f30 msgr2=0x7fa8a41a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.327+0000 7fa8a3200640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8a4104f30 0x7fa8a41a0c60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009fd0 tx=0x7fa89c00ef10 comp rx=0 tx=0).stop 2024-02-26T10:44:51.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.327+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 shutdown_connections 2024-02-26T10:44:51.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.327+0000 7fa8a3200640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8880a1e00 0x7fa8880a42c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.327+0000 7fa8a3200640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8a4104570 0x7fa8a4199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.327+0000 7fa8a3200640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8a4104f30 0x7fa8a41a0c60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.327+0000 7fa8a3200640 1 --2- 172.21.15.60:0/450099701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8a4103380 0x7fa8a41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.328+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 >> 172.21.15.60:0/450099701 conn(0x7fa8a40fe910 msgr2=0x7fa8a4107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.328+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 shutdown_connections 2024-02-26T10:44:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.328+0000 7fa8a3200640 1 -- 172.21.15.60:0/450099701 wait complete. 2024-02-26T10:44:51.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:44:51.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.568+0000 7fc5f4a00640 1 -- 172.21.15.60:0/3514972867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 msgr2=0x7fc5f01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.568+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/3514972867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 0x7fc5f01049f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009900 tx=0x7fc5d802bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:51.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.568+0000 7fc5f4a00640 1 -- 172.21.15.60:0/3514972867 shutdown_connections 2024-02-26T10:44:51.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.568+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/3514972867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 0x7fc5f01049f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.568+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/3514972867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0104f30 0x7fc5f010b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.568+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/3514972867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f0103380 0x7fc5f01037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.568+0000 7fc5f4a00640 1 -- 172.21.15.60:0/3514972867 >> 172.21.15.60:0/3514972867 conn(0x7fc5f00fe910 msgr2=0x7fc5f0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.568+0000 7fc5f4a00640 1 -- 172.21.15.60:0/3514972867 shutdown_connections 2024-02-26T10:44:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.569+0000 7fc5f4a00640 1 -- 172.21.15.60:0/3514972867 wait complete. 2024-02-26T10:44:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.570+0000 7fc5f4a00640 1 Processor -- start 2024-02-26T10:44:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.570+0000 7fc5f4a00640 1 -- start start 2024-02-26T10:44:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.570+0000 7fc5f4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f0103380 0x7fc5f0079010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.570+0000 7fc5f4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 0x7fc5f0079550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.570+0000 7fc5f4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0104f30 0x7fc5f0075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.570+0000 7fc5f4a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f010f010 con 0x7fc5f0103380 2024-02-26T10:44:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.570+0000 7fc5f4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f010ee90 con 0x7fc5f0104570 2024-02-26T10:44:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.570+0000 7fc5f4a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f010f190 con 0x7fc5f0104f30 2024-02-26T10:44:51.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5ee800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f0103380 0x7fc5f0079010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5e7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 0x7fc5f0079550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5e7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 0x7fc5f0079550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49326/0 (socket says 172.21.15.60:49326) 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5e7e00640 1 -- 172.21.15.60:0/1404268316 learned_addr learned my addr 172.21.15.60:0/1404268316 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5ef200640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0104f30 0x7fc5f0075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5e7e00640 1 -- 172.21.15.60:0/1404268316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0104f30 msgr2=0x7fc5f0075b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5e7e00640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0104f30 0x7fc5f0075b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5e7e00640 1 -- 172.21.15.60:0/1404268316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f0103380 msgr2=0x7fc5f0079010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5e7e00640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f0103380 0x7fc5f0079010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5e7e00640 1 -- 172.21.15.60:0/1404268316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f0076440 con 0x7fc5f0104570 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.571+0000 7fc5e7e00640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 0x7fc5f0079550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009230 tx=0x7fc5d802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.572+0000 7fc5e7400640 1 -- 172.21.15.60:0/1404268316 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8041070 con 0x7fc5f0104570 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.572+0000 7fc5e7400640 1 -- 172.21.15.60:0/1404268316 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5d8004450 con 0x7fc5f0104570 2024-02-26T10:44:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.572+0000 7fc5e7400640 1 -- 172.21.15.60:0/1404268316 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8003e20 con 0x7fc5f0104570 2024-02-26T10:44:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.572+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f01a5a90 con 0x7fc5f0104570 2024-02-26T10:44:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.572+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01a5e40 con 0x7fc5f0104570 2024-02-26T10:44:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.573+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f010a6a0 con 0x7fc5f0104570 2024-02-26T10:44:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.575+0000 7fc5e7400640 1 -- 172.21.15.60:0/1404268316 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc5d8044070 con 0x7fc5f0104570 2024-02-26T10:44:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.576+0000 7fc5e7400640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc5d40a1d80 0x7fc5d40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.576+0000 7fc5e7400640 1 -- 172.21.15.60:0/1404268316 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fc5d80efd70 con 0x7fc5f0104570 2024-02-26T10:44:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.577+0000 7fc5ee800640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc5d40a1d80 0x7fc5d40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.577+0000 7fc5ee800640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc5d40a1d80 0x7fc5d40a4240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc5f01043d0 tx=0x7fc5e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:51.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.580+0000 7fc5e7400640 1 -- 172.21.15.60:0/1404268316 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc5d80bc740 con 0x7fc5f0104570 2024-02-26T10:44:51.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:51 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3715801084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:44:51.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:51 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2928835688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:44:51.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:51 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/450099701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:44:51.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:51 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3715801084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:44:51.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:51 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2928835688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:44:51.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:51 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/450099701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:44:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.781+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5f0061e60 con 0x7fc5f0104570 2024-02-26T10:44:51.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.782+0000 7fc5e7400640 1 -- 172.21.15.60:0/1404268316 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5d80c07e0 con 0x7fc5f0104570 2024-02-26T10:44:51.777 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:44:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc5d40a1d80 msgr2=0x7fc5d40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc5d40a1d80 0x7fc5d40a4240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc5f01043d0 tx=0x7fc5e0005f90 comp rx=0 tx=0).stop 2024-02-26T10:44:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 msgr2=0x7fc5f0079550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 0x7fc5f0079550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009230 tx=0x7fc5d802c650 comp rx=0 tx=0).stop 2024-02-26T10:44:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 shutdown_connections 2024-02-26T10:44:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc5d40a1d80 0x7fc5d40a4240 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f0104570 0x7fc5f0079550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0104f30 0x7fc5f0075b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 --2- 172.21.15.60:0/1404268316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f0103380 0x7fc5f0079010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.784+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 >> 172.21.15.60:0/1404268316 conn(0x7fc5f00fe910 msgr2=0x7fc5f0107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.785+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 shutdown_connections 2024-02-26T10:44:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:51.785+0000 7fc5f4a00640 1 -- 172.21.15.60:0/1404268316 wait complete. 2024-02-26T10:44:51.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:44:51.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:51 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3715801084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:44:51.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:51 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2928835688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:44:51.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:51 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/450099701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:44:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.036+0000 7f9765a00640 1 -- 172.21.15.60:0/3599542470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9760104550 msgr2=0x7f97601049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.036+0000 7f9765a00640 1 --2- 172.21.15.60:0/3599542470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9760104550 0x7f97601049d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9748009860 tx=0x7f974802bb70 comp rx=0 tx=0).stop 2024-02-26T10:44:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.036+0000 7f9765a00640 1 -- 172.21.15.60:0/3599542470 shutdown_connections 2024-02-26T10:44:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.036+0000 7f9765a00640 1 --2- 172.21.15.60:0/3599542470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9760104550 0x7f97601049d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.036+0000 7f9765a00640 1 --2- 172.21.15.60:0/3599542470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9760104f10 0x7f976010b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.036+0000 7f9765a00640 1 --2- 172.21.15.60:0/3599542470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9760103360 0x7f9760103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.036+0000 7f9765a00640 1 -- 172.21.15.60:0/3599542470 >> 172.21.15.60:0/3599542470 conn(0x7f97600fe8d0 msgr2=0x7f9760100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.036+0000 7f9765a00640 1 -- 172.21.15.60:0/3599542470 shutdown_connections 2024-02-26T10:44:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.036+0000 7f9765a00640 1 -- 172.21.15.60:0/3599542470 wait complete. 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.037+0000 7f9765a00640 1 Processor -- start 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.037+0000 7f9765a00640 1 -- start start 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.037+0000 7f9765a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9760103360 0x7f9760199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.037+0000 7f9765a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9760104550 0x7f9760199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.037+0000 7f9765a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9760104f10 0x7f97601a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.037+0000 7f9765a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976010eff0 con 0x7f9760103360 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.037+0000 7f9765a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976010ee70 con 0x7f9760104f10 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.037+0000 7f9765a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976010f170 con 0x7f9760104550 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9760103360 0x7f9760199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9760103360 0x7f9760199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34056/0 (socket says 172.21.15.60:34056) 2024-02-26T10:44:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975fe00640 1 -- 172.21.15.60:0/3670845351 learned_addr learned my addr 172.21.15.60:0/3670845351 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975f400640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9760104550 0x7f9760199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f9765000640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9760104f10 0x7f97601a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975fe00640 1 -- 172.21.15.60:0/3670845351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9760104550 msgr2=0x7f9760199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975fe00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9760104550 0x7f9760199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975fe00640 1 -- 172.21.15.60:0/3670845351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9760104f10 msgr2=0x7f97601a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975fe00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9760104f10 0x7f97601a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975fe00640 1 -- 172.21.15.60:0/3670845351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97601a1240 con 0x7f9760103360 2024-02-26T10:44:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.038+0000 7f975fe00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9760103360 0x7f9760199620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f975000e9c0 tx=0x7f975000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:52.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.039+0000 7f975cc00640 1 -- 172.21.15.60:0/3670845351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975000cd80 con 0x7f9760103360 2024-02-26T10:44:52.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.039+0000 7f975cc00640 1 -- 172.21.15.60:0/3670845351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97500035f0 con 0x7f9760103360 2024-02-26T10:44:52.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.039+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97601a1530 con 0x7f9760103360 2024-02-26T10:44:52.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.039+0000 7f975cc00640 1 -- 172.21.15.60:0/3670845351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750010690 con 0x7f9760103360 2024-02-26T10:44:52.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.039+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97601a1aa0 con 0x7f9760103360 2024-02-26T10:44:52.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.040+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976010a680 con 0x7f9760103360 2024-02-26T10:44:52.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.042+0000 7f975cc00640 1 -- 172.21.15.60:0/3670845351 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f97500037a0 con 0x7f9760103360 2024-02-26T10:44:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.043+0000 7f975cc00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97440a1e00 0x7f97440a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.044+0000 7f975cc00640 1 -- 172.21.15.60:0/3670845351 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f97500ce460 con 0x7f9760103360 2024-02-26T10:44:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.044+0000 7f975f400640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97440a1e00 0x7f97440a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.045+0000 7f975f400640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97440a1e00 0x7f97440a42c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f974802c180 tx=0x7f9748034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.048+0000 7f975cc00640 1 -- 172.21.15.60:0/3670845351 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f975009b4e0 con 0x7f9760103360 2024-02-26T10:44:52.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.249+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9760061e60 con 0x7f9760103360 2024-02-26T10:44:52.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.250+0000 7f975cc00640 1 -- 172.21.15.60:0/3670845351 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f975009ade0 con 0x7f9760103360 2024-02-26T10:44:52.246 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:44:52.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.252+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97440a1e00 msgr2=0x7f97440a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.252+0000 7f9765a00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97440a1e00 0x7f97440a42c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f974802c180 tx=0x7f9748034ad0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.252+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9760103360 msgr2=0x7f9760199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.252+0000 7f9765a00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9760103360 0x7f9760199620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f975000e9c0 tx=0x7f975000ee80 comp rx=0 tx=0).stop 2024-02-26T10:44:52.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.253+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 shutdown_connections 2024-02-26T10:44:52.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.253+0000 7f9765a00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f97440a1e00 0x7f97440a42c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.253+0000 7f9765a00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9760104f10 0x7f97601a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.253+0000 7f9765a00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9760104550 0x7f9760199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.253+0000 7f9765a00640 1 --2- 172.21.15.60:0/3670845351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9760103360 0x7f9760199620 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.253+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 >> 172.21.15.60:0/3670845351 conn(0x7f97600fe8d0 msgr2=0x7f9760107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:52.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.253+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 shutdown_connections 2024-02-26T10:44:52.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.253+0000 7f9765a00640 1 -- 172.21.15.60:0/3670845351 wait complete. 2024-02-26T10:44:52.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:44:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.499+0000 7fc3fa600640 1 -- 172.21.15.60:0/216118152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f4073cc0 msgr2=0x7fc3f4074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.499+0000 7fc3fa600640 1 --2- 172.21.15.60:0/216118152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f4073cc0 0x7fc3f4074140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009900 tx=0x7fc3dc02bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.499+0000 7fc3fa600640 1 -- 172.21.15.60:0/216118152 shutdown_connections 2024-02-26T10:44:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.499+0000 7fc3fa600640 1 --2- 172.21.15.60:0/216118152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3f410d840 0x7fc3f410fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.499+0000 7fc3fa600640 1 --2- 172.21.15.60:0/216118152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f4073cc0 0x7fc3f4074140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.499+0000 7fc3fa600640 1 --2- 172.21.15.60:0/216118152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f40752a0 0x7fc3f40736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.499+0000 7fc3fa600640 1 -- 172.21.15.60:0/216118152 >> 172.21.15.60:0/216118152 conn(0x7fc3f40fc840 msgr2=0x7fc3f40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.499+0000 7fc3fa600640 1 -- 172.21.15.60:0/216118152 shutdown_connections 2024-02-26T10:44:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.499+0000 7fc3fa600640 1 -- 172.21.15.60:0/216118152 wait complete. 2024-02-26T10:44:52.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.500+0000 7fc3fa600640 1 Processor -- start 2024-02-26T10:44:52.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.500+0000 7fc3fa600640 1 -- start start 2024-02-26T10:44:52.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.500+0000 7fc3fa600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4073cc0 0x7fc3f419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3fa600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3f40752a0 0x7fc3f419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3fa600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f410d840 0x7fc3f41a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3fa600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f4113530 con 0x7fc3f40752a0 2024-02-26T10:44:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3fa600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41133b0 con 0x7fc3f4073cc0 2024-02-26T10:44:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3fa600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41136b0 con 0x7fc3f410d840 2024-02-26T10:44:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3f3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4073cc0 0x7fc3f419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3f3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4073cc0 0x7fc3f419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49364/0 (socket says 172.21.15.60:49364) 2024-02-26T10:44:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3f40752a0 0x7fc3f419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3f3400640 1 -- 172.21.15.60:0/634969375 learned_addr learned my addr 172.21.15.60:0/634969375 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3f3e00640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f410d840 0x7fc3f41a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3f3e00640 1 -- 172.21.15.60:0/634969375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4073cc0 msgr2=0x7fc3f419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3f3e00640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4073cc0 0x7fc3f419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3f3e00640 1 -- 172.21.15.60:0/634969375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3f40752a0 msgr2=0x7fc3f419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3f3e00640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3f40752a0 0x7fc3f419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.501+0000 7fc3f3e00640 1 -- 172.21.15.60:0/634969375 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f41a57f0 con 0x7fc3f410d840 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.502+0000 7fc3f3e00640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f410d840 0x7fc3f41a5180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00da10 tx=0x7fc3ec00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.502+0000 7fc3f0c00640 1 -- 172.21.15.60:0/634969375 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00bd70 con 0x7fc3f410d840 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.502+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f41a5ae0 con 0x7fc3f410d840 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.502+0000 7fc3f0c00640 1 -- 172.21.15.60:0/634969375 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3ec003e50 con 0x7fc3f410d840 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.502+0000 7fc3f0c00640 1 -- 172.21.15.60:0/634969375 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0106a0 con 0x7fc3f410d840 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.502+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f41a6000 con 0x7fc3f410d840 2024-02-26T10:44:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.503+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f4103900 con 0x7fc3f410d840 2024-02-26T10:44:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.505+0000 7fc3f0c00640 1 -- 172.21.15.60:0/634969375 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc3ec003970 con 0x7fc3f410d840 2024-02-26T10:44:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.507+0000 7fc3f0c00640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3d80a1d80 0x7fc3d80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.507+0000 7fc3f0c00640 1 -- 172.21.15.60:0/634969375 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fc3ec0ce500 con 0x7fc3f410d840 2024-02-26T10:44:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.507+0000 7fc3f3400640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3d80a1d80 0x7fc3d80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.507+0000 7fc3f3400640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3d80a1d80 0x7fc3d80a4240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009cd0 tx=0x7fc3e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:52.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.511+0000 7fc3f0c00640 1 -- 172.21.15.60:0/634969375 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3ec09b580 con 0x7fc3f410d840 2024-02-26T10:44:52.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:52 smithi060 ceph-mon[31539]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-26T10:44:52.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:52 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1404268316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:44:52.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:52 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3670845351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:44:52.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:52 smithi100 ceph-mon[36051]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-26T10:44:52.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:52 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1404268316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:44:52.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:52 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3670845351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:44:52.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.754+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3f4061e60 con 0x7fc3f410d840 2024-02-26T10:44:52.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.755+0000 7fc3f0c00640 1 -- 172.21.15.60:0/634969375 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11359 (secure 0 0 0) 0x7fc3ec09ae80 con 0x7fc3f410d840 2024-02-26T10:44:52.752 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:44:52.752 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:43:16.029212+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:44:52.752 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3d80a1d80 msgr2=0x7fc3d80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3d80a1d80 0x7fc3d80a4240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009cd0 tx=0x7fc3e4005f90 comp rx=0 tx=0).stop 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f410d840 msgr2=0x7fc3f41a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f410d840 0x7fc3f41a5180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00da10 tx=0x7fc3ec00ded0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 shutdown_connections 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc3d80a1d80 0x7fc3d80a4240 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f410d840 0x7fc3f41a5180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3f40752a0 0x7fc3f419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 --2- 172.21.15.60:0/634969375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4073cc0 0x7fc3f419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 >> 172.21.15.60:0/634969375 conn(0x7fc3f40fc840 msgr2=0x7fc3f4100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 shutdown_connections 2024-02-26T10:44:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.759+0000 7fc3fa600640 1 -- 172.21.15.60:0/634969375 wait complete. 2024-02-26T10:44:52.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:44:52.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:52 smithi113 ceph-mon[36126]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-26T10:44:52.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:52 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1404268316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:44:52.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:52 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3670845351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:44:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.995+0000 7f9292c00640 1 -- 172.21.15.60:0/2375567887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c1044a0 msgr2=0x7f928c104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.995+0000 7f9292c00640 1 --2- 172.21.15.60:0/2375567887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c1044a0 0x7f928c104920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9278009900 tx=0x7f927802bc40 comp rx=0 tx=0).stop 2024-02-26T10:44:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.995+0000 7f9292c00640 1 -- 172.21.15.60:0/2375567887 shutdown_connections 2024-02-26T10:44:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.995+0000 7f9292c00640 1 --2- 172.21.15.60:0/2375567887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c1044a0 0x7f928c104920 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.995+0000 7f9292c00640 1 --2- 172.21.15.60:0/2375567887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928c104e60 0x7f928c10b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.995+0000 7f9292c00640 1 --2- 172.21.15.60:0/2375567887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928c1032b0 0x7f928c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.995+0000 7f9292c00640 1 -- 172.21.15.60:0/2375567887 >> 172.21.15.60:0/2375567887 conn(0x7f928c0fe840 msgr2=0x7f928c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.995+0000 7f9292c00640 1 -- 172.21.15.60:0/2375567887 shutdown_connections 2024-02-26T10:44:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.995+0000 7f9292c00640 1 -- 172.21.15.60:0/2375567887 wait complete. 2024-02-26T10:44:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.996+0000 7f9292c00640 1 Processor -- start 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292c00640 1 -- start start 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928c1032b0 0x7f928c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928c1044a0 0x7f928c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c104e60 0x7f928c1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c10ef40 con 0x7f928c1044a0 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c10edc0 con 0x7f928c104e60 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c10f0c0 con 0x7f928c1032b0 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c104e60 0x7f928c1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9290e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928c1044a0 0x7f928c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c104e60 0x7f928c1a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49376/0 (socket says 172.21.15.60:49376) 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9292200640 1 -- 172.21.15.60:0/304080768 learned_addr learned my addr 172.21.15.60:0/304080768 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.997+0000 7f9291800640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928c1032b0 0x7f928c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f9292200640 1 -- 172.21.15.60:0/304080768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928c1032b0 msgr2=0x7f928c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f9292200640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928c1032b0 0x7f928c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f9292200640 1 -- 172.21.15.60:0/304080768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928c1044a0 msgr2=0x7f928c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:52.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f9292200640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928c1044a0 0x7f928c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:52.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f9292200640 1 -- 172.21.15.60:0/304080768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c1a1160 con 0x7f928c104e60 2024-02-26T10:44:52.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f9291800640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928c1032b0 0x7f928c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:44:52.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f9292200640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c104e60 0x7f928c1a0af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f928400eaa0 tx=0x7f928400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:52.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f928a000640 1 -- 172.21.15.60:0/304080768 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928400cdb0 con 0x7f928c104e60 2024-02-26T10:44:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f928a000640 1 -- 172.21.15.60:0/304080768 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9284005020 con 0x7f928c104e60 2024-02-26T10:44:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f928a000640 1 -- 172.21.15.60:0/304080768 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92840116b0 con 0x7f928c104e60 2024-02-26T10:44:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.998+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928c1a1450 con 0x7f928c104e60 2024-02-26T10:44:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:52.999+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c1a1920 con 0x7f928c104e60 2024-02-26T10:44:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.000+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928c10a5d0 con 0x7f928c104e60 2024-02-26T10:44:52.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.002+0000 7f928a000640 1 -- 172.21.15.60:0/304080768 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9284005560 con 0x7f928c104e60 2024-02-26T10:44:52.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.003+0000 7f928a000640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f92740a1d80 0x7f92740a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:52.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.003+0000 7f928a000640 1 -- 172.21.15.60:0/304080768 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f92840ce440 con 0x7f928c104e60 2024-02-26T10:44:52.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.003+0000 7f9291800640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f92740a1d80 0x7f92740a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.004+0000 7f9291800640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f92740a1d80 0x7f92740a4240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f928c104300 tx=0x7f9280005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:53.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.007+0000 7f928a000640 1 -- 172.21.15.60:0/304080768 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9284016030 con 0x7f928c104e60 2024-02-26T10:44:53.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.208+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f928c061e60 con 0x7f928c104e60 2024-02-26T10:44:53.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.211+0000 7f928a000640 1 -- 172.21.15.60:0/304080768 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7f928409b460 con 0x7f928c104e60 2024-02-26T10:44:53.207 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:44:53.207 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:44:53.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f92740a1d80 msgr2=0x7f92740a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f92740a1d80 0x7f92740a4240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f928c104300 tx=0x7f9280005f90 comp rx=0 tx=0).stop 2024-02-26T10:44:53.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c104e60 msgr2=0x7f928c1a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c104e60 0x7f928c1a0af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f928400eaa0 tx=0x7f928400ef60 comp rx=0 tx=0).stop 2024-02-26T10:44:53.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 shutdown_connections 2024-02-26T10:44:53.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f92740a1d80 0x7f92740a4240 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928c104e60 0x7f928c1a0af0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928c1044a0 0x7f928c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 --2- 172.21.15.60:0/304080768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f928c1032b0 0x7f928c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.213+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 >> 172.21.15.60:0/304080768 conn(0x7f928c0fe840 msgr2=0x7f928c1077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:53.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.214+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 shutdown_connections 2024-02-26T10:44:53.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.214+0000 7f9292c00640 1 -- 172.21.15.60:0/304080768 wait complete. 2024-02-26T10:44:53.227 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:44:53.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:44:53.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.460+0000 7fbc6e200640 1 -- 172.21.15.60:0/1772479703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 msgr2=0x7fbc681068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.460+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1772479703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 0x7fbc681068e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009900 tx=0x7fbc5002bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:53.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.460+0000 7fbc6e200640 1 -- 172.21.15.60:0/1772479703 shutdown_connections 2024-02-26T10:44:53.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.460+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1772479703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 0x7fbc681068e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.460+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1772479703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc68106e20 0x7fbc68109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.460+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1772479703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc68101bc0 0x7fbc68103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.460+0000 7fbc6e200640 1 -- 172.21.15.60:0/1772479703 >> 172.21.15.60:0/1772479703 conn(0x7fbc680fb5d0 msgr2=0x7fbc680fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:53.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.461+0000 7fbc6e200640 1 -- 172.21.15.60:0/1772479703 shutdown_connections 2024-02-26T10:44:53.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.461+0000 7fbc6e200640 1 -- 172.21.15.60:0/1772479703 wait complete. 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.462+0000 7fbc6e200640 1 Processor -- start 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.462+0000 7fbc6e200640 1 -- start start 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.462+0000 7fbc6e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc68101bc0 0x7fbc68197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.462+0000 7fbc6e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 0x7fbc68197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.462+0000 7fbc6e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc68106e20 0x7fbc6819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.462+0000 7fbc6e200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6810cdd0 con 0x7fbc68101bc0 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.462+0000 7fbc6e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6810cc50 con 0x7fbc681044f0 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.462+0000 7fbc6e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6810cf50 con 0x7fbc68106e20 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc67e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 0x7fbc68197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc6ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc68101bc0 0x7fbc68197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc67e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 0x7fbc68197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49386/0 (socket says 172.21.15.60:49386) 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc67e00640 1 -- 172.21.15.60:0/1579242232 learned_addr learned my addr 172.21.15.60:0/1579242232 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:53.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc6d800640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc68106e20 0x7fbc6819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:53.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc67e00640 1 -- 172.21.15.60:0/1579242232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc68106e20 msgr2=0x7fbc6819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc67e00640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc68106e20 0x7fbc6819ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc67e00640 1 -- 172.21.15.60:0/1579242232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc68101bc0 msgr2=0x7fbc68197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc67e00640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc68101bc0 0x7fbc68197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc67e00640 1 -- 172.21.15.60:0/1579242232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6819f070 con 0x7fbc681044f0 2024-02-26T10:44:53.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.463+0000 7fbc67e00640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 0x7fbc68197990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc50000c00 tx=0x7fbc5002c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:53.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.464+0000 7fbc65600640 1 -- 172.21.15.60:0/1579242232 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50041070 con 0x7fbc681044f0 2024-02-26T10:44:53.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.464+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6819f300 con 0x7fbc681044f0 2024-02-26T10:44:53.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.464+0000 7fbc65600640 1 -- 172.21.15.60:0/1579242232 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc50004450 con 0x7fbc681044f0 2024-02-26T10:44:53.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.464+0000 7fbc65600640 1 -- 172.21.15.60:0/1579242232 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50034440 con 0x7fbc681044f0 2024-02-26T10:44:53.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.464+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6819f7f0 con 0x7fbc681044f0 2024-02-26T10:44:53.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.465+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc680fd0c0 con 0x7fbc681044f0 2024-02-26T10:44:53.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.467+0000 7fbc65600640 1 -- 172.21.15.60:0/1579242232 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbc50044070 con 0x7fbc681044f0 2024-02-26T10:44:53.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.468+0000 7fbc65600640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbc4c0a1d80 0x7fbc4c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:53.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.468+0000 7fbc65600640 1 -- 172.21.15.60:0/1579242232 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fbc500efc30 con 0x7fbc681044f0 2024-02-26T10:44:53.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.468+0000 7fbc6ce00640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbc4c0a1d80 0x7fbc4c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:53.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.469+0000 7fbc6ce00640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbc4c0a1d80 0x7fbc4c0a4240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc58005fd0 tx=0x7fbc58009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:53.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.472+0000 7fbc65600640 1 -- 172.21.15.60:0/1579242232 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc500bc690 con 0x7fbc681044f0 2024-02-26T10:44:53.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:53 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/634969375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:44:53.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:53 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/304080768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:44:53.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:53 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/634969375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:44:53.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:53 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/304080768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:44:53.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.662+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc68061e60 con 0x7fbc681044f0 2024-02-26T10:44:53.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.663+0000 7fbc65600640 1 -- 172.21.15.60:0/1579242232 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc500c0730 con 0x7fbc681044f0 2024-02-26T10:44:53.659 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:44:53.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbc4c0a1d80 msgr2=0x7fbc4c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbc4c0a1d80 0x7fbc4c0a4240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc58005fd0 tx=0x7fbc58009500 comp rx=0 tx=0).stop 2024-02-26T10:44:53.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 msgr2=0x7fbc68197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 0x7fbc68197990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc50000c00 tx=0x7fbc5002c640 comp rx=0 tx=0).stop 2024-02-26T10:44:53.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 shutdown_connections 2024-02-26T10:44:53.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbc4c0a1d80 0x7fbc4c0a4240 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc681044f0 0x7fbc68197990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc68106e20 0x7fbc6819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 --2- 172.21.15.60:0/1579242232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc68101bc0 0x7fbc68197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 >> 172.21.15.60:0/1579242232 conn(0x7fbc680fb5d0 msgr2=0x7fbc68102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:53.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 shutdown_connections 2024-02-26T10:44:53.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.666+0000 7fbc6e200640 1 -- 172.21.15.60:0/1579242232 wait complete. 2024-02-26T10:44:53.681 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:44:53.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:53 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/634969375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:44:53.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:53 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/304080768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:44:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.908+0000 7fb3fde00640 1 -- 172.21.15.60:0/1554448873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8109430 msgr2=0x7fb3f810fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.908+0000 7fb3fde00640 1 --2- 172.21.15.60:0/1554448873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8109430 0x7fb3f810fcc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009ca0 tx=0x7fb3f002c100 comp rx=0 tx=0).stop 2024-02-26T10:44:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.908+0000 7fb3fde00640 1 -- 172.21.15.60:0/1554448873 shutdown_connections 2024-02-26T10:44:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.908+0000 7fb3fde00640 1 --2- 172.21.15.60:0/1554448873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8109430 0x7fb3f810fcc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.908+0000 7fb3fde00640 1 --2- 172.21.15.60:0/1554448873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8108a70 0x7fb3f8108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.908+0000 7fb3fde00640 1 --2- 172.21.15.60:0/1554448873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f8107880 0x7fb3f8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.908+0000 7fb3fde00640 1 -- 172.21.15.60:0/1554448873 >> 172.21.15.60:0/1554448873 conn(0x7fb3f8076820 msgr2=0x7fb3f8078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:53.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.908+0000 7fb3fde00640 1 -- 172.21.15.60:0/1554448873 shutdown_connections 2024-02-26T10:44:53.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.908+0000 7fb3fde00640 1 -- 172.21.15.60:0/1554448873 wait complete. 2024-02-26T10:44:53.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.910+0000 7fb3fde00640 1 Processor -- start 2024-02-26T10:44:53.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.910+0000 7fb3fde00640 1 -- start start 2024-02-26T10:44:53.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.910+0000 7fb3fde00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8107880 0x7fb3f819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.910+0000 7fb3fde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8108a70 0x7fb3f819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.910+0000 7fb3fde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f8109430 0x7fb3f81a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.910+0000 7fb3fde00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8113500 con 0x7fb3f8107880 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.910+0000 7fb3fde00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8113380 con 0x7fb3f8109430 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.910+0000 7fb3fde00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8113680 con 0x7fb3f8108a70 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8107880 0x7fb3f819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8107880 0x7fb3f819db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35188/0 (socket says 172.21.15.60:35188) 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f6a00640 1 -- 172.21.15.60:0/2115785413 learned_addr learned my addr 172.21.15.60:0/2115785413 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f6000640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8108a70 0x7fb3f819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f7400640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f8109430 0x7fb3f81a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f6a00640 1 -- 172.21.15.60:0/2115785413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8108a70 msgr2=0x7fb3f819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f6a00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8108a70 0x7fb3f819e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f6a00640 1 -- 172.21.15.60:0/2115785413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f8109430 msgr2=0x7fb3f81a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f6a00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f8109430 0x7fb3f81a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.911+0000 7fb3f6a00640 1 -- 172.21.15.60:0/2115785413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f81a5740 con 0x7fb3f8107880 2024-02-26T10:44:53.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.912+0000 7fb3f6a00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8107880 0x7fb3f819db20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800d770 tx=0x7fb3e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:53.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.912+0000 7fb3fca00640 1 -- 172.21.15.60:0/2115785413 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8017070 con 0x7fb3f8107880 2024-02-26T10:44:53.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.912+0000 7fb3fca00640 1 -- 172.21.15.60:0/2115785413 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3e8009c30 con 0x7fb3f8107880 2024-02-26T10:44:53.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.912+0000 7fb3fca00640 1 -- 172.21.15.60:0/2115785413 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8003dd0 con 0x7fb3f8107880 2024-02-26T10:44:53.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.912+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f81a5a30 con 0x7fb3f8107880 2024-02-26T10:44:53.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.912+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f81a5f80 con 0x7fb3f8107880 2024-02-26T10:44:53.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.913+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f810eb40 con 0x7fb3f8107880 2024-02-26T10:44:53.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.916+0000 7fb3fca00640 1 -- 172.21.15.60:0/2115785413 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb3e8003f70 con 0x7fb3f8107880 2024-02-26T10:44:53.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.918+0000 7fb3fca00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb3dc0a1e00 0x7fb3dc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:53.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.918+0000 7fb3fca00640 1 -- 172.21.15.60:0/2115785413 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb3e80cda00 con 0x7fb3f8107880 2024-02-26T10:44:53.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.918+0000 7fb3f6000640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb3dc0a1e00 0x7fb3dc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:53.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.919+0000 7fb3f6000640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb3dc0a1e00 0x7fb3dc0a42c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb3f819f030 tx=0x7fb3e400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:53.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:53.923+0000 7fb3fca00640 1 -- 172.21.15.60:0/2115785413 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb3e8009de0 con 0x7fb3f8107880 2024-02-26T10:44:54.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.116+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3f802d0b0 con 0x7fb3f8107880 2024-02-26T10:44:54.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.117+0000 7fb3fca00640 1 -- 172.21.15.60:0/2115785413 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3e809a3d0 con 0x7fb3f8107880 2024-02-26T10:44:54.113 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:44:54.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb3dc0a1e00 msgr2=0x7fb3dc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb3dc0a1e00 0x7fb3dc0a42c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb3f819f030 tx=0x7fb3e400b390 comp rx=0 tx=0).stop 2024-02-26T10:44:54.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8107880 msgr2=0x7fb3f819db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8107880 0x7fb3f819db20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800d770 tx=0x7fb3e800dc30 comp rx=0 tx=0).stop 2024-02-26T10:44:54.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 shutdown_connections 2024-02-26T10:44:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb3dc0a1e00 0x7fb3dc0a42c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f8109430 0x7fb3f81a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8108a70 0x7fb3f819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 --2- 172.21.15.60:0/2115785413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f8107880 0x7fb3f819db20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.120+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 >> 172.21.15.60:0/2115785413 conn(0x7fb3f8076820 msgr2=0x7fb3f810bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.121+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 shutdown_connections 2024-02-26T10:44:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.121+0000 7fb3fde00640 1 -- 172.21.15.60:0/2115785413 wait complete. 2024-02-26T10:44:54.133 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:44:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.361+0000 7f55bda00640 1 -- 172.21.15.60:0/2034252533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 msgr2=0x7f55b8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.361+0000 7f55bda00640 1 --2- 172.21.15.60:0/2034252533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 0x7f55b8107ba0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009900 tx=0x7f55a802bc50 comp rx=0 tx=0).stop 2024-02-26T10:44:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.362+0000 7f55bda00640 1 -- 172.21.15.60:0/2034252533 shutdown_connections 2024-02-26T10:44:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.362+0000 7f55bda00640 1 --2- 172.21.15.60:0/2034252533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55b8108970 0x7f55b8108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.362+0000 7f55bda00640 1 --2- 172.21.15.60:0/2034252533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55b8109330 0x7f55b810fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.362+0000 7f55bda00640 1 --2- 172.21.15.60:0/2034252533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 0x7f55b8107ba0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.362+0000 7f55bda00640 1 -- 172.21.15.60:0/2034252533 >> 172.21.15.60:0/2034252533 conn(0x7f55b8076730 msgr2=0x7f55b8078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.362+0000 7f55bda00640 1 -- 172.21.15.60:0/2034252533 shutdown_connections 2024-02-26T10:44:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.362+0000 7f55bda00640 1 -- 172.21.15.60:0/2034252533 wait complete. 2024-02-26T10:44:54.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55bda00640 1 Processor -- start 2024-02-26T10:44:54.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55bda00640 1 -- start start 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55bda00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 0x7f55b819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55bda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55b8108970 0x7f55b819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55bda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55b8109330 0x7f55b81a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55bda00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b81133d0 con 0x7f55b8107780 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55bda00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8113250 con 0x7f55b8108970 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55bda00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8113550 con 0x7f55b8109330 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55b7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 0x7f55b819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55b7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 0x7f55b819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35200/0 (socket says 172.21.15.60:35200) 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55b7e00640 1 -- 172.21.15.60:0/193049184 learned_addr learned my addr 172.21.15.60:0/193049184 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55bd000640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55b8109330 0x7f55b81a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.363+0000 7f55b7400640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55b8108970 0x7f55b819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55bd000640 1 -- 172.21.15.60:0/193049184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55b8108970 msgr2=0x7f55b819e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55bd000640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55b8108970 0x7f55b819e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55bd000640 1 -- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 msgr2=0x7f55b819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55bd000640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 0x7f55b819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55bd000640 1 -- 172.21.15.60:0/193049184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b81a5730 con 0x7f55b8109330 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55b7e00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 0x7f55b819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55bd000640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55b8109330 0x7f55b81a50c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e940 tx=0x7f55b000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55b4c00640 1 -- 172.21.15.60:0/193049184 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0004d20 con 0x7f55b8109330 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55b4c00640 1 -- 172.21.15.60:0/193049184 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55b0005020 con 0x7f55b8109330 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.364+0000 7f55b4c00640 1 -- 172.21.15.60:0/193049184 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00116b0 con 0x7f55b8109330 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.365+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b81a5a20 con 0x7f55b8109330 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.365+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b81a5f20 con 0x7f55b8109330 2024-02-26T10:44:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.366+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b810eaa0 con 0x7f55b8109330 2024-02-26T10:44:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.367+0000 7f55b4c00640 1 -- 172.21.15.60:0/193049184 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f55b00054d0 con 0x7f55b8109330 2024-02-26T10:44:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.369+0000 7f55b4c00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f559c0a1e00 0x7f559c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.369+0000 7f55b4c00640 1 -- 172.21.15.60:0/193049184 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f55b00ce190 con 0x7f55b8109330 2024-02-26T10:44:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.369+0000 7f55b7e00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f559c0a1e00 0x7f559c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.370+0000 7f55b7e00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f559c0a1e00 0x7f559c0a42c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f55a802c900 tx=0x7f55a8005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.373+0000 7f55b4c00640 1 -- 172.21.15.60:0/193049184 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55b0016030 con 0x7f55b8109330 2024-02-26T10:44:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.575+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55b8061e60 con 0x7f55b8109330 2024-02-26T10:44:54.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.576+0000 7f55b4c00640 1 -- 172.21.15.60:0/193049184 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55b009ab60 con 0x7f55b8109330 2024-02-26T10:44:54.571 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.578+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f559c0a1e00 msgr2=0x7f559c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.578+0000 7f55bda00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f559c0a1e00 0x7f559c0a42c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f55a802c900 tx=0x7f55a8005b50 comp rx=0 tx=0).stop 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.578+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55b8109330 msgr2=0x7f55b81a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.578+0000 7f55bda00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55b8109330 0x7f55b81a50c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e940 tx=0x7f55b000ee00 comp rx=0 tx=0).stop 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.579+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 shutdown_connections 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.579+0000 7f55bda00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f559c0a1e00 0x7f559c0a42c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.579+0000 7f55bda00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55b8108970 0x7f55b819e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.579+0000 7f55bda00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55b8109330 0x7f55b81a50c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.579+0000 7f55bda00640 1 --2- 172.21.15.60:0/193049184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55b8107780 0x7f55b819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.579+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 >> 172.21.15.60:0/193049184 conn(0x7f55b8076730 msgr2=0x7f55b810bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.579+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 shutdown_connections 2024-02-26T10:44:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.579+0000 7f55bda00640 1 -- 172.21.15.60:0/193049184 wait complete. 2024-02-26T10:44:54.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:54 smithi060 ceph-mon[31539]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:54.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:54 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1579242232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:44:54.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:54 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2115785413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:44:54.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:44:54.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:54 smithi100 ceph-mon[36051]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:54.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:54 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1579242232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:44:54.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:54 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2115785413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:44:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.821+0000 7facafe00640 1 -- 172.21.15.60:0/1562830063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 msgr2=0x7facb0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.821+0000 7facafe00640 1 --2- 172.21.15.60:0/1562830063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 0x7facb0101560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faca0009900 tx=0x7faca002bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.821+0000 7facafe00640 1 -- 172.21.15.60:0/1562830063 shutdown_connections 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.821+0000 7facafe00640 1 --2- 172.21.15.60:0/1562830063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb0102330 0x7facb01027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.821+0000 7facafe00640 1 --2- 172.21.15.60:0/1562830063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facb0102cf0 0x7facb0109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.821+0000 7facafe00640 1 --2- 172.21.15.60:0/1562830063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 0x7facb0101560 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.821+0000 7facafe00640 1 -- 172.21.15.60:0/1562830063 >> 172.21.15.60:0/1562830063 conn(0x7facb00fc6d0 msgr2=0x7facb00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.821+0000 7facafe00640 1 -- 172.21.15.60:0/1562830063 shutdown_connections 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.821+0000 7facafe00640 1 -- 172.21.15.60:0/1562830063 wait complete. 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.822+0000 7facafe00640 1 Processor -- start 2024-02-26T10:44:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.822+0000 7facafe00640 1 -- start start 2024-02-26T10:44:54.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.822+0000 7facafe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 0x7facb0197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:54.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.822+0000 7facafe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facb0102330 0x7facb01979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:54.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.822+0000 7facafe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb0102cf0 0x7facb019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:54.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facafe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb010cdd0 con 0x7facb0101140 2024-02-26T10:44:54.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facafe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb010cc50 con 0x7facb0102cf0 2024-02-26T10:44:54.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facafe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb010cf50 con 0x7facb0102330 2024-02-26T10:44:54.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facaea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 0x7facb0197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facaea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 0x7facb0197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35218/0 (socket says 172.21.15.60:35218) 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facaea00640 1 -- 172.21.15.60:0/2147401670 learned_addr learned my addr 172.21.15.60:0/2147401670 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facaf400640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb0102cf0 0x7facb019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7faca7e00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facb0102330 0x7facb01979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facaea00640 1 -- 172.21.15.60:0/2147401670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facb0102330 msgr2=0x7facb01979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facaea00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facb0102330 0x7facb01979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facaea00640 1 -- 172.21.15.60:0/2147401670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb0102cf0 msgr2=0x7facb019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facaea00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb0102cf0 0x7facb019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.823+0000 7facaea00640 1 -- 172.21.15.60:0/2147401670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb019f080 con 0x7facb0101140 2024-02-26T10:44:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.824+0000 7facaea00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 0x7facb0197460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faca0000c00 tx=0x7faca002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:54.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.824+0000 7faca7400640 1 -- 172.21.15.60:0/2147401670 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0041070 con 0x7facb0101140 2024-02-26T10:44:54.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.824+0000 7faca7400640 1 -- 172.21.15.60:0/2147401670 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faca0004450 con 0x7facb0101140 2024-02-26T10:44:54.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.824+0000 7faca7400640 1 -- 172.21.15.60:0/2147401670 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0034440 con 0x7facb0101140 2024-02-26T10:44:54.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.824+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb019f310 con 0x7facb0101140 2024-02-26T10:44:54.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.824+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb019f7b0 con 0x7facb0101140 2024-02-26T10:44:54.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.825+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb0108460 con 0x7facb0101140 2024-02-26T10:44:54.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.827+0000 7faca7400640 1 -- 172.21.15.60:0/2147401670 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7faca0044070 con 0x7facb0101140 2024-02-26T10:44:54.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.829+0000 7faca7400640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac940a1d80 0x7fac940a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:54.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.829+0000 7faca7400640 1 -- 172.21.15.60:0/2147401670 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7faca00efc30 con 0x7facb0101140 2024-02-26T10:44:54.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.829+0000 7faca7e00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac940a1d80 0x7fac940a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:54.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.830+0000 7faca7e00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac940a1d80 0x7fac940a4240 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac980095f0 tx=0x7fac98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:54.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:54.833+0000 7faca7400640 1 -- 172.21.15.60:0/2147401670 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faca00bc690 con 0x7facb0101140 2024-02-26T10:44:54.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:54 smithi113 ceph-mon[36126]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:44:54.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:54 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1579242232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:44:54.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:54 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2115785413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:44:55.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.033+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7facb0061e60 con 0x7facb0101140 2024-02-26T10:44:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.033+0000 7faca7400640 1 -- 172.21.15.60:0/2147401670 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7faca00c0730 con 0x7facb0101140 2024-02-26T10:44:55.029 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:44:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac940a1d80 msgr2=0x7fac940a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac940a1d80 0x7fac940a4240 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac980095f0 tx=0x7fac98005f90 comp rx=0 tx=0).stop 2024-02-26T10:44:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 msgr2=0x7facb0197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 0x7facb0197460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faca0000c00 tx=0x7faca002c640 comp rx=0 tx=0).stop 2024-02-26T10:44:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 shutdown_connections 2024-02-26T10:44:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac940a1d80 0x7fac940a4240 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facb0102cf0 0x7facb019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facb0102330 0x7facb01979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 --2- 172.21.15.60:0/2147401670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb0101140 0x7facb0197460 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 >> 172.21.15.60:0/2147401670 conn(0x7facb00fc6d0 msgr2=0x7facb0105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.036+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 shutdown_connections 2024-02-26T10:44:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.037+0000 7facafe00640 1 -- 172.21.15.60:0/2147401670 wait complete. 2024-02-26T10:44:55.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:44:55.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.291+0000 7fc9d3200640 1 -- 172.21.15.60:0/83310048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d41032b0 msgr2=0x7fc9d41036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.291+0000 7fc9d3200640 1 --2- 172.21.15.60:0/83310048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d41032b0 0x7fc9d41036d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009900 tx=0x7fc9c402bc40 comp rx=0 tx=0).stop 2024-02-26T10:44:55.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.292+0000 7fc9d3200640 1 -- 172.21.15.60:0/83310048 shutdown_connections 2024-02-26T10:44:55.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.292+0000 7fc9d3200640 1 --2- 172.21.15.60:0/83310048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41044a0 0x7fc9d4104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.292+0000 7fc9d3200640 1 --2- 172.21.15.60:0/83310048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d4104e60 0x7fc9d410b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.292+0000 7fc9d3200640 1 --2- 172.21.15.60:0/83310048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d41032b0 0x7fc9d41036d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.292+0000 7fc9d3200640 1 -- 172.21.15.60:0/83310048 >> 172.21.15.60:0/83310048 conn(0x7fc9d40fe840 msgr2=0x7fc9d4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:55.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.292+0000 7fc9d3200640 1 -- 172.21.15.60:0/83310048 shutdown_connections 2024-02-26T10:44:55.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.292+0000 7fc9d3200640 1 -- 172.21.15.60:0/83310048 wait complete. 2024-02-26T10:44:55.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.293+0000 7fc9d3200640 1 Processor -- start 2024-02-26T10:44:55.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.293+0000 7fc9d3200640 1 -- start start 2024-02-26T10:44:55.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.293+0000 7fc9d3200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d41032b0 0x7fc9d41995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:55.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.293+0000 7fc9d3200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d41044a0 0x7fc9d4199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:55.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.293+0000 7fc9d3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d4104e60 0x7fc9d41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:55.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.293+0000 7fc9d3200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d410ef90 con 0x7fc9d41044a0 2024-02-26T10:44:55.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.293+0000 7fc9d3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d410ee10 con 0x7fc9d4104e60 2024-02-26T10:44:55.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.293+0000 7fc9d3200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d410f110 con 0x7fc9d41032b0 2024-02-26T10:44:55.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d41044a0 0x7fc9d4199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:55.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9d1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d41032b0 0x7fc9d41995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:55.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d41044a0 0x7fc9d4199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35222/0 (socket says 172.21.15.60:35222) 2024-02-26T10:44:55.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9cbe00640 1 -- 172.21.15.60:0/4092069531 learned_addr learned my addr 172.21.15.60:0/4092069531 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9d2800640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d4104e60 0x7fc9d41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9d1e00640 1 -- 172.21.15.60:0/4092069531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d4104e60 msgr2=0x7fc9d41a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9d1e00640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d4104e60 0x7fc9d41a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9d1e00640 1 -- 172.21.15.60:0/4092069531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d41044a0 msgr2=0x7fc9d4199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9d1e00640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d41044a0 0x7fc9d4199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9d1e00640 1 -- 172.21.15.60:0/4092069531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d41a1200 con 0x7fc9d41032b0 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.294+0000 7fc9d1e00640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d41032b0 0x7fc9d41995e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009230 tx=0x7fc9c402c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.295+0000 7fc9caa00640 1 -- 172.21.15.60:0/4092069531 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4041070 con 0x7fc9d41032b0 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.295+0000 7fc9caa00640 1 -- 172.21.15.60:0/4092069531 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9c40054e0 con 0x7fc9d41032b0 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.295+0000 7fc9caa00640 1 -- 172.21.15.60:0/4092069531 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4003e40 con 0x7fc9d41032b0 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.295+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d41a1490 con 0x7fc9d41032b0 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.295+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d41a1900 con 0x7fc9d41032b0 2024-02-26T10:44:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.296+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d410a5d0 con 0x7fc9d41032b0 2024-02-26T10:44:55.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.298+0000 7fc9caa00640 1 -- 172.21.15.60:0/4092069531 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc9c4044070 con 0x7fc9d41032b0 2024-02-26T10:44:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.299+0000 7fc9caa00640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b80a1d80 0x7fc9b80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.299+0000 7fc9caa00640 1 -- 172.21.15.60:0/4092069531 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fc9c40efed0 con 0x7fc9d41032b0 2024-02-26T10:44:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.300+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b80a1d80 0x7fc9b80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:55.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.300+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b80a1d80 0x7fc9b80a4240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc9d419aaf0 tx=0x7fc9bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:55.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.302+0000 7fc9caa00640 1 -- 172.21.15.60:0/4092069531 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9c40bc900 con 0x7fc9d41032b0 2024-02-26T10:44:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.544+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9d402ce30 con 0x7fc9d41032b0 2024-02-26T10:44:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.545+0000 7fc9caa00640 1 -- 172.21.15.60:0/4092069531 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11359 (secure 0 0 0) 0x7fc9c40c09a0 con 0x7fc9d41032b0 2024-02-26T10:44:55.542 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:44:55.542 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:43:16.029212+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:44:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T10:44:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b80a1d80 msgr2=0x7fc9b80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b80a1d80 0x7fc9b80a4240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc9d419aaf0 tx=0x7fc9bc005f90 comp rx=0 tx=0).stop 2024-02-26T10:44:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d41032b0 msgr2=0x7fc9d41995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d41032b0 0x7fc9d41995e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009230 tx=0x7fc9c402c660 comp rx=0 tx=0).stop 2024-02-26T10:44:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 shutdown_connections 2024-02-26T10:44:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b80a1d80 0x7fc9b80a4240 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d4104e60 0x7fc9d41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d41044a0 0x7fc9d4199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 --2- 172.21.15.60:0/4092069531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9d41032b0 0x7fc9d41995e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 >> 172.21.15.60:0/4092069531 conn(0x7fc9d40fe840 msgr2=0x7fc9d41077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 shutdown_connections 2024-02-26T10:44:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.548+0000 7fc9d3200640 1 -- 172.21.15.60:0/4092069531 wait complete. 2024-02-26T10:44:55.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:44:55.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:55 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/193049184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:44:55.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:55 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2147401670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:44:55.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:55 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/193049184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:44:55.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:55 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2147401670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:44:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.796+0000 7fb2f6c00640 1 -- 172.21.15.60:0/3314884741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 msgr2=0x7fb2f0076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.796+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/3314884741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 0x7fb2f0076040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009900 tx=0x7fb2e002bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:55.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.796+0000 7fb2f6c00640 1 -- 172.21.15.60:0/3314884741 shutdown_connections 2024-02-26T10:44:55.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.796+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/3314884741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0077800 0x7fb2f00780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.796+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/3314884741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f0076e60 0x7fb2f00772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.796+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/3314884741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 0x7fb2f0076040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.796+0000 7fb2f6c00640 1 -- 172.21.15.60:0/3314884741 >> 172.21.15.60:0/3314884741 conn(0x7fb2f00fea10 msgr2=0x7fb2f0100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:55.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.796+0000 7fb2f6c00640 1 -- 172.21.15.60:0/3314884741 shutdown_connections 2024-02-26T10:44:55.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.796+0000 7fb2f6c00640 1 -- 172.21.15.60:0/3314884741 wait complete. 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.797+0000 7fb2f6c00640 1 Processor -- start 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.797+0000 7fb2f6c00640 1 -- start start 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.797+0000 7fb2f6c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 0x7fb2f019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.797+0000 7fb2f6c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0076e60 0x7fb2f019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.797+0000 7fb2f6c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f0077800 0x7fb2f01a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2f6c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f0113570 con 0x7fb2f0077800 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2f6c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f01133f0 con 0x7fb2f0076e60 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2f6c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f01136f0 con 0x7fb2f0075c20 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2f4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f0077800 0x7fb2f01a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2efe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 0x7fb2f019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:55.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2ef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0076e60 0x7fb2f019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2efe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 0x7fb2f019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:57620/0 (socket says 172.21.15.60:57620) 2024-02-26T10:44:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2f4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f0077800 0x7fb2f01a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35246/0 (socket says 172.21.15.60:35246) 2024-02-26T10:44:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2efe00640 1 -- 172.21.15.60:0/737838351 learned_addr learned my addr 172.21.15.60:0/737838351 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2efe00640 1 -- 172.21.15.60:0/737838351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0076e60 msgr2=0x7fb2f019e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2efe00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0076e60 0x7fb2f019e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2efe00640 1 -- 172.21.15.60:0/737838351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f0077800 msgr2=0x7fb2f01a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2efe00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f0077800 0x7fb2f01a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2efe00640 1 -- 172.21.15.60:0/737838351 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f01a5780 con 0x7fb2f0075c20 2024-02-26T10:44:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.798+0000 7fb2ef400640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0076e60 0x7fb2f019e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.799+0000 7fb2efe00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 0x7fb2f019db60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000c00 tx=0x7fb2e002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.799+0000 7fb2ecc00640 1 -- 172.21.15.60:0/737838351 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0043020 con 0x7fb2f0075c20 2024-02-26T10:44:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.799+0000 7fb2ecc00640 1 -- 172.21.15.60:0/737838351 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2e00054e0 con 0x7fb2f0075c20 2024-02-26T10:44:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.799+0000 7fb2ecc00640 1 -- 172.21.15.60:0/737838351 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0003c40 con 0x7fb2f0075c20 2024-02-26T10:44:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.799+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f01a5a10 con 0x7fb2f0075c20 2024-02-26T10:44:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.799+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f01a5f10 con 0x7fb2f0075c20 2024-02-26T10:44:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.800+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f00684a0 con 0x7fb2f0075c20 2024-02-26T10:44:55.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.802+0000 7fb2ecc00640 1 -- 172.21.15.60:0/737838351 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb2e0046020 con 0x7fb2f0075c20 2024-02-26T10:44:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.804+0000 7fb2ecc00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb2d40a1d80 0x7fb2d40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.804+0000 7fb2ecc00640 1 -- 172.21.15.60:0/737838351 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb2e00f0360 con 0x7fb2f0075c20 2024-02-26T10:44:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.804+0000 7fb2ef400640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb2d40a1d80 0x7fb2d40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.804+0000 7fb2ef400640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb2d40a1d80 0x7fb2d40a4240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb2f019f070 tx=0x7fb2dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:55.806+0000 7fb2ecc00640 1 -- 172.21.15.60:0/737838351 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb2e004d020 con 0x7fb2f0075c20 2024-02-26T10:44:55.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:55 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/193049184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:44:55.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:55 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2147401670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:44:56.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.006+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2f002ce60 con 0x7fb2f0075c20 2024-02-26T10:44:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.008+0000 7fb2ecc00640 1 -- 172.21.15.60:0/737838351 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7fb2e00bd380 con 0x7fb2f0075c20 2024-02-26T10:44:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:44:56.004 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb2d40a1d80 msgr2=0x7fb2d40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb2d40a1d80 0x7fb2d40a4240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb2f019f070 tx=0x7fb2dc005f90 comp rx=0 tx=0).stop 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 msgr2=0x7fb2f019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 0x7fb2f019db60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000c00 tx=0x7fb2e002c650 comp rx=0 tx=0).stop 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 shutdown_connections 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb2d40a1d80 0x7fb2d40a4240 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0076e60 0x7fb2f019e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f0077800 0x7fb2f01a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 --2- 172.21.15.60:0/737838351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2f0075c20 0x7fb2f019db60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 >> 172.21.15.60:0/737838351 conn(0x7fb2f00fea10 msgr2=0x7fb2f0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 shutdown_connections 2024-02-26T10:44:56.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.011+0000 7fb2f6c00640 1 -- 172.21.15.60:0/737838351 wait complete. 2024-02-26T10:44:56.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:44:56.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.250+0000 7fbcf2200640 1 -- 172.21.15.60:0/2006017106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 msgr2=0x7fbcec1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.250+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2006017106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 0x7fbcec1014e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009900 tx=0x7fbce002bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:56.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.250+0000 7fbcf2200640 1 -- 172.21.15.60:0/2006017106 shutdown_connections 2024-02-26T10:44:56.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.250+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2006017106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102c70 0x7fbcec109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.250+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2006017106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec1022b0 0x7fbcec102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.250+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2006017106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 0x7fbcec1014e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.250+0000 7fbcf2200640 1 -- 172.21.15.60:0/2006017106 >> 172.21.15.60:0/2006017106 conn(0x7fbcec0fc630 msgr2=0x7fbcec0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:56.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.250+0000 7fbcf2200640 1 -- 172.21.15.60:0/2006017106 shutdown_connections 2024-02-26T10:44:56.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.250+0000 7fbcf2200640 1 -- 172.21.15.60:0/2006017106 wait complete. 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.251+0000 7fbcf2200640 1 Processor -- start 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf2200640 1 -- start start 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 0x7fbcec106e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf2200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec1022b0 0x7fbcec107380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf2200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102c70 0x7fbcec1039b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf2200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec10cda0 con 0x7fbcec1022b0 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec10cc20 con 0x7fbcec1010c0 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf2200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec10cf20 con 0x7fbcec102c70 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 0x7fbcec106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 0x7fbcec106e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35674/0 (socket says 172.21.15.60:35674) 2024-02-26T10:44:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf0e00640 1 -- 172.21.15.60:0/2569823843 learned_addr learned my addr 172.21.15.60:0/2569823843 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:56.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcebe00640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec1022b0 0x7fbcec107380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.252+0000 7fbcf1800640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102c70 0x7fbcec1039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbcf1800640 1 -- 172.21.15.60:0/2569823843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 msgr2=0x7fbcec106e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbcf1800640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 0x7fbcec106e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbcf1800640 1 -- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec1022b0 msgr2=0x7fbcec107380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbcf1800640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec1022b0 0x7fbcec107380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbcf1800640 1 -- 172.21.15.60:0/2569823843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec104270 con 0x7fbcec102c70 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbcf0e00640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 0x7fbcec106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbcf1800640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102c70 0x7fbcec1039b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009fd0 tx=0x7fbce400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbce9600640 1 -- 172.21.15.60:0/2569823843 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce400ce20 con 0x7fbcec102c70 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbce9600640 1 -- 172.21.15.60:0/2569823843 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbce4005020 con 0x7fbcec102c70 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec1a5a20 con 0x7fbcec102c70 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbce9600640 1 -- 172.21.15.60:0/2569823843 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4010720 con 0x7fbcec102c70 2024-02-26T10:44:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.253+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec1a5e30 con 0x7fbcec102c70 2024-02-26T10:44:56.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.255+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec1a6120 con 0x7fbcec102c70 2024-02-26T10:44:56.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.257+0000 7fbce9600640 1 -- 172.21.15.60:0/2569823843 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbce4005540 con 0x7fbcec102c70 2024-02-26T10:44:56.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.258+0000 7fbce9600640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.258+0000 7fbce9600640 1 -- 172.21.15.60:0/2569823843 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fbce40ce7d0 con 0x7fbcec102c70 2024-02-26T10:44:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.258+0000 7fbcf0e00640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.259+0000 7fbcf0e00640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006010 tx=0x7fbce002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:56.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.262+0000 7fbce9600640 1 -- 172.21.15.60:0/2569823843 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbce409b7f0 con 0x7fbcec102c70 2024-02-26T10:44:56.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.512+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcec1a62f0 con 0x7fbcec102c70 2024-02-26T10:44:56.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.513+0000 7fbce9600640 1 -- 172.21.15.60:0/2569823843 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11359 (secure 0 0 0) 0x7fbce409b0f0 con 0x7fbcec102c70 2024-02-26T10:44:56.509 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:44:56.509 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:43:16.029212+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:44:56.509 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T10:44:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 msgr2=0x7fbcd40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006010 tx=0x7fbce002df90 comp rx=0 tx=0).stop 2024-02-26T10:44:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102c70 msgr2=0x7fbcec1039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102c70 0x7fbcec1039b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009fd0 tx=0x7fbce400ef10 comp rx=0 tx=0).stop 2024-02-26T10:44:56.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 shutdown_connections 2024-02-26T10:44:56.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102c70 0x7fbcec1039b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec1022b0 0x7fbcec107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 --2- 172.21.15.60:0/2569823843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec1010c0 0x7fbcec106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 >> 172.21.15.60:0/2569823843 conn(0x7fbcec0fc630 msgr2=0x7fbcec073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 shutdown_connections 2024-02-26T10:44:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.517+0000 7fbcf2200640 1 -- 172.21.15.60:0/2569823843 wait complete. 2024-02-26T10:44:56.533 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-26T10:42:42.076607+0000', 'modified': '2024-02-26T10:43:16.029212+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24587, 'mds_1': 24553, 'mds_2': 24530}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24506': {'gid': 24506, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6835/1201544557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 1201544557}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 1201544557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6835/2109318506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 2109318506}, {'type': 'v1', 'addr': '172.21.15.113:6835', 'nonce': 2109318506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6837/3968092815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3968092815}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3968092815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.100:6837/1347855255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 1347855255}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 1347855255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.113:6839/2057195735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6838', 'nonce': 2057195735}, {'type': 'v1', 'addr': '172.21.15.113:6839', 'nonce': 2057195735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6839/2377114018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 2377114018}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 2377114018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T10:44:56.533 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-26T10:44:56.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-26T10:44:56.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:56 smithi060 ceph-mon[31539]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:56.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:56 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4092069531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:44:56.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:56 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/737838351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:44:56.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:56 smithi100 ceph-mon[36051]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:56.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:56 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/4092069531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:44:56.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:56 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/737838351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:44:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.735+0000 7f967d400640 1 -- 172.21.15.60:0/2369430778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9678107880 msgr2=0x7f9678107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.735+0000 7f967d400640 1 --2- 172.21.15.60:0/2369430778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9678107880 0x7f9678107ca0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9668009900 tx=0x7f966802bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.735+0000 7f967d400640 1 -- 172.21.15.60:0/2369430778 shutdown_connections 2024-02-26T10:44:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.735+0000 7f967d400640 1 --2- 172.21.15.60:0/2369430778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9678109430 0x7f967810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.735+0000 7f967d400640 1 --2- 172.21.15.60:0/2369430778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9678108a70 0x7f9678108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.735+0000 7f967d400640 1 --2- 172.21.15.60:0/2369430778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9678107880 0x7f9678107ca0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.735+0000 7f967d400640 1 -- 172.21.15.60:0/2369430778 >> 172.21.15.60:0/2369430778 conn(0x7f9678076820 msgr2=0x7f9678078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:56.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.735+0000 7f967d400640 1 -- 172.21.15.60:0/2369430778 shutdown_connections 2024-02-26T10:44:56.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.736+0000 7f967d400640 1 -- 172.21.15.60:0/2369430778 wait complete. 2024-02-26T10:44:56.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.737+0000 7f967d400640 1 Processor -- start 2024-02-26T10:44:56.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.737+0000 7f967d400640 1 -- start start 2024-02-26T10:44:56.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.737+0000 7f967d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9678107880 0x7f967819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.737+0000 7f967d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9678108a70 0x7f967819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.737+0000 7f967d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9678109430 0x7f96781a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.737+0000 7f967d400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96781134f0 con 0x7f9678109430 2024-02-26T10:44:56.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.737+0000 7f967d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9678113370 con 0x7f9678107880 2024-02-26T10:44:56.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.737+0000 7f967d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9678113670 con 0x7f9678108a70 2024-02-26T10:44:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.737+0000 7f9677200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9678109430 0x7f96781a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f966de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9678108a70 0x7f967819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f9677200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9678109430 0x7f96781a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35288/0 (socket says 172.21.15.60:35288) 2024-02-26T10:44:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f9677200640 1 -- 172.21.15.60:0/1827602513 learned_addr learned my addr 172.21.15.60:0/1827602513 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f9676800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9678107880 0x7f967819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f966de00640 1 -- 172.21.15.60:0/1827602513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9678107880 msgr2=0x7f967819dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f966de00640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9678107880 0x7f967819dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f966de00640 1 -- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9678109430 msgr2=0x7f96781a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f966de00640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9678109430 0x7f96781a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f966de00640 1 -- 172.21.15.60:0/1827602513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96781a5810 con 0x7f9678108a70 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f9677200640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9678109430 0x7f96781a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.738+0000 7f966de00640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9678108a70 0x7f967819e130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f966000e9f0 tx=0x7f966000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.739+0000 7f966fe00640 1 -- 172.21.15.60:0/1827602513 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966000ccc0 con 0x7f9678108a70 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.739+0000 7f967d400640 1 -- 172.21.15.60:0/1827602513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96781a5b00 con 0x7f9678108a70 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.739+0000 7f966fe00640 1 -- 172.21.15.60:0/1827602513 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9660005020 con 0x7f9678108a70 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.739+0000 7f966fe00640 1 -- 172.21.15.60:0/1827602513 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96600049e0 con 0x7f9678108a70 2024-02-26T10:44:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.739+0000 7f967d400640 1 -- 172.21.15.60:0/1827602513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96781a6000 con 0x7f9678108a70 2024-02-26T10:44:56.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.743+0000 7f96803d7740 1 Processor -- start 2024-02-26T10:44:56.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f96803d7740 1 -- start start 2024-02-26T10:44:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f966fe00640 1 -- 172.21.15.60:0/1827602513 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9660005560 con 0x7f9678108a70 2024-02-26T10:44:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f96803d7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a8caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f96803d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 0x559af6a91bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f96803d7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a92230 0x559af6a946c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f96803d7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af69f15d0 con 0x559af6a92230 2024-02-26T10:44:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f96803d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af699a900 con 0x559af6a8d0c0 2024-02-26T10:44:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f96803d7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af69ed670 con 0x559af6a8c6d0 2024-02-26T10:44:56.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f966de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a92230 0x559af6a946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f9676800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 0x559af6a91bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f9677200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a8caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f9676800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 0x559af6a91bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35710/0 (socket says 172.21.15.60:35710) 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.744+0000 7f9676800640 1 -- 172.21.15.60:0/1777937561 learned_addr learned my addr 172.21.15.60:0/1777937561 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.745+0000 7f9677200640 1 -- 172.21.15.60:0/1777937561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 msgr2=0x559af6a91bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.745+0000 7f9677200640 1 --2- 172.21.15.60:0/1777937561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 0x559af6a91bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.745+0000 7f9677200640 1 -- 172.21.15.60:0/1777937561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a92230 msgr2=0x559af6a946c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.745+0000 7f9677200640 1 --2- 172.21.15.60:0/1777937561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a92230 0x559af6a946c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.745+0000 7f9677200640 1 -- 172.21.15.60:0/1777937561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af67062d0 con 0x559af6a8c6d0 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.745+0000 7f9677200640 1 --2- 172.21.15.60:0/1777937561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a8caf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96700117e0 tx=0x7f9670011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=716b87e5cfa1b712 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.745+0000 7f966fe00640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f965c0a1d80 0x7f965c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.746+0000 7f9667400640 1 -- 172.21.15.60:0/1777937561 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670011310 con 0x559af6a8c6d0 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.746+0000 7f9667400640 1 -- 172.21.15.60:0/1777937561 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f967000f9a0 con 0x559af6a8c6d0 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.746+0000 7f9667400640 1 -- 172.21.15.60:0/1777937561 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96700057d0 con 0x559af6a8c6d0 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.746+0000 7f966fe00640 1 -- 172.21.15.60:0/1827602513 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f96600ce0e0 con 0x7f9678108a70 2024-02-26T10:44:56.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.746+0000 7f9676800640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f965c0a1d80 0x7f965c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.746+0000 7f96803d7740 1 -- 172.21.15.60:0/1777937561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 msgr2=0x559af6a8caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.746+0000 7f96803d7740 1 --2- 172.21.15.60:0/1777937561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a8caf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96700117e0 tx=0x7f9670011ca0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f9676800640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f965c0a1d80 0x7f965c0a4240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f96680096c0 tx=0x7f9668005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:56.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f96803d7740 1 -- 172.21.15.60:0/1777937561 shutdown_connections 2024-02-26T10:44:56.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f96803d7740 1 --2- 172.21.15.60:0/1777937561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 0x559af6a91bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f96803d7740 1 --2- 172.21.15.60:0/1777937561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a92230 0x559af6a946c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f96803d7740 1 --2- 172.21.15.60:0/1777937561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a8caf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f96803d7740 1 -- 172.21.15.60:0/1777937561 >> 172.21.15.60:0/1777937561 conn(0x559af6a87c40 msgr2=0x559af6a88040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f96803d7740 1 -- 172.21.15.60:0/1777937561 shutdown_connections 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f96803d7740 1 -- 172.21.15.60:0/1777937561 wait complete. 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f96803d7740 1 -- start start 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.747+0000 7f96803d7740 1 Processor -- start 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f96803d7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a96880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f96803d7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 0x559af6a96dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f96803d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 0x559af6a9de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f96803d7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af6a951e0 con 0x559af6a8d0c0 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f96803d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af69add60 con 0x559af6a92230 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f96803d7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af6a95360 con 0x559af6a8c6d0 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f96803d7740 10 client.0 _ll_register_callbacks cb 0x559af69e9320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:44:56.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f966de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 0x559af6a96dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f9676800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a96880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f9676800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a96880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:57680/0 (socket says 172.21.15.60:57680) 2024-02-26T10:44:56.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f9676800640 1 -- 172.21.15.60:0/3960024138 learned_addr learned my addr 172.21.15.60:0/3960024138 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:56.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.748+0000 7f9677200640 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 0x559af6a9de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f9676800640 1 -- 172.21.15.60:0/3960024138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 msgr2=0x559af6a9de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:56.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f9676800640 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 0x559af6a9de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f9676800640 1 -- 172.21.15.60:0/3960024138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 msgr2=0x559af6a96dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:56.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f9676800640 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 0x559af6a96dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f9676800640 1 -- 172.21.15.60:0/3960024138 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559af67d06b0 con 0x559af6a8c6d0 2024-02-26T10:44:56.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f9676800640 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a96880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9668000c00 tx=0x7f9668009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8865dcc30d6bf0d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:56.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f965be00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f965be00640 1 -- 172.21.15.60:0/3960024138 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668034440 con 0x559af6a8c6d0 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f965be00640 1 -- 172.21.15.60:0/3960024138 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f96680345e0 con 0x559af6a8c6d0 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.749+0000 7f96803d7740 1 -- 172.21.15.60:0/3960024138 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f967000f9a0 con 0x559af6a8c6d0 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.750+0000 7f965be00640 1 -- 172.21.15.60:0/3960024138 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f966805b050 con 0x559af6a8c6d0 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.750+0000 7f96803d7740 10 client.25165 fetch_fsmap learned FSMap version 23 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.750+0000 7f96803d7740 1 -- 172.21.15.60:0/3960024138 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559af6aa1230 con 0x559af6a8c6d0 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.750+0000 7f965be00640 1 -- 172.21.15.60:0/3960024138 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f967000f9a0 con 0x559af6a8c6d0 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.750+0000 7f965be00640 1 -- 172.21.15.60:0/3960024138 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9668052740 con 0x559af6a8c6d0 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.750+0000 7f96803d7740 10 client.25165 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.750+0000 7f96803d7740 10 client.25165 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.751+0000 7f96803d7740 20 client.25165 populate_metadata read hostname 'smithi060' 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.751+0000 7f96803d7740 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x559af6aa35f0 0x559af6aa26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.751+0000 7f96803d7740 4 client.25165 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:44:56.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.751+0000 7f96803d7740 1 -- 172.21.15.60:0/3960024138 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559af69feb10 con 0x559af6aa35f0 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.751+0000 7f966de00640 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x559af6aa35f0 0x559af6aa26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.751+0000 7f966de00640 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x559af6aa35f0 0x559af6aa26f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb832aa452c0552d server_cookie=93002c7a14d1ab98 in_seq=0 out_seq=0 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.751+0000 7f965be00640 10 client.25165 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.752+0000 7f965be00640 1 -- 172.21.15.60:0/3960024138 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559af69feb10 con 0x559af6aa35f0 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.752+0000 7f965be00640 10 client.25165 handle_command_reply: tid=0 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.752+0000 7f96803d7740 1 client.25165 shutdown 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 -- 172.21.15.60:0/3960024138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 msgr2=0x559af6a96880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a96880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9668000c00 tx=0x7f9668009260 comp rx=0 tx=0).stop 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 -- 172.21.15.60:0/3960024138 shutdown_connections 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x559af6aa35f0 0x559af6aa26f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 0x559af6a9de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f965be00640 0 client.25165 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:44:56.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 0x559af6a96dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 --2- 172.21.15.60:0/3960024138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a96880 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 -- 172.21.15.60:0/3960024138 >> 172.21.15.60:0/3960024138 conn(0x559af6a87c40 msgr2=0x559af6a86e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:56.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 -- 172.21.15.60:0/3960024138 shutdown_connections 2024-02-26T10:44:56.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 1 -- 172.21.15.60:0/3960024138 wait complete. 2024-02-26T10:44:56.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.753+0000 7f96803d7740 20 client.25165 trim_cache size 0 max 16384 2024-02-26T10:44:56.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f96803d7740 1 Processor -- start 2024-02-26T10:44:56.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f96803d7740 1 -- start start 2024-02-26T10:44:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f96803d7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a91850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f96803d7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 0x559af6a9dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f96803d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 0x559af6aaba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f96803d7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af6aa2e10 con 0x559af6a8d0c0 2024-02-26T10:44:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f96803d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af6aa3470 con 0x559af6a92230 2024-02-26T10:44:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f96803d7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af6aa3ad0 con 0x559af6a8c6d0 2024-02-26T10:44:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f9676800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a91850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f9676800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a91850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:57688/0 (socket says 172.21.15.60:57688) 2024-02-26T10:44:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f9676800640 1 -- 172.21.15.60:0/3444378174 learned_addr learned my addr 172.21.15.60:0/3444378174 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f9677200640 1 --2- 172.21.15.60:0/3444378174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 0x559af6aaba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.786+0000 7f966de00640 1 --2- 172.21.15.60:0/3444378174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 0x559af6a9dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.787+0000 7f9676800640 1 -- 172.21.15.60:0/3444378174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 msgr2=0x559af6aaba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.787+0000 7f9676800640 1 --2- 172.21.15.60:0/3444378174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 0x559af6aaba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.787+0000 7f9676800640 1 -- 172.21.15.60:0/3444378174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 msgr2=0x559af6a9dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.787+0000 7f9676800640 1 --2- 172.21.15.60:0/3444378174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 0x559af6a9dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.787+0000 7f9676800640 1 -- 172.21.15.60:0/3444378174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af6a86290 con 0x559af6a8c6d0 2024-02-26T10:44:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.787+0000 7f9676800640 1 --2- 172.21.15.60:0/3444378174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a91850 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9668009230 tx=0x7f9668005990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e1a1a45eefec0f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.787+0000 7f9666a00640 1 -- 172.21.15.60:0/3444378174 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668064070 con 0x559af6a8c6d0 2024-02-26T10:44:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.787+0000 7f9666a00640 1 -- 172.21.15.60:0/3444378174 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9668034440 con 0x559af6a8c6d0 2024-02-26T10:44:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.787+0000 7f9666a00640 1 -- 172.21.15.60:0/3444378174 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680043e0 con 0x559af6a8c6d0 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.788+0000 7f96803d7740 1 -- 172.21.15.60:0/3444378174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 msgr2=0x559af6a91850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.788+0000 7f96803d7740 1 --2- 172.21.15.60:0/3444378174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a91850 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9668009230 tx=0x7f9668005990 comp rx=0 tx=0).stop 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.788+0000 7f96803d7740 1 -- 172.21.15.60:0/3444378174 shutdown_connections 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.788+0000 7f96803d7740 1 --2- 172.21.15.60:0/3444378174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a92230 0x559af6aaba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.788+0000 7f96803d7740 1 --2- 172.21.15.60:0/3444378174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8d0c0 0x559af6a9dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.788+0000 7f96803d7740 1 --2- 172.21.15.60:0/3444378174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a8c6d0 0x559af6a91850 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.788+0000 7f96803d7740 1 -- 172.21.15.60:0/3444378174 >> 172.21.15.60:0/3444378174 conn(0x559af6a87c40 msgr2=0x559af6a88020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.788+0000 7f96803d7740 1 -- 172.21.15.60:0/3444378174 shutdown_connections 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.788+0000 7f96803d7740 1 -- 172.21.15.60:0/3444378174 wait complete. 2024-02-26T10:44:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.789+0000 7f96803d7740 1 -- start start 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.789+0000 7f96803d7740 1 Processor -- start 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.789+0000 7f96803d7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8c6d0 0x559af6a9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.789+0000 7f96803d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 0x559af6a9b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f96803d7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a92230 0x559af6af91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f96803d7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af6aa4790 con 0x559af6a8c6d0 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f96803d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af6aa4130 con 0x559af6a8d0c0 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f96803d7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af6aa4df0 con 0x559af6a92230 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f96803d7740 10 client.0 _ll_register_callbacks cb 0x7f9668009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f9676800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8c6d0 0x559af6a9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f9677200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a92230 0x559af6af91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f966de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 0x559af6a9b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f9677200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a92230 0x559af6af91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:57696/0 (socket says 172.21.15.60:57696) 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f9677200640 1 -- 172.21.15.60:0/1272878814 learned_addr learned my addr 172.21.15.60:0/1272878814 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f9677200640 1 -- 172.21.15.60:0/1272878814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 msgr2=0x559af6a9b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f9677200640 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 0x559af6a9b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f9677200640 1 -- 172.21.15.60:0/1272878814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8c6d0 msgr2=0x559af6a9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:56.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f9677200640 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8c6d0 0x559af6a9ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.790+0000 7f9677200640 1 -- 172.21.15.60:0/1272878814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559af6a9b830 con 0x559af6a92230 2024-02-26T10:44:56.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.791+0000 7f9677200640 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a92230 0x559af6af91f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96700117e0 tx=0x7f967000fbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f63944a87aba088 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:56.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.791+0000 7f965be00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:44:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.791+0000 7f965be00640 1 -- 172.21.15.60:0/1272878814 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967002a020 con 0x559af6a92230 2024-02-26T10:44:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.791+0000 7f96803d7740 1 -- 172.21.15.60:0/1272878814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9668034440 con 0x559af6a92230 2024-02-26T10:44:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.791+0000 7f965be00640 1 -- 172.21.15.60:0/1272878814 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f9670026670 con 0x559af6a92230 2024-02-26T10:44:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.791+0000 7f965be00640 1 -- 172.21.15.60:0/1272878814 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9670035050 con 0x559af6a92230 2024-02-26T10:44:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.791+0000 7f96803d7740 10 client.25171 fetch_fsmap learned FSMap version 23 2024-02-26T10:44:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.791+0000 7f96803d7740 1 -- 172.21.15.60:0/1272878814 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559af6afa880 con 0x559af6a92230 2024-02-26T10:44:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.792+0000 7f965be00640 1 -- 172.21.15.60:0/1272878814 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9668034440 con 0x559af6a92230 2024-02-26T10:44:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.792+0000 7f965be00640 1 -- 172.21.15.60:0/1272878814 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9670024040 con 0x559af6a92230 2024-02-26T10:44:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.792+0000 7f96803d7740 10 client.25171 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.792+0000 7f96803d7740 10 client.25171 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.792+0000 7f96803d7740 20 client.25171 populate_metadata read hostname 'smithi060' 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.792+0000 7f96803d7740 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x559af6afed60 0x559af6afab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.792+0000 7f96803d7740 4 client.25171 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.792+0000 7f96803d7740 1 -- 172.21.15.60:0/1272878814 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559af69f8e60 con 0x559af6afed60 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.792+0000 7f9676800640 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x559af6afed60 0x559af6afab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f9676800640 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x559af6afed60 0x559af6afab10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d8fbcb2f91e817e server_cookie=5d39a80f73af3eb in_seq=0 out_seq=0 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f965be00640 10 client.25171 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f965be00640 1 -- 172.21.15.60:0/1272878814 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x559af69f8e60 con 0x559af6afed60 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f965be00640 10 client.25171 handle_command_reply: tid=0 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f96803d7740 1 client.25171 shutdown 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f96803d7740 1 -- 172.21.15.60:0/1272878814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a92230 msgr2=0x559af6af91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f96803d7740 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a92230 0x559af6af91f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96700117e0 tx=0x7f967000fbb0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f96803d7740 1 -- 172.21.15.60:0/1272878814 shutdown_connections 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f96803d7740 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x559af6afed60 0x559af6afab10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f96803d7740 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559af6a8d0c0 0x559af6a9b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f96803d7740 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559af6a92230 0x559af6af91f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f965be00640 0 client.25171 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f96803d7740 1 --2- 172.21.15.60:0/1272878814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559af6a8c6d0 0x559af6a9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.793+0000 7f96803d7740 1 -- 172.21.15.60:0/1272878814 >> 172.21.15.60:0/1272878814 conn(0x559af6a87c40 msgr2=0x559af6aaba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f96803d7740 1 -- 172.21.15.60:0/1272878814 shutdown_connections 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f96803d7740 1 -- 172.21.15.60:0/1272878814 wait complete. 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f96803d7740 20 client.25171 trim_cache size 0 max 16384 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:44:56.792 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 2, 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24530, 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 101.45074712, 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 23, 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 95, 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 91, 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 154.74528404 2024-02-26T10:44:56.793 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:44:56.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 -- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f965c0a1d80 msgr2=0x7f965c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f965c0a1d80 0x7f965c0a4240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f96680096c0 tx=0x7f9668005fd0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 -- 172.21.15.60:0/1827602513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9678108a70 msgr2=0x7f967819e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:56.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9678108a70 0x7f967819e130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f966000e9f0 tx=0x7f966000eeb0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 -- 172.21.15.60:0/1827602513 shutdown_connections 2024-02-26T10:44:56.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f965c0a1d80 0x7f965c0a4240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9678109430 0x7f96781a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9678108a70 0x7f967819e130 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 --2- 172.21.15.60:0/1827602513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9678107880 0x7f967819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:56.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.794+0000 7f967d400640 1 -- 172.21.15.60:0/1827602513 >> 172.21.15.60:0/1827602513 conn(0x7f9678076820 msgr2=0x7f967810bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:56.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.795+0000 7f967d400640 1 -- 172.21.15.60:0/1827602513 shutdown_connections 2024-02-26T10:44:56.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:56.795+0000 7f967d400640 1 -- 172.21.15.60:0/1827602513 wait complete. 2024-02-26T10:44:56.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-26T10:44:56.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:56 smithi113 ceph-mon[36126]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:44:56.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:56 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/4092069531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:44:56.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:56 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/737838351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:44:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.029+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1337847095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102450 msgr2=0x7fbcec1048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.029+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1337847095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102450 0x7fbcec1048c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009900 tx=0x7fbcd802bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.029+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1337847095 shutdown_connections 2024-02-26T10:44:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.029+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1337847095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec104e00 0x7fbcec105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.029+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1337847095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec102450 0x7fbcec1048c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.029+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1337847095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec0ffae0 0x7fbcec101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.029+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1337847095 >> 172.21.15.60:0/1337847095 conn(0x7fbcec0fb700 msgr2=0x7fbcec0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.029+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1337847095 shutdown_connections 2024-02-26T10:44:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.029+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1337847095 wait complete. 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.030+0000 7fbcf3a00640 1 Processor -- start 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.030+0000 7fbcf3a00640 1 -- start start 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.030+0000 7fbcf3a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec0ffae0 0x7fbcec19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.030+0000 7fbcf3a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec102450 0x7fbcec19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.030+0000 7fbcf3a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec104e00 0x7fbcec1a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.030+0000 7fbcf3a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec113530 con 0x7fbcec102450 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.030+0000 7fbcf3a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec1133b0 con 0x7fbcec0ffae0 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.030+0000 7fbcf3a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec1136b0 con 0x7fbcec104e00 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec102450 0x7fbcec19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcf1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec0ffae0 0x7fbcec19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcf1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec0ffae0 0x7fbcec19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35736/0 (socket says 172.21.15.60:35736) 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcf1000640 1 -- 172.21.15.60:0/1522804289 learned_addr learned my addr 172.21.15.60:0/1522804289 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcf1a00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec104e00 0x7fbcec1a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcebe00640 1 -- 172.21.15.60:0/1522804289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec104e00 msgr2=0x7fbcec1a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcebe00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec104e00 0x7fbcec1a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcebe00640 1 -- 172.21.15.60:0/1522804289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec0ffae0 msgr2=0x7fbcec19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcebe00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec0ffae0 0x7fbcec19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcebe00640 1 -- 172.21.15.60:0/1522804289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec1a57c0 con 0x7fbcec102450 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcf1000640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec0ffae0 0x7fbcec19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.031+0000 7fbcebe00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec102450 0x7fbcec19e0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8036040 tx=0x7fbcd802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.032+0000 7fbce9600640 1 -- 172.21.15.60:0/1522804289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8043020 con 0x7fbcec102450 2024-02-26T10:44:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.032+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1522804289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec1a5a50 con 0x7fbcec102450 2024-02-26T10:44:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.032+0000 7fbce9600640 1 -- 172.21.15.60:0/1522804289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcd80054e0 con 0x7fbcec102450 2024-02-26T10:44:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.032+0000 7fbce9600640 1 -- 172.21.15.60:0/1522804289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8003c30 con 0x7fbcec102450 2024-02-26T10:44:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.032+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1522804289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec1a5f70 con 0x7fbcec102450 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.036+0000 7fbcf6981740 1 Processor -- start 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.036+0000 7fbcf6981740 1 -- start start 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbce9600640 1 -- 172.21.15.60:0/1522804289 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbcd8046020 con 0x7fbcec102450 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c92baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c930bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 0x55de5c9336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcf6981740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c8905d0 con 0x55de5c931230 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcf6981740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c839900 con 0x55de5c92b6d0 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcf6981740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c88c670 con 0x55de5c92c0c0 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 0x55de5c9336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 0x55de5c9336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35332/0 (socket says 172.21.15.60:35332) 2024-02-26T10:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcebe00640 1 -- 172.21.15.60:0/1187281650 learned_addr learned my addr 172.21.15.60:0/1187281650 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcf1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c92baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.037+0000 7fbcf1000640 1 --2- 172.21.15.60:0/1187281650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c930bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbcf1000640 1 -- 172.21.15.60:0/1187281650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 msgr2=0x55de5c92baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbcf1000640 1 --2- 172.21.15.60:0/1187281650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c92baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbcf1000640 1 -- 172.21.15.60:0/1187281650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 msgr2=0x55de5c9336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbcf1000640 1 --2- 172.21.15.60:0/1187281650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 0x55de5c9336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbcf1000640 1 -- 172.21.15.60:0/1187281650 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de5c5a52d0 con 0x55de5c92c0c0 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbcf1000640 1 --2- 172.21.15.60:0/1187281650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c930bc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbce00114f0 tx=0x7fbce00119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a51f676ad066ab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbcde000640 1 -- 172.21.15.60:0/1187281650 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce001e070 con 0x55de5c92c0c0 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbcde000640 1 -- 172.21.15.60:0/1187281650 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbce00035f0 con 0x55de5c92c0c0 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbcde000640 1 -- 172.21.15.60:0/1187281650 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce000f820 con 0x55de5c92c0c0 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.038+0000 7fbce9600640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbce9600640 1 -- 172.21.15.60:0/1522804289 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fbcd80f0180 con 0x7fbcec102450 2024-02-26T10:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbcf1a00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbcf6981740 1 -- 172.21.15.60:0/1187281650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 msgr2=0x55de5c930bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbcf6981740 1 --2- 172.21.15.60:0/1187281650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c930bc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbce00114f0 tx=0x7fbce00119b0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbcf6981740 1 -- 172.21.15.60:0/1187281650 shutdown_connections 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbcf6981740 1 --2- 172.21.15.60:0/1187281650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 0x55de5c9336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbcf6981740 1 --2- 172.21.15.60:0/1187281650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c930bc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbcf6981740 1 --2- 172.21.15.60:0/1187281650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c92baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbcf6981740 1 -- 172.21.15.60:0/1187281650 >> 172.21.15.60:0/1187281650 conn(0x55de5c926c40 msgr2=0x55de5c927040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.039+0000 7fbcf1a00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55de5c92c920 tx=0x7fbce4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.040+0000 7fbcf6981740 1 -- 172.21.15.60:0/1187281650 shutdown_connections 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.040+0000 7fbcf6981740 1 -- 172.21.15.60:0/1187281650 wait complete. 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.040+0000 7fbcf6981740 1 -- start start 2024-02-26T10:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.040+0000 7fbcf6981740 1 Processor -- start 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c935df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 0x55de5c93ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcf6981740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c9341e0 con 0x55de5c931230 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcf6981740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c84cd60 con 0x55de5c92b6d0 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcf6981740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c934360 con 0x55de5c92c0c0 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcf6981740 10 client.0 _ll_register_callbacks cb 0x55de5c888320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c935df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcf1a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 0x55de5c93ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c935df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:57718/0 (socket says 172.21.15.60:57718) 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcebe00640 1 -- 172.21.15.60:0/3951847502 learned_addr learned my addr 172.21.15.60:0/3951847502 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcf1000640 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcebe00640 1 -- 172.21.15.60:0/3951847502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 msgr2=0x55de5c9358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcebe00640 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcebe00640 1 -- 172.21.15.60:0/3951847502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 msgr2=0x55de5c93ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcebe00640 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 0x55de5c93ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.041+0000 7fbcebe00640 1 -- 172.21.15.60:0/3951847502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de5c66f6b0 con 0x55de5c92c0c0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.042+0000 7fbcebe00640 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c935df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80f0420 tx=0x7fbcd80ed7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e9bce5ef3d06f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.042+0000 7fbcd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.042+0000 7fbcd2a00640 1 -- 172.21.15.60:0/3951847502 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80b8d40 con 0x55de5c92c0c0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.042+0000 7fbcd2a00640 1 -- 172.21.15.60:0/3951847502 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fbcd80bada0 con 0x55de5c92c0c0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.042+0000 7fbcf6981740 1 -- 172.21.15.60:0/3951847502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbce00035f0 con 0x55de5c92c0c0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.042+0000 7fbcd2a00640 1 -- 172.21.15.60:0/3951847502 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcd80bc400 con 0x55de5c92c0c0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.042+0000 7fbcf6981740 10 client.25183 fetch_fsmap learned FSMap version 23 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.042+0000 7fbcf6981740 1 -- 172.21.15.60:0/3951847502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55de5c934300 con 0x55de5c92c0c0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.042+0000 7fbcd2a00640 1 -- 172.21.15.60:0/3951847502 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbce00035f0 con 0x55de5c92c0c0 2024-02-26T10:44:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.043+0000 7fbcd2a00640 1 -- 172.21.15.60:0/3951847502 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbcd80c4db0 con 0x55de5c92c0c0 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.043+0000 7fbcf6981740 10 client.25183 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.043+0000 7fbcf6981740 10 client.25183 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.043+0000 7fbcf6981740 20 client.25183 populate_metadata read hostname 'smithi060' 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.043+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x55de5c9426a0 0x55de5c941440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.043+0000 7fbcf6981740 4 client.25183 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.043+0000 7fbcf6981740 1 -- 172.21.15.60:0/3951847502 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de5c89db10 con 0x55de5c9426a0 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.043+0000 7fbcf1000640 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x55de5c9426a0 0x55de5c941440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.044+0000 7fbcf1000640 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x55de5c9426a0 0x55de5c941440 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3ccb3e2793c50b7 server_cookie=470b9413f6205941 in_seq=0 out_seq=0 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.044+0000 7fbcd2a00640 10 client.25183 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.044+0000 7fbcd2a00640 1 -- 172.21.15.60:0/3951847502 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de5c89db10 con 0x55de5c9426a0 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.044+0000 7fbcd2a00640 10 client.25183 handle_command_reply: tid=0 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.044+0000 7fbcf6981740 1 client.25183 shutdown 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 -- 172.21.15.60:0/3951847502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 msgr2=0x55de5c935df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c935df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80f0420 tx=0x7fbcd80ed7b0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 -- 172.21.15.60:0/3951847502 shutdown_connections 2024-02-26T10:44:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x55de5c9426a0 0x55de5c941440 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c931230 0x55de5c93ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92c0c0 0x55de5c935df0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3951847502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcd2a00640 0 client.25183 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:44:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 -- 172.21.15.60:0/3951847502 >> 172.21.15.60:0/3951847502 conn(0x55de5c926c40 msgr2=0x55de5c925e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 -- 172.21.15.60:0/3951847502 shutdown_connections 2024-02-26T10:44:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 1 -- 172.21.15.60:0/3951847502 wait complete. 2024-02-26T10:44:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.045+0000 7fbcf6981740 20 client.25183 trim_cache size 0 max 16384 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.062+0000 7fbcf6981740 1 Processor -- start 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.062+0000 7fbcf6981740 1 -- start start 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c93cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c931230 0x55de5c935df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf6981740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c941ec0 con 0x55de5c92c0c0 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf6981740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c942520 con 0x55de5c92b6d0 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf6981740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c942b80 con 0x55de5c931230 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c93cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35762/0 (socket says 172.21.15.60:35762) 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf1000640 1 -- 172.21.15.60:0/3656496085 learned_addr learned my addr 172.21.15.60:0/3656496085 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.063+0000 7fbcf1a00640 1 --2- 172.21.15.60:0/3656496085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c931230 0x55de5c935df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.064+0000 7fbcf1000640 1 -- 172.21.15.60:0/3656496085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c931230 msgr2=0x55de5c935df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.064+0000 7fbcf1000640 1 --2- 172.21.15.60:0/3656496085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c931230 0x55de5c935df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.064+0000 7fbcf1000640 1 -- 172.21.15.60:0/3656496085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 msgr2=0x55de5c93cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.064+0000 7fbcf1000640 1 --2- 172.21.15.60:0/3656496085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c93cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.064+0000 7fbcf1000640 1 -- 172.21.15.60:0/3656496085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de5c926a60 con 0x55de5c92b6d0 2024-02-26T10:44:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.064+0000 7fbcf1000640 1 --2- 172.21.15.60:0/3656496085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbce00114f0 tx=0x7fbce0011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a276f441d584ee9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.064+0000 7fbcdd600640 1 -- 172.21.15.60:0/3656496085 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce001e040 con 0x55de5c92b6d0 2024-02-26T10:44:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.064+0000 7fbcdd600640 1 -- 172.21.15.60:0/3656496085 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbce002d020 con 0x55de5c92b6d0 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.064+0000 7fbcdd600640 1 -- 172.21.15.60:0/3656496085 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00053c0 con 0x55de5c92b6d0 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.065+0000 7fbcf6981740 1 -- 172.21.15.60:0/3656496085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 msgr2=0x55de5c9358b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.065+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3656496085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbce00114f0 tx=0x7fbce0011440 comp rx=0 tx=0).stop 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.065+0000 7fbcf6981740 1 -- 172.21.15.60:0/3656496085 shutdown_connections 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.065+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3656496085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c931230 0x55de5c935df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.065+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3656496085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c93cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.065+0000 7fbcf6981740 1 --2- 172.21.15.60:0/3656496085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c92b6d0 0x55de5c9358b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.065+0000 7fbcf6981740 1 -- 172.21.15.60:0/3656496085 >> 172.21.15.60:0/3656496085 conn(0x55de5c926c40 msgr2=0x55de5c941440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.065+0000 7fbcf6981740 1 -- 172.21.15.60:0/3656496085 shutdown_connections 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.065+0000 7fbcf6981740 1 -- 172.21.15.60:0/3656496085 wait complete. 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf6981740 1 -- start start 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf6981740 1 Processor -- start 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92b6d0 0x55de5c93cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c9396e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf6981740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c931230 0x55de5c939c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf6981740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c943840 con 0x55de5c92c0c0 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf6981740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c9431e0 con 0x55de5c931230 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf6981740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5c943ea0 con 0x55de5c92b6d0 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf6981740 10 client.0 _ll_register_callbacks cb 0x7fbce0009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c9396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c931230 0x55de5c939c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf1000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92b6d0 0x55de5c93cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcf1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c931230 0x55de5c939c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35766/0 (socket says 172.21.15.60:35766) 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.066+0000 7fbcebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c9396e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35352/0 (socket says 172.21.15.60:35352) 2024-02-26T10:44:57.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcf1a00640 1 -- 172.21.15.60:0/2409616997 learned_addr learned my addr 172.21.15.60:0/2409616997 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcebe00640 1 -- 172.21.15.60:0/2409616997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92b6d0 msgr2=0x55de5c93cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcebe00640 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92b6d0 0x55de5c93cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcebe00640 1 -- 172.21.15.60:0/2409616997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c931230 msgr2=0x55de5c939c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcebe00640 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c931230 0x55de5c939c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcebe00640 1 -- 172.21.15.60:0/2409616997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de5c9982a0 con 0x55de5c92c0c0 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcf1a00640 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c931230 0x55de5c939c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcf1000640 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92b6d0 0x55de5c93cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcebe00640 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c9396e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b9470 tx=0x7fbcd8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb75c413f390147 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:44:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcd2a00640 1 -- 172.21.15.60:0/2409616997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80bc5e0 con 0x55de5c92c0c0 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcd2a00640 1 -- 172.21.15.60:0/2409616997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fbcd80bbef0 con 0x55de5c92c0c0 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.067+0000 7fbcf6981740 1 -- 172.21.15.60:0/2409616997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbce002d020 con 0x55de5c92c0c0 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcd2a00640 1 -- 172.21.15.60:0/2409616997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcd80bc870 con 0x55de5c92c0c0 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcf6981740 10 client.15072 fetch_fsmap learned FSMap version 23 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcf6981740 1 -- 172.21.15.60:0/2409616997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55de5c999460 con 0x55de5c92c0c0 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcd2a00640 1 -- 172.21.15.60:0/2409616997 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbce002d020 con 0x55de5c92c0c0 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcd2a00640 1 -- 172.21.15.60:0/2409616997 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbcd80b87f0 con 0x55de5c92c0c0 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcf6981740 10 client.15072 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcf6981740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcf6981740 20 client.15072 populate_metadata read hostname 'smithi060' 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcf6981740 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x55de5c99d880 0x55de5c999690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcf6981740 4 client.15072 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.068+0000 7fbcf6981740 1 -- 172.21.15.60:0/2409616997 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55de5c897e60 con 0x55de5c99d880 2024-02-26T10:44:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.069+0000 7fbcf1000640 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x55de5c99d880 0x55de5c999690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.069+0000 7fbcf1000640 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x55de5c99d880 0x55de5c999690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf7ed92963ce67d3 server_cookie=30ccac1fd73ed69b in_seq=0 out_seq=0 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.069+0000 7fbcd2a00640 10 client.15072 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcd2a00640 1 -- 172.21.15.60:0/2409616997 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55de5c897e60 con 0x55de5c99d880 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcd2a00640 10 client.15072 handle_command_reply: tid=0 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 client.15072 shutdown 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 -- 172.21.15.60:0/2409616997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 msgr2=0x55de5c9396e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c9396e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b9470 tx=0x7fbcd8005e40 comp rx=0 tx=0).stop 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 -- 172.21.15.60:0/2409616997 shutdown_connections 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de5c931230 0x55de5c939c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x55de5c99d880 0x55de5c999690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55de5c92c0c0 0x55de5c9396e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 --2- 172.21.15.60:0/2409616997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55de5c92b6d0 0x55de5c93cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 -- 172.21.15.60:0/2409616997 >> 172.21.15.60:0/2409616997 conn(0x55de5c926c40 msgr2=0x55de5c9358b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcd2a00640 0 client.15072 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 -- 172.21.15.60:0/2409616997 shutdown_connections 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 1 -- 172.21.15.60:0/2409616997 wait complete. 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.070+0000 7fbcf6981740 20 client.15072 trim_cache size 0 max 16384 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 1, 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24553, 2024-02-26T10:44:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T10:44:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T10:44:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T10:44:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 102.73958156800001, 2024-02-26T10:44:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 23, 2024-02-26T10:44:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 95, 2024-02-26T10:44:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 91, 2024-02-26T10:44:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 152.03703317399999 2024-02-26T10:44:57.071 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 msgr2=0x7fbcd40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55de5c92c920 tx=0x7fbce4009500 comp rx=0 tx=0).stop 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec102450 msgr2=0x7fbcec19e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec102450 0x7fbcec19e0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8036040 tx=0x7fbcd802c650 comp rx=0 tx=0).stop 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1522804289 shutdown_connections 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcd40a1d80 0x7fbcd40a4240 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec104e00 0x7fbcec1a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcec102450 0x7fbcec19e0e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 --2- 172.21.15.60:0/1522804289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcec0ffae0 0x7fbcec19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1522804289 >> 172.21.15.60:0/1522804289 conn(0x7fbcec0fb700 msgr2=0x7fbcec100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1522804289 shutdown_connections 2024-02-26T10:44:57.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.071+0000 7fbcf3a00640 1 -- 172.21.15.60:0/1522804289 wait complete. 2024-02-26T10:44:57.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T10:44:57.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.287+0000 7fc98f200640 1 -- 172.21.15.60:0/1608396097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 msgr2=0x7fc990101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.287+0000 7fc98f200640 1 --2- 172.21.15.60:0/1608396097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 0x7fc990101580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc980009900 tx=0x7fc98002bc30 comp rx=0 tx=0).stop 2024-02-26T10:44:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.287+0000 7fc98f200640 1 -- 172.21.15.60:0/1608396097 shutdown_connections 2024-02-26T10:44:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.287+0000 7fc98f200640 1 --2- 172.21.15.60:0/1608396097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc990102350 0x7fc9901027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.287+0000 7fc98f200640 1 --2- 172.21.15.60:0/1608396097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990102d10 0x7fc9901095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.287+0000 7fc98f200640 1 --2- 172.21.15.60:0/1608396097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 0x7fc990101580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.287+0000 7fc98f200640 1 -- 172.21.15.60:0/1608396097 >> 172.21.15.60:0/1608396097 conn(0x7fc9900fc6d0 msgr2=0x7fc9900feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.287+0000 7fc98f200640 1 -- 172.21.15.60:0/1608396097 shutdown_connections 2024-02-26T10:44:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.288+0000 7fc98f200640 1 -- 172.21.15.60:0/1608396097 wait complete. 2024-02-26T10:44:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.288+0000 7fc98f200640 1 Processor -- start 2024-02-26T10:44:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98f200640 1 -- start start 2024-02-26T10:44:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 0x7fc990075010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc990102350 0x7fc990073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990102d10 0x7fc990073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99010cdf0 con 0x7fc990101160 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99010cc70 con 0x7fc990102350 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99010cf70 con 0x7fc990102d10 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 0x7fc990075010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 0x7fc990075010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35368/0 (socket says 172.21.15.60:35368) 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98de00640 1 -- 172.21.15.60:0/3875057013 learned_addr learned my addr 172.21.15.60:0/3875057013 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98d400640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc990102350 0x7fc990073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.289+0000 7fc98e800640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990102d10 0x7fc990073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.290+0000 7fc98de00640 1 -- 172.21.15.60:0/3875057013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990102d10 msgr2=0x7fc990073b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.290+0000 7fc98de00640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990102d10 0x7fc990073b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.290+0000 7fc98de00640 1 -- 172.21.15.60:0/3875057013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc990102350 msgr2=0x7fc990073650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.290+0000 7fc98de00640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc990102350 0x7fc990073650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.290+0000 7fc98de00640 1 -- 172.21.15.60:0/3875057013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990074420 con 0x7fc990101160 2024-02-26T10:44:57.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.290+0000 7fc98e800640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990102d10 0x7fc990073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.290+0000 7fc98de00640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 0x7fc990075010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc98002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.290+0000 7fc986a00640 1 -- 172.21.15.60:0/3875057013 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980041070 con 0x7fc990101160 2024-02-26T10:44:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.291+0000 7fc98f200640 1 -- 172.21.15.60:0/3875057013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9901a1540 con 0x7fc990101160 2024-02-26T10:44:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.291+0000 7fc986a00640 1 -- 172.21.15.60:0/3875057013 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc980004450 con 0x7fc990101160 2024-02-26T10:44:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.291+0000 7fc986a00640 1 -- 172.21.15.60:0/3875057013 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980034440 con 0x7fc990101160 2024-02-26T10:44:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.291+0000 7fc98f200640 1 -- 172.21.15.60:0/3875057013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9901a1a90 con 0x7fc990101160 2024-02-26T10:44:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.294+0000 7fc99822f740 1 Processor -- start 2024-02-26T10:44:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.294+0000 7fc99822f740 1 -- start start 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.294+0000 7fc99822f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e538b0 0x557976e53cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.294+0000 7fc99822f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e542a0 0x557976e58da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.294+0000 7fc99822f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 0x557976e5b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.294+0000 7fc99822f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976db85d0 con 0x557976e538b0 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.294+0000 7fc99822f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976d61900 con 0x557976e59410 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.294+0000 7fc99822f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976db4670 con 0x557976e542a0 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e538b0 0x557976e53cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e538b0 0x557976e53cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35370/0 (socket says 172.21.15.60:35370) 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98e800640 1 -- 172.21.15.60:0/4099064575 learned_addr learned my addr 172.21.15.60:0/4099064575 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98d400640 1 --2- 172.21.15.60:0/4099064575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 0x557976e5b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98e800640 1 -- 172.21.15.60:0/4099064575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e542a0 msgr2=0x557976e58da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc986a00640 1 -- 172.21.15.60:0/3875057013 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc980044070 con 0x7fc990101160 2024-02-26T10:44:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98e800640 1 --2- 172.21.15.60:0/4099064575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e542a0 0x557976e58da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98e800640 1 -- 172.21.15.60:0/4099064575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 msgr2=0x557976e5b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98e800640 1 --2- 172.21.15.60:0/4099064575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 0x557976e5b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98e800640 1 -- 172.21.15.60:0/4099064575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557976acd2d0 con 0x557976e538b0 2024-02-26T10:44:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.295+0000 7fc98e800640 1 --2- 172.21.15.60:0/4099064575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e538b0 0x557976e53cd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc9880117e0 tx=0x7fc988011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e56d892f5ee96bc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.296+0000 7fc97b400640 1 -- 172.21.15.60:0/4099064575 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988011310 con 0x557976e538b0 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.296+0000 7fc97b400640 1 -- 172.21.15.60:0/4099064575 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc988003e50 con 0x557976e538b0 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.296+0000 7fc97b400640 1 -- 172.21.15.60:0/4099064575 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988004560 con 0x557976e538b0 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.296+0000 7fc986a00640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9740a1d80 0x7fc9740a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 -- 172.21.15.60:0/4099064575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e538b0 msgr2=0x557976e53cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 --2- 172.21.15.60:0/4099064575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e538b0 0x557976e53cd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc9880117e0 tx=0x7fc988011ca0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc986a00640 1 -- 172.21.15.60:0/3875057013 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fc9800efc10 con 0x7fc990101160 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 -- 172.21.15.60:0/4099064575 shutdown_connections 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 --2- 172.21.15.60:0/4099064575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 0x557976e5b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 --2- 172.21.15.60:0/4099064575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e542a0 0x557976e58da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 --2- 172.21.15.60:0/4099064575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e538b0 0x557976e53cd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 -- 172.21.15.60:0/4099064575 >> 172.21.15.60:0/4099064575 conn(0x557976e4ee20 msgr2=0x557976e4f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 -- 172.21.15.60:0/4099064575 shutdown_connections 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 -- 172.21.15.60:0/4099064575 wait complete. 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc98de00640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9740a1d80 0x7fc9740a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.297+0000 7fc99822f740 1 -- start start 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.298+0000 7fc98de00640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9740a1d80 0x7fc9740a4240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9800b8b30 tx=0x7fc980005670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.298+0000 7fc99822f740 1 Processor -- start 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.298+0000 7fc99822f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 0x557976e5db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.298+0000 7fc99822f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e5e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.298+0000 7fc99822f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 0x557976e65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.298+0000 7fc99822f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976e5c3c0 con 0x557976e542a0 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.298+0000 7fc99822f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976d74d60 con 0x557976e59410 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.298+0000 7fc99822f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976e5c540 con 0x557976e538b0 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc99822f740 10 client.0 _ll_register_callbacks cb 0x557976db0320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e5e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e5e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35384/0 (socket says 172.21.15.60:35384) 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98d400640 1 -- 172.21.15.60:0/1817371100 learned_addr learned my addr 172.21.15.60:0/1817371100 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98e800640 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 0x557976e5db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98e800640 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 0x557976e65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98d400640 1 -- 172.21.15.60:0/1817371100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 msgr2=0x557976e5db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98d400640 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 0x557976e5db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98d400640 1 -- 172.21.15.60:0/1817371100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 msgr2=0x557976e65110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98d400640 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 0x557976e65110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98d400640 1 -- 172.21.15.60:0/1817371100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557976b976b0 con 0x557976e542a0 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98e800640 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 0x557976e5db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc98d400640 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e5e0d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc97c012700 tx=0x7fc97c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ae01ff8553afd0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.299+0000 7fc96fe00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.300+0000 7fc96fe00640 1 -- 172.21.15.60:0/1817371100 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c00f8c0 con 0x557976e542a0 2024-02-26T10:44:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.300+0000 7fc96fe00640 1 -- 172.21.15.60:0/1817371100 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fc97c00fa60 con 0x557976e542a0 2024-02-26T10:44:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.300+0000 7fc99822f740 1 -- 172.21.15.60:0/1817371100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc988003e50 con 0x557976e542a0 2024-02-26T10:44:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.300+0000 7fc96fe00640 1 -- 172.21.15.60:0/1817371100 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc97c025050 con 0x557976e542a0 2024-02-26T10:44:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.300+0000 7fc99822f740 10 client.15084 fetch_fsmap learned FSMap version 23 2024-02-26T10:44:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.300+0000 7fc99822f740 1 -- 172.21.15.60:0/1817371100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557976e68550 con 0x557976e542a0 2024-02-26T10:44:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.300+0000 7fc96fe00640 1 -- 172.21.15.60:0/1817371100 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc988003e50 con 0x557976e542a0 2024-02-26T10:44:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.301+0000 7fc96fe00640 1 -- 172.21.15.60:0/1817371100 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc97c010430 con 0x557976e542a0 2024-02-26T10:44:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.301+0000 7fc99822f740 10 client.15084 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:44:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.301+0000 7fc99822f740 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T10:44:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.301+0000 7fc99822f740 20 client.15084 populate_metadata read hostname 'smithi060' 2024-02-26T10:44:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.301+0000 7fc99822f740 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557976e6a930 0x557976e69a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.301+0000 7fc99822f740 4 client.15084 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:44:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.301+0000 7fc99822f740 1 -- 172.21.15.60:0/1817371100 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557976dc5b10 con 0x557976e6a930 2024-02-26T10:44:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.301+0000 7fc98e800640 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557976e6a930 0x557976e69a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.302+0000 7fc98e800640 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557976e6a930 0x557976e69a10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2079c33db258d6e server_cookie=7f03e7523681dfe1 in_seq=0 out_seq=0 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.302+0000 7fc96fe00640 10 client.15084 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.302+0000 7fc96fe00640 1 -- 172.21.15.60:0/1817371100 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557976dc5b10 con 0x557976e6a930 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.302+0000 7fc96fe00640 10 client.15084 handle_command_reply: tid=0 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.302+0000 7fc99822f740 1 client.15084 shutdown 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 -- 172.21.15.60:0/1817371100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 msgr2=0x557976e5e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e5e0d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc97c012700 tx=0x7fc97c012bc0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 -- 172.21.15.60:0/1817371100 shutdown_connections 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557976e6a930 0x557976e69a10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e59410 0x557976e65110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e5e0d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc96fe00640 0 client.15084 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 --2- 172.21.15.60:0/1817371100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 0x557976e5db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 -- 172.21.15.60:0/1817371100 >> 172.21.15.60:0/1817371100 conn(0x557976e4ee20 msgr2=0x557976e4e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 -- 172.21.15.60:0/1817371100 shutdown_connections 2024-02-26T10:44:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 1 -- 172.21.15.60:0/1817371100 wait complete. 2024-02-26T10:44:57.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.303+0000 7fc99822f740 20 client.15084 trim_cache size 0 max 16384 2024-02-26T10:44:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.320+0000 7fc99822f740 1 Processor -- start 2024-02-26T10:44:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.320+0000 7fc99822f740 1 -- start start 2024-02-26T10:44:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.320+0000 7fc99822f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e4ee20 0x557976e72dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.320+0000 7fc99822f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 0x557976e650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.320+0000 7fc99822f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e52080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.320+0000 7fc99822f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976e6a150 con 0x557976e542a0 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.320+0000 7fc99822f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976e6a7b0 con 0x557976e4ee20 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.320+0000 7fc99822f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976e6ae10 con 0x557976e538b0 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e52080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e52080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35394/0 (socket says 172.21.15.60:35394) 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 -- 172.21.15.60:0/636999464 learned_addr learned my addr 172.21.15.60:0/636999464 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98d400640 1 --2- 172.21.15.60:0/636999464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 0x557976e650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 --2- 172.21.15.60:0/636999464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e4ee20 0x557976e72dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 -- 172.21.15.60:0/636999464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 msgr2=0x557976e650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 --2- 172.21.15.60:0/636999464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 0x557976e650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 -- 172.21.15.60:0/636999464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e4ee20 msgr2=0x557976e72dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 --2- 172.21.15.60:0/636999464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e4ee20 0x557976e72dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 -- 172.21.15.60:0/636999464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557976e534f0 con 0x557976e542a0 2024-02-26T10:44:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.321+0000 7fc98e800640 1 --2- 172.21.15.60:0/636999464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e52080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc9880117b0 tx=0x7fc988005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda9e24ec1eca793 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc97aa00640 1 -- 172.21.15.60:0/636999464 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988025040 con 0x557976e542a0 2024-02-26T10:44:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc97aa00640 1 -- 172.21.15.60:0/636999464 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc98800fdb0 con 0x557976e542a0 2024-02-26T10:44:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc97aa00640 1 -- 172.21.15.60:0/636999464 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988005070 con 0x557976e542a0 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc99822f740 1 -- 172.21.15.60:0/636999464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 msgr2=0x557976e52080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc99822f740 1 --2- 172.21.15.60:0/636999464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e52080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc9880117b0 tx=0x7fc988005fd0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc99822f740 1 -- 172.21.15.60:0/636999464 shutdown_connections 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc99822f740 1 --2- 172.21.15.60:0/636999464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e542a0 0x557976e52080 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc99822f740 1 --2- 172.21.15.60:0/636999464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e538b0 0x557976e650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc99822f740 1 --2- 172.21.15.60:0/636999464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e4ee20 0x557976e72dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc99822f740 1 -- 172.21.15.60:0/636999464 >> 172.21.15.60:0/636999464 conn(0x557976e58f50 msgr2=0x557976e4f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.322+0000 7fc99822f740 1 -- 172.21.15.60:0/636999464 shutdown_connections 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.323+0000 7fc99822f740 1 -- 172.21.15.60:0/636999464 wait complete. 2024-02-26T10:44:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.323+0000 7fc99822f740 1 -- start start 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.323+0000 7fc99822f740 1 Processor -- start 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.323+0000 7fc99822f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e4ee20 0x557976e61dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.323+0000 7fc99822f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e538b0 0x557976e62310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.323+0000 7fc99822f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e542a0 0x557976ec8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.323+0000 7fc99822f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976e6bad0 con 0x557976e4ee20 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.323+0000 7fc99822f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976e6b470 con 0x557976e538b0 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.323+0000 7fc99822f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557976e6c130 con 0x557976e542a0 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc99822f740 10 client.0 _ll_register_callbacks cb 0x7fc988009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e4ee20 0x557976e61dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e4ee20 0x557976e61dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35400/0 (socket says 172.21.15.60:35400) 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98e800640 1 -- 172.21.15.60:0/450050767 learned_addr learned my addr 172.21.15.60:0/450050767 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98d400640 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e538b0 0x557976e62310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98e800640 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e542a0 0x557976ec8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98e800640 1 -- 172.21.15.60:0/450050767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e542a0 msgr2=0x557976ec8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98e800640 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e542a0 0x557976ec8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98e800640 1 -- 172.21.15.60:0/450050767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e538b0 msgr2=0x557976e62310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98e800640 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e538b0 0x557976e62310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98e800640 1 -- 172.21.15.60:0/450050767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557976e62900 con 0x557976e4ee20 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.324+0000 7fc98d400640 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e538b0 0x557976e62310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.325+0000 7fc98e800640 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e4ee20 0x557976e61dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc98800a010 tx=0x7fc98801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ecd07dc72523e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.325+0000 7fc96fe00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.325+0000 7fc96fe00640 1 -- 172.21.15.60:0/450050767 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98802a350 con 0x557976e4ee20 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.325+0000 7fc96fe00640 1 -- 172.21.15.60:0/450050767 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fc98802a7d0 con 0x557976e4ee20 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.325+0000 7fc99822f740 1 -- 172.21.15.60:0/450050767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc98800fdb0 con 0x557976e4ee20 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.325+0000 7fc96fe00640 1 -- 172.21.15.60:0/450050767 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc98803c050 con 0x557976e4ee20 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.325+0000 7fc99822f740 10 client.15090 fetch_fsmap learned FSMap version 23 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.325+0000 7fc99822f740 1 -- 172.21.15.60:0/450050767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557976eca2f0 con 0x557976e4ee20 2024-02-26T10:44:57.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.326+0000 7fc96fe00640 1 -- 172.21.15.60:0/450050767 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc98800fdb0 con 0x557976e4ee20 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.326+0000 7fc96fe00640 1 -- 172.21.15.60:0/450050767 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc988018020 con 0x557976e4ee20 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.326+0000 7fc99822f740 10 client.15090 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.326+0000 7fc99822f740 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.326+0000 7fc99822f740 20 client.15090 populate_metadata read hostname 'smithi060' 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.326+0000 7fc99822f740 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557976e59410 0x557976eca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.326+0000 7fc99822f740 4 client.15090 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.326+0000 7fc99822f740 1 -- 172.21.15.60:0/450050767 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557976dbfe60 con 0x557976e59410 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.326+0000 7fc98d400640 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557976e59410 0x557976eca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc98d400640 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557976e59410 0x557976eca4c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c5b21f014c01f4 server_cookie=6949a4a39e8af601 in_seq=0 out_seq=0 2024-02-26T10:44:57.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc96fe00640 10 client.15090 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:44:57.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc96fe00640 1 -- 172.21.15.60:0/450050767 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557976dbfe60 con 0x557976e59410 2024-02-26T10:44:57.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc96fe00640 10 client.15090 handle_command_reply: tid=0 2024-02-26T10:44:57.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 client.15090 shutdown 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 -- 172.21.15.60:0/450050767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e4ee20 msgr2=0x557976e61dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e4ee20 0x557976e61dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc98800a010 tx=0x7fc98801c040 comp rx=0 tx=0).stop 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 -- 172.21.15.60:0/450050767 shutdown_connections 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557976e59410 0x557976eca4c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557976e538b0 0x557976e62310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557976e542a0 0x557976ec8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 --2- 172.21.15.60:0/450050767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557976e4ee20 0x557976e61dd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 -- 172.21.15.60:0/450050767 >> 172.21.15.60:0/450050767 conn(0x557976e58f50 msgr2=0x557976e72dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc96fe00640 0 client.15090 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 -- 172.21.15.60:0/450050767 shutdown_connections 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 1 -- 172.21.15.60:0/450050767 wait complete. 2024-02-26T10:44:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.327+0000 7fc99822f740 20 client.15090 trim_cache size 0 max 16384 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24587, 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 134.26231970800001, 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 23, 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 95, 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 91, 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 141.20646782599999 2024-02-26T10:44:57.328 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:44:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 -- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9740a1d80 msgr2=0x7fc9740a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9740a1d80 0x7fc9740a4240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9800b8b30 tx=0x7fc980005670 comp rx=0 tx=0).stop 2024-02-26T10:44:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 -- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 msgr2=0x7fc990075010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:44:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 0x7fc990075010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc98002c640 comp rx=0 tx=0).stop 2024-02-26T10:44:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 -- 172.21.15.60:0/3875057013 shutdown_connections 2024-02-26T10:44:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9740a1d80 0x7fc9740a4240 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc990102350 0x7fc990073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc990102d10 0x7fc990073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 --2- 172.21.15.60:0/3875057013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc990101160 0x7fc990075010 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:44:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 -- 172.21.15.60:0/3875057013 >> 172.21.15.60:0/3875057013 conn(0x7fc9900fc6d0 msgr2=0x7fc990105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:44:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 -- 172.21.15.60:0/3875057013 shutdown_connections 2024-02-26T10:44:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:44:57.328+0000 7fc98f200640 1 -- 172.21.15.60:0/3875057013 wait complete. 2024-02-26T10:44:57.341 INFO:tasks.cephadm:Setup complete, yielding 2024-02-26T10:44:57.341 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-26T10:44:57.350 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T10:44:57.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- bash -c 'ceph orch status' 2024-02-26T10:44:57.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:57 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2569823843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:44:57.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:57 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2569823843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:44:57.779 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:44:57.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:57 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2569823843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:44:58.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:44:58 smithi060 ceph-mon[31539]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:58.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:44:58 smithi100 ceph-mon[36051]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:44:58.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:44:58 smithi113 ceph-mon[36126]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.088+0000 7f694ee00700 1 -- 172.21.15.60:0/1205357893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948102810 msgr2=0x7f6948104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.088+0000 7f694ee00700 1 --2- 172.21.15.60:0/1205357893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948102810 0x7f6948104c00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6938009990 tx=0x7f6938009ca0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.088+0000 7f694ee00700 1 -- 172.21.15.60:0/1205357893 shutdown_connections 2024-02-26T10:45:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.088+0000 7f694ee00700 1 --2- 172.21.15.60:0/1205357893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948107a70 0x7f6948109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.088+0000 7f694ee00700 1 --2- 172.21.15.60:0/1205357893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948102810 0x7f6948104c00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.088+0000 7f694ee00700 1 --2- 172.21.15.60:0/1205357893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6948105140 0x7f6948107530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.088+0000 7f694ee00700 1 -- 172.21.15.60:0/1205357893 >> 172.21.15.60:0/1205357893 conn(0x7f69480fc220 msgr2=0x7f69480fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.088+0000 7f694ee00700 1 -- 172.21.15.60:0/1205357893 shutdown_connections 2024-02-26T10:45:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.089+0000 7f694ee00700 1 -- 172.21.15.60:0/1205357893 wait complete. 2024-02-26T10:45:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.090+0000 7f694ee00700 1 Processor -- start 2024-02-26T10:45:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.090+0000 7f694ee00700 1 -- start start 2024-02-26T10:45:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.090+0000 7f694ee00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102810 0x7f694819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.090+0000 7f694ee00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6948105140 0x7f694819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.090+0000 7f694ee00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948107a70 0x7f69481a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.090+0000 7f694ee00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694810d980 con 0x7f6948105140 2024-02-26T10:45:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.090+0000 7f694ee00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694810d800 con 0x7f6948107a70 2024-02-26T10:45:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.090+0000 7f694ee00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694810db00 con 0x7f6948102810 2024-02-26T10:45:00.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f694ce00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948107a70 0x7f69481a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f6947400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6948105140 0x7f694819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f694ce00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948107a70 0x7f69481a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35834/0 (socket says 172.21.15.60:35834) 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f694ce00700 1 -- 172.21.15.60:0/1790519089 learned_addr learned my addr 172.21.15.60:0/1790519089 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f6947e00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102810 0x7f694819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f694ce00700 1 -- 172.21.15.60:0/1790519089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102810 msgr2=0x7f694819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f694ce00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102810 0x7f694819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f694ce00700 1 -- 172.21.15.60:0/1790519089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6948105140 msgr2=0x7f694819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f694ce00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6948105140 0x7f694819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.091+0000 7f694ce00700 1 -- 172.21.15.60:0/1790519089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69481a1e20 con 0x7f6948107a70 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.092+0000 7f694ce00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948107a70 0x7f69481a1780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f694000ef90 tx=0x7f694000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.092+0000 7f6944c00700 1 -- 172.21.15.60:0/1790519089 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69400092e0 con 0x7f6948107a70 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.092+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69481a20b0 con 0x7f6948107a70 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.092+0000 7f6944c00700 1 -- 172.21.15.60:0/1790519089 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6940004bb0 con 0x7f6948107a70 2024-02-26T10:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.092+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69481a2640 con 0x7f6948107a70 2024-02-26T10:45:00.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.092+0000 7f6944c00700 1 -- 172.21.15.60:0/1790519089 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940004280 con 0x7f6948107a70 2024-02-26T10:45:00.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.093+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6948068a10 con 0x7f6948107a70 2024-02-26T10:45:00.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.095+0000 7f6944c00700 1 -- 172.21.15.60:0/1790519089 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f694001b030 con 0x7f6948107a70 2024-02-26T10:45:00.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.097+0000 7f6944c00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f692c0a1e90 0x7f692c0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:00.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.097+0000 7f6944c00700 1 -- 172.21.15.60:0/1790519089 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f69400cdc80 con 0x7f6948107a70 2024-02-26T10:45:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.097+0000 7f6947e00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f692c0a1e90 0x7f692c0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.098+0000 7f6947e00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f692c0a1e90 0x7f692c0a4350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f693800b7b0 tx=0x7f693800b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.101+0000 7f6944c00700 1 -- 172.21.15.60:0/1790519089 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6940022020 con 0x7f6948107a70 2024-02-26T10:45:00.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.346+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6948062190 con 0x7f692c0a1e90 2024-02-26T10:45:00.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.347+0000 7f6944c00700 1 -- 172.21.15.60:0/1790519089 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6948062190 con 0x7f692c0a1e90 2024-02-26T10:45:00.343 INFO:teuthology.orchestra.run.smithi060.stdout:Backend: cephadm 2024-02-26T10:45:00.343 INFO:teuthology.orchestra.run.smithi060.stdout:Available: Yes 2024-02-26T10:45:00.344 INFO:teuthology.orchestra.run.smithi060.stdout:Paused: No 2024-02-26T10:45:00.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.350+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f692c0a1e90 msgr2=0x7f692c0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:00.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.350+0000 7f694ee00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f692c0a1e90 0x7f692c0a4350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f693800b7b0 tx=0x7f693800b880 comp rx=0 tx=0).stop 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.350+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948107a70 msgr2=0x7f69481a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.350+0000 7f694ee00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948107a70 0x7f69481a1780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f694000ef90 tx=0x7f694000c3a0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.351+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 shutdown_connections 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.351+0000 7f694ee00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f692c0a1e90 0x7f692c0a4350 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.351+0000 7f694ee00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6948107a70 0x7f69481a1780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.351+0000 7f694ee00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102810 0x7f694819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.351+0000 7f694ee00700 1 --2- 172.21.15.60:0/1790519089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6948105140 0x7f694819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.351+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 >> 172.21.15.60:0/1790519089 conn(0x7f69480fc220 msgr2=0x7f6948102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.351+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 shutdown_connections 2024-02-26T10:45:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:00.351+0000 7f694ee00700 1 -- 172.21.15.60:0/1790519089 wait complete. 2024-02-26T10:45:00.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:00 smithi060 ceph-mon[31539]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:00.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:00 smithi100 ceph-mon[36051]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:00.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:00 smithi113 ceph-mon[36126]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:01.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-26T10:45:01.658 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:45:01.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:01 smithi060 ceph-mon[31539]: from='client.25211 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:01.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:01 smithi113 ceph-mon[36126]: from='client.25211 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:01.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:01 smithi100 ceph-mon[36051]: from='client.25211 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:02.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:02 smithi060 ceph-mon[31539]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:02.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:02 smithi113 ceph-mon[36126]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:02.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:02 smithi100 ceph-mon[36051]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:03.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.964+0000 7f319f400700 1 -- 172.21.15.60:0/3384229632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3198105110 msgr2=0x7f3198107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:03.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.964+0000 7f319f400700 1 --2- 172.21.15.60:0/3384229632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3198105110 0x7f3198107500 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3184009990 tx=0x7f3184009ca0 comp rx=0 tx=0).stop 2024-02-26T10:45:03.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.964+0000 7f319f400700 1 -- 172.21.15.60:0/3384229632 shutdown_connections 2024-02-26T10:45:03.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.964+0000 7f319f400700 1 --2- 172.21.15.60:0/3384229632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3198107a40 0x7f3198109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:03.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.964+0000 7f319f400700 1 --2- 172.21.15.60:0/3384229632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31981027e0 0x7f3198104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:03.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.964+0000 7f319f400700 1 --2- 172.21.15.60:0/3384229632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3198105110 0x7f3198107500 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:03.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.964+0000 7f319f400700 1 -- 172.21.15.60:0/3384229632 >> 172.21.15.60:0/3384229632 conn(0x7f31980fc1f0 msgr2=0x7f31980fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:03.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.964+0000 7f319f400700 1 -- 172.21.15.60:0/3384229632 shutdown_connections 2024-02-26T10:45:03.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.964+0000 7f319f400700 1 -- 172.21.15.60:0/3384229632 wait complete. 2024-02-26T10:45:03.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.965+0000 7f319f400700 1 Processor -- start 2024-02-26T10:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.965+0000 7f319f400700 1 -- start start 2024-02-26T10:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319f400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31981027e0 0x7f31981969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319f400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3198105110 0x7f3198196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319f400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3198107a40 0x7f3198193570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319f400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319810d9a0 con 0x7f31981027e0 2024-02-26T10:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319f400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319810d820 con 0x7f3198105110 2024-02-26T10:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319f400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319810db20 con 0x7f3198107a40 2024-02-26T10:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319ca00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31981027e0 0x7f31981969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319ca00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31981027e0 0x7f31981969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33700/0 (socket says 172.21.15.60:33700) 2024-02-26T10:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319ca00700 1 -- 172.21.15.60:0/2582397985 learned_addr learned my addr 172.21.15.60:0/2582397985 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f319d400700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3198107a40 0x7f3198193570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.966+0000 7f3197e00700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3198105110 0x7f3198196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f319ca00700 1 -- 172.21.15.60:0/2582397985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3198107a40 msgr2=0x7f3198193570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f319ca00700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3198107a40 0x7f3198193570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f319ca00700 1 -- 172.21.15.60:0/2582397985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3198105110 msgr2=0x7f3198196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f319ca00700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3198105110 0x7f3198196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f319ca00700 1 -- 172.21.15.60:0/2582397985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3198193e40 con 0x7f31981027e0 2024-02-26T10:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f3197e00700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3198105110 0x7f3198196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f319ca00700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31981027e0 0x7f31981969f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f318c00ef10 tx=0x7f318c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f3195600700 1 -- 172.21.15.60:0/2582397985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c00c9c0 con 0x7f31981027e0 2024-02-26T10:45:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f3195600700 1 -- 172.21.15.60:0/2582397985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f318c019070 con 0x7f31981027e0 2024-02-26T10:45:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31981a6680 con 0x7f31981027e0 2024-02-26T10:45:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f3195600700 1 -- 172.21.15.60:0/2582397985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c007e60 con 0x7f31981027e0 2024-02-26T10:45:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.967+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31981a6a70 con 0x7f31981027e0 2024-02-26T10:45:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.968+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3198068a10 con 0x7f31981027e0 2024-02-26T10:45:03.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.970+0000 7f3195600700 1 -- 172.21.15.60:0/2582397985 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f318c012050 con 0x7f31981027e0 2024-02-26T10:45:03.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.972+0000 7f3195600700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31800a1e10 0x7f31800a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:03.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.972+0000 7f3195600700 1 -- 172.21.15.60:0/2582397985 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f318c0ce130 con 0x7f31981027e0 2024-02-26T10:45:03.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.972+0000 7f3197e00700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31800a1e10 0x7f31800a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:03.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.973+0000 7f3197e00700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31800a1e10 0x7f31800a42d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f318400b7b0 tx=0x7f318400b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:03.977+0000 7f3195600700 1 -- 172.21.15.60:0/2582397985 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f318c010040 con 0x7f31981027e0 2024-02-26T10:45:04.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.213+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3198062190 con 0x7f31800a1e10 2024-02-26T10:45:04.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.257+0000 7f3195600700 1 -- 172.21.15.60:0/2582397985 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7f3198062190 con 0x7f31800a1e10 2024-02-26T10:45:04.255 INFO:teuthology.orchestra.run.smithi060.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-26T10:45:04.255 INFO:teuthology.orchestra.run.smithi060.stdout:mds.a smithi060 running (2m) 28s ago 2m 12.4M - 19.0.0-1686-g6085b91f 07b056410b37 5647321050e4 2024-02-26T10:45:04.255 INFO:teuthology.orchestra.run.smithi060.stdout:mds.b smithi100 running (2m) 79s ago 2m 25.2M - 19.0.0-1686-g6085b91f 07b056410b37 ccbf93bac5b9 2024-02-26T10:45:04.255 INFO:teuthology.orchestra.run.smithi060.stdout:mds.c smithi113 running (2m) 79s ago 2m 18.0M - 19.0.0-1686-g6085b91f 07b056410b37 6f16811c6824 2024-02-26T10:45:04.255 INFO:teuthology.orchestra.run.smithi060.stdout:mds.d smithi060 running (2m) 28s ago 2m 12.4M - 19.0.0-1686-g6085b91f 07b056410b37 15451d5bb680 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mds.e smithi100 running (2m) 79s ago 2m 15.6M - 19.0.0-1686-g6085b91f 07b056410b37 e36bf7a5fde2 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mds.f smithi113 running (2m) 79s ago 2m 17.1M - 19.0.0-1686-g6085b91f 07b056410b37 8d5f20c4882e 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mds.g smithi060 running (2m) 28s ago 2m 12.3M - 19.0.0-1686-g6085b91f 07b056410b37 07cd66bddb7a 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mds.h smithi100 running (2m) 79s ago 2m 15.7M - 19.0.0-1686-g6085b91f 07b056410b37 bc785a3a01e3 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mds.i smithi113 running (2m) 79s ago 2m 16.6M - 19.0.0-1686-g6085b91f 07b056410b37 a587f82898b4 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mds.j smithi060 running (2m) 28s ago 2m 12.4M - 19.0.0-1686-g6085b91f 07b056410b37 d938fc1db71a 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mds.k smithi100 running (3m) 79s ago 3m 14.2M - 19.0.0-1686-g6085b91f 07b056410b37 25c2b7bad73b 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mds.l smithi113 running (3m) 79s ago 3m 14.2M - 19.0.0-1686-g6085b91f 07b056410b37 2110a133da3d 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.x smithi060 *:9283,8765 running (11m) 28s ago 11m 513M - 19.0.0-1686-g6085b91f 07b056410b37 397be5014c6a 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.y smithi100 *:8443,8765 running (9m) 79s ago 9m 453M - 19.0.0-1686-g6085b91f 07b056410b37 1cf2df4ef95e 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.z smithi113 *:8443,8765 running (9m) 79s ago 9m 451M - 19.0.0-1686-g6085b91f 07b056410b37 9242054c1bbf 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mon.a smithi060 running (11m) 28s ago 11m 47.3M 2048M 19.0.0-1686-g6085b91f 07b056410b37 0eaaccfac6f9 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mon.b smithi100 running (9m) 79s ago 9m 40.6M 2048M 19.0.0-1686-g6085b91f 07b056410b37 2aaee5209894 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:mon.c smithi113 running (9m) 79s ago 9m 41.0M 2048M 19.0.0-1686-g6085b91f 07b056410b37 800ead2bfef6 2024-02-26T10:45:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 smithi060 running (8m) 28s ago 8m 45.2M 4273M 19.0.0-1686-g6085b91f 07b056410b37 c74db9e103be 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 smithi100 running (8m) 79s ago 8m 57.8M 4270M 19.0.0-1686-g6085b91f 07b056410b37 f1b23795e739 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 smithi113 running (7m) 79s ago 7m 58.5M 4270M 19.0.0-1686-g6085b91f 07b056410b37 fd97e01dd665 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 smithi060 running (7m) 28s ago 7m 53.5M 4273M 19.0.0-1686-g6085b91f 07b056410b37 5647faff3888 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 smithi100 running (6m) 79s ago 6m 57.8M 4270M 19.0.0-1686-g6085b91f 07b056410b37 a7db84214aac 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 smithi113 running (6m) 79s ago 6m 58.7M 4270M 19.0.0-1686-g6085b91f 07b056410b37 d55eb5fd563e 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 smithi060 running (5m) 28s ago 5m 52.4M 4273M 19.0.0-1686-g6085b91f 07b056410b37 3c4eead1d896 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 smithi100 running (5m) 79s ago 5m 45.5M 4270M 19.0.0-1686-g6085b91f 07b056410b37 a741ec77fcab 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8 smithi113 running (4m) 79s ago 4m 53.2M 4270M 19.0.0-1686-g6085b91f 07b056410b37 1043ad3a4402 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9 smithi060 running (4m) 28s ago 4m 42.0M 4273M 19.0.0-1686-g6085b91f 07b056410b37 f8d98b53910a 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10 smithi100 running (3m) 79s ago 3m 41.8M 4270M 19.0.0-1686-g6085b91f 07b056410b37 dd719bd87672 2024-02-26T10:45:04.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11 smithi113 running (3m) 79s ago 3m 42.1M 4270M 19.0.0-1686-g6085b91f 07b056410b37 f7aed9b784c2 2024-02-26T10:45:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.260+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31800a1e10 msgr2=0x7f31800a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.260+0000 7f319f400700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31800a1e10 0x7f31800a42d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f318400b7b0 tx=0x7f318400b880 comp rx=0 tx=0).stop 2024-02-26T10:45:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.260+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31981027e0 msgr2=0x7f31981969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.260+0000 7f319f400700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31981027e0 0x7f31981969f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f318c00ef10 tx=0x7f318c00c3a0 comp rx=0 tx=0).stop 2024-02-26T10:45:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.260+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 shutdown_connections 2024-02-26T10:45:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.260+0000 7f319f400700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31800a1e10 0x7f31800a42d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.260+0000 7f319f400700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3198107a40 0x7f3198193570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.260+0000 7f319f400700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31981027e0 0x7f31981969f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.260+0000 7f319f400700 1 --2- 172.21.15.60:0/2582397985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3198105110 0x7f3198196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.261+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 >> 172.21.15.60:0/2582397985 conn(0x7f31980fc1f0 msgr2=0x7f3198101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.261+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 shutdown_connections 2024-02-26T10:45:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:04.261+0000 7f319f400700 1 -- 172.21.15.60:0/2582397985 wait complete. 2024-02-26T10:45:04.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:04 smithi060 ceph-mon[31539]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:04.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:04 smithi113 ceph-mon[36126]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:04.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:04 smithi100 ceph-mon[36051]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:05.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-26T10:45:05.630 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:45:05.778 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:05 smithi060 ceph-mon[31539]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:05.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:05 smithi113 ceph-mon[36126]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:05.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:05 smithi100 ceph-mon[36051]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:06.657 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:06 smithi060 ceph-mon[31539]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:06.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:06 smithi113 ceph-mon[36126]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:06.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:06 smithi100 ceph-mon[36051]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:07.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.938+0000 7faab3400700 1 -- 172.21.15.60:0/2476442757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac103950 msgr2=0x7faaac10a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:07.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.938+0000 7faab3400700 1 --2- 172.21.15.60:0/2476442757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac103950 0x7faaac10a200 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faaa400b0d0 tx=0x7faaa400b3e0 comp rx=0 tx=0).stop 2024-02-26T10:45:07.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.938+0000 7faab3400700 1 -- 172.21.15.60:0/2476442757 shutdown_connections 2024-02-26T10:45:07.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.938+0000 7faab3400700 1 --2- 172.21.15.60:0/2476442757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac103950 0x7faaac10a200 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:07.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.938+0000 7faab3400700 1 --2- 172.21.15.60:0/2476442757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaac101db0 0x7faaac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.938+0000 7faab3400700 1 --2- 172.21.15.60:0/2476442757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaac102fb0 0x7faaac103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.938+0000 7faab3400700 1 -- 172.21.15.60:0/2476442757 >> 172.21.15.60:0/2476442757 conn(0x7faaac0fd350 msgr2=0x7faaac0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.938+0000 7faab3400700 1 -- 172.21.15.60:0/2476442757 shutdown_connections 2024-02-26T10:45:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.939+0000 7faab3400700 1 -- 172.21.15.60:0/2476442757 wait complete. 2024-02-26T10:45:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab3400700 1 Processor -- start 2024-02-26T10:45:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab3400700 1 -- start start 2024-02-26T10:45:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab3400700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaac101db0 0x7faaac19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab3400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaac102fb0 0x7faaac19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab3400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac103950 0x7faaac1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab3400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac10d9f0 con 0x7faaac103950 2024-02-26T10:45:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab3400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac10d870 con 0x7faaac102fb0 2024-02-26T10:45:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab3400700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac10db70 con 0x7faaac101db0 2024-02-26T10:45:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab0a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaac101db0 0x7faaac19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab1400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac103950 0x7faaac1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faaabe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaac102fb0 0x7faaac19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab0a00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaac101db0 0x7faaac19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:48392/0 (socket says 172.21.15.60:48392) 2024-02-26T10:45:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.940+0000 7faab0a00700 1 -- 172.21.15.60:0/2123609719 learned_addr learned my addr 172.21.15.60:0/2123609719 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.941+0000 7faab0a00700 1 -- 172.21.15.60:0/2123609719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaac102fb0 msgr2=0x7faaac19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.941+0000 7faab0a00700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaac102fb0 0x7faaac19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.941+0000 7faab0a00700 1 -- 172.21.15.60:0/2123609719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac103950 msgr2=0x7faaac1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.941+0000 7faab0a00700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac103950 0x7faaac1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.941+0000 7faab0a00700 1 -- 172.21.15.60:0/2123609719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac1a1db0 con 0x7faaac101db0 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.941+0000 7faab0a00700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaac101db0 0x7faaac19a150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faaa000e980 tx=0x7faaa000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.941+0000 7faaa9600700 1 -- 172.21.15.60:0/2123609719 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0004230 con 0x7faaac101db0 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.941+0000 7faaa9600700 1 -- 172.21.15.60:0/2123609719 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faaa00043d0 con 0x7faaac101db0 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.941+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac1a20a0 con 0x7faaac101db0 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.942+0000 7faaa9600700 1 -- 172.21.15.60:0/2123609719 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0003dc0 con 0x7faaac101db0 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.942+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac1a2590 con 0x7faaac101db0 2024-02-26T10:45:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.943+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac068a10 con 0x7faaac101db0 2024-02-26T10:45:07.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.945+0000 7faaa9600700 1 -- 172.21.15.60:0/2123609719 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7faaa000c810 con 0x7faaac101db0 2024-02-26T10:45:07.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.946+0000 7faaa9600700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa940a1e10 0x7faa940a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:07.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.946+0000 7faaa9600700 1 -- 172.21.15.60:0/2123609719 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7faaa00ce400 con 0x7faaac101db0 2024-02-26T10:45:07.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.946+0000 7faaabe00700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa940a1e10 0x7faa940a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:07.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.947+0000 7faaabe00700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa940a1e10 0x7faa940a42d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faa98005950 tx=0x7faa9800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:07.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:07.950+0000 7faaa9600700 1 -- 172.21.15.60:0/2123609719 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faaa009b420 con 0x7faaac101db0 2024-02-26T10:45:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.187+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faaac0756a0 con 0x7faa940a1e10 2024-02-26T10:45:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.204+0000 7faaa9600700 1 -- 172.21.15.60:0/2123609719 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7faaac0756a0 con 0x7faa940a1e10 2024-02-26T10:45:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-26T10:45:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:mds.all 12/12 83s ago 3m smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l;count:12 2024-02-26T10:45:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:mgr 3/3 83s ago 9m smithi060=x;smithi100=y;smithi113=z;count:3 2024-02-26T10:45:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:mon 3/3 83s ago 9m smithi060:172.21.15.60=a;smithi100:172.21.15.100=b;smithi113:172.21.15.113=c;count:3 2024-02-26T10:45:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:osd 12 83s ago - 2024-02-26T10:45:08.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa940a1e10 msgr2=0x7faa940a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:08.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa940a1e10 0x7faa940a42d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faa98005950 tx=0x7faa9800b410 comp rx=0 tx=0).stop 2024-02-26T10:45:08.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaac101db0 msgr2=0x7faaac19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:08.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaac101db0 0x7faaac19a150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faaa000e980 tx=0x7faaa000ec90 comp rx=0 tx=0).stop 2024-02-26T10:45:08.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 shutdown_connections 2024-02-26T10:45:08.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa940a1e10 0x7faa940a42d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:08.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac103950 0x7faaac1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaac101db0 0x7faaac19a150 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 --2- 172.21.15.60:0/2123609719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaac102fb0 0x7faaac19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 >> 172.21.15.60:0/2123609719 conn(0x7faaac0fd350 msgr2=0x7faaac073f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 shutdown_connections 2024-02-26T10:45:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:08.207+0000 7faab3400700 1 -- 172.21.15.60:0/2123609719 wait complete. 2024-02-26T10:45:08.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:08 smithi060 ceph-mon[31539]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:08.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:08 smithi113 ceph-mon[36126]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:08.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:08 smithi100 ceph-mon[36051]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:09.056 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-26T10:45:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:45:09.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:09 smithi060 ceph-mon[31539]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:09.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:09 smithi113 ceph-mon[36126]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:09.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:09 smithi100 ceph-mon[36051]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:10.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:10 smithi060 ceph-mon[31539]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:10.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:10 smithi113 ceph-mon[36126]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:10.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:10 smithi100 ceph-mon[36051]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.803+0000 7f7b07000700 1 -- 172.21.15.60:0/256364987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00074750 msgr2=0x7f7b00074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.803+0000 7f7b07000700 1 --2- 172.21.15.60:0/256364987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00074750 0x7f7b00074bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0098f0 tx=0x7f7aec009c00 comp rx=0 tx=0).stop 2024-02-26T10:45:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.803+0000 7f7b07000700 1 -- 172.21.15.60:0/256364987 shutdown_connections 2024-02-26T10:45:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.803+0000 7f7b07000700 1 --2- 172.21.15.60:0/256364987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b00109c90 0x7f7b0010c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.803+0000 7f7b07000700 1 --2- 172.21.15.60:0/256364987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b00075d20 0x7f7b00074180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.803+0000 7f7b07000700 1 --2- 172.21.15.60:0/256364987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00074750 0x7f7b00074bd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.803+0000 7f7b07000700 1 -- 172.21.15.60:0/256364987 >> 172.21.15.60:0/256364987 conn(0x7f7b000fd350 msgr2=0x7f7b000ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.803+0000 7f7b07000700 1 -- 172.21.15.60:0/256364987 shutdown_connections 2024-02-26T10:45:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.803+0000 7f7b07000700 1 -- 172.21.15.60:0/256364987 wait complete. 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.805+0000 7f7b07000700 1 Processor -- start 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.805+0000 7f7b07000700 1 -- start start 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.805+0000 7f7b07000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b00074750 0x7f7b0019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.805+0000 7f7b07000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b00075d20 0x7f7b0019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.805+0000 7f7b07000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00109c90 0x7f7b001a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.805+0000 7f7b07000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0010fbb0 con 0x7f7b00075d20 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.805+0000 7f7b07000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0010fa30 con 0x7f7b00109c90 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.805+0000 7f7b07000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0010fd30 con 0x7f7b00074750 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7aff400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b00075d20 0x7f7b0019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7affe00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b00074750 0x7f7b0019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7aff400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b00075d20 0x7f7b0019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33742/0 (socket says 172.21.15.60:33742) 2024-02-26T10:45:11.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7aff400700 1 -- 172.21.15.60:0/2293573898 learned_addr learned my addr 172.21.15.60:0/2293573898 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:11.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7b05000700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00109c90 0x7f7b001a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:11.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7aff400700 1 -- 172.21.15.60:0/2293573898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b00074750 msgr2=0x7f7b0019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:11.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7aff400700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b00074750 0x7f7b0019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:11.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7aff400700 1 -- 172.21.15.60:0/2293573898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00109c90 msgr2=0x7f7b001a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:11.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7aff400700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00109c90 0x7f7b001a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:11.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.806+0000 7f7aff400700 1 -- 172.21.15.60:0/2293573898 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b001a1df0 con 0x7f7b00075d20 2024-02-26T10:45:11.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.807+0000 7f7aff400700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b00075d20 0x7f7b0019a6d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00fca0 tx=0x7f7aec005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:11.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.807+0000 7f7afcc00700 1 -- 172.21.15.60:0/2293573898 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec025070 con 0x7f7b00075d20 2024-02-26T10:45:11.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.807+0000 7f7afcc00700 1 -- 172.21.15.60:0/2293573898 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aec005650 con 0x7f7b00075d20 2024-02-26T10:45:11.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.807+0000 7f7afcc00700 1 -- 172.21.15.60:0/2293573898 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec003c70 con 0x7f7b00075d20 2024-02-26T10:45:11.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.807+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b001a2080 con 0x7f7b00075d20 2024-02-26T10:45:11.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.807+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b001a2480 con 0x7f7b00075d20 2024-02-26T10:45:11.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.808+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b00068a50 con 0x7f7b00075d20 2024-02-26T10:45:11.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.810+0000 7f7afcc00700 1 -- 172.21.15.60:0/2293573898 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f7aec028070 con 0x7f7b00075d20 2024-02-26T10:45:11.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.811+0000 7f7afcc00700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7ae40a1e90 0x7f7ae40a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:11.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.812+0000 7f7afcc00700 1 -- 172.21.15.60:0/2293573898 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f7aec0cea40 con 0x7f7b00075d20 2024-02-26T10:45:11.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.812+0000 7f7affe00700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7ae40a1e90 0x7f7ae40a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:11.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.812+0000 7f7affe00700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7ae40a1e90 0x7f7ae40a4350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009680 tx=0x7f7af0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:11.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:11.816+0000 7f7afcc00700 1 -- 172.21.15.60:0/2293573898 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7aec05bd20 con 0x7f7b00075d20 2024-02-26T10:45:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.048+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b00062190 con 0x7f7ae40a1e90 2024-02-26T10:45:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.051+0000 7f7afcc00700 1 -- 172.21.15.60:0/2293573898 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f7b00062190 con 0x7f7ae40a1e90 2024-02-26T10:45:12.047 INFO:teuthology.orchestra.run.smithi060.stdout:HOST ADDR LABELS STATUS 2024-02-26T10:45:12.047 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060 172.21.15.60 2024-02-26T10:45:12.047 INFO:teuthology.orchestra.run.smithi060.stdout:smithi100 172.21.15.100 2024-02-26T10:45:12.047 INFO:teuthology.orchestra.run.smithi060.stdout:smithi113 172.21.15.113 2024-02-26T10:45:12.047 INFO:teuthology.orchestra.run.smithi060.stdout:3 hosts in cluster 2024-02-26T10:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.054+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7ae40a1e90 msgr2=0x7f7ae40a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.054+0000 7f7b07000700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7ae40a1e90 0x7f7ae40a4350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009680 tx=0x7f7af0009260 comp rx=0 tx=0).stop 2024-02-26T10:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.054+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b00075d20 msgr2=0x7f7b0019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.054+0000 7f7b07000700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b00075d20 0x7f7b0019a6d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00fca0 tx=0x7f7aec005250 comp rx=0 tx=0).stop 2024-02-26T10:45:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.055+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 shutdown_connections 2024-02-26T10:45:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.055+0000 7f7b07000700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f7ae40a1e90 0x7f7ae40a4350 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.055+0000 7f7b07000700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b00109c90 0x7f7b001a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.055+0000 7f7b07000700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b00074750 0x7f7b0019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.055+0000 7f7b07000700 1 --2- 172.21.15.60:0/2293573898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b00075d20 0x7f7b0019a6d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.055+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 >> 172.21.15.60:0/2293573898 conn(0x7f7b000fd350 msgr2=0x7f7b00105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.055+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 shutdown_connections 2024-02-26T10:45:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:12.055+0000 7f7b07000700 1 -- 172.21.15.60:0/2293573898 wait complete. 2024-02-26T10:45:12.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:12 smithi113 ceph-mon[36126]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:12.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:12 smithi060 ceph-mon[31539]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:12.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:12 smithi100 ceph-mon[36051]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:12.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-26T10:45:13.465 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:45:13.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:13 smithi060 ceph-mon[31539]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:13.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:13 smithi113 ceph-mon[36126]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:13.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:13 smithi100 ceph-mon[36051]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:14.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:14 smithi060 ceph-mon[31539]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:14.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:14 smithi113 ceph-mon[36126]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:14.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:14 smithi100 ceph-mon[36051]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:15.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.841+0000 7f8e33e00700 1 -- 172.21.15.60:0/2442565370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e34100ef0 msgr2=0x7f8e34101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:15.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.841+0000 7f8e33e00700 1 --2- 172.21.15.60:0/2442565370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e34100ef0 0x7f8e34101310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8e240099c0 tx=0x7f8e24009cd0 comp rx=0 tx=0).stop 2024-02-26T10:45:15.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.841+0000 7f8e33e00700 1 -- 172.21.15.60:0/2442565370 shutdown_connections 2024-02-26T10:45:15.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.841+0000 7f8e33e00700 1 --2- 172.21.15.60:0/2442565370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e34100310 0x7f8e34101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:15.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.841+0000 7f8e33e00700 1 --2- 172.21.15.60:0/2442565370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e34100ef0 0x7f8e34101310 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:15.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.841+0000 7f8e33e00700 1 --2- 172.21.15.60:0/2442565370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e340ff950 0x7f8e340ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:15.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.841+0000 7f8e33e00700 1 -- 172.21.15.60:0/2442565370 >> 172.21.15.60:0/2442565370 conn(0x7f8e340fb4c0 msgr2=0x7f8e340fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:15.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.841+0000 7f8e33e00700 1 -- 172.21.15.60:0/2442565370 shutdown_connections 2024-02-26T10:45:15.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.841+0000 7f8e33e00700 1 -- 172.21.15.60:0/2442565370 wait complete. 2024-02-26T10:45:15.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.843+0000 7f8e33e00700 1 Processor -- start 2024-02-26T10:45:15.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.843+0000 7f8e33e00700 1 -- start start 2024-02-26T10:45:15.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.843+0000 7f8e33e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e340ff950 0x7f8e3419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:15.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.843+0000 7f8e33e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e34100310 0x7f8e3419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.843+0000 7f8e33e00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e34100ef0 0x7f8e341a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.843+0000 7f8e33e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3410fae0 con 0x7f8e34100310 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.843+0000 7f8e33e00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3410f960 con 0x7f8e340ff950 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.843+0000 7f8e33e00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3410fc60 con 0x7f8e34100ef0 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e340ff950 0x7f8e3419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e34100310 0x7f8e3419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e340ff950 0x7f8e3419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52924/0 (socket says 172.21.15.60:52924) 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32a00700 1 -- 172.21.15.60:0/245810819 learned_addr learned my addr 172.21.15.60:0/245810819 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e33400700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e34100ef0 0x7f8e341a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32000700 1 -- 172.21.15.60:0/245810819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e34100ef0 msgr2=0x7f8e341a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32000700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e34100ef0 0x7f8e341a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32000700 1 -- 172.21.15.60:0/245810819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e340ff950 msgr2=0x7f8e3419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:15.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32000700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e340ff950 0x7f8e3419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:15.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32000700 1 -- 172.21.15.60:0/245810819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e341a1cf0 con 0x7f8e34100310 2024-02-26T10:45:15.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32a00700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e340ff950 0x7f8e3419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:15.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.844+0000 7f8e32000700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e34100310 0x7f8e3419a5d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000ea20 tx=0x7f8e2000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:15.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.845+0000 7f8e2b400700 1 -- 172.21.15.60:0/245810819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000cbd0 con 0x7f8e34100310 2024-02-26T10:45:15.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.845+0000 7f8e2b400700 1 -- 172.21.15.60:0/245810819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e20004500 con 0x7f8e34100310 2024-02-26T10:45:15.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.845+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e341a1fe0 con 0x7f8e34100310 2024-02-26T10:45:15.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.845+0000 7f8e2b400700 1 -- 172.21.15.60:0/245810819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20005020 con 0x7f8e34100310 2024-02-26T10:45:15.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.845+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e341a2440 con 0x7f8e34100310 2024-02-26T10:45:15.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.846+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e34068a10 con 0x7f8e34100310 2024-02-26T10:45:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.848+0000 7f8e2b400700 1 -- 172.21.15.60:0/245810819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8e200051c0 con 0x7f8e34100310 2024-02-26T10:45:15.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.850+0000 7f8e2b400700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8e180a1e10 0x7f8e180a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:15.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.850+0000 7f8e2b400700 1 -- 172.21.15.60:0/245810819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f8e200ce0d0 con 0x7f8e34100310 2024-02-26T10:45:15.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.850+0000 7f8e32a00700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8e180a1e10 0x7f8e180a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:15.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.851+0000 7f8e32a00700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8e180a1e10 0x7f8e180a42d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8e240038c0 tx=0x7f8e24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:15.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:15.852+0000 7f8e2b400700 1 -- 172.21.15.60:0/245810819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8e2009b180 con 0x7f8e34100310 2024-02-26T10:45:16.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.088+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e34062190 con 0x7f8e180a1e10 2024-02-26T10:45:16.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.091+0000 7f8e2b400700 1 -- 172.21.15.60:0/245810819 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f8e34062190 con 0x7f8e180a1e10 2024-02-26T10:45:16.087 INFO:teuthology.orchestra.run.smithi060.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-26T10:45:16.087 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476008A480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-02-26T10:45:16.087 INFO:teuthology.orchestra.run.smithi060.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-02-26T10:45:16.087 INFO:teuthology.orchestra.run.smithi060.stdout:smithi113 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD5171000M2P0SGN 1863G No 3m ago Has a FileSystem, LVM detected 2024-02-26T10:45:16.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8e180a1e10 msgr2=0x7f8e180a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8e180a1e10 0x7f8e180a42d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8e240038c0 tx=0x7f8e24005fb0 comp rx=0 tx=0).stop 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e34100310 msgr2=0x7f8e3419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e34100310 0x7f8e3419a5d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000ea20 tx=0x7f8e2000ed30 comp rx=0 tx=0).stop 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 shutdown_connections 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8e180a1e10 0x7f8e180a42d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e34100ef0 0x7f8e341a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e340ff950 0x7f8e3419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 --2- 172.21.15.60:0/245810819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e34100310 0x7f8e3419a5d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 >> 172.21.15.60:0/245810819 conn(0x7f8e340fb4c0 msgr2=0x7f8e340fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.095+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 shutdown_connections 2024-02-26T10:45:16.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:16.096+0000 7f8e33e00700 1 -- 172.21.15.60:0/245810819 wait complete. 2024-02-26T10:45:16.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:16 smithi113 ceph-mon[36126]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:16.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:16 smithi060 ceph-mon[31539]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:16.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:16 smithi100 ceph-mon[36051]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:17.042 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-26T10:45:17.051 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T10:45:17.051 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-26T10:45:17.537 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:45:17.689 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:17 smithi060 ceph-mon[31539]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:17.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:17 smithi113 ceph-mon[36126]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:17.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:17 smithi100 ceph-mon[36051]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:18.734 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:18 smithi113 ceph-mon[36126]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:18.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:18 smithi060 ceph-mon[31539]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:18.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:18 smithi100 ceph-mon[36051]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.918+0000 7f4802000700 1 -- 172.21.15.60:0/235496493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47fc102f40 msgr2=0x7f47fc1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.918+0000 7f4802000700 1 --2- 172.21.15.60:0/235496493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47fc102f40 0x7f47fc1053d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009990 tx=0x7f47e4009ca0 comp rx=0 tx=0).stop 2024-02-26T10:45:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.919+0000 7f4802000700 1 -- 172.21.15.60:0/235496493 shutdown_connections 2024-02-26T10:45:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.919+0000 7f4802000700 1 --2- 172.21.15.60:0/235496493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47fc105910 0x7f47fc105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.919+0000 7f4802000700 1 --2- 172.21.15.60:0/235496493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47fc1005d0 0x7f47fc102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.919+0000 7f4802000700 1 --2- 172.21.15.60:0/235496493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47fc102f40 0x7f47fc1053d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.919+0000 7f4802000700 1 -- 172.21.15.60:0/235496493 >> 172.21.15.60:0/235496493 conn(0x7f47fc0fc1f0 msgr2=0x7f47fc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.919+0000 7f4802000700 1 -- 172.21.15.60:0/235496493 shutdown_connections 2024-02-26T10:45:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.919+0000 7f4802000700 1 -- 172.21.15.60:0/235496493 wait complete. 2024-02-26T10:45:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.920+0000 7f4802000700 1 Processor -- start 2024-02-26T10:45:19.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f4802000700 1 -- start start 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f4802000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47fc1005d0 0x7f47fc19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f4802000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47fc102f40 0x7f47fc19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f4802000700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47fc105910 0x7f47fc1a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f4802000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc113f00 con 0x7f47fc102f40 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f4802000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc113d80 con 0x7f47fc1005d0 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f4802000700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc114080 con 0x7f47fc105910 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f47fb400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47fc1005d0 0x7f47fc19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f47ebe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47fc102f40 0x7f47fc19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f47fb400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47fc1005d0 0x7f47fc19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52938/0 (socket says 172.21.15.60:52938) 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f47fb400700 1 -- 172.21.15.60:0/1912529353 learned_addr learned my addr 172.21.15.60:0/1912529353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.921+0000 7f47fbe00700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47fc105910 0x7f47fc1a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.922+0000 7f47fb400700 1 -- 172.21.15.60:0/1912529353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47fc105910 msgr2=0x7f47fc1a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.922+0000 7f47fb400700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47fc105910 0x7f47fc1a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.922+0000 7f47fb400700 1 -- 172.21.15.60:0/1912529353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47fc102f40 msgr2=0x7f47fc19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.922+0000 7f47fb400700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47fc102f40 0x7f47fc19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.922+0000 7f47fb400700 1 -- 172.21.15.60:0/1912529353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc1a61e0 con 0x7f47fc1005d0 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.922+0000 7f47fb400700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47fc1005d0 0x7f47fc19e580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00ef10 tx=0x7f47ec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.922+0000 7f47f8c00700 1 -- 172.21.15.60:0/1912529353 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00c9c0 con 0x7f47fc1005d0 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.922+0000 7f47f8c00700 1 -- 172.21.15.60:0/1912529353 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47ec019070 con 0x7f47fc1005d0 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.922+0000 7f47f8c00700 1 -- 172.21.15.60:0/1912529353 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0146e0 con 0x7f47fc1005d0 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.923+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc1a6470 con 0x7f47fc1005d0 2024-02-26T10:45:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.923+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc1a6960 con 0x7f47fc1005d0 2024-02-26T10:45:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.924+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47fc068a10 con 0x7f47fc1005d0 2024-02-26T10:45:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.926+0000 7f47f8c00700 1 -- 172.21.15.60:0/1912529353 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f47ec012050 con 0x7f47fc1005d0 2024-02-26T10:45:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.927+0000 7f47f8c00700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47dc0a1e90 0x7f47dc0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.927+0000 7f47f8c00700 1 -- 172.21.15.60:0/1912529353 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f47ec0cdff0 con 0x7f47fc1005d0 2024-02-26T10:45:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.927+0000 7f47ebe00700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47dc0a1e90 0x7f47dc0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.928+0000 7f47ebe00700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47dc0a1e90 0x7f47dc0a4350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47e4006010 tx=0x7f47e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:19.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:19.932+0000 7f47f8c00700 1 -- 172.21.15.60:0/1912529353 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47ec010040 con 0x7f47fc1005d0 2024-02-26T10:45:20.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.206+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f47fc062960 con 0x7f47fc1005d0 2024-02-26T10:45:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.207+0000 7f47f8c00700 1 -- 172.21.15.60:0/1912529353 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2953 (secure 0 0 0) 0x7f47ec09b070 con 0x7f47fc1005d0 2024-02-26T10:45:20.203 INFO:teuthology.orchestra.run.smithi060.stdout:e23 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:legacy client fscid: 1 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:Filesystem 'cephfs' (1) 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:fs_name cephfs 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 23 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-02-26T10:42:42.076607+0000 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-02-26T10:43:16.029212+0000 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:tableserver 0 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:root 0 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:session_timeout 300 2024-02-26T10:45:20.204 INFO:teuthology.orchestra.run.smithi060.stdout:session_autoclose 300 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:max_file_size 1099511627776 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:max_xattr_size 65536 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:required_client_features {} 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:last_failure 0 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:last_failure_osd_epoch 0 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:max_mds 3 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:in 0,1,2 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:up {0=24587,1=24553,2=24530} 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:failed 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:damaged 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:stopped 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:data_pools [3,4] 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:metadata_pool 2 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:inline_data disabled 2024-02-26T10:45:20.205 INFO:teuthology.orchestra.run.smithi060.stdout:balancer 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:bal_rank_mask -1 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:standby_count_wanted 0 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b{0:24587} state up:active seq 3 addr [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c{0:24529} state up:standby-replay seq 1 addr [v2:172.21.15.113:6836/3968092815,v1:172.21.15.113:6837/3968092815] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.i{1:24553} state up:active seq 14 addr [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.f{1:24517} state up:standby-replay seq 1 addr [v2:172.21.15.113:6834/2109318506,v1:172.21.15.113:6835/2109318506] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.h{2:24530} state up:active seq 15 addr [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.e{2:24506} state up:standby-replay seq 1 addr [v2:172.21.15.100:6834/1201544557,v1:172.21.15.100:6835/1201544557] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:Standby daemons: 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:20.206 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.g{-1:14451} state up:standby seq 1 addr [v2:172.21.15.60:6838/2551740513,v1:172.21.15.60:6839/2551740513] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.207 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a{-1:14469} state up:standby seq 1 addr [v2:172.21.15.60:6840/1960047882,v1:172.21.15.60:6841/1960047882] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.207 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.l{-1:24448} state up:standby seq 1 addr [v2:172.21.15.113:6832/1265971356,v1:172.21.15.113:6833/1265971356] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.207 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.k{-1:24458} state up:standby seq 1 addr [v2:172.21.15.100:6832/1828064682,v1:172.21.15.100:6833/1828064682] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.207 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.j{-1:24478} state up:standby seq 1 addr [v2:172.21.15.60:6834/2576791717,v1:172.21.15.60:6835/2576791717] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.207 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d{-1:24494} state up:standby seq 1 addr [v2:172.21.15.60:6836/2267710624,v1:172.21.15.60:6837/2267710624] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47dc0a1e90 msgr2=0x7f47dc0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47dc0a1e90 0x7f47dc0a4350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47e4006010 tx=0x7f47e40058e0 comp rx=0 tx=0).stop 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47fc1005d0 msgr2=0x7f47fc19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47fc1005d0 0x7f47fc19e580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00ef10 tx=0x7f47ec00c3a0 comp rx=0 tx=0).stop 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 shutdown_connections 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47dc0a1e90 0x7f47dc0a4350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47fc105910 0x7f47fc1a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47fc1005d0 0x7f47fc19e580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 --2- 172.21.15.60:0/1912529353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47fc102f40 0x7f47fc19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 >> 172.21.15.60:0/1912529353 conn(0x7f47fc0fc1f0 msgr2=0x7f47fc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 shutdown_connections 2024-02-26T10:45:20.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:20.211+0000 7f4802000700 1 -- 172.21.15.60:0/1912529353 wait complete. 2024-02-26T10:45:20.209 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T10:45:20.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:20 smithi060 ceph-mon[31539]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:20.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:20 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1912529353' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T10:45:20.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:20 smithi113 ceph-mon[36126]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:20.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:20 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1912529353' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T10:45:20.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:20 smithi100 ceph-mon[36051]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:20.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:20 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1912529353' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T10:45:21.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-26T10:45:21.567 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:45:22.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:22 smithi060 ceph-mon[31539]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:22.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:22 smithi113 ceph-mon[36126]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:22.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:22 smithi100 ceph-mon[36051]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.937+0000 7f4e48200700 1 -- 172.21.15.60:0/1895728300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e40105110 msgr2=0x7f4e40107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.937+0000 7f4e48200700 1 --2- 172.21.15.60:0/1895728300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e40105110 0x7f4e40107500 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009990 tx=0x7f4e2c009ca0 comp rx=0 tx=0).stop 2024-02-26T10:45:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.937+0000 7f4e48200700 1 -- 172.21.15.60:0/1895728300 shutdown_connections 2024-02-26T10:45:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.937+0000 7f4e48200700 1 --2- 172.21.15.60:0/1895728300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e40107a40 0x7f4e40109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.937+0000 7f4e48200700 1 --2- 172.21.15.60:0/1895728300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e401027e0 0x7f4e40104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.937+0000 7f4e48200700 1 --2- 172.21.15.60:0/1895728300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e40105110 0x7f4e40107500 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.937+0000 7f4e48200700 1 -- 172.21.15.60:0/1895728300 >> 172.21.15.60:0/1895728300 conn(0x7f4e400fc1f0 msgr2=0x7f4e400fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.937+0000 7f4e48200700 1 -- 172.21.15.60:0/1895728300 shutdown_connections 2024-02-26T10:45:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.937+0000 7f4e48200700 1 -- 172.21.15.60:0/1895728300 wait complete. 2024-02-26T10:45:23.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.938+0000 7f4e48200700 1 Processor -- start 2024-02-26T10:45:23.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.939+0000 7f4e48200700 1 -- start start 2024-02-26T10:45:23.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.939+0000 7f4e48200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e401027e0 0x7f4e40197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:23.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.939+0000 7f4e48200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e40105110 0x7f4e40198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.939+0000 7f4e48200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e40107a40 0x7f4e4019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.939+0000 7f4e48200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4010d9a0 con 0x7f4e401027e0 2024-02-26T10:45:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.939+0000 7f4e48200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4010d820 con 0x7f4e40105110 2024-02-26T10:45:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.939+0000 7f4e48200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4010db20 con 0x7f4e40107a40 2024-02-26T10:45:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.939+0000 7f4e45800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e401027e0 0x7f4e40197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e45800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e401027e0 0x7f4e40197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51598/0 (socket says 172.21.15.60:51598) 2024-02-26T10:45:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e44e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e40105110 0x7f4e40198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e45800700 1 -- 172.21.15.60:0/295742862 learned_addr learned my addr 172.21.15.60:0/295742862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e46200700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e40107a40 0x7f4e4019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e45800700 1 -- 172.21.15.60:0/295742862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e40107a40 msgr2=0x7f4e4019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e45800700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e40107a40 0x7f4e4019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e45800700 1 -- 172.21.15.60:0/295742862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e40105110 msgr2=0x7f4e40198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e45800700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e40105110 0x7f4e40198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e45800700 1 -- 172.21.15.60:0/295742862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4019fc40 con 0x7f4e401027e0 2024-02-26T10:45:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.940+0000 7f4e45800700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e401027e0 0x7f4e40197fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400e9d0 tx=0x7f4e3400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.941+0000 7f4e3e000700 1 -- 172.21.15.60:0/295742862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400cbd0 con 0x7f4e401027e0 2024-02-26T10:45:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.941+0000 7f4e3e000700 1 -- 172.21.15.60:0/295742862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e3400cd70 con 0x7f4e401027e0 2024-02-26T10:45:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.941+0000 7f4e3e000700 1 -- 172.21.15.60:0/295742862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34018660 con 0x7f4e401027e0 2024-02-26T10:45:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.941+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4019ff30 con 0x7f4e401027e0 2024-02-26T10:45:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.941+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e401a0380 con 0x7f4e401027e0 2024-02-26T10:45:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.942+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e40068a10 con 0x7f4e401027e0 2024-02-26T10:45:23.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.944+0000 7f4e3e000700 1 -- 172.21.15.60:0/295742862 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4e34018820 con 0x7f4e401027e0 2024-02-26T10:45:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.946+0000 7f4e3e000700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e280a1e10 0x7f4e280a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.946+0000 7f4e3e000700 1 -- 172.21.15.60:0/295742862 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f4e340ce8e0 con 0x7f4e401027e0 2024-02-26T10:45:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.946+0000 7f4e44e00700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e280a1e10 0x7f4e280a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.947+0000 7f4e44e00700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e280a1e10 0x7f4e280a42d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b7b0 tx=0x7f4e2c00b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:23.948+0000 7f4e3e000700 1 -- 172.21.15.60:0/295742862 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e3409b900 con 0x7f4e401027e0 2024-02-26T10:45:24.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.173+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f4e4002da60 con 0x7f4e401027e0 2024-02-26T10:45:24.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.176+0000 7f4e3e000700 1 -- 172.21.15.60:0/295742862 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) v1 ==== 56+0+5681 (secure 0 0 0) 0x7f4e3409b200 con 0x7f4e401027e0 2024-02-26T10:45:24.172 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 95 2024-02-26T10:45:24.172 INFO:teuthology.orchestra.run.smithi060.stdout:fsid 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:45:24.172 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-02-26T10:33:28.696158+0000 2024-02-26T10:45:24.172 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-02-26T10:44:39.229830+0000 2024-02-26T10:45:24.172 INFO:teuthology.orchestra.run.smithi060.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:crush_version 27 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:full_ratio 0.95 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:backfillfull_ratio 0.9 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:nearfull_ratio 0.85 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:require_min_compat_client luminous 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:min_compat_client luminous 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:require_osd_release squid 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:stretch_mode_enabled false 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 91 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:max_osd 12 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6802/810878839,v1:172.21.15.60:6803/810878839] [v2:172.21.15.60:6804/810878839,v1:172.21.15.60:6805/810878839] exists,up f23d95e5-b12d-4215-b22a-879bb7807b94 2024-02-26T10:45:24.173 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/3893830913,v1:172.21.15.100:6801/3893830913] [v2:172.21.15.100:6802/3893830913,v1:172.21.15.100:6803/3893830913] exists,up 63269103-ac90-4d99-af6e-e73cd5b8846d 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 up in weight 1 up_from 18 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6800/1221563222,v1:172.21.15.113:6801/1221563222] [v2:172.21.15.113:6802/1221563222,v1:172.21.15.113:6803/1221563222] exists,up 857e383d-a190-46b0-a523-79126cf32a69 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 up in weight 1 up_from 25 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/3095490526,v1:172.21.15.60:6811/3095490526] [v2:172.21.15.60:6812/3095490526,v1:172.21.15.60:6813/3095490526] exists,up ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 up in weight 1 up_from 33 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6808/2467117088,v1:172.21.15.100:6809/2467117088] [v2:172.21.15.100:6810/2467117088,v1:172.21.15.100:6811/2467117088] exists,up c3719710-9869-4caf-bfcf-cbe453306583 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 up in weight 1 up_from 39 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6808/3892917193,v1:172.21.15.113:6809/3892917193] [v2:172.21.15.113:6810/3892917193,v1:172.21.15.113:6811/3892917193] exists,up ecd0dba3-6cb1-4e97-9b63-c83d0485c759 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 up in weight 1 up_from 47 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6818/1002313688,v1:172.21.15.60:6819/1002313688] [v2:172.21.15.60:6820/1002313688,v1:172.21.15.60:6821/1002313688] exists,up cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 up in weight 1 up_from 52 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6816/1677982377,v1:172.21.15.100:6817/1677982377] [v2:172.21.15.100:6818/1677982377,v1:172.21.15.100:6819/1677982377] exists,up c07a3c8c-b460-46fd-8b14-7e042a98495d 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8 up in weight 1 up_from 60 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6816/3414484490,v1:172.21.15.113:6817/3414484490] [v2:172.21.15.113:6818/3414484490,v1:172.21.15.113:6819/3414484490] exists,up a26d9b49-c7af-40f2-96b1-25018e1e9ddf 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9 up in weight 1 up_from 68 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6826/3674402617,v1:172.21.15.60:6827/3674402617] [v2:172.21.15.60:6828/3674402617,v1:172.21.15.60:6829/3674402617] exists,up 2beba404-c823-40fc-9102-990ad3810ce5 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10 up in weight 1 up_from 73 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6824/304375829,v1:172.21.15.100:6825/304375829] [v2:172.21.15.100:6826/304375829,v1:172.21.15.100:6827/304375829] exists,up 1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11 up in weight 1 up_from 81 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6824/2587204649,v1:172.21.15.113:6825/2587204649] [v2:172.21.15.113:6826/2587204649,v1:172.21.15.113:6827/2587204649] exists,up 7ecbb968-c217-40ea-b64a-91e5de0c7327 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.5 [8,11] 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.c [2,4] 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.10 [8,11] 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.14 [2,4] 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.5 [3,9] 2024-02-26T10:45:24.174 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.13 [4,9] 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.17 [4,8] 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.1a [5,6] 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.3c [4,9] 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.f [9,8] 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.14 [9,8] 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.1a [5,8] 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.1c [9,10] 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/603652410 expires 2024-02-27T10:34:36.733932+0000 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3463459063 expires 2024-02-27T10:34:36.733932+0000 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/2587364905 expires 2024-02-27T10:34:36.733932+0000 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/603652410 expires 2024-02-27T10:34:36.733932+0000 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/116674454 expires 2024-02-27T10:34:01.770755+0000 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/61419351 expires 2024-02-27T10:34:01.770755+0000 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3598708487 expires 2024-02-27T10:34:36.733932+0000 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1853267621 expires 2024-02-27T10:34:01.770755+0000 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/2948072642 expires 2024-02-27T10:34:01.770755+0000 2024-02-26T10:45:24.175 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/2948072642 expires 2024-02-27T10:34:01.770755+0000 2024-02-26T10:45:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e280a1e10 msgr2=0x7f4e280a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e280a1e10 0x7f4e280a42d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b7b0 tx=0x7f4e2c00b880 comp rx=0 tx=0).stop 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e401027e0 msgr2=0x7f4e40197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e401027e0 0x7f4e40197fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400e9d0 tx=0x7f4e3400ece0 comp rx=0 tx=0).stop 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 shutdown_connections 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e280a1e10 0x7f4e280a42d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e40107a40 0x7f4e4019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e401027e0 0x7f4e40197fe0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 --2- 172.21.15.60:0/295742862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e40105110 0x7f4e40198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 >> 172.21.15.60:0/295742862 conn(0x7f4e400fc1f0 msgr2=0x7f4e40103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 shutdown_connections 2024-02-26T10:45:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:24.179+0000 7f4e48200700 1 -- 172.21.15.60:0/295742862 wait complete. 2024-02-26T10:45:24.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:24 smithi113 ceph-mon[36126]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:24.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:24 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/295742862' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T10:45:24.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:24 smithi060 ceph-mon[31539]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:24.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:24 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/295742862' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T10:45:24.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:24 smithi100 ceph-mon[36051]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:24.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:24 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/295742862' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T10:45:25.056 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-26T10:45:25.068 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:45:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.309+0000 7f24afe00640 1 -- 172.21.15.60:0/948382710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8101280 msgr2=0x7f24a81016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.309+0000 7f24afe00640 1 --2- 172.21.15.60:0/948382710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8101280 0x7f24a81016a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f249c009ca0 tx=0x7f249c02c100 comp rx=0 tx=0).stop 2024-02-26T10:45:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.309+0000 7f24afe00640 1 -- 172.21.15.60:0/948382710 shutdown_connections 2024-02-26T10:45:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.309+0000 7f24afe00640 1 --2- 172.21.15.60:0/948382710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a8102e30 0x7f24a81096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.309+0000 7f24afe00640 1 --2- 172.21.15.60:0/948382710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24a8102470 0x7f24a81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.309+0000 7f24afe00640 1 --2- 172.21.15.60:0/948382710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8101280 0x7f24a81016a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.309+0000 7f24afe00640 1 -- 172.21.15.60:0/948382710 >> 172.21.15.60:0/948382710 conn(0x7f24a80fc810 msgr2=0x7f24a80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.309+0000 7f24afe00640 1 -- 172.21.15.60:0/948382710 shutdown_connections 2024-02-26T10:45:25.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.309+0000 7f24afe00640 1 -- 172.21.15.60:0/948382710 wait complete. 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.310+0000 7f24afe00640 1 Processor -- start 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24afe00640 1 -- start start 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24afe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a8101280 0x7f24a81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24afe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24a8102470 0x7f24a8199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102e30 0x7f24a81a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24afe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a810cf10 con 0x7f24a8102470 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24afe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a810cd90 con 0x7f24a8102e30 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24afe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a810d090 con 0x7f24a8101280 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24aca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24a8102470 0x7f24a8199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24aca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24a8102470 0x7f24a8199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51618/0 (socket says 172.21.15.60:51618) 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24aca00640 1 -- 172.21.15.60:0/1379840107 learned_addr learned my addr 172.21.15.60:0/1379840107 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:25.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24ade00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102e30 0x7f24a81a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.311+0000 7f24ad400640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a8101280 0x7f24a81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24ade00640 1 -- 172.21.15.60:0/1379840107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a8101280 msgr2=0x7f24a81997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24ade00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a8101280 0x7f24a81997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24ade00640 1 -- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24a8102470 msgr2=0x7f24a8199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24ade00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24a8102470 0x7f24a8199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24ade00640 1 -- 172.21.15.60:0/1379840107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a81a13d0 con 0x7f24a8102e30 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24aca00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24a8102470 0x7f24a8199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24ad400640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a8101280 0x7f24a81997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24ade00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102e30 0x7f24a81a0d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e970 tx=0x7f24a000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24a6000640 1 -- 172.21.15.60:0/1379840107 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a000ccc0 con 0x7f24a8102e30 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24a6000640 1 -- 172.21.15.60:0/1379840107 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24a0005020 con 0x7f24a8102e30 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a81a16c0 con 0x7f24a8102e30 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.312+0000 7f24a6000640 1 -- 172.21.15.60:0/1379840107 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0004a00 con 0x7f24a8102e30 2024-02-26T10:45:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.313+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a81a1c10 con 0x7f24a8102e30 2024-02-26T10:45:25.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.314+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a81a1ed0 con 0x7f24a8102e30 2024-02-26T10:45:25.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.316+0000 7f24a6000640 1 -- 172.21.15.60:0/1379840107 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f24a00051d0 con 0x7f24a8102e30 2024-02-26T10:45:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.317+0000 7f24a6000640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f24900a1e00 0x7f24900a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.317+0000 7f24a6000640 1 -- 172.21.15.60:0/1379840107 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f24a00ce0a0 con 0x7f24a8102e30 2024-02-26T10:45:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.317+0000 7f24ad400640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f24900a1e00 0x7f24900a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:25.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.318+0000 7f24ad400640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f24900a1e00 0x7f24900a42c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f249c02c710 tx=0x7f249c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:25.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.319+0000 7f24a6000640 1 -- 172.21.15.60:0/1379840107 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24a009b0c0 con 0x7f24a8102e30 2024-02-26T10:45:25.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.524+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24a81a20a0 con 0x7f24a8102e30 2024-02-26T10:45:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.527+0000 7f24a6000640 1 -- 172.21.15.60:0/1379840107 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7f24a009a9c0 con 0x7f24a8102e30 2024-02-26T10:45:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:25.522 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:45:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f24900a1e00 msgr2=0x7f24900a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f24900a1e00 0x7f24900a42c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f249c02c710 tx=0x7f249c032590 comp rx=0 tx=0).stop 2024-02-26T10:45:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102e30 msgr2=0x7f24a81a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102e30 0x7f24a81a0d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e970 tx=0x7f24a000ee30 comp rx=0 tx=0).stop 2024-02-26T10:45:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 shutdown_connections 2024-02-26T10:45:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f24900a1e00 0x7f24900a42c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24a8102e30 0x7f24a81a0d60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24a8102470 0x7f24a8199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 --2- 172.21.15.60:0/1379840107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a8101280 0x7f24a81997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 >> 172.21.15.60:0/1379840107 conn(0x7f24a80fc810 msgr2=0x7f24a8073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 shutdown_connections 2024-02-26T10:45:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.529+0000 7f24afe00640 1 -- 172.21.15.60:0/1379840107 wait complete. 2024-02-26T10:45:25.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:45:25.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:45:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.784+0000 7f5a3e800640 1 -- 172.21.15.60:0/4086012201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38101bc0 msgr2=0x7f5a38103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.784+0000 7f5a3e800640 1 --2- 172.21.15.60:0/4086012201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38101bc0 0x7f5a38103fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009ca0 tx=0x7f5a2802c100 comp rx=0 tx=0).stop 2024-02-26T10:45:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.784+0000 7f5a3e800640 1 -- 172.21.15.60:0/4086012201 shutdown_connections 2024-02-26T10:45:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.784+0000 7f5a3e800640 1 --2- 172.21.15.60:0/4086012201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a38106e20 0x7f5a38109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.784+0000 7f5a3e800640 1 --2- 172.21.15.60:0/4086012201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a381044f0 0x7f5a381068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.784+0000 7f5a3e800640 1 --2- 172.21.15.60:0/4086012201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38101bc0 0x7f5a38103fb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.784+0000 7f5a3e800640 1 -- 172.21.15.60:0/4086012201 >> 172.21.15.60:0/4086012201 conn(0x7f5a380fb5d0 msgr2=0x7f5a380fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.784+0000 7f5a3e800640 1 -- 172.21.15.60:0/4086012201 shutdown_connections 2024-02-26T10:45:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.784+0000 7f5a3e800640 1 -- 172.21.15.60:0/4086012201 wait complete. 2024-02-26T10:45:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.786+0000 7f5a3e800640 1 Processor -- start 2024-02-26T10:45:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.786+0000 7f5a3e800640 1 -- start start 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.786+0000 7f5a3e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a38101bc0 0x7f5a38199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.786+0000 7f5a3e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a381044f0 0x7f5a38199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.786+0000 7f5a3e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38106e20 0x7f5a381a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.786+0000 7f5a3e800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3810cdd0 con 0x7f5a381044f0 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.786+0000 7f5a3e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3810cc50 con 0x7f5a38106e20 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.786+0000 7f5a3e800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3810cf50 con 0x7f5a38101bc0 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a381044f0 0x7f5a38199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a38101bc0 0x7f5a38199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a38101bc0 0x7f5a38199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:39416/0 (socket says 172.21.15.60:39416) 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3d400640 1 -- 172.21.15.60:0/1255417423 learned_addr learned my addr 172.21.15.60:0/1255417423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a381044f0 0x7f5a38199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51636/0 (socket says 172.21.15.60:51636) 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3de00640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38106e20 0x7f5a381a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3d400640 1 -- 172.21.15.60:0/1255417423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38106e20 msgr2=0x7f5a381a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3d400640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38106e20 0x7f5a381a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3d400640 1 -- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a381044f0 msgr2=0x7f5a38199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:25.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3d400640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a381044f0 0x7f5a38199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:25.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3d400640 1 -- 172.21.15.60:0/1255417423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a381a1240 con 0x7f5a38101bc0 2024-02-26T10:45:25.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.787+0000 7f5a3ca00640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a381044f0 0x7f5a38199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:25.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.788+0000 7f5a3d400640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a38101bc0 0x7f5a38199620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5a280325f0 tx=0x7f5a28035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:25.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.788+0000 7f5a36000640 1 -- 172.21.15.60:0/1255417423 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28039040 con 0x7f5a38101bc0 2024-02-26T10:45:25.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.788+0000 7f5a36000640 1 -- 172.21.15.60:0/1255417423 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a28044070 con 0x7f5a38101bc0 2024-02-26T10:45:25.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.788+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a381a14d0 con 0x7f5a38101bc0 2024-02-26T10:45:25.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.788+0000 7f5a36000640 1 -- 172.21.15.60:0/1255417423 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28004540 con 0x7f5a38101bc0 2024-02-26T10:45:25.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.788+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a381a19e0 con 0x7f5a38101bc0 2024-02-26T10:45:25.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.789+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a38108970 con 0x7f5a38101bc0 2024-02-26T10:45:25.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.791+0000 7f5a36000640 1 -- 172.21.15.60:0/1255417423 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f5a28032890 con 0x7f5a38101bc0 2024-02-26T10:45:25.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.792+0000 7f5a36000640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5a1c0a1e00 0x7f5a1c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:25.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.793+0000 7f5a36000640 1 -- 172.21.15.60:0/1255417423 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f5a280f0760 con 0x7f5a38101bc0 2024-02-26T10:45:25.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.793+0000 7f5a3ca00640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5a1c0a1e00 0x7f5a1c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:25.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.793+0000 7f5a3ca00640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5a1c0a1e00 0x7f5a1c0a42c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5a240095f0 tx=0x7f5a24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:25.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:25.797+0000 7f5a36000640 1 -- 172.21.15.60:0/1255417423 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a280bd780 con 0x7f5a38101bc0 2024-02-26T10:45:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.000+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a38061e60 con 0x7f5a38101bc0 2024-02-26T10:45:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.001+0000 7f5a36000640 1 -- 172.21.15.60:0/1255417423 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a280bd080 con 0x7f5a38101bc0 2024-02-26T10:45:25.997 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:45:25.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5a1c0a1e00 msgr2=0x7f5a1c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5a1c0a1e00 0x7f5a1c0a42c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5a240095f0 tx=0x7f5a24005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a38101bc0 msgr2=0x7f5a38199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a38101bc0 0x7f5a38199620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5a280325f0 tx=0x7f5a28035d10 comp rx=0 tx=0).stop 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 shutdown_connections 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f5a1c0a1e00 0x7f5a1c0a42c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a38106e20 0x7f5a381a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a381044f0 0x7f5a38199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 --2- 172.21.15.60:0/1255417423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a38101bc0 0x7f5a38199620 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 >> 172.21.15.60:0/1255417423 conn(0x7f5a380fb5d0 msgr2=0x7f5a381012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 shutdown_connections 2024-02-26T10:45:26.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.004+0000 7f5a3e800640 1 -- 172.21.15.60:0/1255417423 wait complete. 2024-02-26T10:45:26.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:45:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.253+0000 7f7896c00640 1 -- 172.21.15.60:0/186200287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 msgr2=0x7f78900736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.253+0000 7f7896c00640 1 --2- 172.21.15.60:0/186200287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 0x7f78900736f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7884009900 tx=0x7f788402bc40 comp rx=0 tx=0).stop 2024-02-26T10:45:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.253+0000 7f7896c00640 1 -- 172.21.15.60:0/186200287 shutdown_connections 2024-02-26T10:45:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.253+0000 7f7896c00640 1 --2- 172.21.15.60:0/186200287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109020 0x7f789010b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.253+0000 7f7896c00640 1 --2- 172.21.15.60:0/186200287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7890073cc0 0x7f7890074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.253+0000 7f7896c00640 1 --2- 172.21.15.60:0/186200287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 0x7f78900736f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.253+0000 7f7896c00640 1 -- 172.21.15.60:0/186200287 >> 172.21.15.60:0/186200287 conn(0x7f78900fc6d0 msgr2=0x7f78900feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.253+0000 7f7896c00640 1 -- 172.21.15.60:0/186200287 shutdown_connections 2024-02-26T10:45:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.253+0000 7f7896c00640 1 -- 172.21.15.60:0/186200287 wait complete. 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.254+0000 7f7896c00640 1 Processor -- start 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.254+0000 7f7896c00640 1 -- start start 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7896c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7890073cc0 0x7f7890199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7896c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 0x7f7890199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7896c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109020 0x7f78901a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7896c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789010efe0 con 0x7f78900752a0 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7896c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789010ee60 con 0x7f7890109020 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7896c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789010f160 con 0x7f7890073cc0 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7894e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 0x7f7890199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7895800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7890073cc0 0x7f7890199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7895800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7890073cc0 0x7f7890199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:39428/0 (socket says 172.21.15.60:39428) 2024-02-26T10:45:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7894e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 0x7f7890199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51644/0 (socket says 172.21.15.60:51644) 2024-02-26T10:45:26.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7895800640 1 -- 172.21.15.60:0/205809119 learned_addr learned my addr 172.21.15.60:0/205809119 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:26.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7896200640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109020 0x7f78901a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:26.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7894e00640 1 -- 172.21.15.60:0/205809119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7890073cc0 msgr2=0x7f7890199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7894e00640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7890073cc0 0x7f7890199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7894e00640 1 -- 172.21.15.60:0/205809119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109020 msgr2=0x7f78901a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7894e00640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109020 0x7f78901a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.255+0000 7f7894e00640 1 -- 172.21.15.60:0/205809119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78901a1270 con 0x7f78900752a0 2024-02-26T10:45:26.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.256+0000 7f7894e00640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 0x7f7890199b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9f0 tx=0x7f787c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:26.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.256+0000 7f788e000640 1 -- 172.21.15.60:0/205809119 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00ccc0 con 0x7f78900752a0 2024-02-26T10:45:26.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.256+0000 7f788e000640 1 -- 172.21.15.60:0/205809119 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f787c005020 con 0x7f78900752a0 2024-02-26T10:45:26.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.256+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78901a1560 con 0x7f78900752a0 2024-02-26T10:45:26.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.256+0000 7f788e000640 1 -- 172.21.15.60:0/205809119 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c0049e0 con 0x7f78900752a0 2024-02-26T10:45:26.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.256+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78901a1a60 con 0x7f78900752a0 2024-02-26T10:45:26.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.258+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7890107d50 con 0x7f78900752a0 2024-02-26T10:45:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.260+0000 7f788e000640 1 -- 172.21.15.60:0/205809119 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f787c005560 con 0x7f78900752a0 2024-02-26T10:45:26.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.261+0000 7f788e000640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f78780a1e00 0x7f78780a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:26.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.261+0000 7f788e000640 1 -- 172.21.15.60:0/205809119 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f787c0ce180 con 0x7f78900752a0 2024-02-26T10:45:26.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.261+0000 7f7895800640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f78780a1e00 0x7f78780a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:26.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.262+0000 7f7895800640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f78780a1e00 0x7f78780a42c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7884000c00 tx=0x7f788402c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:26.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.265+0000 7f788e000640 1 -- 172.21.15.60:0/205809119 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f787c0051d0 con 0x7f78900752a0 2024-02-26T10:45:26.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.469+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7890061e60 con 0x7f78900752a0 2024-02-26T10:45:26.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.470+0000 7f788e000640 1 -- 172.21.15.60:0/205809119 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f787c09b1a0 con 0x7f78900752a0 2024-02-26T10:45:26.466 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:26.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f78780a1e00 msgr2=0x7f78780a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f78780a1e00 0x7f78780a42c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7884000c00 tx=0x7f788402c890 comp rx=0 tx=0).stop 2024-02-26T10:45:26.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 msgr2=0x7f7890199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 0x7f7890199b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9f0 tx=0x7f787c00eeb0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 shutdown_connections 2024-02-26T10:45:26.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f78780a1e00 0x7f78780a42c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7890109020 0x7f78901a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78900752a0 0x7f7890199b90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 --2- 172.21.15.60:0/205809119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7890073cc0 0x7f7890199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 >> 172.21.15.60:0/205809119 conn(0x7f78900fc6d0 msgr2=0x7f7890104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:26.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 shutdown_connections 2024-02-26T10:45:26.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.473+0000 7f7896c00640 1 -- 172.21.15.60:0/205809119 wait complete. 2024-02-26T10:45:26.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:45:26.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.717+0000 7f4e3f000640 1 -- 172.21.15.60:0/4179971809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 msgr2=0x7f4e38103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.717+0000 7f4e3f000640 1 --2- 172.21.15.60:0/4179971809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 0x7f4e38103780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009900 tx=0x7f4e2c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:26.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.717+0000 7f4e3f000640 1 -- 172.21.15.60:0/4179971809 shutdown_connections 2024-02-26T10:45:26.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.717+0000 7f4e3f000640 1 --2- 172.21.15.60:0/4179971809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e38104f10 0x7f4e3810b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.717+0000 7f4e3f000640 1 --2- 172.21.15.60:0/4179971809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e38104550 0x7f4e381049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.717+0000 7f4e3f000640 1 --2- 172.21.15.60:0/4179971809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 0x7f4e38103780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.717+0000 7f4e3f000640 1 -- 172.21.15.60:0/4179971809 >> 172.21.15.60:0/4179971809 conn(0x7f4e380fe8d0 msgr2=0x7f4e38100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:26.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.717+0000 7f4e3f000640 1 -- 172.21.15.60:0/4179971809 shutdown_connections 2024-02-26T10:45:26.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.717+0000 7f4e3f000640 1 -- 172.21.15.60:0/4179971809 wait complete. 2024-02-26T10:45:26.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.718+0000 7f4e3f000640 1 Processor -- start 2024-02-26T10:45:26.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3f000640 1 -- start start 2024-02-26T10:45:26.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 0x7f4e38199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:26.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3f000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e38104550 0x7f4e38199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:26.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3f000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e38104f10 0x7f4e381a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:26.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3f000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3810f040 con 0x7f4e38104550 2024-02-26T10:45:26.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3810eec0 con 0x7f4e38103360 2024-02-26T10:45:26.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3f000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3810f1c0 con 0x7f4e38104f10 2024-02-26T10:45:26.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e38104550 0x7f4e38199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:26.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 0x7f4e38199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:26.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 0x7f4e38199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51394/0 (socket says 172.21.15.60:51394) 2024-02-26T10:45:26.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e38104550 0x7f4e38199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51652/0 (socket says 172.21.15.60:51652) 2024-02-26T10:45:26.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3dc00640 1 -- 172.21.15.60:0/512786730 learned_addr learned my addr 172.21.15.60:0/512786730 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:26.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.719+0000 7f4e3e600640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e38104f10 0x7f4e381a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.720+0000 7f4e3dc00640 1 -- 172.21.15.60:0/512786730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e38104f10 msgr2=0x7f4e381a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.720+0000 7f4e3dc00640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e38104f10 0x7f4e381a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.720+0000 7f4e3dc00640 1 -- 172.21.15.60:0/512786730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e38104550 msgr2=0x7f4e38199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.720+0000 7f4e3dc00640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e38104550 0x7f4e38199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.720+0000 7f4e3dc00640 1 -- 172.21.15.60:0/512786730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e381a1280 con 0x7f4e38103360 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.720+0000 7f4e3dc00640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 0x7f4e38199660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c000c00 tx=0x7f4e2c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.720+0000 7f4e36000640 1 -- 172.21.15.60:0/512786730 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c043020 con 0x7f4e38103360 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.721+0000 7f4e36000640 1 -- 172.21.15.60:0/512786730 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e2c0054e0 con 0x7f4e38103360 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.721+0000 7f4e36000640 1 -- 172.21.15.60:0/512786730 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c003c40 con 0x7f4e38103360 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.721+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e381a1510 con 0x7f4e38103360 2024-02-26T10:45:26.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.721+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e381a1a60 con 0x7f4e38103360 2024-02-26T10:45:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.722+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3810a680 con 0x7f4e38103360 2024-02-26T10:45:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.724+0000 7f4e36000640 1 -- 172.21.15.60:0/512786730 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4e2c046020 con 0x7f4e38103360 2024-02-26T10:45:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.725+0000 7f4e36000640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e200a1d80 0x7f4e200a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:26.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.725+0000 7f4e36000640 1 -- 172.21.15.60:0/512786730 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f4e2c0f0170 con 0x7f4e38103360 2024-02-26T10:45:26.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.725+0000 7f4e3d200640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e200a1d80 0x7f4e200a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:26.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.726+0000 7f4e3d200640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e200a1d80 0x7f4e200a4240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4e3819ab70 tx=0x7f4e24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:26.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.729+0000 7f4e36000640 1 -- 172.21.15.60:0/512786730 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e2c04d020 con 0x7f4e38103360 2024-02-26T10:45:26.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:26 smithi060 ceph-mon[31539]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:26.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:26 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1379840107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:26.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:26 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1255417423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:26.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:26 smithi113 ceph-mon[36126]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:26.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:26 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1379840107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:26.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:26 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1255417423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:26.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:26 smithi100 ceph-mon[36051]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:26.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:26 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1379840107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:26.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:26 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1255417423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.933+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e3802d050 con 0x7f4e38103360 2024-02-26T10:45:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.934+0000 7f4e36000640 1 -- 172.21.15.60:0/512786730 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e2c0bd190 con 0x7f4e38103360 2024-02-26T10:45:26.929 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:45:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.936+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e200a1d80 msgr2=0x7f4e200a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e200a1d80 0x7f4e200a4240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4e3819ab70 tx=0x7f4e24005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 msgr2=0x7f4e38199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 0x7f4e38199660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c000c00 tx=0x7f4e2c02c650 comp rx=0 tx=0).stop 2024-02-26T10:45:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 shutdown_connections 2024-02-26T10:45:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e200a1d80 0x7f4e200a4240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e38104f10 0x7f4e381a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e38104550 0x7f4e38199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 --2- 172.21.15.60:0/512786730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38103360 0x7f4e38199660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 >> 172.21.15.60:0/512786730 conn(0x7f4e380fe8d0 msgr2=0x7f4e38107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 shutdown_connections 2024-02-26T10:45:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:26.937+0000 7f4e3f000640 1 -- 172.21.15.60:0/512786730 wait complete. 2024-02-26T10:45:26.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:45:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.169+0000 7f176f000640 1 -- 172.21.15.60:0/1961870071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1768102460 msgr2=0x7f17681028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.169+0000 7f176f000640 1 --2- 172.21.15.60:0/1961870071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1768102460 0x7f17681028e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1754009900 tx=0x7f175402bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.169+0000 7f176f000640 1 -- 172.21.15.60:0/1961870071 shutdown_connections 2024-02-26T10:45:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.169+0000 7f176f000640 1 --2- 172.21.15.60:0/1961870071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1768102e20 0x7f17681096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.169+0000 7f176f000640 1 --2- 172.21.15.60:0/1961870071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1768102460 0x7f17681028e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.169+0000 7f176f000640 1 --2- 172.21.15.60:0/1961870071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1768101270 0x7f1768101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.169+0000 7f176f000640 1 -- 172.21.15.60:0/1961870071 >> 172.21.15.60:0/1961870071 conn(0x7f17680fc800 msgr2=0x7f17680fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.170+0000 7f176f000640 1 -- 172.21.15.60:0/1961870071 shutdown_connections 2024-02-26T10:45:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.170+0000 7f176f000640 1 -- 172.21.15.60:0/1961870071 wait complete. 2024-02-26T10:45:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.171+0000 7f176f000640 1 Processor -- start 2024-02-26T10:45:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.171+0000 7f176f000640 1 -- start start 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f176f000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1768101270 0x7f17681997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f176f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1768102460 0x7f1768199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f176f000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1768102e20 0x7f17681a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f176f000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176810cf50 con 0x7f1768101270 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f176f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176810cdd0 con 0x7f1768102460 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f176f000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176810d0d0 con 0x7f1768102e20 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f1767e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1768101270 0x7f17681997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f1767e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1768101270 0x7f17681997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51678/0 (socket says 172.21.15.60:51678) 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f1767e00640 1 -- 172.21.15.60:0/1125868140 learned_addr learned my addr 172.21.15.60:0/1125868140 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f176d000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1768102e20 0x7f17681a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.172+0000 7f1767400640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1768102460 0x7f1768199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f176d000640 1 -- 172.21.15.60:0/1125868140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1768102460 msgr2=0x7f1768199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f176d000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1768102460 0x7f1768199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f176d000640 1 -- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1768101270 msgr2=0x7f17681997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f176d000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1768101270 0x7f17681997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f176d000640 1 -- 172.21.15.60:0/1125868140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17681a13f0 con 0x7f1768102e20 2024-02-26T10:45:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f1767e00640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1768101270 0x7f17681997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:27.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f176d000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1768102e20 0x7f17681a0d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1760009fd0 tx=0x7f176000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:27.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f1764c00640 1 -- 172.21.15.60:0/1125868140 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176000ce20 con 0x7f1768102e20 2024-02-26T10:45:27.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f1764c00640 1 -- 172.21.15.60:0/1125868140 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1760005020 con 0x7f1768102e20 2024-02-26T10:45:27.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f1764c00640 1 -- 172.21.15.60:0/1125868140 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760010720 con 0x7f1768102e20 2024-02-26T10:45:27.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.173+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17681a16e0 con 0x7f1768102e20 2024-02-26T10:45:27.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.174+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17681a1bb0 con 0x7f1768102e20 2024-02-26T10:45:27.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.175+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17681a1e70 con 0x7f1768102e20 2024-02-26T10:45:27.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.177+0000 7f1764c00640 1 -- 172.21.15.60:0/1125868140 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1760005540 con 0x7f1768102e20 2024-02-26T10:45:27.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.178+0000 7f1764c00640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f174c0a1e00 0x7f174c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:27.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.178+0000 7f1764c00640 1 -- 172.21.15.60:0/1125868140 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f17600ce820 con 0x7f1768102e20 2024-02-26T10:45:27.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.179+0000 7f1767e00640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f174c0a1e00 0x7f174c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:27.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.179+0000 7f1767e00640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f174c0a1e00 0x7f174c0a42c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17580058c0 tx=0x7f1758005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:27.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.180+0000 7f1764c00640 1 -- 172.21.15.60:0/1125868140 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f176009b840 con 0x7f1768102e20 2024-02-26T10:45:27.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.390+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f17681a2040 con 0x7f1768102e20 2024-02-26T10:45:27.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.390+0000 7f1764c00640 1 -- 172.21.15.60:0/1125868140 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f176009b140 con 0x7f1768102e20 2024-02-26T10:45:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:27.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.393+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f174c0a1e00 msgr2=0x7f174c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.393+0000 7f176f000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f174c0a1e00 0x7f174c0a42c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17580058c0 tx=0x7f1758005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:27.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.393+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1768102e20 msgr2=0x7f17681a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.393+0000 7f176f000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1768102e20 0x7f17681a0d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1760009fd0 tx=0x7f176000ef10 comp rx=0 tx=0).stop 2024-02-26T10:45:27.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.394+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 shutdown_connections 2024-02-26T10:45:27.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.394+0000 7f176f000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f174c0a1e00 0x7f174c0a42c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.394+0000 7f176f000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1768102460 0x7f1768199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.394+0000 7f176f000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1768102e20 0x7f17681a0d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.394+0000 7f176f000640 1 --2- 172.21.15.60:0/1125868140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1768101270 0x7f17681997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.394+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 >> 172.21.15.60:0/1125868140 conn(0x7f17680fc800 msgr2=0x7f1768073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.394+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 shutdown_connections 2024-02-26T10:45:27.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.394+0000 7f176f000640 1 -- 172.21.15.60:0/1125868140 wait complete. 2024-02-26T10:45:27.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:45:27.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.619+0000 7f274b200640 1 -- 172.21.15.60:0/2064276097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 msgr2=0x7f274c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.619+0000 7f274b200640 1 --2- 172.21.15.60:0/2064276097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 0x7f274c1036e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f273c009900 tx=0x7f273c02bc50 comp rx=0 tx=0).stop 2024-02-26T10:45:27.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.619+0000 7f274b200640 1 -- 172.21.15.60:0/2064276097 shutdown_connections 2024-02-26T10:45:27.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.619+0000 7f274b200640 1 --2- 172.21.15.60:0/2064276097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c104e70 0x7f274c10b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.619+0000 7f274b200640 1 --2- 172.21.15.60:0/2064276097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c1044b0 0x7f274c104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.619+0000 7f274b200640 1 --2- 172.21.15.60:0/2064276097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 0x7f274c1036e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.619+0000 7f274b200640 1 -- 172.21.15.60:0/2064276097 >> 172.21.15.60:0/2064276097 conn(0x7f274c0fe830 msgr2=0x7f274c100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.619+0000 7f274b200640 1 -- 172.21.15.60:0/2064276097 shutdown_connections 2024-02-26T10:45:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.619+0000 7f274b200640 1 -- 172.21.15.60:0/2064276097 wait complete. 2024-02-26T10:45:27.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.620+0000 7f274b200640 1 Processor -- start 2024-02-26T10:45:27.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.620+0000 7f274b200640 1 -- start start 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f274b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 0x7f274c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f274b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c1044b0 0x7f274c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f274b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104e70 0x7f274c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f274b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c10efa0 con 0x7f274c1032c0 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f274b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c10ee20 con 0x7f274c1044b0 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f274b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c10f120 con 0x7f274c104e70 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f2749e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 0x7f274c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f2749e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 0x7f274c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51692/0 (socket says 172.21.15.60:51692) 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f2749e00640 1 -- 172.21.15.60:0/1754789298 learned_addr learned my addr 172.21.15.60:0/1754789298 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f274a800640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104e70 0x7f274c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f2749400640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c1044b0 0x7f274c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f2749e00640 1 -- 172.21.15.60:0/1754789298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104e70 msgr2=0x7f274c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f2749e00640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104e70 0x7f274c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f2749e00640 1 -- 172.21.15.60:0/1754789298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c1044b0 msgr2=0x7f274c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f2749e00640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c1044b0 0x7f274c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.621+0000 7f2749e00640 1 -- 172.21.15.60:0/1754789298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c1a1210 con 0x7f274c1032c0 2024-02-26T10:45:27.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.622+0000 7f2749e00640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 0x7f274c1995f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f273c02c260 tx=0x7f273c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:27.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.622+0000 7f2742a00640 1 -- 172.21.15.60:0/1754789298 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c041070 con 0x7f274c1032c0 2024-02-26T10:45:27.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.622+0000 7f2742a00640 1 -- 172.21.15.60:0/1754789298 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f273c004450 con 0x7f274c1032c0 2024-02-26T10:45:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.622+0000 7f2742a00640 1 -- 172.21.15.60:0/1754789298 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c034440 con 0x7f274c1032c0 2024-02-26T10:45:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.622+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c1a14a0 con 0x7f274c1032c0 2024-02-26T10:45:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.622+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c1a19f0 con 0x7f274c1032c0 2024-02-26T10:45:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.623+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c10a5e0 con 0x7f274c1032c0 2024-02-26T10:45:27.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.627+0000 7f2742a00640 1 -- 172.21.15.60:0/1754789298 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f273c044070 con 0x7f274c1032c0 2024-02-26T10:45:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.628+0000 7f2742a00640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f27300a1d80 0x7f27300a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.628+0000 7f2742a00640 1 -- 172.21.15.60:0/1754789298 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f273c0efc70 con 0x7f274c1032c0 2024-02-26T10:45:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.628+0000 7f2749400640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f27300a1d80 0x7f27300a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:27.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.629+0000 7f2749400640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f27300a1d80 0x7f27300a4240 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f274c19ab00 tx=0x7f2738005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.632+0000 7f2742a00640 1 -- 172.21.15.60:0/1754789298 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f273c0bc640 con 0x7f274c1032c0 2024-02-26T10:45:27.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:27 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/205809119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:27.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:27 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/512786730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:27.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:27 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1125868140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:27.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.869+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f274c02ce30 con 0x7f274c1032c0 2024-02-26T10:45:27.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:27 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/205809119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:27.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:27 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/512786730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:27.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:27 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1125868140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:27.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.870+0000 7f2742a00640 1 -- 172.21.15.60:0/1754789298 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11359 (secure 0 0 0) 0x7f273c0b8020 con 0x7f274c1032c0 2024-02-26T10:45:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:27.866 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:43:16.029212+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:45:27.866 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T10:45:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.873+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f27300a1d80 msgr2=0x7f27300a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.873+0000 7f274b200640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f27300a1d80 0x7f27300a4240 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f274c19ab00 tx=0x7f2738005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.873+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 msgr2=0x7f274c1995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.873+0000 7f274b200640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 0x7f274c1995f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f273c02c260 tx=0x7f273c02c670 comp rx=0 tx=0).stop 2024-02-26T10:45:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.873+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 shutdown_connections 2024-02-26T10:45:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.873+0000 7f274b200640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f27300a1d80 0x7f27300a4240 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.873+0000 7f274b200640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f274c1044b0 0x7f274c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.873+0000 7f274b200640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f274c104e70 0x7f274c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.873+0000 7f274b200640 1 --2- 172.21.15.60:0/1754789298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f274c1032c0 0x7f274c1995f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:27.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.874+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 >> 172.21.15.60:0/1754789298 conn(0x7f274c0fe830 msgr2=0x7f274c1077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:27.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.874+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 shutdown_connections 2024-02-26T10:45:27.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:27.874+0000 7f274b200640 1 -- 172.21.15.60:0/1754789298 wait complete. 2024-02-26T10:45:27.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:27 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/205809119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:27.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:27 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/512786730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:27.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:27 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1125868140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:27.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:45:28.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.091+0000 7fd401e00640 1 -- 172.21.15.60:0/1454873135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc104e70 msgr2=0x7fd3fc10b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.091+0000 7fd401e00640 1 --2- 172.21.15.60:0/1454873135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc104e70 0x7fd3fc10b700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009ca0 tx=0x7fd3f402c100 comp rx=0 tx=0).stop 2024-02-26T10:45:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.091+0000 7fd401e00640 1 -- 172.21.15.60:0/1454873135 shutdown_connections 2024-02-26T10:45:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.091+0000 7fd401e00640 1 --2- 172.21.15.60:0/1454873135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc1044b0 0x7fd3fc104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.091+0000 7fd401e00640 1 --2- 172.21.15.60:0/1454873135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc104e70 0x7fd3fc10b700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.091+0000 7fd401e00640 1 --2- 172.21.15.60:0/1454873135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc1032c0 0x7fd3fc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.091+0000 7fd401e00640 1 -- 172.21.15.60:0/1454873135 >> 172.21.15.60:0/1454873135 conn(0x7fd3fc0fe830 msgr2=0x7fd3fc100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.091+0000 7fd401e00640 1 -- 172.21.15.60:0/1454873135 shutdown_connections 2024-02-26T10:45:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.091+0000 7fd401e00640 1 -- 172.21.15.60:0/1454873135 wait complete. 2024-02-26T10:45:28.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.092+0000 7fd401e00640 1 Processor -- start 2024-02-26T10:45:28.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.092+0000 7fd401e00640 1 -- start start 2024-02-26T10:45:28.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd401e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc1032c0 0x7fd3fc199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:28.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd401e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc1044b0 0x7fd3fc199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd401e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc104e70 0x7fd3fc1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd401e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc10ef20 con 0x7fd3fc1044b0 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd401e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc10eda0 con 0x7fd3fc104e70 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd401e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc10f0a0 con 0x7fd3fc1032c0 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd3fbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc1044b0 0x7fd3fc199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd3fbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc1044b0 0x7fd3fc199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51702/0 (socket says 172.21.15.60:51702) 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd3fbe00640 1 -- 172.21.15.60:0/2510339143 learned_addr learned my addr 172.21.15.60:0/2510339143 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd400a00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc1032c0 0x7fd3fc199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.093+0000 7fd401400640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc104e70 0x7fd3fc1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd3fbe00640 1 -- 172.21.15.60:0/2510339143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc1032c0 msgr2=0x7fd3fc199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd3fbe00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc1032c0 0x7fd3fc199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd3fbe00640 1 -- 172.21.15.60:0/2510339143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc104e70 msgr2=0x7fd3fc1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd3fbe00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc104e70 0x7fd3fc1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd3fbe00640 1 -- 172.21.15.60:0/2510339143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc1a1180 con 0x7fd3fc1044b0 2024-02-26T10:45:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd3fbe00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc1044b0 0x7fd3fc199aa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e9f0 tx=0x7fd3e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd3f9600640 1 -- 172.21.15.60:0/2510339143 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800cd80 con 0x7fd3fc1044b0 2024-02-26T10:45:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd3f9600640 1 -- 172.21.15.60:0/2510339143 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3e8010910 con 0x7fd3fc1044b0 2024-02-26T10:45:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3fc1a1470 con 0x7fd3fc1044b0 2024-02-26T10:45:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd3f9600640 1 -- 172.21.15.60:0/2510339143 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8018990 con 0x7fd3fc1044b0 2024-02-26T10:45:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.094+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3fc1a1a10 con 0x7fd3fc1044b0 2024-02-26T10:45:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.096+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3fc10a5e0 con 0x7fd3fc1044b0 2024-02-26T10:45:28.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.097+0000 7fd3f9600640 1 -- 172.21.15.60:0/2510339143 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd3e8010ac0 con 0x7fd3fc1044b0 2024-02-26T10:45:28.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.099+0000 7fd3f9600640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3e40a1e00 0x7fd3e40a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:28.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.099+0000 7fd3f9600640 1 -- 172.21.15.60:0/2510339143 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fd3e80ce360 con 0x7fd3fc1044b0 2024-02-26T10:45:28.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.099+0000 7fd400a00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3e40a1e00 0x7fd3e40a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.100+0000 7fd400a00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3e40a1e00 0x7fd3e40a42c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc104310 tx=0x7fd3f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:28.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.103+0000 7fd3f9600640 1 -- 172.21.15.60:0/2510339143 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd3e809ae30 con 0x7fd3fc1044b0 2024-02-26T10:45:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.305+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3fc02d070 con 0x7fd3fc1044b0 2024-02-26T10:45:28.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.310+0000 7fd3f9600640 1 -- 172.21.15.60:0/2510339143 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7fd3e809eed0 con 0x7fd3fc1044b0 2024-02-26T10:45:28.305 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:28.305 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:45:28.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.312+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3e40a1e00 msgr2=0x7fd3e40a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.312+0000 7fd401e00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3e40a1e00 0x7fd3e40a42c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc104310 tx=0x7fd3f0005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:28.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc1044b0 msgr2=0x7fd3fc199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc1044b0 0x7fd3fc199aa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e9f0 tx=0x7fd3e800eeb0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 shutdown_connections 2024-02-26T10:45:28.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3e40a1e00 0x7fd3e40a42c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc104e70 0x7fd3fc1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc1044b0 0x7fd3fc199aa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 --2- 172.21.15.60:0/2510339143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3fc1032c0 0x7fd3fc199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 >> 172.21.15.60:0/2510339143 conn(0x7fd3fc0fe830 msgr2=0x7fd3fc1077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:28.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 shutdown_connections 2024-02-26T10:45:28.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.313+0000 7fd401e00640 1 -- 172.21.15.60:0/2510339143 wait complete. 2024-02-26T10:45:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:45:28.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:45:28.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.557+0000 7f1e81000640 1 -- 172.21.15.60:0/3220307362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7c1034a0 msgr2=0x7f1e7c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.557+0000 7f1e81000640 1 --2- 172.21.15.60:0/3220307362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7c1034a0 0x7f1e7c1038c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009be0 tx=0x7f1e6c02bef0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.557+0000 7f1e81000640 1 -- 172.21.15.60:0/3220307362 shutdown_connections 2024-02-26T10:45:28.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.557+0000 7f1e81000640 1 --2- 172.21.15.60:0/3220307362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e7c105050 0x7f1e7c10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.557+0000 7f1e81000640 1 --2- 172.21.15.60:0/3220307362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c104690 0x7f1e7c104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.557+0000 7f1e81000640 1 --2- 172.21.15.60:0/3220307362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7c1034a0 0x7f1e7c1038c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.557+0000 7f1e81000640 1 -- 172.21.15.60:0/3220307362 >> 172.21.15.60:0/3220307362 conn(0x7f1e7c0fea10 msgr2=0x7f1e7c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:28.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.557+0000 7f1e81000640 1 -- 172.21.15.60:0/3220307362 shutdown_connections 2024-02-26T10:45:28.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.557+0000 7f1e81000640 1 -- 172.21.15.60:0/3220307362 wait complete. 2024-02-26T10:45:28.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.558+0000 7f1e81000640 1 Processor -- start 2024-02-26T10:45:28.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e81000640 1 -- start start 2024-02-26T10:45:28.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e81000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c1034a0 0x7f1e7c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:28.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e81000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7c104690 0x7f1e7c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:28.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e81000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e7c105050 0x7f1e7c1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:28.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e81000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c10f180 con 0x7f1e7c104690 2024-02-26T10:45:28.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e81000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c10f000 con 0x7f1e7c105050 2024-02-26T10:45:28.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e81000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c10f300 con 0x7f1e7c1034a0 2024-02-26T10:45:28.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e79400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7c104690 0x7f1e7c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e79e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c1034a0 0x7f1e7c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e79e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c1034a0 0x7f1e7c1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:39488/0 (socket says 172.21.15.60:39488) 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e79e00640 1 -- 172.21.15.60:0/686844430 learned_addr learned my addr 172.21.15.60:0/686844430 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.559+0000 7f1e7a800640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e7c105050 0x7f1e7c1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e79400640 1 -- 172.21.15.60:0/686844430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c1034a0 msgr2=0x7f1e7c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e79400640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c1034a0 0x7f1e7c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e79400640 1 -- 172.21.15.60:0/686844430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e7c105050 msgr2=0x7f1e7c1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e79400640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e7c105050 0x7f1e7c1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e79400640 1 -- 172.21.15.60:0/686844430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c1a1400 con 0x7f1e7c104690 2024-02-26T10:45:28.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e79400640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7c104690 0x7f1e7c199d20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400e9f0 tx=0x7f1e6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:28.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e72a00640 1 -- 172.21.15.60:0/686844430 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6400ccc0 con 0x7f1e7c104690 2024-02-26T10:45:28.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e72a00640 1 -- 172.21.15.60:0/686844430 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e64005020 con 0x7f1e7c104690 2024-02-26T10:45:28.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e72a00640 1 -- 172.21.15.60:0/686844430 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e640049e0 con 0x7f1e7c104690 2024-02-26T10:45:28.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.560+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c1a16f0 con 0x7f1e7c104690 2024-02-26T10:45:28.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.561+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c1a1bf0 con 0x7f1e7c104690 2024-02-26T10:45:28.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.562+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c10a7e0 con 0x7f1e7c104690 2024-02-26T10:45:28.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.564+0000 7f1e72a00640 1 -- 172.21.15.60:0/686844430 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1e64005560 con 0x7f1e7c104690 2024-02-26T10:45:28.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.566+0000 7f1e72a00640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e600a1d80 0x7f1e600a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:28.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.566+0000 7f1e72a00640 1 -- 172.21.15.60:0/686844430 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f1e640ce150 con 0x7f1e7c104690 2024-02-26T10:45:28.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.566+0000 7f1e79e00640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e600a1d80 0x7f1e600a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:28.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.567+0000 7f1e79e00640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e600a1d80 0x7f1e600a4240 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c006010 tx=0x7f1e6c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:28.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.570+0000 7f1e72a00640 1 -- 172.21.15.60:0/686844430 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e640051d0 con 0x7f1e7c104690 2024-02-26T10:45:28.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:28 smithi060 ceph-mon[31539]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:28.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:28 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1754789298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:28.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:28 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2510339143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:28.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.772+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e7c061e60 con 0x7f1e7c104690 2024-02-26T10:45:28.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.773+0000 7f1e72a00640 1 -- 172.21.15.60:0/686844430 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e6409b170 con 0x7f1e7c104690 2024-02-26T10:45:28.768 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:45:28.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e600a1d80 msgr2=0x7f1e600a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e600a1d80 0x7f1e600a4240 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c006010 tx=0x7f1e6c000f40 comp rx=0 tx=0).stop 2024-02-26T10:45:28.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7c104690 msgr2=0x7f1e7c199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:28.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7c104690 0x7f1e7c199d20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400e9f0 tx=0x7f1e6400eeb0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 shutdown_connections 2024-02-26T10:45:28.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e600a1d80 0x7f1e600a4240 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e7c105050 0x7f1e7c1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7c104690 0x7f1e7c199d20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 --2- 172.21.15.60:0/686844430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c1034a0 0x7f1e7c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:28.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 >> 172.21.15.60:0/686844430 conn(0x7f1e7c0fea10 msgr2=0x7f1e7c1079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:28.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.775+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 shutdown_connections 2024-02-26T10:45:28.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:28.776+0000 7f1e81000640 1 -- 172.21.15.60:0/686844430 wait complete. 2024-02-26T10:45:28.777 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:28 smithi100 ceph-mon[36051]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:28.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:28 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1754789298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:28.778 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:28 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2510339143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:28.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:45:28.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:28 smithi113 ceph-mon[36126]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:28.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:28 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1754789298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:28.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:28 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2510339143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:29.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.025+0000 7f91af200640 1 -- 172.21.15.60:0/1813357842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 msgr2=0x7f91b0103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.025+0000 7f91af200640 1 --2- 172.21.15.60:0/1813357842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 0x7f91b0103780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009900 tx=0x7f91a002bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:29.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.025+0000 7f91af200640 1 -- 172.21.15.60:0/1813357842 shutdown_connections 2024-02-26T10:45:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.025+0000 7f91af200640 1 --2- 172.21.15.60:0/1813357842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b0104f10 0x7f91b010b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.025+0000 7f91af200640 1 --2- 172.21.15.60:0/1813357842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0104550 0x7f91b01049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.025+0000 7f91af200640 1 --2- 172.21.15.60:0/1813357842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 0x7f91b0103780 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.025+0000 7f91af200640 1 -- 172.21.15.60:0/1813357842 >> 172.21.15.60:0/1813357842 conn(0x7f91b00fe8d0 msgr2=0x7f91b0100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.025+0000 7f91af200640 1 -- 172.21.15.60:0/1813357842 shutdown_connections 2024-02-26T10:45:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.025+0000 7f91af200640 1 -- 172.21.15.60:0/1813357842 wait complete. 2024-02-26T10:45:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.027+0000 7f91af200640 1 Processor -- start 2024-02-26T10:45:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.027+0000 7f91af200640 1 -- start start 2024-02-26T10:45:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.027+0000 7f91af200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 0x7f91b0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.027+0000 7f91af200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b0104550 0x7f91b0199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.027+0000 7f91af200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0104f10 0x7f91b01a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.027+0000 7f91af200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b010eff0 con 0x7f91b0103360 2024-02-26T10:45:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91af200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b010ee70 con 0x7f91b0104550 2024-02-26T10:45:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91af200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b010f170 con 0x7f91b0104f10 2024-02-26T10:45:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ade00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 0x7f91b0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ade00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 0x7f91b0199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51738/0 (socket says 172.21.15.60:51738) 2024-02-26T10:45:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ade00640 1 -- 172.21.15.60:0/4286503552 learned_addr learned my addr 172.21.15.60:0/4286503552 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ad400640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b0104550 0x7f91b0199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ae800640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0104f10 0x7f91b01a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ade00640 1 -- 172.21.15.60:0/4286503552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0104f10 msgr2=0x7f91b01a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ade00640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0104f10 0x7f91b01a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ade00640 1 -- 172.21.15.60:0/4286503552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b0104550 msgr2=0x7f91b0199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ade00640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b0104550 0x7f91b0199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ade00640 1 -- 172.21.15.60:0/4286503552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b01a1290 con 0x7f91b0103360 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.028+0000 7f91ad400640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b0104550 0x7f91b0199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.029+0000 7f91ade00640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 0x7f91b0199670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f91a0000c00 tx=0x7f91a002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.029+0000 7f91a6a00640 1 -- 172.21.15.60:0/4286503552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0041070 con 0x7f91b0103360 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.029+0000 7f91a6a00640 1 -- 172.21.15.60:0/4286503552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91a0004450 con 0x7f91b0103360 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.029+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b01a1520 con 0x7f91b0103360 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.029+0000 7f91a6a00640 1 -- 172.21.15.60:0/4286503552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0034440 con 0x7f91b0103360 2024-02-26T10:45:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.029+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b01a1a10 con 0x7f91b0103360 2024-02-26T10:45:29.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.030+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b010a680 con 0x7f91b0103360 2024-02-26T10:45:29.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.032+0000 7f91a6a00640 1 -- 172.21.15.60:0/4286503552 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f91a0044070 con 0x7f91b0103360 2024-02-26T10:45:29.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.033+0000 7f91a6a00640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f91900a1d80 0x7f91900a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.034+0000 7f91a6a00640 1 -- 172.21.15.60:0/4286503552 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f91a00efc70 con 0x7f91b0103360 2024-02-26T10:45:29.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.034+0000 7f91ad400640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f91900a1d80 0x7f91900a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.035+0000 7f91ad400640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f91900a1d80 0x7f91900a4240 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91b019ab80 tx=0x7f9198009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:29.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.038+0000 7f91a6a00640 1 -- 172.21.15.60:0/4286503552 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91a00bc6d0 con 0x7f91b0103360 2024-02-26T10:45:29.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.239+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91b002d050 con 0x7f91b0103360 2024-02-26T10:45:29.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.240+0000 7f91a6a00640 1 -- 172.21.15.60:0/4286503552 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f91a00c0770 con 0x7f91b0103360 2024-02-26T10:45:29.235 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f91900a1d80 msgr2=0x7f91900a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f91900a1d80 0x7f91900a4240 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91b019ab80 tx=0x7f9198009500 comp rx=0 tx=0).stop 2024-02-26T10:45:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 msgr2=0x7f91b0199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 0x7f91b0199670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f91a0000c00 tx=0x7f91a002c640 comp rx=0 tx=0).stop 2024-02-26T10:45:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 shutdown_connections 2024-02-26T10:45:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f91900a1d80 0x7f91900a4240 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91b0104550 0x7f91b0199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91b0104f10 0x7f91b01a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 --2- 172.21.15.60:0/4286503552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91b0103360 0x7f91b0199670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 >> 172.21.15.60:0/4286503552 conn(0x7f91b00fe8d0 msgr2=0x7f91b0107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:29.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.242+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 shutdown_connections 2024-02-26T10:45:29.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.243+0000 7f91af200640 1 -- 172.21.15.60:0/4286503552 wait complete. 2024-02-26T10:45:29.256 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:45:29.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.488+0000 7fb7efe00640 1 -- 172.21.15.60:0/2799824799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f01044f0 msgr2=0x7fb7f01068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.488+0000 7fb7efe00640 1 --2- 172.21.15.60:0/2799824799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f01044f0 0x7fb7f01068e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009900 tx=0x7fb7dc02bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:29.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.489+0000 7fb7efe00640 1 -- 172.21.15.60:0/2799824799 shutdown_connections 2024-02-26T10:45:29.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.489+0000 7fb7efe00640 1 --2- 172.21.15.60:0/2799824799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f0106e20 0x7fb7f0109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.489+0000 7fb7efe00640 1 --2- 172.21.15.60:0/2799824799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f01044f0 0x7fb7f01068e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.489+0000 7fb7efe00640 1 --2- 172.21.15.60:0/2799824799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7f0101bc0 0x7fb7f0103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.489+0000 7fb7efe00640 1 -- 172.21.15.60:0/2799824799 >> 172.21.15.60:0/2799824799 conn(0x7fb7f00fb5d0 msgr2=0x7fb7f00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:29.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.489+0000 7fb7efe00640 1 -- 172.21.15.60:0/2799824799 shutdown_connections 2024-02-26T10:45:29.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.489+0000 7fb7efe00640 1 -- 172.21.15.60:0/2799824799 wait complete. 2024-02-26T10:45:29.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.490+0000 7fb7efe00640 1 Processor -- start 2024-02-26T10:45:29.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.490+0000 7fb7efe00640 1 -- start start 2024-02-26T10:45:29.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7efe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7efe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f01044f0 0x7fb7f0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7efe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7f0106e20 0x7fb7f019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7efe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f010ce20 con 0x7fb7f01044f0 2024-02-26T10:45:29.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7efe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f010cca0 con 0x7fb7f0101bc0 2024-02-26T10:45:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7efe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f010cfa0 con 0x7fb7f0106e20 2024-02-26T10:45:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7ee000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f01044f0 0x7fb7f0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7eea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7ef400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7f0106e20 0x7fb7f019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7eea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51516/0 (socket says 172.21.15.60:51516) 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7eea00640 1 -- 172.21.15.60:0/580342473 learned_addr learned my addr 172.21.15.60:0/580342473 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7ee000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f01044f0 0x7fb7f0197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51758/0 (socket says 172.21.15.60:51758) 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7eea00640 1 -- 172.21.15.60:0/580342473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7f0106e20 msgr2=0x7fb7f019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7eea00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7f0106e20 0x7fb7f019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7eea00640 1 -- 172.21.15.60:0/580342473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f01044f0 msgr2=0x7fb7f0197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.491+0000 7fb7eea00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f01044f0 0x7fb7f0197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.492+0000 7fb7eea00640 1 -- 172.21.15.60:0/580342473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f019f070 con 0x7fb7f0101bc0 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.492+0000 7fb7ef400640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7f0106e20 0x7fb7f019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.492+0000 7fb7eea00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01974f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e9f0 tx=0x7fb7e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:29.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.492+0000 7fb7f4a00640 1 -- 172.21.15.60:0/580342473 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000cd80 con 0x7fb7f0101bc0 2024-02-26T10:45:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.492+0000 7fb7f4a00640 1 -- 172.21.15.60:0/580342473 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7e0010910 con 0x7fb7f0101bc0 2024-02-26T10:45:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.492+0000 7fb7f4a00640 1 -- 172.21.15.60:0/580342473 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0018990 con 0x7fb7f0101bc0 2024-02-26T10:45:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.492+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f00ffa10 con 0x7fb7f0101bc0 2024-02-26T10:45:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.492+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f00fff80 con 0x7fb7f0101bc0 2024-02-26T10:45:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.494+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f00fd0c0 con 0x7fb7f0101bc0 2024-02-26T10:45:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.495+0000 7fb7f4a00640 1 -- 172.21.15.60:0/580342473 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb7e0010ac0 con 0x7fb7f0101bc0 2024-02-26T10:45:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.497+0000 7fb7f4a00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb7d40a1e00 0x7fb7d40a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.497+0000 7fb7f4a00640 1 -- 172.21.15.60:0/580342473 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb7e00ce400 con 0x7fb7f0101bc0 2024-02-26T10:45:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.497+0000 7fb7ee000640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb7d40a1e00 0x7fb7d40a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.498+0000 7fb7ee000640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb7d40a1e00 0x7fb7d40a42c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc02c170 tx=0x7fb7dc03f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.501+0000 7fb7f4a00640 1 -- 172.21.15.60:0/580342473 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb7e009ae30 con 0x7fb7f0101bc0 2024-02-26T10:45:29.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:29 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/686844430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:29.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:29 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4286503552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.706+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7f0101360 con 0x7fb7f0101bc0 2024-02-26T10:45:29.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.707+0000 7fb7f4a00640 1 -- 172.21.15.60:0/580342473 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7e009eed0 con 0x7fb7f0101bc0 2024-02-26T10:45:29.702 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.709+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb7d40a1e00 msgr2=0x7fb7d40a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.709+0000 7fb7efe00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb7d40a1e00 0x7fb7d40a42c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc02c170 tx=0x7fb7dc03f040 comp rx=0 tx=0).stop 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.709+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f0101bc0 msgr2=0x7fb7f01974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.709+0000 7fb7efe00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01974f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e9f0 tx=0x7fb7e000eeb0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.710+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 shutdown_connections 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.710+0000 7fb7efe00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb7d40a1e00 0x7fb7d40a42c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.710+0000 7fb7efe00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7f0106e20 0x7fb7f019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.710+0000 7fb7efe00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f01044f0 0x7fb7f0197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.710+0000 7fb7efe00640 1 --2- 172.21.15.60:0/580342473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7f0101bc0 0x7fb7f01974f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.710+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 >> 172.21.15.60:0/580342473 conn(0x7fb7f00fb5d0 msgr2=0x7fb7f0102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.710+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 shutdown_connections 2024-02-26T10:45:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.710+0000 7fb7efe00640 1 -- 172.21.15.60:0/580342473 wait complete. 2024-02-26T10:45:29.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:45:29.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:29 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/686844430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:29.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:29 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/4286503552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:29.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:29 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/686844430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:29.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:29 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/4286503552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.961+0000 7f25c7200640 1 -- 172.21.15.60:0/2781294720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 msgr2=0x7f25c8101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.961+0000 7f25c7200640 1 --2- 172.21.15.60:0/2781294720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 0x7f25c8101580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009900 tx=0x7f25b802bc40 comp rx=0 tx=0).stop 2024-02-26T10:45:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.961+0000 7f25c7200640 1 -- 172.21.15.60:0/2781294720 shutdown_connections 2024-02-26T10:45:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.961+0000 7f25c7200640 1 --2- 172.21.15.60:0/2781294720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c8102350 0x7f25c81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.961+0000 7f25c7200640 1 --2- 172.21.15.60:0/2781294720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25c8102d10 0x7f25c81095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.961+0000 7f25c7200640 1 --2- 172.21.15.60:0/2781294720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 0x7f25c8101580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.961+0000 7f25c7200640 1 -- 172.21.15.60:0/2781294720 >> 172.21.15.60:0/2781294720 conn(0x7f25c80fc6d0 msgr2=0x7f25c80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.961+0000 7f25c7200640 1 -- 172.21.15.60:0/2781294720 shutdown_connections 2024-02-26T10:45:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.961+0000 7f25c7200640 1 -- 172.21.15.60:0/2781294720 wait complete. 2024-02-26T10:45:29.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.963+0000 7f25c7200640 1 Processor -- start 2024-02-26T10:45:29.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.963+0000 7f25c7200640 1 -- start start 2024-02-26T10:45:29.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.963+0000 7f25c7200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 0x7f25c8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.963+0000 7f25c7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c8102350 0x7f25c8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.963+0000 7f25c7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25c8102d10 0x7f25c81a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.963+0000 7f25c7200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c810ce40 con 0x7f25c8101160 2024-02-26T10:45:29.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.963+0000 7f25c7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c810ccc0 con 0x7f25c8102350 2024-02-26T10:45:29.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.963+0000 7f25c7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c810cfc0 con 0x7f25c8102d10 2024-02-26T10:45:29.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 0x7f25c8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 0x7f25c8199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51786/0 (socket says 172.21.15.60:51786) 2024-02-26T10:45:29.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5e00640 1 -- 172.21.15.60:0/3631104712 learned_addr learned my addr 172.21.15.60:0/3631104712 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:29.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c6800640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25c8102d10 0x7f25c81a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5400640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c8102350 0x7f25c8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5e00640 1 -- 172.21.15.60:0/3631104712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25c8102d10 msgr2=0x7f25c81a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5e00640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25c8102d10 0x7f25c81a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5e00640 1 -- 172.21.15.60:0/3631104712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c8102350 msgr2=0x7f25c8199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:29.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5e00640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c8102350 0x7f25c8199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:29.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5e00640 1 -- 172.21.15.60:0/3631104712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c81a1290 con 0x7f25c8101160 2024-02-26T10:45:29.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.964+0000 7f25c5400640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c8102350 0x7f25c8199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:29.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.965+0000 7f25c5e00640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 0x7f25c8199670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25b802efd0 tx=0x7f25b802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:29.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.965+0000 7f25bea00640 1 -- 172.21.15.60:0/3631104712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8041070 con 0x7f25c8101160 2024-02-26T10:45:29.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.965+0000 7f25bea00640 1 -- 172.21.15.60:0/3631104712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b8004450 con 0x7f25c8101160 2024-02-26T10:45:29.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.965+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c81a1520 con 0x7f25c8101160 2024-02-26T10:45:29.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.965+0000 7f25bea00640 1 -- 172.21.15.60:0/3631104712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8003e50 con 0x7f25c8101160 2024-02-26T10:45:29.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.965+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c81a1a30 con 0x7f25c8101160 2024-02-26T10:45:29.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.966+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c81a1c90 con 0x7f25c8101160 2024-02-26T10:45:29.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.968+0000 7f25bea00640 1 -- 172.21.15.60:0/3631104712 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f25b8044070 con 0x7f25c8101160 2024-02-26T10:45:29.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.970+0000 7f25bea00640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f25ac0a1fa0 0x7f25ac0a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:29.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.970+0000 7f25bea00640 1 -- 172.21.15.60:0/3631104712 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f25b80eff90 con 0x7f25c8101160 2024-02-26T10:45:29.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.970+0000 7f25c5400640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f25ac0a1fa0 0x7f25ac0a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:29.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.971+0000 7f25c5400640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f25ac0a1fa0 0x7f25ac0a4460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f25c819ab80 tx=0x7f25b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:29.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:29.974+0000 7f25bea00640 1 -- 172.21.15.60:0/3631104712 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f25b80bc9c0 con 0x7f25c8101160 2024-02-26T10:45:30.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.177+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f25c81a1e60 con 0x7f25c8101160 2024-02-26T10:45:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.178+0000 7f25bea00640 1 -- 172.21.15.60:0/3631104712 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f25b80c0a60 con 0x7f25c8101160 2024-02-26T10:45:30.175 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f25ac0a1fa0 msgr2=0x7f25ac0a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f25ac0a1fa0 0x7f25ac0a4460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f25c819ab80 tx=0x7f25b0005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 msgr2=0x7f25c8199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 0x7f25c8199670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25b802efd0 tx=0x7f25b802c660 comp rx=0 tx=0).stop 2024-02-26T10:45:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 shutdown_connections 2024-02-26T10:45:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f25ac0a1fa0 0x7f25ac0a4460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c8102350 0x7f25c8199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25c8102d10 0x7f25c81a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 --2- 172.21.15.60:0/3631104712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c8101160 0x7f25c8199670 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.181+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 >> 172.21.15.60:0/3631104712 conn(0x7f25c80fc6d0 msgr2=0x7f25c8073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.182+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 shutdown_connections 2024-02-26T10:45:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.182+0000 7f25c7200640 1 -- 172.21.15.60:0/3631104712 wait complete. 2024-02-26T10:45:30.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:45:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.427+0000 7ff254a00640 1 -- 172.21.15.60:0/173994928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250108f50 msgr2=0x7ff25010b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.427+0000 7ff254a00640 1 --2- 172.21.15.60:0/173994928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250108f50 0x7ff25010b3e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff248009ca0 tx=0x7ff24802c180 comp rx=0 tx=0).stop 2024-02-26T10:45:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.427+0000 7ff254a00640 1 -- 172.21.15.60:0/173994928 shutdown_connections 2024-02-26T10:45:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.427+0000 7ff254a00640 1 --2- 172.21.15.60:0/173994928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250108f50 0x7ff25010b3e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.427+0000 7ff254a00640 1 --2- 172.21.15.60:0/173994928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff250073c40 0x7ff2500740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.427+0000 7ff254a00640 1 --2- 172.21.15.60:0/173994928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff250075220 0x7ff250073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.427+0000 7ff254a00640 1 -- 172.21.15.60:0/173994928 >> 172.21.15.60:0/173994928 conn(0x7ff2500fc5f0 msgr2=0x7ff2500fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.427+0000 7ff254a00640 1 -- 172.21.15.60:0/173994928 shutdown_connections 2024-02-26T10:45:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.427+0000 7ff254a00640 1 -- 172.21.15.60:0/173994928 wait complete. 2024-02-26T10:45:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.428+0000 7ff254a00640 1 Processor -- start 2024-02-26T10:45:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.428+0000 7ff254a00640 1 -- start start 2024-02-26T10:45:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.428+0000 7ff254a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff250073c40 0x7ff250199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.428+0000 7ff254a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250075220 0x7ff250199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.428+0000 7ff254a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff250108f50 0x7ff2501a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.428+0000 7ff254a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25010ef40 con 0x7ff250075220 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.428+0000 7ff254a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25010edc0 con 0x7ff250073c40 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.428+0000 7ff254a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25010f0c0 con 0x7ff250108f50 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250075220 0x7ff250199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff250073c40 0x7ff250199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff250073c40 0x7ff250199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51542/0 (socket says 172.21.15.60:51542) 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24f400640 1 -- 172.21.15.60:0/3555262199 learned_addr learned my addr 172.21.15.60:0/3555262199 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250075220 0x7ff250199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51810/0 (socket says 172.21.15.60:51810) 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24fe00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff250108f50 0x7ff2501a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24f400640 1 -- 172.21.15.60:0/3555262199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff250108f50 msgr2=0x7ff2501a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24f400640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff250108f50 0x7ff2501a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24f400640 1 -- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250075220 msgr2=0x7ff250199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24f400640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250075220 0x7ff250199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24f400640 1 -- 172.21.15.60:0/3555262199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2501a1190 con 0x7ff250073c40 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.429+0000 7ff24ea00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250075220 0x7ff250199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.430+0000 7ff24f400640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff250073c40 0x7ff250199570 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff24000e9f0 tx=0x7ff24000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.430+0000 7ff247e00640 1 -- 172.21.15.60:0/3555262199 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24000cd80 con 0x7ff250073c40 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.430+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2501a1480 con 0x7ff250073c40 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.430+0000 7ff247e00640 1 -- 172.21.15.60:0/3555262199 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff240010910 con 0x7ff250073c40 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.430+0000 7ff247e00640 1 -- 172.21.15.60:0/3555262199 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2400188f0 con 0x7ff250073c40 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.430+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2501a19d0 con 0x7ff250073c40 2024-02-26T10:45:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.431+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff250107d10 con 0x7ff250073c40 2024-02-26T10:45:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.433+0000 7ff247e00640 1 -- 172.21.15.60:0/3555262199 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff240010ac0 con 0x7ff250073c40 2024-02-26T10:45:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.434+0000 7ff247e00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff2340a1d80 0x7ff2340a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.435+0000 7ff247e00640 1 -- 172.21.15.60:0/3555262199 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7ff2400ce940 con 0x7ff250073c40 2024-02-26T10:45:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.435+0000 7ff24ea00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff2340a1d80 0x7ff2340a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.435+0000 7ff24ea00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff2340a1d80 0x7ff2340a4240 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff25019aa80 tx=0x7ff23c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.436+0000 7ff247e00640 1 -- 172.21.15.60:0/3555262199 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff24009b310 con 0x7ff250073c40 2024-02-26T10:45:30.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.682+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff25002ce30 con 0x7ff250073c40 2024-02-26T10:45:30.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.684+0000 7ff247e00640 1 -- 172.21.15.60:0/3555262199 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11359 (secure 0 0 0) 0x7ff24009f3b0 con 0x7ff250073c40 2024-02-26T10:45:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:30.679 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:43:16.029212+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:45:30.679 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff2340a1d80 msgr2=0x7ff2340a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff2340a1d80 0x7ff2340a4240 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff25019aa80 tx=0x7ff23c005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff250073c40 msgr2=0x7ff250199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff250073c40 0x7ff250199570 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff24000e9f0 tx=0x7ff24000eeb0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 shutdown_connections 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff2340a1d80 0x7ff2340a4240 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff250108f50 0x7ff2501a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff250075220 0x7ff250199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 --2- 172.21.15.60:0/3555262199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff250073c40 0x7ff250199570 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 >> 172.21.15.60:0/3555262199 conn(0x7ff2500fc5f0 msgr2=0x7ff250104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.686+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 shutdown_connections 2024-02-26T10:45:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.687+0000 7ff254a00640 1 -- 172.21.15.60:0/3555262199 wait complete. 2024-02-26T10:45:30.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:45:30.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:30 smithi060 ceph-mon[31539]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:30.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:30 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/580342473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:30.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:30 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3631104712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:30.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:30 smithi113 ceph-mon[36126]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:30.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:30 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/580342473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:30.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:30 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3631104712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:30.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:30 smithi100 ceph-mon[36051]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:30.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:30 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/580342473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:30.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:30 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3631104712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:30.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.937+0000 7f2e1f200640 1 -- 172.21.15.60:0/3852081363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 msgr2=0x7f2e20108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.937+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3852081363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 0x7f2e20108d30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009900 tx=0x7f2e0c02bc50 comp rx=0 tx=0).stop 2024-02-26T10:45:30.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.937+0000 7f2e1f200640 1 -- 172.21.15.60:0/3852081363 shutdown_connections 2024-02-26T10:45:30.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.937+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3852081363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e20109270 0x7f2e2010fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.937+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3852081363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 0x7f2e20108d30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.937+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3852081363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e201076c0 0x7f2e20107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.937+0000 7f2e1f200640 1 -- 172.21.15.60:0/3852081363 >> 172.21.15.60:0/3852081363 conn(0x7f2e20076670 msgr2=0x7f2e20078a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:30.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.937+0000 7f2e1f200640 1 -- 172.21.15.60:0/3852081363 shutdown_connections 2024-02-26T10:45:30.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.937+0000 7f2e1f200640 1 -- 172.21.15.60:0/3852081363 wait complete. 2024-02-26T10:45:30.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.938+0000 7f2e1f200640 1 Processor -- start 2024-02-26T10:45:30.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.938+0000 7f2e1f200640 1 -- start start 2024-02-26T10:45:30.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.938+0000 7f2e1f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e201076c0 0x7f2e2019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:30.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.938+0000 7f2e1f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 0x7f2e2019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:30.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.938+0000 7f2e1f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e20109270 0x7f2e201a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:30.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.938+0000 7f2e1f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e20113340 con 0x7f2e201088b0 2024-02-26T10:45:30.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.938+0000 7f2e1f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e201131c0 con 0x7f2e201076c0 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.938+0000 7f2e1f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e201134c0 con 0x7f2e20109270 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 0x7f2e2019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e20109270 0x7f2e201a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 0x7f2e2019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51822/0 (socket says 172.21.15.60:51822) 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1d400640 1 -- 172.21.15.60:0/3844587382 learned_addr learned my addr 172.21.15.60:0/3844587382 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1de00640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e201076c0 0x7f2e2019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1e800640 1 -- 172.21.15.60:0/3844587382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e201076c0 msgr2=0x7f2e2019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1e800640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e201076c0 0x7f2e2019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1e800640 1 -- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 msgr2=0x7f2e2019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1e800640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 0x7f2e2019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1e800640 1 -- 172.21.15.60:0/3844587382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e201a5670 con 0x7f2e20109270 2024-02-26T10:45:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.939+0000 7f2e1d400640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 0x7f2e2019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:30.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.940+0000 7f2e1e800640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e20109270 0x7f2e201a5000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009fd0 tx=0x7f2e1800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:30.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.940+0000 7f2e16a00640 1 -- 172.21.15.60:0/3844587382 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800ce20 con 0x7f2e20109270 2024-02-26T10:45:30.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.940+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e201a5960 con 0x7f2e20109270 2024-02-26T10:45:30.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.940+0000 7f2e16a00640 1 -- 172.21.15.60:0/3844587382 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e18005020 con 0x7f2e20109270 2024-02-26T10:45:30.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.940+0000 7f2e16a00640 1 -- 172.21.15.60:0/3844587382 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18010720 con 0x7f2e20109270 2024-02-26T10:45:30.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.940+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e201a5ed0 con 0x7f2e20109270 2024-02-26T10:45:30.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.941+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2010e9e0 con 0x7f2e20109270 2024-02-26T10:45:30.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.943+0000 7f2e16a00640 1 -- 172.21.15.60:0/3844587382 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2e18005540 con 0x7f2e20109270 2024-02-26T10:45:30.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.944+0000 7f2e16a00640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e040a1e00 0x7f2e040a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:30.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.945+0000 7f2e16a00640 1 -- 172.21.15.60:0/3844587382 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2e180ce770 con 0x7f2e20109270 2024-02-26T10:45:30.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.945+0000 7f2e1de00640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e040a1e00 0x7f2e040a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:30.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.945+0000 7f2e1de00640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e040a1e00 0x7f2e040a42c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2e20108710 tx=0x7f2e10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:30.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:30.949+0000 7f2e16a00640 1 -- 172.21.15.60:0/3844587382 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e1809b790 con 0x7f2e20109270 2024-02-26T10:45:31.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.156+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e2002ce30 con 0x7f2e20109270 2024-02-26T10:45:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.158+0000 7f2e16a00640 1 -- 172.21.15.60:0/3844587382 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7f2e1809b090 con 0x7f2e20109270 2024-02-26T10:45:31.154 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:31.154 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:45:31.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e040a1e00 msgr2=0x7f2e040a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e040a1e00 0x7f2e040a42c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2e20108710 tx=0x7f2e10005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:31.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e20109270 msgr2=0x7f2e201a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e20109270 0x7f2e201a5000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009fd0 tx=0x7f2e1800ef10 comp rx=0 tx=0).stop 2024-02-26T10:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 shutdown_connections 2024-02-26T10:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2e040a1e00 0x7f2e040a42c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e20109270 0x7f2e201a5000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e201088b0 0x7f2e2019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 --2- 172.21.15.60:0/3844587382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e201076c0 0x7f2e2019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 >> 172.21.15.60:0/3844587382 conn(0x7f2e20076670 msgr2=0x7f2e2010bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 shutdown_connections 2024-02-26T10:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.161+0000 7f2e1f200640 1 -- 172.21.15.60:0/3844587382 wait complete. 2024-02-26T10:45:31.175 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-26T10:42:42.076607+0000', 'modified': '2024-02-26T10:43:16.029212+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24587, 'mds_1': 24553, 'mds_2': 24530}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24506': {'gid': 24506, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6835/1201544557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 1201544557}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 1201544557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6835/2109318506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 2109318506}, {'type': 'v1', 'addr': '172.21.15.113:6835', 'nonce': 2109318506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6837/3968092815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3968092815}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3968092815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.100:6837/1347855255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 1347855255}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 1347855255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.113:6839/2057195735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6838', 'nonce': 2057195735}, {'type': 'v1', 'addr': '172.21.15.113:6839', 'nonce': 2057195735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6839/2377114018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 2377114018}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 2377114018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T10:45:31.175 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-26T10:45:31.175 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-26T10:45:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.401+0000 7fb0fac00640 1 -- 172.21.15.60:0/1762561754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f4104f10 msgr2=0x7fb0f410b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.401+0000 7fb0fac00640 1 --2- 172.21.15.60:0/1762561754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f4104f10 0x7fb0f410b7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009ca0 tx=0x7fb0ec02c100 comp rx=0 tx=0).stop 2024-02-26T10:45:31.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.401+0000 7fb0fac00640 1 -- 172.21.15.60:0/1762561754 shutdown_connections 2024-02-26T10:45:31.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.401+0000 7fb0fac00640 1 --2- 172.21.15.60:0/1762561754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f4104550 0x7fb0f41049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.401+0000 7fb0fac00640 1 --2- 172.21.15.60:0/1762561754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f4104f10 0x7fb0f410b7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.401+0000 7fb0fac00640 1 --2- 172.21.15.60:0/1762561754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0f4103360 0x7fb0f4103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.401+0000 7fb0fac00640 1 -- 172.21.15.60:0/1762561754 >> 172.21.15.60:0/1762561754 conn(0x7fb0f40fe8d0 msgr2=0x7fb0f4100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.401+0000 7fb0fac00640 1 -- 172.21.15.60:0/1762561754 shutdown_connections 2024-02-26T10:45:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.401+0000 7fb0fac00640 1 -- 172.21.15.60:0/1762561754 wait complete. 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.402+0000 7fb0fac00640 1 Processor -- start 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.402+0000 7fb0fac00640 1 -- start start 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.402+0000 7fb0fac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f4103360 0x7fb0f4199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.402+0000 7fb0fac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f4104550 0x7fb0f4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fac00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0f4104f10 0x7fb0f41a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fac00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f410eff0 con 0x7fb0f4104f10 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f410ee70 con 0x7fb0f4103360 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f410f170 con 0x7fb0f4104550 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fa200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0f4104f10 0x7fb0f41a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0f9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f4103360 0x7fb0f4199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fa200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0f4104f10 0x7fb0f41a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51830/0 (socket says 172.21.15.60:51830) 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fa200640 1 -- 172.21.15.60:0/3716382890 learned_addr learned my addr 172.21.15.60:0/3716382890 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0f8e00640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f4104550 0x7fb0f4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fa200640 1 -- 172.21.15.60:0/3716382890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f4104550 msgr2=0x7fb0f4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fa200640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f4104550 0x7fb0f4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fa200640 1 -- 172.21.15.60:0/3716382890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f4103360 msgr2=0x7fb0f4199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fa200640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f4103360 0x7fb0f4199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.403+0000 7fb0fa200640 1 -- 172.21.15.60:0/3716382890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f41a1230 con 0x7fb0f4104f10 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.404+0000 7fb0fa200640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0f4104f10 0x7fb0f41a0bc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec02c640 tx=0x7fb0ec035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.404+0000 7fb0f2000640 1 -- 172.21.15.60:0/3716382890 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec039040 con 0x7fb0f4104f10 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.404+0000 7fb0f2000640 1 -- 172.21.15.60:0/3716382890 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0ec044070 con 0x7fb0f4104f10 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.404+0000 7fb0f2000640 1 -- 172.21.15.60:0/3716382890 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0043c0 con 0x7fb0f4104f10 2024-02-26T10:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.404+0000 7fb0fac00640 1 -- 172.21.15.60:0/3716382890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f41a14c0 con 0x7fb0f4104f10 2024-02-26T10:45:31.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.404+0000 7fb0fac00640 1 -- 172.21.15.60:0/3716382890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f41a1960 con 0x7fb0f4104f10 2024-02-26T10:45:31.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.407+0000 7fb0ff030740 1 Processor -- start 2024-02-26T10:45:31.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.407+0000 7fb0f2000640 1 -- 172.21.15.60:0/3716382890 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb0ec004560 con 0x7fb0f4104f10 2024-02-26T10:45:31.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0ff030740 1 -- start start 2024-02-26T10:45:31.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f30c8b0 0x56059f30ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f311da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f312410 0x56059f3148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0ff030740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f2715d0 con 0x56059f30d2a0 2024-02-26T10:45:31.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0ff030740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f21a900 con 0x56059f312410 2024-02-26T10:45:31.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0ff030740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f26d670 con 0x56059f30c8b0 2024-02-26T10:45:31.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0f9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f311da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0f8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f30c8b0 0x56059f30ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0f9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f311da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51846/0 (socket says 172.21.15.60:51846) 2024-02-26T10:45:31.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.408+0000 7fb0f9800640 1 -- 172.21.15.60:0/3800791922 learned_addr learned my addr 172.21.15.60:0/3800791922 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.409+0000 7fb0f8e00640 1 --2- 172.21.15.60:0/3800791922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f312410 0x56059f3148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.409+0000 7fb0f9800640 1 -- 172.21.15.60:0/3800791922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f30c8b0 msgr2=0x56059f30ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.409+0000 7fb0f9800640 1 --2- 172.21.15.60:0/3800791922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f30c8b0 0x56059f30ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.409+0000 7fb0f9800640 1 -- 172.21.15.60:0/3800791922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f312410 msgr2=0x56059f3148a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.409+0000 7fb0f9800640 1 --2- 172.21.15.60:0/3800791922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f312410 0x56059f3148a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.409+0000 7fb0f9800640 1 -- 172.21.15.60:0/3800791922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56059ef862d0 con 0x56059f30d2a0 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.409+0000 7fb0f8e00640 1 --2- 172.21.15.60:0/3800791922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f30c8b0 0x56059f30ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.409+0000 7fb0f9800640 1 --2- 172.21.15.60:0/3800791922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f311da0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009ca0 tx=0x7fb0e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e400bb8a4b333 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.410+0000 7fb0f2000640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0d80a1e00 0x7fb0d80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.410+0000 7fb0eaa00640 1 -- 172.21.15.60:0/3800791922 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4004220 con 0x56059f30d2a0 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.410+0000 7fb0eaa00640 1 -- 172.21.15.60:0/3800791922 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0e40043c0 con 0x56059f30d2a0 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.410+0000 7fb0eaa00640 1 -- 172.21.15.60:0/3800791922 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4019870 con 0x56059f30d2a0 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.410+0000 7fb0f2000640 1 -- 172.21.15.60:0/3716382890 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb0ec0f0680 con 0x7fb0f4104f10 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.410+0000 7fb0f8e00640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0d80a1e00 0x7fb0d80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 -- 172.21.15.60:0/3800791922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 msgr2=0x56059f311da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 --2- 172.21.15.60:0/3800791922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f311da0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009ca0 tx=0x7fb0e4011810 comp rx=0 tx=0).stop 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0f8e00640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0d80a1e00 0x7fb0d80a42c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x56059f30db00 tx=0x7fb0e00046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 -- 172.21.15.60:0/3800791922 shutdown_connections 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 --2- 172.21.15.60:0/3800791922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f312410 0x56059f3148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 --2- 172.21.15.60:0/3800791922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f311da0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 --2- 172.21.15.60:0/3800791922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f30c8b0 0x56059f30ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 -- 172.21.15.60:0/3800791922 >> 172.21.15.60:0/3800791922 conn(0x56059f307e20 msgr2=0x56059f308220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 -- 172.21.15.60:0/3800791922 shutdown_connections 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 -- 172.21.15.60:0/3800791922 wait complete. 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.411+0000 7fb0ff030740 1 -- start start 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0ff030740 1 Processor -- start 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 0x56059f316a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f316fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0ff030740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f3153c0 con 0x56059f30d2a0 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0ff030740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f22dd60 con 0x56059f30c8b0 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0ff030740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f315540 con 0x56059f312410 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0ff030740 10 client.0 _ll_register_callbacks cb 0x56059f269320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0f8e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f316fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0f8e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f316fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51852/0 (socket says 172.21.15.60:51852) 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0f8e00640 1 -- 172.21.15.60:0/625049 learned_addr learned my addr 172.21.15.60:0/625049 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0fa200640 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.412+0000 7fb0f9800640 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 0x56059f316a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0fa200640 1 -- 172.21.15.60:0/625049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 msgr2=0x56059f316a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0fa200640 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 0x56059f316a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0fa200640 1 -- 172.21.15.60:0/625049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 msgr2=0x56059f316fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0fa200640 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f316fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0fa200640 1 -- 172.21.15.60:0/625049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56059f0506b0 con 0x56059f312410 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0fa200640 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31e010 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006010 tx=0x7fb0ec0bcc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e4bb136dd91b5cf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0df400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0df400640 1 -- 172.21.15.60:0/625049 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0098b0 con 0x56059f312410 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0df400640 1 -- 172.21.15.60:0/625049 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb0ec0bd260 con 0x56059f312410 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.413+0000 7fb0ff030740 1 -- 172.21.15.60:0/625049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0e40043c0 con 0x56059f312410 2024-02-26T10:45:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0df400640 1 -- 172.21.15.60:0/625049 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0ec0bd000 con 0x56059f312410 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0ff030740 10 client.25336 fetch_fsmap learned FSMap version 23 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0ff030740 1 -- 172.21.15.60:0/625049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56059f315490 con 0x56059f312410 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0df400640 1 -- 172.21.15.60:0/625049 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0e40043c0 con 0x56059f312410 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0df400640 1 -- 172.21.15.60:0/625049 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb0ec03c030 con 0x56059f312410 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0ff030740 10 client.25336 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0ff030740 10 client.25336 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0ff030740 20 client.25336 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0ff030740 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x56059f323800 0x56059f3225c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0ff030740 4 client.25336 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.414+0000 7fb0ff030740 1 -- 172.21.15.60:0/625049 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56059f27eb10 con 0x56059f323800 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.415+0000 7fb0f9800640 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x56059f323800 0x56059f3225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.415+0000 7fb0f9800640 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x56059f323800 0x56059f3225c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6374fc141cd0c80b server_cookie=e3268a04e1c7edef in_seq=0 out_seq=0 2024-02-26T10:45:31.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.415+0000 7fb0df400640 10 client.25336 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:45:31.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0df400640 1 -- 172.21.15.60:0/625049 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56059f27eb10 con 0x56059f323800 2024-02-26T10:45:31.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0df400640 10 client.25336 handle_command_reply: tid=0 2024-02-26T10:45:31.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0ff030740 1 client.25336 shutdown 2024-02-26T10:45:31.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0ff030740 1 -- 172.21.15.60:0/625049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 msgr2=0x56059f31e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0ff030740 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31e010 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006010 tx=0x7fb0ec0bcc00 comp rx=0 tx=0).stop 2024-02-26T10:45:31.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0ff030740 1 -- 172.21.15.60:0/625049 shutdown_connections 2024-02-26T10:45:31.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0ff030740 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x56059f323800 0x56059f3225c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0ff030740 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31e010 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0ff030740 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f316fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0df400640 0 client.25336 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:45:31.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0ff030740 1 --2- 172.21.15.60:0/625049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 0x56059f316a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.416+0000 7fb0ff030740 1 -- 172.21.15.60:0/625049 >> 172.21.15.60:0/625049 conn(0x56059f307e20 msgr2=0x56059f307020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.417+0000 7fb0ff030740 1 -- 172.21.15.60:0/625049 shutdown_connections 2024-02-26T10:45:31.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.417+0000 7fb0ff030740 1 -- 172.21.15.60:0/625049 wait complete. 2024-02-26T10:45:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.417+0000 7fb0ff030740 20 client.25336 trim_cache size 0 max 16384 2024-02-26T10:45:31.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.434+0000 7fb0ff030740 1 Processor -- start 2024-02-26T10:45:31.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.434+0000 7fb0ff030740 1 -- start start 2024-02-26T10:45:31.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.434+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30c8b0 0x56059f3225c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.434+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30d2a0 0x56059f31dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.434+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f32bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.434+0000 7fb0ff030740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f323020 con 0x56059f30c8b0 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.434+0000 7fb0ff030740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f323680 con 0x56059f30d2a0 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.434+0000 7fb0ff030740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f323ce0 con 0x56059f312410 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30c8b0 0x56059f3225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30c8b0 0x56059f3225c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51868/0 (socket says 172.21.15.60:51868) 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f9800640 1 -- 172.21.15.60:0/3378004831 learned_addr learned my addr 172.21.15.60:0/3378004831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f8e00640 1 --2- 172.21.15.60:0/3378004831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30d2a0 0x56059f31dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0fa200640 1 --2- 172.21.15.60:0/3378004831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f32bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f9800640 1 -- 172.21.15.60:0/3378004831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 msgr2=0x56059f32bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f9800640 1 --2- 172.21.15.60:0/3378004831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f32bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f9800640 1 -- 172.21.15.60:0/3378004831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30d2a0 msgr2=0x56059f31dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f9800640 1 --2- 172.21.15.60:0/3378004831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30d2a0 0x56059f31dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f9800640 1 -- 172.21.15.60:0/3378004831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56059f30db30 con 0x56059f30c8b0 2024-02-26T10:45:31.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0f9800640 1 --2- 172.21.15.60:0/3378004831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30c8b0 0x56059f3225c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4011d10 tx=0x7fb0e4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465e0530f3d6e1f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.435+0000 7fb0ea000640 1 -- 172.21.15.60:0/3378004831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4009e50 con 0x56059f30c8b0 2024-02-26T10:45:31.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ea000640 1 -- 172.21.15.60:0/3378004831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0e4003e60 con 0x56059f30c8b0 2024-02-26T10:45:31.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ea000640 1 -- 172.21.15.60:0/3378004831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4005210 con 0x56059f30c8b0 2024-02-26T10:45:31.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ff030740 1 -- 172.21.15.60:0/3378004831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30c8b0 msgr2=0x56059f3225c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ff030740 1 --2- 172.21.15.60:0/3378004831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30c8b0 0x56059f3225c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4011d10 tx=0x7fb0e4009310 comp rx=0 tx=0).stop 2024-02-26T10:45:31.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ff030740 1 -- 172.21.15.60:0/3378004831 shutdown_connections 2024-02-26T10:45:31.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ff030740 1 --2- 172.21.15.60:0/3378004831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30d2a0 0x56059f31dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ff030740 1 --2- 172.21.15.60:0/3378004831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f32bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ff030740 1 --2- 172.21.15.60:0/3378004831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30c8b0 0x56059f3225c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ff030740 1 -- 172.21.15.60:0/3378004831 >> 172.21.15.60:0/3378004831 conn(0x56059f307e20 msgr2=0x56059f308200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ff030740 1 -- 172.21.15.60:0/3378004831 shutdown_connections 2024-02-26T10:45:31.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.436+0000 7fb0ff030740 1 -- 172.21.15.60:0/3378004831 wait complete. 2024-02-26T10:45:31.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.437+0000 7fb0ff030740 1 -- start start 2024-02-26T10:45:31.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.437+0000 7fb0ff030740 1 Processor -- start 2024-02-26T10:45:31.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.437+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 0x56059f31db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.437+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f31a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.437+0000 7fb0ff030740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.437+0000 7fb0ff030740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f3249a0 con 0x56059f30d2a0 2024-02-26T10:45:31.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.437+0000 7fb0ff030740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f324340 con 0x56059f30c8b0 2024-02-26T10:45:31.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.437+0000 7fb0ff030740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56059f325000 con 0x56059f312410 2024-02-26T10:45:31.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.437+0000 7fb0ff030740 10 client.0 _ll_register_callbacks cb 0x7fb0e40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:31.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f8e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f31a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f8e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f31a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51870/0 (socket says 172.21.15.60:51870) 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 0x56059f31db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f8e00640 1 -- 172.21.15.60:0/269851200 learned_addr learned my addr 172.21.15.60:0/269851200 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0fa200640 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f8e00640 1 -- 172.21.15.60:0/269851200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 msgr2=0x56059f31ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f8e00640 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f8e00640 1 -- 172.21.15.60:0/269851200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 msgr2=0x56059f31db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f8e00640 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 0x56059f31db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f8e00640 1 -- 172.21.15.60:0/269851200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56059f31e170 con 0x56059f30d2a0 2024-02-26T10:45:31.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0fa200640 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.438+0000 7fb0f9800640 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 0x56059f31db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.439+0000 7fb0f8e00640 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f31a630 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0011a80 tx=0x7fb0e0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b43aadce200d547 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.439+0000 7fb0df400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.439+0000 7fb0df400640 1 -- 172.21.15.60:0/269851200 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0005020 con 0x56059f30d2a0 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.439+0000 7fb0df400640 1 -- 172.21.15.60:0/269851200 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb0e00051c0 con 0x56059f30d2a0 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.439+0000 7fb0ff030740 1 -- 172.21.15.60:0/269851200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0e4003e60 con 0x56059f30d2a0 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.439+0000 7fb0df400640 1 -- 172.21.15.60:0/269851200 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e0038050 con 0x56059f30d2a0 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.439+0000 7fb0ff030740 10 client.15228 fetch_fsmap learned FSMap version 23 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.439+0000 7fb0ff030740 1 -- 172.21.15.60:0/269851200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56059f37a3c0 con 0x56059f30d2a0 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.439+0000 7fb0df400640 1 -- 172.21.15.60:0/269851200 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0e4003e60 con 0x56059f30d2a0 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.440+0000 7fb0df400640 1 -- 172.21.15.60:0/269851200 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb0e0004a00 con 0x56059f30d2a0 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.440+0000 7fb0ff030740 10 client.15228 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.440+0000 7fb0ff030740 10 client.15228 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T10:45:31.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.440+0000 7fb0ff030740 20 client.15228 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.440+0000 7fb0ff030740 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x56059f37eb50 0x56059f37a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.440+0000 7fb0ff030740 4 client.15228 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.440+0000 7fb0ff030740 1 -- 172.21.15.60:0/269851200 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56059f278e60 con 0x56059f37eb50 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.440+0000 7fb0f9800640 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x56059f37eb50 0x56059f37a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0f9800640 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x56059f37eb50 0x56059f37a700 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c7bd3a4f91af857 server_cookie=732df2cf4e8a1cdf in_seq=0 out_seq=0 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0df400640 10 client.15228 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0df400640 1 -- 172.21.15.60:0/269851200 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56059f278e60 con 0x56059f37eb50 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0df400640 10 client.15228 handle_command_reply: tid=0 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 client.15228 shutdown 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 -- 172.21.15.60:0/269851200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 msgr2=0x56059f31a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f31a630 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0011a80 tx=0x7fb0e0005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 -- 172.21.15.60:0/269851200 shutdown_connections 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x56059f37eb50 0x56059f37a700 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56059f312410 0x56059f31ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56059f30d2a0 0x56059f31a630 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 --2- 172.21.15.60:0/269851200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059f30c8b0 0x56059f31db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0df400640 0 client.15228 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 -- 172.21.15.60:0/269851200 >> 172.21.15.60:0/269851200 conn(0x56059f307e20 msgr2=0x56059f32bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.441+0000 7fb0ff030740 1 -- 172.21.15.60:0/269851200 shutdown_connections 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0ff030740 1 -- 172.21.15.60:0/269851200 wait complete. 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0ff030740 20 client.15228 trim_cache size 0 max 16384 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 2, 2024-02-26T10:45:31.444 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24530, 2024-02-26T10:45:31.445 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T10:45:31.445 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T10:45:31.445 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T10:45:31.445 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 136.09865158700001, 2024-02-26T10:45:31.445 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 23, 2024-02-26T10:45:31.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 95, 2024-02-26T10:45:31.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 91, 2024-02-26T10:45:31.445 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 189.393188622 2024-02-26T10:45:31.445 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:45:31.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 -- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0d80a1e00 msgr2=0x7fb0d80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0d80a1e00 0x7fb0d80a42c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x56059f30db00 tx=0x7fb0e00046c0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 -- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0f4104f10 msgr2=0x7fb0f41a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0f4104f10 0x7fb0f41a0bc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec02c640 tx=0x7fb0ec035d10 comp rx=0 tx=0).stop 2024-02-26T10:45:31.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 -- 172.21.15.60:0/3716382890 shutdown_connections 2024-02-26T10:45:31.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb0d80a1e00 0x7fb0d80a42c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0f4104f10 0x7fb0f41a0bc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f4104550 0x7fb0f4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 --2- 172.21.15.60:0/3716382890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0f4103360 0x7fb0f4199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.442+0000 7fb0fac00640 1 -- 172.21.15.60:0/3716382890 >> 172.21.15.60:0/3716382890 conn(0x7fb0f40fe8d0 msgr2=0x7fb0f4107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.443+0000 7fb0fac00640 1 -- 172.21.15.60:0/3716382890 shutdown_connections 2024-02-26T10:45:31.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.443+0000 7fb0fac00640 1 -- 172.21.15.60:0/3716382890 wait complete. 2024-02-26T10:45:31.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-26T10:45:31.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.676+0000 7f2c43200640 1 -- 172.21.15.60:0/2616142264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c44075220 msgr2=0x7f2c44073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.676+0000 7f2c43200640 1 --2- 172.21.15.60:0/2616142264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c44075220 0x7f2c44073670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009900 tx=0x7f2c3402bcc0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.676+0000 7f2c43200640 1 -- 172.21.15.60:0/2616142264 shutdown_connections 2024-02-26T10:45:31.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.676+0000 7f2c43200640 1 --2- 172.21.15.60:0/2616142264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c44073c40 0x7f2c440740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.676+0000 7f2c43200640 1 --2- 172.21.15.60:0/2616142264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c44108ff0 0x7f2c4410b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.676+0000 7f2c43200640 1 --2- 172.21.15.60:0/2616142264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c44075220 0x7f2c44073670 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.676+0000 7f2c43200640 1 -- 172.21.15.60:0/2616142264 >> 172.21.15.60:0/2616142264 conn(0x7f2c440fc6d0 msgr2=0x7f2c440feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.676+0000 7f2c43200640 1 -- 172.21.15.60:0/2616142264 shutdown_connections 2024-02-26T10:45:31.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.676+0000 7f2c43200640 1 -- 172.21.15.60:0/2616142264 wait complete. 2024-02-26T10:45:31.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.677+0000 7f2c43200640 1 Processor -- start 2024-02-26T10:45:31.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.677+0000 7f2c43200640 1 -- start start 2024-02-26T10:45:31.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.677+0000 7f2c43200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c44073c40 0x7f2c44199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.677+0000 7f2c43200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c44075220 0x7f2c44199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.677+0000 7f2c43200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c44108ff0 0x7f2c441a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.677+0000 7f2c43200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4410f030 con 0x7f2c44108ff0 2024-02-26T10:45:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.677+0000 7f2c43200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4410eeb0 con 0x7f2c44073c40 2024-02-26T10:45:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.677+0000 7f2c43200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4410f1b0 con 0x7f2c44075220 2024-02-26T10:45:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c41e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c44073c40 0x7f2c44199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c41e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c44073c40 0x7f2c44199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51626/0 (socket says 172.21.15.60:51626) 2024-02-26T10:45:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c41e00640 1 -- 172.21.15.60:0/2457198228 learned_addr learned my addr 172.21.15.60:0/2457198228 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c42800640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c44108ff0 0x7f2c441a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c3be00640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c44075220 0x7f2c44199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c41e00640 1 -- 172.21.15.60:0/2457198228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c44075220 msgr2=0x7f2c44199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c41e00640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c44075220 0x7f2c44199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c41e00640 1 -- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c44108ff0 msgr2=0x7f2c441a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c41e00640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c44108ff0 0x7f2c441a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c41e00640 1 -- 172.21.15.60:0/2457198228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c441a1240 con 0x7f2c44073c40 2024-02-26T10:45:31.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.678+0000 7f2c3be00640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c44075220 0x7f2c44199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.679+0000 7f2c41e00640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c44073c40 0x7f2c44199620 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005ab0 tx=0x7f2c3402c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.679+0000 7f2c3aa00640 1 -- 172.21.15.60:0/2457198228 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34043020 con 0x7f2c44073c40 2024-02-26T10:45:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.679+0000 7f2c43200640 1 -- 172.21.15.60:0/2457198228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c441a14d0 con 0x7f2c44073c40 2024-02-26T10:45:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.679+0000 7f2c3aa00640 1 -- 172.21.15.60:0/2457198228 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c340054e0 con 0x7f2c44073c40 2024-02-26T10:45:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.679+0000 7f2c3aa00640 1 -- 172.21.15.60:0/2457198228 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34003ca0 con 0x7f2c44073c40 2024-02-26T10:45:31.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.679+0000 7f2c43200640 1 -- 172.21.15.60:0/2457198228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c441a1a70 con 0x7f2c44073c40 2024-02-26T10:45:31.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.682+0000 7f2c4b9ce740 1 Processor -- start 2024-02-26T10:45:31.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.682+0000 7f2c4b9ce740 1 -- start start 2024-02-26T10:45:31.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.682+0000 7f2c3aa00640 1 -- 172.21.15.60:0/2457198228 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2c34046020 con 0x7f2c44073c40 2024-02-26T10:45:31.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d49cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 0x556fc8d518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8cae5d0 con 0x556fc8d4a2a0 2024-02-26T10:45:31.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8c57900 con 0x556fc8d498b0 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8caa670 con 0x556fc8d4f410 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c42800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d49cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c41e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c42800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d49cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51630/0 (socket says 172.21.15.60:51630) 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c42800640 1 -- 172.21.15.60:0/1831643421 learned_addr learned my addr 172.21.15.60:0/1831643421 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.683+0000 7f2c3be00640 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 0x556fc8d518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c42800640 1 -- 172.21.15.60:0/1831643421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 msgr2=0x556fc8d518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c42800640 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 0x556fc8d518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c42800640 1 -- 172.21.15.60:0/1831643421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 msgr2=0x556fc8d4eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c42800640 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d4eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c42800640 1 -- 172.21.15.60:0/1831643421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fc89c32d0 con 0x556fc8d498b0 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c41e00640 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d4eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:31.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c3be00640 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 0x556fc8d518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c42800640 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d49cd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c010370 tx=0x7f2c3c010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dba79ef6451bb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c33000640 1 -- 172.21.15.60:0/1831643421 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c016070 con 0x556fc8d498b0 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c33000640 1 -- 172.21.15.60:0/1831643421 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c3c004d20 con 0x556fc8d498b0 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c33000640 1 -- 172.21.15.60:0/1831643421 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c005480 con 0x556fc8d498b0 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.684+0000 7f2c3aa00640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c280a1d80 0x7f2c280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c3aa00640 1 -- 172.21.15.60:0/2457198228 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2c340f0370 con 0x7f2c44073c40 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c41e00640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c280a1d80 0x7f2c280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/1831643421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 msgr2=0x556fc8d49cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d49cd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c010370 tx=0x7f2c3c010830 comp rx=0 tx=0).stop 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/1831643421 shutdown_connections 2024-02-26T10:45:31.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 0x556fc8d518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d4eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/1831643421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d49cd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/1831643421 >> 172.21.15.60:0/1831643421 conn(0x556fc8d44e20 msgr2=0x556fc8d45220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/1831643421 shutdown_connections 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/1831643421 wait complete. 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.685+0000 7f2c41e00640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c280a1d80 0x7f2c280a4240 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c340f3db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.686+0000 7f2c4b9ce740 1 -- start start 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.686+0000 7f2c4b9ce740 1 Processor -- start 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.686+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d498b0 0x556fc8d53af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 0x556fc8d54030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d4f410 0x556fc8d5b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8d523c0 con 0x556fc8d498b0 2024-02-26T10:45:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8c6ad60 con 0x556fc8d4f410 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8d52540 con 0x556fc8d4a2a0 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c4b9ce740 10 client.0 _ll_register_callbacks cb 0x556fc8ca6320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c42800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d498b0 0x556fc8d53af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c42800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d498b0 0x556fc8d53af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51910/0 (socket says 172.21.15.60:51910) 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c42800640 1 -- 172.21.15.60:0/2108572615 learned_addr learned my addr 172.21.15.60:0/2108572615 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c3be00640 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 0x556fc8d54030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c42800640 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d4f410 0x556fc8d5b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c42800640 1 -- 172.21.15.60:0/2108572615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 msgr2=0x556fc8d54030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c42800640 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 0x556fc8d54030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c42800640 1 -- 172.21.15.60:0/2108572615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d4f410 msgr2=0x556fc8d5b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c42800640 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d4f410 0x556fc8d5b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.687+0000 7f2c42800640 1 -- 172.21.15.60:0/2108572615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fc8a8d6b0 con 0x556fc8d498b0 2024-02-26T10:45:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.688+0000 7f2c42800640 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d498b0 0x556fc8d53af0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c010fe0 tx=0x7f2c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f85da087b44b820 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.688+0000 7f2c27400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.688+0000 7f2c27400640 1 -- 172.21.15.60:0/2108572615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c016070 con 0x556fc8d498b0 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.688+0000 7f2c27400640 1 -- 172.21.15.60:0/2108572615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2c3c022040 con 0x556fc8d498b0 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.688+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/2108572615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c3c004d20 con 0x556fc8d498b0 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.688+0000 7f2c27400640 1 -- 172.21.15.60:0/2108572615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c3c032050 con 0x556fc8d498b0 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.688+0000 7f2c4b9ce740 10 client.15240 fetch_fsmap learned FSMap version 23 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.688+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/2108572615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556fc8d5e3e0 con 0x556fc8d498b0 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.688+0000 7f2c27400640 1 -- 172.21.15.60:0/2108572615 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c3c004d20 con 0x556fc8d498b0 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.689+0000 7f2c27400640 1 -- 172.21.15.60:0/2108572615 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c3c025070 con 0x556fc8d498b0 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.689+0000 7f2c4b9ce740 10 client.15240 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:45:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.689+0000 7f2c4b9ce740 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.689+0000 7f2c4b9ce740 20 client.15240 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.689+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x556fc8d5f7e0 0x556fc8d648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.689+0000 7f2c4b9ce740 4 client.15240 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.689+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/2108572615 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fc8cbbb10 con 0x556fc8d5f7e0 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.689+0000 7f2c3be00640 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x556fc8d5f7e0 0x556fc8d648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.690+0000 7f2c3be00640 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x556fc8d5f7e0 0x556fc8d648f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=321ca4a2a487752c server_cookie=319a9b94744eac9b in_seq=0 out_seq=0 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.690+0000 7f2c27400640 10 client.15240 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c27400640 1 -- 172.21.15.60:0/2108572615 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556fc8cbbb10 con 0x556fc8d5f7e0 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c27400640 10 client.15240 handle_command_reply: tid=0 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 client.15240 shutdown 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/2108572615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d498b0 msgr2=0x556fc8d53af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d498b0 0x556fc8d53af0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c010fe0 tx=0x7f2c3c009bb0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/2108572615 shutdown_connections 2024-02-26T10:45:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d4f410 0x556fc8d5b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x556fc8d5f7e0 0x556fc8d648f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 0x556fc8d54030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/2108572615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d498b0 0x556fc8d53af0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c27400640 0 client.15240 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:45:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/2108572615 >> 172.21.15.60:0/2108572615 conn(0x556fc8d44e20 msgr2=0x556fc8d44020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/2108572615 shutdown_connections 2024-02-26T10:45:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/2108572615 wait complete. 2024-02-26T10:45:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.691+0000 7f2c4b9ce740 20 client.15240 trim_cache size 0 max 16384 2024-02-26T10:45:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c4b9ce740 1 Processor -- start 2024-02-26T10:45:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c4b9ce740 1 -- start start 2024-02-26T10:45:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d4eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 0x556fc8d5b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4f410 0x556fc8d68910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8d5fc90 con 0x556fc8d4f410 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8d602f0 con 0x556fc8d498b0 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8d60950 con 0x556fc8d4a2a0 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c42800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4f410 0x556fc8d68910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c42800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4f410 0x556fc8d68910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51926/0 (socket says 172.21.15.60:51926) 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c42800640 1 -- 172.21.15.60:0/3377230570 learned_addr learned my addr 172.21.15.60:0/3377230570 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c3be00640 1 --2- 172.21.15.60:0/3377230570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 0x556fc8d5b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.709+0000 7f2c42800640 1 --2- 172.21.15.60:0/3377230570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d4eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.710+0000 7f2c42800640 1 -- 172.21.15.60:0/3377230570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 msgr2=0x556fc8d5b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.710+0000 7f2c42800640 1 --2- 172.21.15.60:0/3377230570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 0x556fc8d5b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.710+0000 7f2c42800640 1 -- 172.21.15.60:0/3377230570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 msgr2=0x556fc8d4eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.710+0000 7f2c42800640 1 --2- 172.21.15.60:0/3377230570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d4eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.710+0000 7f2c42800640 1 -- 172.21.15.60:0/3377230570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fc8d54ae0 con 0x556fc8d4f410 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.710+0000 7f2c42800640 1 --2- 172.21.15.60:0/3377230570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4f410 0x556fc8d68910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009770 tx=0x7f2c3c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980c8232da402340 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.710+0000 7f2c32600640 1 -- 172.21.15.60:0/3377230570 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c016070 con 0x556fc8d4f410 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.710+0000 7f2c32600640 1 -- 172.21.15.60:0/3377230570 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c3c005bf0 con 0x556fc8d4f410 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.710+0000 7f2c32600640 1 -- 172.21.15.60:0/3377230570 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c012750 con 0x556fc8d4f410 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/3377230570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4f410 msgr2=0x556fc8d68910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/3377230570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4f410 0x556fc8d68910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009770 tx=0x7f2c3c010830 comp rx=0 tx=0).stop 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/3377230570 shutdown_connections 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/3377230570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4f410 0x556fc8d68910 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/3377230570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4a2a0 0x556fc8d5b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/3377230570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d4eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/3377230570 >> 172.21.15.60:0/3377230570 conn(0x556fc8d44e20 msgr2=0x556fc8d45200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/3377230570 shutdown_connections 2024-02-26T10:45:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/3377230570 wait complete. 2024-02-26T10:45:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.711+0000 7f2c4b9ce740 1 -- start start 2024-02-26T10:45:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c4b9ce740 1 Processor -- start 2024-02-26T10:45:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d57160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c4b9ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 0x556fc8d5a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8d61610 con 0x556fc8d4a2a0 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8d60fb0 con 0x556fc8d498b0 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c4b9ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc8d61c70 con 0x556fc8d4f410 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c4b9ce740 10 client.0 _ll_register_callbacks cb 0x7f2c3c009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c3be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c3be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51940/0 (socket says 172.21.15.60:51940) 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c3be00640 1 -- 172.21.15.60:0/4089886592 learned_addr learned my addr 172.21.15.60:0/4089886592 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.712+0000 7f2c42800640 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 0x556fc8d5a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c42800640 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d57160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c3be00640 1 -- 172.21.15.60:0/4089886592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 msgr2=0x556fc8d5a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c3be00640 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 0x556fc8d5a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c3be00640 1 -- 172.21.15.60:0/4089886592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 msgr2=0x556fc8d57160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c3be00640 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d57160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c3be00640 1 -- 172.21.15.60:0/4089886592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fc8d5ac40 con 0x556fc8d4a2a0 2024-02-26T10:45:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c42800640 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d57160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:45:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c3be00640 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d576a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009fd0 tx=0x7f2c2c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5cdb6b151fd166 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c27400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:45:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c27400640 1 -- 172.21.15.60:0/4089886592 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c004460 con 0x556fc8d4a2a0 2024-02-26T10:45:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/4089886592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c3c005bf0 con 0x556fc8d4a2a0 2024-02-26T10:45:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.713+0000 7f2c27400640 1 -- 172.21.15.60:0/4089886592 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2c2c01f400 con 0x556fc8d4a2a0 2024-02-26T10:45:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c27400640 1 -- 172.21.15.60:0/4089886592 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2c01f600 con 0x556fc8d4a2a0 2024-02-26T10:45:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c4b9ce740 10 client.15246 fetch_fsmap learned FSMap version 23 2024-02-26T10:45:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/4089886592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556fc8db7060 con 0x556fc8d4a2a0 2024-02-26T10:45:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c27400640 1 -- 172.21.15.60:0/4089886592 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c3c005bf0 con 0x556fc8d4a2a0 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c27400640 1 -- 172.21.15.60:0/4089886592 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c2c003680 con 0x556fc8d4a2a0 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c4b9ce740 10 client.15246 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c4b9ce740 10 client.15246 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c4b9ce740 20 client.15246 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x556fc8dbb740 0x556fc8db73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c4b9ce740 4 client.15246 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.714+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/4089886592 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556fc8cb5e60 con 0x556fc8dbb740 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.715+0000 7f2c42800640 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x556fc8dbb740 0x556fc8db73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.715+0000 7f2c42800640 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x556fc8dbb740 0x556fc8db73a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41f00f0ee71a0fec server_cookie=d9eff6694a78a709 in_seq=0 out_seq=0 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.715+0000 7f2c27400640 10 client.15246 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.715+0000 7f2c27400640 1 -- 172.21.15.60:0/4089886592 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556fc8cb5e60 con 0x556fc8dbb740 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.715+0000 7f2c27400640 10 client.15246 handle_command_reply: tid=0 2024-02-26T10:45:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.715+0000 7f2c4b9ce740 1 client.15246 shutdown 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/4089886592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 msgr2=0x556fc8d576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d576a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009fd0 tx=0x7f2c2c012bc0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/4089886592 shutdown_connections 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x556fc8dbb740 0x556fc8db73a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556fc8d4f410 0x556fc8d5a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c27400640 0 client.15246 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc8d4a2a0 0x556fc8d576a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 --2- 172.21.15.60:0/4089886592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc8d498b0 0x556fc8d57160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/4089886592 >> 172.21.15.60:0/4089886592 conn(0x556fc8d44e20 msgr2=0x556fc8d44040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/4089886592 shutdown_connections 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 1 -- 172.21.15.60:0/4089886592 wait complete. 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c4b9ce740 20 client.15246 trim_cache size 0 max 16384 2024-02-26T10:45:31.717 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 1, 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24553, 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 137.38645962999999, 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 23, 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 95, 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 91, 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 186.68389951200001 2024-02-26T10:45:31.718 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:45:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.716+0000 7f2c43200640 1 -- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c280a1d80 msgr2=0x7f2c280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c280a1d80 0x7f2c280a4240 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c340f3db0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 -- 172.21.15.60:0/2457198228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c44073c40 msgr2=0x7f2c44199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c44073c40 0x7f2c44199620 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005ab0 tx=0x7f2c3402c750 comp rx=0 tx=0).stop 2024-02-26T10:45:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 -- 172.21.15.60:0/2457198228 shutdown_connections 2024-02-26T10:45:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c280a1d80 0x7f2c280a4240 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c44108ff0 0x7f2c441a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c44075220 0x7f2c44199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 --2- 172.21.15.60:0/2457198228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c44073c40 0x7f2c44199620 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 -- 172.21.15.60:0/2457198228 >> 172.21.15.60:0/2457198228 conn(0x7f2c440fc6d0 msgr2=0x7f2c44104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 -- 172.21.15.60:0/2457198228 shutdown_connections 2024-02-26T10:45:31.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.717+0000 7f2c43200640 1 -- 172.21.15.60:0/2457198228 wait complete. 2024-02-26T10:45:31.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:31 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3555262199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:31.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:31 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3844587382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:31.730 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T10:45:31.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:31 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3555262199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:31 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3844587382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:31.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:31 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3555262199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:31.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:31 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3844587382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.944+0000 7f1fdc800640 1 -- 172.21.15.60:0/2704141836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4076e60 msgr2=0x7f1fd40772c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.944+0000 7f1fdc800640 1 --2- 172.21.15.60:0/2704141836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4076e60 0x7f1fd40772c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009900 tx=0x7f1fc402bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.944+0000 7f1fdc800640 1 -- 172.21.15.60:0/2704141836 shutdown_connections 2024-02-26T10:45:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.944+0000 7f1fdc800640 1 --2- 172.21.15.60:0/2704141836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd4077800 0x7f1fd40780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.944+0000 7f1fdc800640 1 --2- 172.21.15.60:0/2704141836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4076e60 0x7f1fd40772c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.944+0000 7f1fdc800640 1 --2- 172.21.15.60:0/2704141836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd4075c20 0x7f1fd4076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.944+0000 7f1fdc800640 1 -- 172.21.15.60:0/2704141836 >> 172.21.15.60:0/2704141836 conn(0x7f1fd40fea10 msgr2=0x7f1fd4100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.944+0000 7f1fdc800640 1 -- 172.21.15.60:0/2704141836 shutdown_connections 2024-02-26T10:45:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.944+0000 7f1fdc800640 1 -- 172.21.15.60:0/2704141836 wait complete. 2024-02-26T10:45:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.945+0000 7f1fdc800640 1 Processor -- start 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fdc800640 1 -- start start 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fdc800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4075c20 0x7f1fd419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fdc800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd4076e60 0x7f1fd419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fdc800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd4077800 0x7f1fd41a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fdc800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd41134f0 con 0x7f1fd4075c20 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fdc800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4113370 con 0x7f1fd4076e60 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fdc800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4113670 con 0x7f1fd4077800 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fd9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4075c20 0x7f1fd419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fd9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4075c20 0x7f1fd419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51964/0 (socket says 172.21.15.60:51964) 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fd9e00640 1 -- 172.21.15.60:0/307485986 learned_addr learned my addr 172.21.15.60:0/307485986 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fd9400640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd4076e60 0x7f1fd419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.946+0000 7f1fda800640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd4077800 0x7f1fd41a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fd9e00640 1 -- 172.21.15.60:0/307485986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd4077800 msgr2=0x7f1fd41a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fd9e00640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd4077800 0x7f1fd41a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fd9e00640 1 -- 172.21.15.60:0/307485986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd4076e60 msgr2=0x7f1fd419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fd9e00640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd4076e60 0x7f1fd419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fd9e00640 1 -- 172.21.15.60:0/307485986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd41a57e0 con 0x7f1fd4075c20 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fd9e00640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4075c20 0x7f1fd419dbc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00ebc0 tx=0x7f1fcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fcaa00640 1 -- 172.21.15.60:0/307485986 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc010430 con 0x7f1fd4075c20 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fcaa00640 1 -- 172.21.15.60:0/307485986 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fcc010ab0 con 0x7f1fd4075c20 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fdc800640 1 -- 172.21.15.60:0/307485986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd41a5ad0 con 0x7f1fd4075c20 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.947+0000 7f1fcaa00640 1 -- 172.21.15.60:0/307485986 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc018b30 con 0x7f1fd4075c20 2024-02-26T10:45:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.948+0000 7f1fdc800640 1 -- 172.21.15.60:0/307485986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd41a5ff0 con 0x7f1fd4075c20 2024-02-26T10:45:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.951+0000 7f1fcaa00640 1 -- 172.21.15.60:0/307485986 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1fcc018cd0 con 0x7f1fd4075c20 2024-02-26T10:45:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.951+0000 7f1fdf663740 1 Processor -- start 2024-02-26T10:45:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.951+0000 7f1fdf663740 1 -- start start 2024-02-26T10:45:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e496d0 0x561496e49af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e4a0c0 0x561496e4ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fdf663740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496dae5d0 con 0x561496e496d0 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fdf663740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496d57900 con 0x561496e4a0c0 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fdf663740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496daa670 con 0x561496e4f230 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fda800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e496d0 0x561496e49af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fda800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e496d0 0x561496e49af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51970/0 (socket says 172.21.15.60:51970) 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fda800640 1 -- 172.21.15.60:0/251079693 learned_addr learned my addr 172.21.15.60:0/251079693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fd9e00640 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e4a0c0 0x561496e4ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fd9400640 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fda800640 1 -- 172.21.15.60:0/251079693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 msgr2=0x561496e516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fda800640 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fda800640 1 -- 172.21.15.60:0/251079693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e4a0c0 msgr2=0x561496e4ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fda800640 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e4a0c0 0x561496e4ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fda800640 1 -- 172.21.15.60:0/251079693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561496ac32d0 con 0x561496e496d0 2024-02-26T10:45:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.952+0000 7f1fd9e00640 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e4a0c0 0x561496e4ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:45:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.953+0000 7f1fcaa00640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1fc00a1d80 0x7f1fc00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.953+0000 7f1fda800640 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e496d0 0x561496e49af0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009c90 tx=0x7f1fd0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f017a6cb814928f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.953+0000 7f1fcaa00640 1 -- 172.21.15.60:0/307485986 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f1fcc0ce880 con 0x7f1fd4075c20 2024-02-26T10:45:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.953+0000 7f1fd9e00640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1fc00a1d80 0x7f1fc00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.953+0000 7f1fbf000640 1 -- 172.21.15.60:0/251079693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0004230 con 0x561496e496d0 2024-02-26T10:45:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.953+0000 7f1fbf000640 1 -- 172.21.15.60:0/251079693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fd0009e40 con 0x561496e496d0 2024-02-26T10:45:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.953+0000 7f1fbf000640 1 -- 172.21.15.60:0/251079693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00197e0 con 0x561496e496d0 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fdf663740 1 -- 172.21.15.60:0/251079693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e496d0 msgr2=0x561496e49af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fd9e00640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1fc00a1d80 0x7f1fc00a4240 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc09aea0 tx=0x7f1fcc009a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fdf663740 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e496d0 0x561496e49af0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009c90 tx=0x7f1fd0011810 comp rx=0 tx=0).stop 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fdf663740 1 -- 172.21.15.60:0/251079693 shutdown_connections 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fdf663740 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e4a0c0 0x561496e4ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fdf663740 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fdf663740 1 --2- 172.21.15.60:0/251079693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e496d0 0x561496e49af0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fdf663740 1 -- 172.21.15.60:0/251079693 >> 172.21.15.60:0/251079693 conn(0x561496e44c40 msgr2=0x561496e45040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fdf663740 1 -- 172.21.15.60:0/251079693 shutdown_connections 2024-02-26T10:45:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.954+0000 7f1fdf663740 1 -- 172.21.15.60:0/251079693 wait complete. 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.955+0000 7f1fdf663740 1 -- start start 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.955+0000 7f1fdf663740 1 Processor -- start 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.955+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e53890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.955+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4a0c0 0x561496e53dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.955+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4f230 0x561496e5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.955+0000 7f1fdf663740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496e521e0 con 0x561496e4f230 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.955+0000 7f1fdf663740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496d6ad60 con 0x561496e496d0 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.955+0000 7f1fdf663740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496e52360 con 0x561496e4a0c0 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.955+0000 7f1fdf663740 10 client.0 _ll_register_callbacks cb 0x561496da6320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fd9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4a0c0 0x561496e53dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fda800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e53890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fd9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4a0c0 0x561496e53dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:39704/0 (socket says 172.21.15.60:39704) 2024-02-26T10:45:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fd9400640 1 -- 172.21.15.60:0/3227723425 learned_addr learned my addr 172.21.15.60:0/3227723425 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fda800640 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4f230 0x561496e5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fd9400640 1 -- 172.21.15.60:0/3227723425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 msgr2=0x561496e53890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fd9400640 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e53890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fd9400640 1 -- 172.21.15.60:0/3227723425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4f230 msgr2=0x561496e5ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fd9400640 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4f230 0x561496e5ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fd9400640 1 -- 172.21.15.60:0/3227723425 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561496b8d6b0 con 0x561496e4a0c0 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fda800640 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4f230 0x561496e5ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.956+0000 7f1fd9400640 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4a0c0 0x561496e53dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009900 tx=0x7f1fc4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a3785d03d882561 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.957+0000 7f1fb3400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.957+0000 7f1fb3400640 1 -- 172.21.15.60:0/3227723425 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4005d50 con 0x561496e4a0c0 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.957+0000 7f1fdf663740 1 -- 172.21.15.60:0/3227723425 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561496dbbb10 con 0x561496e4a0c0 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.957+0000 7f1fb3400640 1 -- 172.21.15.60:0/3227723425 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f1fc4005230 con 0x561496e4a0c0 2024-02-26T10:45:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.957+0000 7f1fb3400640 1 -- 172.21.15.60:0/3227723425 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fc4005430 con 0x561496e4a0c0 2024-02-26T10:45:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.957+0000 7f1fdf663740 10 client.25372 fetch_fsmap learned FSMap version 23 2024-02-26T10:45:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.957+0000 7f1fdf663740 1 -- 172.21.15.60:0/3227723425 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561496e5e240 con 0x561496e4a0c0 2024-02-26T10:45:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.958+0000 7f1fb3400640 1 -- 172.21.15.60:0/3227723425 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561496dbbb10 con 0x561496e4a0c0 2024-02-26T10:45:31.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.958+0000 7f1fb3400640 1 -- 172.21.15.60:0/3227723425 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1fc4003a10 con 0x561496e4a0c0 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.958+0000 7f1fdf663740 10 client.25372 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.958+0000 7f1fdf663740 10 client.25372 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.958+0000 7f1fdf663740 20 client.25372 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.958+0000 7f1fdf663740 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x561496e60670 0x561496e5f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.958+0000 7f1fdf663740 4 client.25372 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.958+0000 7f1fdf663740 1 -- 172.21.15.60:0/3227723425 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561496db5e60 con 0x561496e60670 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.959+0000 7f1fda800640 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x561496e60670 0x561496e5f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.959+0000 7f1fda800640 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x561496e60670 0x561496e5f700 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d195eeb051cb2d server_cookie=bc4469a46dd8855b in_seq=0 out_seq=0 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.959+0000 7f1fb3400640 10 client.25372 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fb3400640 1 -- 172.21.15.60:0/3227723425 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561496db5e60 con 0x561496e60670 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fb3400640 10 client.25372 handle_command_reply: tid=0 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 client.25372 shutdown 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 -- 172.21.15.60:0/3227723425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4a0c0 msgr2=0x561496e53dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4a0c0 0x561496e53dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009900 tx=0x7f1fc4004270 comp rx=0 tx=0).stop 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 -- 172.21.15.60:0/3227723425 shutdown_connections 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x561496e60670 0x561496e5f700 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4f230 0x561496e5ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fb3400640 0 client.25372 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4a0c0 0x561496e53dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 --2- 172.21.15.60:0/3227723425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e53890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 -- 172.21.15.60:0/3227723425 >> 172.21.15.60:0/3227723425 conn(0x561496e44c40 msgr2=0x561496e43e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.960+0000 7f1fdf663740 1 -- 172.21.15.60:0/3227723425 shutdown_connections 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.961+0000 7f1fdf663740 1 -- 172.21.15.60:0/3227723425 wait complete. 2024-02-26T10:45:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.961+0000 7f1fdf663740 20 client.25372 trim_cache size 0 max 16384 2024-02-26T10:45:31.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.978+0000 7f1fdf663740 1 Processor -- start 2024-02-26T10:45:31.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.978+0000 7f1fdf663740 1 -- start start 2024-02-26T10:45:31.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.978+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e45020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.978+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e5add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.978+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e4e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.978+0000 7f1fdf663740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496e5fe90 con 0x561496e4a0c0 2024-02-26T10:45:31.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.978+0000 7f1fdf663740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496e604f0 con 0x561496e496d0 2024-02-26T10:45:31.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.978+0000 7f1fdf663740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496e60b50 con 0x561496e4f230 2024-02-26T10:45:31.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fda800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e45020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fda800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e45020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51740/0 (socket says 172.21.15.60:51740) 2024-02-26T10:45:31.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fda800640 1 -- 172.21.15.60:0/936984276 learned_addr learned my addr 172.21.15.60:0/936984276 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fd9400640 1 --2- 172.21.15.60:0/936984276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e5add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fda800640 1 --2- 172.21.15.60:0/936984276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e4e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fda800640 1 -- 172.21.15.60:0/936984276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 msgr2=0x561496e45020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fda800640 1 --2- 172.21.15.60:0/936984276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e45020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fda800640 1 -- 172.21.15.60:0/936984276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 msgr2=0x561496e5add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fda800640 1 --2- 172.21.15.60:0/936984276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e5add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.979+0000 7f1fda800640 1 -- 172.21.15.60:0/936984276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561496e43290 con 0x561496e4f230 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fda800640 1 --2- 172.21.15.60:0/936984276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e4e840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0003ff0 tx=0x7f1fd0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19b9cbb482f29bb7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fbe600640 1 -- 172.21.15.60:0/936984276 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0031040 con 0x561496e4f230 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fbe600640 1 -- 172.21.15.60:0/936984276 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fd00055d0 con 0x561496e4f230 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fbe600640 1 -- 172.21.15.60:0/936984276 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd001a6c0 con 0x561496e4f230 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fdf663740 1 -- 172.21.15.60:0/936984276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 msgr2=0x561496e4e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fdf663740 1 --2- 172.21.15.60:0/936984276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e4e840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0003ff0 tx=0x7f1fd0000bc0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fdf663740 1 -- 172.21.15.60:0/936984276 shutdown_connections 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fdf663740 1 --2- 172.21.15.60:0/936984276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e4e840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fdf663740 1 --2- 172.21.15.60:0/936984276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e5add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fdf663740 1 --2- 172.21.15.60:0/936984276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e45020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.980+0000 7f1fdf663740 1 -- 172.21.15.60:0/936984276 >> 172.21.15.60:0/936984276 conn(0x561496e44c40 msgr2=0x561496e3f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.981+0000 7f1fdf663740 1 -- 172.21.15.60:0/936984276 shutdown_connections 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.981+0000 7f1fdf663740 1 -- 172.21.15.60:0/936984276 wait complete. 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.981+0000 7f1fdf663740 1 -- start start 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.981+0000 7f1fdf663740 1 Processor -- start 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e5ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e57700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fdf663740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e57c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fdf663740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496e61810 con 0x561496e4a0c0 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fdf663740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496e611b0 con 0x561496e496d0 2024-02-26T10:45:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fdf663740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561496e61e70 con 0x561496e4f230 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fdf663740 10 client.0 _ll_register_callbacks cb 0x7f1fd0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fd9400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e57700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fda800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e5ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fd9400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e57700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51994/0 (socket says 172.21.15.60:51994) 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fd9400640 1 -- 172.21.15.60:0/2582578274 learned_addr learned my addr 172.21.15.60:0/2582578274 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fda800640 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e57c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fd9400640 1 -- 172.21.15.60:0/2582578274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 msgr2=0x561496e57c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fd9400640 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e57c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fd9400640 1 -- 172.21.15.60:0/2582578274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 msgr2=0x561496e5ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fd9400640 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e5ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.982+0000 7f1fd9400640 1 -- 172.21.15.60:0/2582578274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561496e582b0 con 0x561496e4a0c0 2024-02-26T10:45:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.983+0000 7f1fd9400640 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e57700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009340 tx=0x7f1fc4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6843a69cc47d9d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.983+0000 7f1fb3400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:45:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.983+0000 7f1fb3400640 1 -- 172.21.15.60:0/2582578274 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4058060 con 0x561496e4a0c0 2024-02-26T10:45:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.983+0000 7f1fdf663740 1 -- 172.21.15.60:0/2582578274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fd00055d0 con 0x561496e4a0c0 2024-02-26T10:45:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.983+0000 7f1fb3400640 1 -- 172.21.15.60:0/2582578274 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f1fc4003a00 con 0x561496e4a0c0 2024-02-26T10:45:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.983+0000 7f1fb3400640 1 -- 172.21.15.60:0/2582578274 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fc405f050 con 0x561496e4a0c0 2024-02-26T10:45:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fdf663740 10 client.15264 fetch_fsmap learned FSMap version 23 2024-02-26T10:45:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fdf663740 1 -- 172.21.15.60:0/2582578274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561496eb73c0 con 0x561496e4a0c0 2024-02-26T10:45:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fb3400640 1 -- 172.21.15.60:0/2582578274 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fd00055d0 con 0x561496e4a0c0 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fb3400640 1 -- 172.21.15.60:0/2582578274 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1fc40048d0 con 0x561496e4a0c0 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fdf663740 10 client.15264 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fdf663740 10 client.15264 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fdf663740 20 client.15264 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fdf663740 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x561496ebb9d0 0x561496eb7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fdf663740 4 client.15264 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.984+0000 7f1fdf663740 1 -- 172.21.15.60:0/2582578274 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561496d34830 con 0x561496ebb9d0 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fda800640 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x561496ebb9d0 0x561496eb7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fda800640 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x561496ebb9d0 0x561496eb7650 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44f406e6ce495b5 server_cookie=d2afbd17549317e5 in_seq=0 out_seq=0 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fb3400640 10 client.15264 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fb3400640 1 -- 172.21.15.60:0/2582578274 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561496d34830 con 0x561496ebb9d0 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fb3400640 10 client.15264 handle_command_reply: tid=0 2024-02-26T10:45:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fdf663740 1 client.15264 shutdown 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fdf663740 1 -- 172.21.15.60:0/2582578274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 msgr2=0x561496e57700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fdf663740 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e57700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009340 tx=0x7f1fc4005800 comp rx=0 tx=0).stop 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fdf663740 1 -- 172.21.15.60:0/2582578274 shutdown_connections 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fdf663740 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x561496ebb9d0 0x561496eb7650 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fdf663740 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561496e4f230 0x561496e57c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fdf663740 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561496e4a0c0 0x561496e57700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fdf663740 1 --2- 172.21.15.60:0/2582578274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561496e496d0 0x561496e5ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.985+0000 7f1fb3400640 0 client.15264 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.986+0000 7f1fdf663740 1 -- 172.21.15.60:0/2582578274 >> 172.21.15.60:0/2582578274 conn(0x561496e44c40 msgr2=0x561496e45020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.986+0000 7f1fdf663740 1 -- 172.21.15.60:0/2582578274 shutdown_connections 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.986+0000 7f1fdf663740 1 -- 172.21.15.60:0/2582578274 wait complete. 2024-02-26T10:45:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.986+0000 7f1fdf663740 20 client.15264 trim_cache size 0 max 16384 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "51464d50-d492-11ee-95c3-87774f69a715", 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24587, 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 168.921727433, 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 23, 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 95, 2024-02-26T10:45:31.987 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 91, 2024-02-26T10:45:31.988 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 175.86587524500001 2024-02-26T10:45:31.988 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:45:31.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.986+0000 7f1fdc800640 1 -- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1fc00a1d80 msgr2=0x7f1fc00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.986+0000 7f1fdc800640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1fc00a1d80 0x7f1fc00a4240 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc09aea0 tx=0x7f1fcc009a70 comp rx=0 tx=0).stop 2024-02-26T10:45:31.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.986+0000 7f1fdc800640 1 -- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4075c20 msgr2=0x7f1fd419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.986+0000 7f1fdc800640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4075c20 0x7f1fd419dbc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00ebc0 tx=0x7f1fcc00c2d0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.987+0000 7f1fdc800640 1 -- 172.21.15.60:0/307485986 shutdown_connections 2024-02-26T10:45:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.987+0000 7f1fdc800640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1fc00a1d80 0x7f1fc00a4240 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.987+0000 7f1fdc800640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fd4076e60 0x7f1fd419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.987+0000 7f1fdc800640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fd4077800 0x7f1fd41a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.987+0000 7f1fdc800640 1 --2- 172.21.15.60:0/307485986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fd4075c20 0x7f1fd419dbc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.987+0000 7f1fdc800640 1 -- 172.21.15.60:0/307485986 >> 172.21.15.60:0/307485986 conn(0x7f1fd40fea10 msgr2=0x7f1fd4101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.987+0000 7f1fdc800640 1 -- 172.21.15.60:0/307485986 shutdown_connections 2024-02-26T10:45:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:31.987+0000 7f1fdc800640 1 -- 172.21.15.60:0/307485986 wait complete. 2024-02-26T10:45:32.000 INFO:teuthology.run_tasks:Running task kclient... 2024-02-26T10:45:32.014 INFO:tasks.kclient:Mounting kernel clients... 2024-02-26T10:45:32.014 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-26T10:45:32.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:32.014 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-02-26T10:45:32.072 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-02-26T10:45:32.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:32.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T10:45:32.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:32.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-02-26T10:45:32.306 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-02-26T10:45:32.306 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-26T10:45:32.306 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:45:32.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph fs ls 2024-02-26T10:45:32.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:32 smithi060 ceph-mon[31539]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:32.782 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:45:32.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:32 smithi113 ceph-mon[36126]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:32 smithi100 ceph-mon[36051]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:34.831 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:34 smithi060 ceph-mon[31539]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:34.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:34 smithi113 ceph-mon[36126]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:34.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:34 smithi100 ceph-mon[36051]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T10:45:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.008+0000 7f57a5200700 1 -- 172.21.15.60:0/2907949129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a0102000 msgr2=0x7f57a01023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.008+0000 7f57a5200700 1 --2- 172.21.15.60:0/2907949129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a0102000 0x7f57a01023e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5790009990 tx=0x7f5790009ca0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.008+0000 7f57a5200700 1 -- 172.21.15.60:0/2907949129 shutdown_connections 2024-02-26T10:45:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.008+0000 7f57a5200700 1 --2- 172.21.15.60:0/2907949129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a0107020 0x7f57a010ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.008+0000 7f57a5200700 1 --2- 172.21.15.60:0/2907949129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a0102000 0x7f57a01023e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.008+0000 7f57a5200700 1 --2- 172.21.15.60:0/2907949129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a01029b0 0x7f57a0106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.008+0000 7f57a5200700 1 -- 172.21.15.60:0/2907949129 >> 172.21.15.60:0/2907949129 conn(0x7f57a0078f10 msgr2=0x7f57a0079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.008+0000 7f57a5200700 1 -- 172.21.15.60:0/2907949129 shutdown_connections 2024-02-26T10:45:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.008+0000 7f57a5200700 1 -- 172.21.15.60:0/2907949129 wait complete. 2024-02-26T10:45:35.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.010+0000 7f57a5200700 1 Processor -- start 2024-02-26T10:45:35.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.010+0000 7f57a5200700 1 -- start start 2024-02-26T10:45:35.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.010+0000 7f57a5200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a0102000 0x7f57a019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.010+0000 7f57a5200700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01029b0 0x7f57a019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.010+0000 7f57a5200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a0107020 0x7f57a019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.010+0000 7f57a5200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a0112000 con 0x7f57a0107020 2024-02-26T10:45:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.010+0000 7f57a5200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a0111e80 con 0x7f57a0102000 2024-02-26T10:45:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.010+0000 7f57a5200700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a0112180 con 0x7f57a01029b0 2024-02-26T10:45:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579e800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a0107020 0x7f57a019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579de00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a0102000 0x7f57a019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579e800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a0107020 0x7f57a019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58048/0 (socket says 172.21.15.60:58048) 2024-02-26T10:45:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579e800700 1 -- 172.21.15.60:0/589716297 learned_addr learned my addr 172.21.15.60:0/589716297 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f5795400700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01029b0 0x7f57a019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579e800700 1 -- 172.21.15.60:0/589716297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01029b0 msgr2=0x7f57a019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579e800700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01029b0 0x7f57a019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579e800700 1 -- 172.21.15.60:0/589716297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a0102000 msgr2=0x7f57a019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579e800700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a0102000 0x7f57a019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579e800700 1 -- 172.21.15.60:0/589716297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a019f6a0 con 0x7f57a0107020 2024-02-26T10:45:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579de00700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a0102000 0x7f57a019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.011+0000 7f579e800700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a0107020 0x7f57a019ef60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f579800ef30 tx=0x7f579800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:35.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.012+0000 7f5797400700 1 -- 172.21.15.60:0/589716297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980092e0 con 0x7f57a0107020 2024-02-26T10:45:35.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.012+0000 7f5797400700 1 -- 172.21.15.60:0/589716297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5798004bb0 con 0x7f57a0107020 2024-02-26T10:45:35.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.012+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a019f930 con 0x7f57a0107020 2024-02-26T10:45:35.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.012+0000 7f5797400700 1 -- 172.21.15.60:0/589716297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798004280 con 0x7f57a0107020 2024-02-26T10:45:35.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.012+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a01a7210 con 0x7f57a0107020 2024-02-26T10:45:35.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.013+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a0103420 con 0x7f57a0107020 2024-02-26T10:45:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.015+0000 7f5797400700 1 -- 172.21.15.60:0/589716297 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f579801b030 con 0x7f57a0107020 2024-02-26T10:45:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.016+0000 7f5797400700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57840a1e90 0x7f57840a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.017+0000 7f5797400700 1 -- 172.21.15.60:0/589716297 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f57980cdb40 con 0x7f57a0107020 2024-02-26T10:45:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.017+0000 7f579de00700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57840a1e90 0x7f57840a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.018+0000 7f579de00700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57840a1e90 0x7f57840a4350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f579000b2e0 tx=0x7f579001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.021+0000 7f5797400700 1 -- 172.21.15.60:0/589716297 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5798022020 con 0x7f57a0107020 2024-02-26T10:45:35.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.302+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57a01a7410 con 0x7f57a0107020 2024-02-26T10:45:35.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.302+0000 7f5797400700 1 -- 172.21.15.60:0/589716297 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5798010040 con 0x7f57a0107020 2024-02-26T10:45:35.299 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-26T10:45:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.305+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57840a1e90 msgr2=0x7f57840a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.305+0000 7f57a5200700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57840a1e90 0x7f57840a4350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f579000b2e0 tx=0x7f579001a040 comp rx=0 tx=0).stop 2024-02-26T10:45:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.305+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a0107020 msgr2=0x7f57a019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.305+0000 7f57a5200700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a0107020 0x7f57a019ef60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f579800ef30 tx=0x7f579800c3a0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.306+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 shutdown_connections 2024-02-26T10:45:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.306+0000 7f57a5200700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f57840a1e90 0x7f57840a4350 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.306+0000 7f57a5200700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a0107020 0x7f57a019ef60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.306+0000 7f57a5200700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a0102000 0x7f57a019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.306+0000 7f57a5200700 1 --2- 172.21.15.60:0/589716297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57a01029b0 0x7f57a019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.306+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 >> 172.21.15.60:0/589716297 conn(0x7f57a0078f10 msgr2=0x7f57a010dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.306+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 shutdown_connections 2024-02-26T10:45:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:35.306+0000 7f57a5200700 1 -- 172.21.15.60:0/589716297 wait complete. 2024-02-26T10:45:35.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:35 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/589716297' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T10:45:35.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:35 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/589716297' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T10:45:35.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:35 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/589716297' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T10:45:36.250 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-26T10:45:36.250 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:45:36.250 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-02-26T10:45:36.250 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-26T10:45:36.250 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:45:36.250 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-26T10:45:36.250 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-26T10:45:36.251 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:45:36.251 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:45:36.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:36.251 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:45:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41806sec preferred_lft 41806sec 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:45:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-02-26T10:45:36.312 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-26T10:45:36.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:36.312 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:45:36.312 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:45:36.312 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-02-26T10:45:36.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-02-26T10:45:36.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-02-26T10:45:36.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-26T10:45:36.313 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:45:36.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:36.560 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:45:36.643 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-02-26T10:45:36.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:36.648 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-02-26T10:45:36.703 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-02-26T10:45:36.703 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:45:36.703 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-26T10:45:36.703 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-26T10:45:36.703 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:45:36.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:36.705 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:45:36.705 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:45:36.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-26T10:45:36.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-26T10:45:36.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-26T10:45:36.705 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:45:36.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:36 smithi060 ceph-mon[31539]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:36.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:36 smithi113 ceph-mon[36126]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:36.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:36 smithi100 ceph-mon[36051]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:36.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:36.900 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-02-26T10:45:36.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:36.953 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-02-26T10:45:37.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:37.007 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:45:37.007 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:45:37.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:45:37.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-26T10:45:37.008 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:45:37.156 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-26T10:45:37.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:37.156 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:45:37.156 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:45:37.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-26T10:45:37.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-26T10:45:37.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-26T10:45:37.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-26T10:45:37.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-26T10:45:37.157 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:45:37.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:37.419 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T10:45:37.419 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T10:45:37.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-02-26T10:45:37.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-02-26T10:45:37.419 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T10:45:37.589 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:45:37.589 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T10:45:37.641 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:45:37.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:45:37.641 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:45:37.690 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-26T10:45:37.690 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-26T10:45:37.690 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72 2024-02-26T10:45:37.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:37.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-02-26T10:45:37.827 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-02-26T10:45:37.827 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-02-26T10:45:37.827 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-02-26T10:45:38.451 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-02-26T10:45:38.451 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-02-26T10:45:38.452 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-02-26T10:45:38.452 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.113:3300" 2024-02-26T10:45:38.452 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:3300,172.21.15.100:3300,172.21.15.113:3300:/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72 2024-02-26T10:45:38.452 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=51464d50-d492-11ee-95c3-87774f69a715" will pass to kernel 2024-02-26T10:45:38.456 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-26T10:45:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:45:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:45:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:45:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:45:38.462 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:45:38.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:38.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:45:38.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:38.550 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:45:38.601 INFO:teuthology.orchestra.run.smithi060.stdout:client25387 2024-02-26T10:45:38.601 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:45:38.601 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:45:38.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:38.645 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:45:38.742 INFO:teuthology.orchestra.run.smithi060.stdout:client25387 2024-02-26T10:45:38.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:38.743 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:45:38.798 INFO:teuthology.orchestra.run.smithi060.stdout:client25387 2024-02-26T10:45:38.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:38.798 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/51464d50-d492-11ee-95c3-87774f69a715.client25387/status') 2024-02-26T10:45:38.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:38 smithi113 ceph-mon[36126]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:38.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:45:38.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:38 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:38 smithi060 ceph-mon[31539]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:38.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:45:38.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:38 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:38 smithi100 ceph-mon[36051]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T10:45:38.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:45:38.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:38 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:38.883 INFO:teuthology.orchestra.run.smithi060.stdout:instance: client.25387 (3)172.21.15.60:0/785918119 2024-02-26T10:45:38.883 INFO:teuthology.orchestra.run.smithi060.stdout:blocklisted: false 2024-02-26T10:45:38.883 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records in 2024-02-26T10:45:38.883 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records out 2024-02-26T10:45:38.883 INFO:teuthology.orchestra.run.smithi060.stderr:70 bytes copied, 6.087e-05 s, 1.1 MB/s 2024-02-26T10:45:38.886 INFO:teuthology.run_tasks:Running task exec... 2024-02-26T10:45:38.897 INFO:teuthology.task.exec:Executing custom commands... 2024-02-26T10:45:38.898 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T10:45:38.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-02-26T10:45:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.094+0000 7fe781c00640 1 -- 172.21.15.60:0/3635734749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1034b0 msgr2=0x7fe77c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.094+0000 7fe781c00640 1 --2- 172.21.15.60:0/3635734749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1034b0 0x7fe77c1038d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009900 tx=0x7fe76c02bc40 comp rx=0 tx=0).stop 2024-02-26T10:45:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.094+0000 7fe781c00640 1 -- 172.21.15.60:0/3635734749 shutdown_connections 2024-02-26T10:45:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.094+0000 7fe781c00640 1 --2- 172.21.15.60:0/3635734749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77c105060 0x7fe77c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.094+0000 7fe781c00640 1 --2- 172.21.15.60:0/3635734749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe77c1046a0 0x7fe77c104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.094+0000 7fe781c00640 1 --2- 172.21.15.60:0/3635734749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1034b0 0x7fe77c1038d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.094+0000 7fe781c00640 1 -- 172.21.15.60:0/3635734749 >> 172.21.15.60:0/3635734749 conn(0x7fe77c0fea20 msgr2=0x7fe77c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.094+0000 7fe781c00640 1 -- 172.21.15.60:0/3635734749 shutdown_connections 2024-02-26T10:45:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.095+0000 7fe781c00640 1 -- 172.21.15.60:0/3635734749 wait complete. 2024-02-26T10:45:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.095+0000 7fe781c00640 1 Processor -- start 2024-02-26T10:45:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe781c00640 1 -- start start 2024-02-26T10:45:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe781c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe77c1034b0 0x7fe77c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe781c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1046a0 0x7fe77c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe781c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77c105060 0x7fe77c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe781c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c10f190 con 0x7fe77c105060 2024-02-26T10:45:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe781c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c10f010 con 0x7fe77c1046a0 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe781c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c10f310 con 0x7fe77c1034b0 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe77a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1046a0 0x7fe77c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe77c1034b0 0x7fe77c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe77a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1046a0 0x7fe77c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56882/0 (socket says 172.21.15.60:56882) 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe77c1034b0 0x7fe77c1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:47332/0 (socket says 172.21.15.60:47332) 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe77a000640 1 -- 172.21.15.60:0/835329667 learned_addr learned my addr 172.21.15.60:0/835329667 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.096+0000 7fe77b400640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77c105060 0x7fe77c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.097+0000 7fe77a000640 1 -- 172.21.15.60:0/835329667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe77c1034b0 msgr2=0x7fe77c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.097+0000 7fe77a000640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe77c1034b0 0x7fe77c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.097+0000 7fe77a000640 1 -- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77c105060 msgr2=0x7fe77c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.097+0000 7fe77a000640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77c105060 0x7fe77c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.097+0000 7fe77a000640 1 -- 172.21.15.60:0/835329667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c1a12c0 con 0x7fe77c1046a0 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.097+0000 7fe77aa00640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe77c1034b0 0x7fe77c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:39.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.097+0000 7fe77a000640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1046a0 0x7fe77c199be0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe76800d950 tx=0x7fe76800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:39.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.098+0000 7fe773400640 1 -- 172.21.15.60:0/835329667 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768017070 con 0x7fe77c1046a0 2024-02-26T10:45:39.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.098+0000 7fe773400640 1 -- 172.21.15.60:0/835329667 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe768009c30 con 0x7fe77c1046a0 2024-02-26T10:45:39.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.098+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c1a15b0 con 0x7fe77c1046a0 2024-02-26T10:45:39.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.098+0000 7fe773400640 1 -- 172.21.15.60:0/835329667 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800f6b0 con 0x7fe77c1046a0 2024-02-26T10:45:39.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.098+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77c1a1b50 con 0x7fe77c1046a0 2024-02-26T10:45:39.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.099+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77c10a700 con 0x7fe77c1046a0 2024-02-26T10:45:39.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.101+0000 7fe773400640 1 -- 172.21.15.60:0/835329667 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe7680109d0 con 0x7fe77c1046a0 2024-02-26T10:45:39.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.102+0000 7fe773400640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7600a1ea0 0x7fe7600a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:39.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.102+0000 7fe773400640 1 -- 172.21.15.60:0/835329667 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe7680cdce0 con 0x7fe77c1046a0 2024-02-26T10:45:39.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.102+0000 7fe77aa00640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7600a1ea0 0x7fe7600a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:39.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.103+0000 7fe77aa00640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7600a1ea0 0x7fe7600a4360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe76c02c180 tx=0x7fe76c02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:39.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.106+0000 7fe773400640 1 -- 172.21.15.60:0/835329667 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe768009de0 con 0x7fe77c1046a0 2024-02-26T10:45:39.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.352+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7fe77c061e60 con 0x7fe77c1046a0 2024-02-26T10:45:39.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.839+0000 7fe773400640 1 -- 172.21.15.60:0/835329667 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe76809a6b0 con 0x7fe77c1046a0 2024-02-26T10:45:39.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7600a1ea0 msgr2=0x7fe7600a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:39.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7600a1ea0 0x7fe7600a4360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe76c02c180 tx=0x7fe76c02cf90 comp rx=0 tx=0).stop 2024-02-26T10:45:39.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1046a0 msgr2=0x7fe77c199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1046a0 0x7fe77c199be0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe76800d950 tx=0x7fe76800de10 comp rx=0 tx=0).stop 2024-02-26T10:45:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 shutdown_connections 2024-02-26T10:45:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe7600a1ea0 0x7fe7600a4360 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c1046a0 0x7fe77c199be0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77c105060 0x7fe77c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 --2- 172.21.15.60:0/835329667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe77c1034b0 0x7fe77c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 >> 172.21.15.60:0/835329667 conn(0x7fe77c0fea20 msgr2=0x7fe77c1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 shutdown_connections 2024-02-26T10:45:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:39.843+0000 7fe781c00640 1 -- 172.21.15.60:0/835329667 wait complete. 2024-02-26T10:45:39.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-02-26T10:45:40.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.056+0000 7fc9cbe00640 1 -- 172.21.15.60:0/124385023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 msgr2=0x7fc9cc103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.056+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/124385023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 0x7fc9cc103700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009900 tx=0x7fc9bc02bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:40.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.057+0000 7fc9cbe00640 1 -- 172.21.15.60:0/124385023 shutdown_connections 2024-02-26T10:45:40.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.057+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/124385023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc104e90 0x7fc9cc10b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.057+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/124385023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9cc1044d0 0x7fc9cc104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.057+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/124385023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 0x7fc9cc103700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.057+0000 7fc9cbe00640 1 -- 172.21.15.60:0/124385023 >> 172.21.15.60:0/124385023 conn(0x7fc9cc0fe870 msgr2=0x7fc9cc100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:40.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.057+0000 7fc9cbe00640 1 -- 172.21.15.60:0/124385023 shutdown_connections 2024-02-26T10:45:40.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.057+0000 7fc9cbe00640 1 -- 172.21.15.60:0/124385023 wait complete. 2024-02-26T10:45:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.058+0000 7fc9cbe00640 1 Processor -- start 2024-02-26T10:45:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.058+0000 7fc9cbe00640 1 -- start start 2024-02-26T10:45:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.058+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 0x7fc9cc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.058+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9cc1044d0 0x7fc9cc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.058+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc104e90 0x7fc9cc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.058+0000 7fc9cbe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc10ef90 con 0x7fc9cc1032e0 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.058+0000 7fc9cbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc10ee10 con 0x7fc9cc104e90 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.058+0000 7fc9cbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc10f110 con 0x7fc9cc1044d0 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9caa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 0x7fc9cc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9cc1044d0 0x7fc9cc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9cc1044d0 0x7fc9cc199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:47350/0 (socket says 172.21.15.60:47350) 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9caa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 0x7fc9cc199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58082/0 (socket says 172.21.15.60:58082) 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3e00640 1 -- 172.21.15.60:0/3320851313 learned_addr learned my addr 172.21.15.60:0/3320851313 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9cb400640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc104e90 0x7fc9cc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3e00640 1 -- 172.21.15.60:0/3320851313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc104e90 msgr2=0x7fc9cc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3e00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc104e90 0x7fc9cc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3e00640 1 -- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 msgr2=0x7fc9cc199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3e00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 0x7fc9cc199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3e00640 1 -- 172.21.15.60:0/3320851313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc1a1220 con 0x7fc9cc1044d0 2024-02-26T10:45:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9caa00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 0x7fc9cc199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:40.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3e00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9cc1044d0 0x7fc9cc199b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400e9f0 tx=0x7fc9b400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:40.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.059+0000 7fc9c3400640 1 -- 172.21.15.60:0/3320851313 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400ccc0 con 0x7fc9cc1044d0 2024-02-26T10:45:40.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.060+0000 7fc9c3400640 1 -- 172.21.15.60:0/3320851313 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9b4010910 con 0x7fc9cc1044d0 2024-02-26T10:45:40.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.060+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc1a1510 con 0x7fc9cc1044d0 2024-02-26T10:45:40.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.060+0000 7fc9c3400640 1 -- 172.21.15.60:0/3320851313 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40188a0 con 0x7fc9cc1044d0 2024-02-26T10:45:40.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.060+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc1a1a60 con 0x7fc9cc1044d0 2024-02-26T10:45:40.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.061+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc10a600 con 0x7fc9cc1044d0 2024-02-26T10:45:40.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.062+0000 7fc9c3400640 1 -- 172.21.15.60:0/3320851313 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc9b4010ac0 con 0x7fc9cc1044d0 2024-02-26T10:45:40.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.064+0000 7fc9c3400640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b00a1d70 0x7fc9b00a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:40.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.064+0000 7fc9c3400640 1 -- 172.21.15.60:0/3320851313 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fc9b40ce5a0 con 0x7fc9cc1044d0 2024-02-26T10:45:40.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.064+0000 7fc9caa00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b00a1d70 0x7fc9b00a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:40.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.065+0000 7fc9caa00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b00a1d70 0x7fc9b00a4230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc02c170 tx=0x7fc9bc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.068+0000 7fc9c3400640 1 -- 172.21.15.60:0/3320851313 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9b409b5c0 con 0x7fc9cc1044d0 2024-02-26T10:45:40.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:39 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:40.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:39 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:40.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:39 smithi113 ceph-mon[36126]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:40.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:39 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/835329667' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-26T10:45:40.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:39 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-26T10:45:40.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:39 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:40.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:39 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:40.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:39 smithi060 ceph-mon[31539]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:40.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:39 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/835329667' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-26T10:45:40.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:39 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-26T10:45:40.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:39 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:40.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:39 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:40.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:39 smithi100 ceph-mon[36051]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-02-26T10:45:40.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:39 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/835329667' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-26T10:45:40.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:39 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-26T10:45:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.323+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9cc061740 con 0x7fc9b00a1d70 2024-02-26T10:45:40.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.344+0000 7fc9c3400640 1 -- 172.21.15.60:0/3320851313 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7fc9cc061740 con 0x7fc9b00a1d70 2024-02-26T10:45:40.340 INFO:teuthology.orchestra.run.smithi060.stdout:{} 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b00a1d70 msgr2=0x7fc9b00a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b00a1d70 0x7fc9b00a4230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc02c170 tx=0x7fc9bc02df90 comp rx=0 tx=0).stop 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9cc1044d0 msgr2=0x7fc9cc199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9cc1044d0 0x7fc9cc199b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400e9f0 tx=0x7fc9b400edf0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 shutdown_connections 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc9b00a1d70 0x7fc9b00a4230 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9cc104e90 0x7fc9cc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9cc1044d0 0x7fc9cc199b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 --2- 172.21.15.60:0/3320851313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc1032e0 0x7fc9cc199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.346+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 >> 172.21.15.60:0/3320851313 conn(0x7fc9cc0fe870 msgr2=0x7fc9cc1077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.347+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 shutdown_connections 2024-02-26T10:45:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.347+0000 7fc9cbe00640 1 -- 172.21.15.60:0/3320851313 wait complete. 2024-02-26T10:45:40.358 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:45:40.330+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:45:40.359 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:45:40.330+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:45:40.359 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:45:40.330+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:45:40.359 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:45:40.330+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:45:40.359 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:45:40.330+0000 7f651d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T10:45:40.362 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-26T10:45:40.375 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-02-26T10:45:40.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:45:40.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.607+0000 7fcfc9e00640 1 -- 172.21.15.60:0/1511310569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfc4109b70 msgr2=0x7fcfc4109ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.607+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/1511310569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfc4109b70 0x7fcfc4109ff0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009900 tx=0x7fcfb002bc40 comp rx=0 tx=0).stop 2024-02-26T10:45:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.607+0000 7fcfc9e00640 1 -- 172.21.15.60:0/1511310569 shutdown_connections 2024-02-26T10:45:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.607+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/1511310569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfc4109b70 0x7fcfc4109ff0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.607+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/1511310569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfc410a530 0x7fcfc4110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.607+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/1511310569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc41089d0 0x7fcfc4108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.607+0000 7fcfc9e00640 1 -- 172.21.15.60:0/1511310569 >> 172.21.15.60:0/1511310569 conn(0x7fcfc406f120 msgr2=0x7fcfc4071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.607+0000 7fcfc9e00640 1 -- 172.21.15.60:0/1511310569 shutdown_connections 2024-02-26T10:45:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.607+0000 7fcfc9e00640 1 -- 172.21.15.60:0/1511310569 wait complete. 2024-02-26T10:45:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.609+0000 7fcfc9e00640 1 Processor -- start 2024-02-26T10:45:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.609+0000 7fcfc9e00640 1 -- start start 2024-02-26T10:45:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.609+0000 7fcfc9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfc41089d0 0x7fcfc419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.609+0000 7fcfc9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc4109b70 0x7fcfc41a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.609+0000 7fcfc9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfc410a530 0x7fcfc41a7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.609+0000 7fcfc9e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc4114630 con 0x7fcfc4109b70 2024-02-26T10:45:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.609+0000 7fcfc9e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc41144b0 con 0x7fcfc410a530 2024-02-26T10:45:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.609+0000 7fcfc9e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc41147b0 con 0x7fcfc41089d0 2024-02-26T10:45:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfc41089d0 0x7fcfc419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfc41089d0 0x7fcfc419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:47396/0 (socket says 172.21.15.60:47396) 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc4109b70 0x7fcfc41a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2a00640 1 -- 172.21.15.60:0/2704116197 learned_addr learned my addr 172.21.15.60:0/2704116197 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc3400640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfc410a530 0x7fcfc41a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2a00640 1 -- 172.21.15.60:0/2704116197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfc410a530 msgr2=0x7fcfc41a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2a00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfc410a530 0x7fcfc41a7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2a00640 1 -- 172.21.15.60:0/2704116197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc4109b70 msgr2=0x7fcfc41a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2a00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc4109b70 0x7fcfc41a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2a00640 1 -- 172.21.15.60:0/2704116197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc41a79f0 con 0x7fcfc41089d0 2024-02-26T10:45:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.610+0000 7fcfc2a00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfc41089d0 0x7fcfc419fdd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400e9c0 tx=0x7fcfb400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.611+0000 7fcfc8a00640 1 -- 172.21.15.60:0/2704116197 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb400ccc0 con 0x7fcfc41089d0 2024-02-26T10:45:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.611+0000 7fcfc8a00640 1 -- 172.21.15.60:0/2704116197 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfb40035f0 con 0x7fcfc41089d0 2024-02-26T10:45:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.611+0000 7fcfc8a00640 1 -- 172.21.15.60:0/2704116197 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb400ce60 con 0x7fcfc41089d0 2024-02-26T10:45:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.611+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc41a7ce0 con 0x7fcfc41089d0 2024-02-26T10:45:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.611+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc41a81a0 con 0x7fcfc41089d0 2024-02-26T10:45:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.613+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc41a8430 con 0x7fcfc41089d0 2024-02-26T10:45:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.614+0000 7fcfc8a00640 1 -- 172.21.15.60:0/2704116197 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fcfb40037a0 con 0x7fcfc41089d0 2024-02-26T10:45:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.616+0000 7fcfc8a00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcfa80a1e00 0x7fcfa80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.616+0000 7fcfc8a00640 1 -- 172.21.15.60:0/2704116197 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fcfb40ce420 con 0x7fcfc41089d0 2024-02-26T10:45:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.616+0000 7fcfc2000640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcfa80a1e00 0x7fcfa80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.617+0000 7fcfc2000640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcfa80a1e00 0x7fcfa80a42c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcfb002c250 tx=0x7fcfb002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:40.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.624+0000 7fcfc8a00640 1 -- 172.21.15.60:0/2704116197 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcfb409b4a0 con 0x7fcfc41089d0 2024-02-26T10:45:40.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.863+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfc41a86a0 con 0x7fcfc41089d0 2024-02-26T10:45:40.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.866+0000 7fcfc8a00640 1 -- 172.21.15.60:0/2704116197 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7fcfb409ada0 con 0x7fcfc41089d0 2024-02-26T10:45:40.861 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:40.861 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.867+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcfa80a1e00 msgr2=0x7fcfa80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.867+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcfa80a1e00 0x7fcfa80a42c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcfb002c250 tx=0x7fcfb002cf90 comp rx=0 tx=0).stop 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.867+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfc41089d0 msgr2=0x7fcfc419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.867+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfc41089d0 0x7fcfc419fdd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400e9c0 tx=0x7fcfb400edc0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.868+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 shutdown_connections 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.868+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcfa80a1e00 0x7fcfa80a42c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.868+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfc410a530 0x7fcfc41a7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.868+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc4109b70 0x7fcfc41a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.868+0000 7fcfc9e00640 1 --2- 172.21.15.60:0/2704116197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfc41089d0 0x7fcfc419fdd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.868+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 >> 172.21.15.60:0/2704116197 conn(0x7fcfc406f120 msgr2=0x7fcfc4070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.868+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 shutdown_connections 2024-02-26T10:45:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:40.868+0000 7fcfc9e00640 1 -- 172.21.15.60:0/2704116197 wait complete. 2024-02-26T10:45:40.883 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:45:40.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:45:41.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.091+0000 7f1671000640 1 -- 172.21.15.60:0/2979916047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 msgr2=0x7f166c102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.091+0000 7f1671000640 1 --2- 172.21.15.60:0/2979916047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 0x7f166c102980 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1658009930 tx=0x7f165802bc20 comp rx=0 tx=0).stop 2024-02-26T10:45:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.091+0000 7f1671000640 1 -- 172.21.15.60:0/2979916047 shutdown_connections 2024-02-26T10:45:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.091+0000 7f1671000640 1 --2- 172.21.15.60:0/2979916047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c102ec0 0x7f166c109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.091+0000 7f1671000640 1 --2- 172.21.15.60:0/2979916047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 0x7f166c102980 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.091+0000 7f1671000640 1 --2- 172.21.15.60:0/2979916047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c0ff530 0x7f166c0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.091+0000 7f1671000640 1 -- 172.21.15.60:0/2979916047 >> 172.21.15.60:0/2979916047 conn(0x7f166c0fae80 msgr2=0x7f166c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.091+0000 7f1671000640 1 -- 172.21.15.60:0/2979916047 shutdown_connections 2024-02-26T10:45:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.091+0000 7f1671000640 1 -- 172.21.15.60:0/2979916047 wait complete. 2024-02-26T10:45:41.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.093+0000 7f1671000640 1 Processor -- start 2024-02-26T10:45:41.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.093+0000 7f1671000640 1 -- start start 2024-02-26T10:45:41.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1671000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c0ff530 0x7f166c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1671000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 0x7f166c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1671000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c102ec0 0x7f166c1a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1671000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c10cf70 con 0x7f166c102ec0 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1671000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c10cdf0 con 0x7f166c0ff530 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1671000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c10d0f0 con 0x7f166c102500 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1669400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 0x7f166c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1669400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 0x7f166c19e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:47402/0 (socket says 172.21.15.60:47402) 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1669400640 1 -- 172.21.15.60:0/3689161350 learned_addr learned my addr 172.21.15.60:0/3689161350 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1669e00640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c0ff530 0x7f166c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f166a800640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c102ec0 0x7f166c1a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1669400640 1 -- 172.21.15.60:0/3689161350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c0ff530 msgr2=0x7f166c19dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1669400640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c0ff530 0x7f166c19dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1669400640 1 -- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c102ec0 msgr2=0x7f166c1a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1669400640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c102ec0 0x7f166c1a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.094+0000 7f1669400640 1 -- 172.21.15.60:0/3689161350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c1a5870 con 0x7f166c102500 2024-02-26T10:45:41.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.095+0000 7f1669400640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 0x7f166c19e1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1658000c00 tx=0x7f165802c630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:41.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.095+0000 7f1662a00640 1 -- 172.21.15.60:0/3689161350 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658043020 con 0x7f166c102500 2024-02-26T10:45:41.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.095+0000 7f1662a00640 1 -- 172.21.15.60:0/3689161350 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16580054e0 con 0x7f166c102500 2024-02-26T10:45:41.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.095+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c1a5b00 con 0x7f166c102500 2024-02-26T10:45:41.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.095+0000 7f1662a00640 1 -- 172.21.15.60:0/3689161350 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658003c10 con 0x7f166c102500 2024-02-26T10:45:41.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.095+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c1a6050 con 0x7f166c102500 2024-02-26T10:45:41.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.097+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166c1a6220 con 0x7f166c102500 2024-02-26T10:45:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.098+0000 7f1662a00640 1 -- 172.21.15.60:0/3689161350 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1658046020 con 0x7f166c102500 2024-02-26T10:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.101+0000 7f1662a00640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f16500a1d80 0x7f16500a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.101+0000 7f1662a00640 1 -- 172.21.15.60:0/3689161350 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f16580f0310 con 0x7f166c102500 2024-02-26T10:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.101+0000 7f1669e00640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f16500a1d80 0x7f16500a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.102+0000 7f1669e00640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f16500a1d80 0x7f16500a4240 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f166c19f130 tx=0x7f165c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:41.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.103+0000 7f1662a00640 1 -- 172.21.15.60:0/3689161350 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f165804d020 con 0x7f166c102500 2024-02-26T10:45:41.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-26T10:45:41.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:45:41.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:40 smithi060 ceph-mon[31539]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:41.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:40 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-26T10:45:41.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:40 smithi100 ceph-mon[36051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:45:41.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:40 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:40 smithi100 ceph-mon[36051]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:41.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.327+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f166c1a6490 con 0x7f166c102500 2024-02-26T10:45:41.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.327+0000 7f1662a00640 1 -- 172.21.15.60:0/3689161350 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16580bd330 con 0x7f166c102500 2024-02-26T10:45:41.323 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:45:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f16500a1d80 msgr2=0x7f16500a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f16500a1d80 0x7f16500a4240 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f166c19f130 tx=0x7f165c005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 msgr2=0x7f166c19e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 0x7f166c19e1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1658000c00 tx=0x7f165802c630 comp rx=0 tx=0).stop 2024-02-26T10:45:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 shutdown_connections 2024-02-26T10:45:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f16500a1d80 0x7f16500a4240 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c102ec0 0x7f166c1a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166c102500 0x7f166c19e1a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 --2- 172.21.15.60:0/3689161350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c0ff530 0x7f166c19dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 >> 172.21.15.60:0/3689161350 conn(0x7f166c0fae80 msgr2=0x7f166c1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 shutdown_connections 2024-02-26T10:45:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.330+0000 7f1671000640 1 -- 172.21.15.60:0/3689161350 wait complete. 2024-02-26T10:45:41.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:40 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-26T10:45:41.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:40 smithi113 ceph-mon[36126]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:45:41.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:40 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:41.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:40 smithi113 ceph-mon[36126]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T10:45:41.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:45:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.540+0000 7f47c2c00640 1 -- 172.21.15.60:0/2569346566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47bc1044f0 msgr2=0x7f47bc1068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.540+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2569346566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47bc1044f0 0x7f47bc1068e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009900 tx=0x7f47a802bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.541+0000 7f47c2c00640 1 -- 172.21.15.60:0/2569346566 shutdown_connections 2024-02-26T10:45:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.541+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2569346566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47bc106e20 0x7f47bc109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.541+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2569346566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47bc1044f0 0x7f47bc1068e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.541+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2569346566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47bc101bc0 0x7f47bc103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.541+0000 7f47c2c00640 1 -- 172.21.15.60:0/2569346566 >> 172.21.15.60:0/2569346566 conn(0x7f47bc0fb5d0 msgr2=0x7f47bc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:41.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.541+0000 7f47c2c00640 1 -- 172.21.15.60:0/2569346566 shutdown_connections 2024-02-26T10:45:41.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.541+0000 7f47c2c00640 1 -- 172.21.15.60:0/2569346566 wait complete. 2024-02-26T10:45:41.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.542+0000 7f47c2c00640 1 Processor -- start 2024-02-26T10:45:41.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.542+0000 7f47c2c00640 1 -- start start 2024-02-26T10:45:41.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.542+0000 7f47c2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47bc101bc0 0x7f47bc199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.542+0000 7f47c2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47bc1044f0 0x7f47bc199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.542+0000 7f47c2c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47bc106e20 0x7f47bc1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.542+0000 7f47c2c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc10ce20 con 0x7f47bc106e20 2024-02-26T10:45:41.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.542+0000 7f47c2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc10cca0 con 0x7f47bc1044f0 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.542+0000 7f47c2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc10cfa0 con 0x7f47bc101bc0 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c2200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47bc106e20 0x7f47bc1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47bc1044f0 0x7f47bc199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47bc1044f0 0x7f47bc199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56966/0 (socket says 172.21.15.60:56966) 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47bc101bc0 0x7f47bc199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c0e00640 1 -- 172.21.15.60:0/2742506241 learned_addr learned my addr 172.21.15.60:0/2742506241 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c0e00640 1 -- 172.21.15.60:0/2742506241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47bc101bc0 msgr2=0x7f47bc199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c0e00640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47bc101bc0 0x7f47bc199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c0e00640 1 -- 172.21.15.60:0/2742506241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47bc106e20 msgr2=0x7f47bc1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c0e00640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47bc106e20 0x7f47bc1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c0e00640 1 -- 172.21.15.60:0/2742506241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc1a1250 con 0x7f47bc1044f0 2024-02-26T10:45:41.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.543+0000 7f47c0e00640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47bc1044f0 0x7f47bc199b70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000c00 tx=0x7f47a802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:41.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.544+0000 7f47ba000640 1 -- 172.21.15.60:0/2742506241 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8043020 con 0x7f47bc1044f0 2024-02-26T10:45:41.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.544+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc1a14e0 con 0x7f47bc1044f0 2024-02-26T10:45:41.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.544+0000 7f47ba000640 1 -- 172.21.15.60:0/2742506241 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47a80054e0 con 0x7f47bc1044f0 2024-02-26T10:45:41.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.544+0000 7f47ba000640 1 -- 172.21.15.60:0/2742506241 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8003c40 con 0x7f47bc1044f0 2024-02-26T10:45:41.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.544+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc1a1a80 con 0x7f47bc1044f0 2024-02-26T10:45:41.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.545+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47bc108970 con 0x7f47bc1044f0 2024-02-26T10:45:41.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.547+0000 7f47ba000640 1 -- 172.21.15.60:0/2742506241 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f47a8046020 con 0x7f47bc1044f0 2024-02-26T10:45:41.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.550+0000 7f47ba000640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47a40a1d80 0x7f47a40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.550+0000 7f47ba000640 1 -- 172.21.15.60:0/2742506241 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f47a80f0190 con 0x7f47bc1044f0 2024-02-26T10:45:41.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.550+0000 7f47c1800640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47a40a1d80 0x7f47a40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.550+0000 7f47c1800640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47a40a1d80 0x7f47a40a4240 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009bd0 tx=0x7f47b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:41.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.553+0000 7f47ba000640 1 -- 172.21.15.60:0/2742506241 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47a804d020 con 0x7f47bc1044f0 2024-02-26T10:45:41.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.763+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47bc061e60 con 0x7f47bc1044f0 2024-02-26T10:45:41.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.764+0000 7f47ba000640 1 -- 172.21.15.60:0/2742506241 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47a80bd1b0 con 0x7f47bc1044f0 2024-02-26T10:45:41.760 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47a40a1d80 msgr2=0x7f47a40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47a40a1d80 0x7f47a40a4240 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009bd0 tx=0x7f47b0005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47bc1044f0 msgr2=0x7f47bc199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47bc1044f0 0x7f47bc199b70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000c00 tx=0x7f47a802c650 comp rx=0 tx=0).stop 2024-02-26T10:45:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 shutdown_connections 2024-02-26T10:45:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47a40a1d80 0x7f47a40a4240 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47bc1044f0 0x7f47bc199b70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47bc106e20 0x7f47bc1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 --2- 172.21.15.60:0/2742506241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47bc101bc0 0x7f47bc199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 >> 172.21.15.60:0/2742506241 conn(0x7f47bc0fb5d0 msgr2=0x7f47bc101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:41.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 shutdown_connections 2024-02-26T10:45:41.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.767+0000 7f47c2c00640 1 -- 172.21.15.60:0/2742506241 wait complete. 2024-02-26T10:45:41.783 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:45:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.996+0000 7f1e4a800640 1 -- 172.21.15.60:0/3844449661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e44103360 msgr2=0x7f1e44103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.996+0000 7f1e4a800640 1 --2- 172.21.15.60:0/3844449661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e44103360 0x7f1e44103780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009900 tx=0x7f1e3802bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.996+0000 7f1e4a800640 1 -- 172.21.15.60:0/3844449661 shutdown_connections 2024-02-26T10:45:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.996+0000 7f1e4a800640 1 --2- 172.21.15.60:0/3844449661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e44104550 0x7f1e441049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.996+0000 7f1e4a800640 1 --2- 172.21.15.60:0/3844449661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e44104f10 0x7f1e4410b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.996+0000 7f1e4a800640 1 --2- 172.21.15.60:0/3844449661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e44103360 0x7f1e44103780 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.996+0000 7f1e4a800640 1 -- 172.21.15.60:0/3844449661 >> 172.21.15.60:0/3844449661 conn(0x7f1e440fe8d0 msgr2=0x7f1e44100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.996+0000 7f1e4a800640 1 -- 172.21.15.60:0/3844449661 shutdown_connections 2024-02-26T10:45:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.996+0000 7f1e4a800640 1 -- 172.21.15.60:0/3844449661 wait complete. 2024-02-26T10:45:41.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.997+0000 7f1e4a800640 1 Processor -- start 2024-02-26T10:45:41.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.997+0000 7f1e4a800640 1 -- start start 2024-02-26T10:45:41.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e4a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e44103360 0x7f1e44199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e4a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e44104550 0x7f1e44199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e4a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e44104f10 0x7f1e441a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:41.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e4a800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4410f010 con 0x7f1e44104550 2024-02-26T10:45:41.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e4a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4410ee90 con 0x7f1e44103360 2024-02-26T10:45:41.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e4a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4410f190 con 0x7f1e44104f10 2024-02-26T10:45:41.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e49e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e44104f10 0x7f1e441a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e48a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e44104550 0x7f1e44199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e49e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e44104f10 0x7f1e441a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:47424/0 (socket says 172.21.15.60:47424) 2024-02-26T10:45:41.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e49e00640 1 -- 172.21.15.60:0/1016305251 learned_addr learned my addr 172.21.15.60:0/1016305251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:41.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.998+0000 7f1e49400640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e44103360 0x7f1e44199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e49e00640 1 -- 172.21.15.60:0/1016305251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e44103360 msgr2=0x7f1e44199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e49e00640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e44103360 0x7f1e44199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e49e00640 1 -- 172.21.15.60:0/1016305251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e44104550 msgr2=0x7f1e44199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e49e00640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e44104550 0x7f1e44199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e49e00640 1 -- 172.21.15.60:0/1016305251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e441a12a0 con 0x7f1e44104f10 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e49e00640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e44104f10 0x7f1e441a0c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00efd0 tx=0x7f1e3c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e42000640 1 -- 172.21.15.60:0/1016305251 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c003b70 con 0x7f1e44104f10 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e42000640 1 -- 172.21.15.60:0/1016305251 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c003d10 con 0x7f1e44104f10 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e441a1590 con 0x7f1e44104f10 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e42000640 1 -- 172.21.15.60:0/1016305251 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0109f0 con 0x7f1e44104f10 2024-02-26T10:45:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:41.999+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e441a1a90 con 0x7f1e44104f10 2024-02-26T10:45:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.000+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4410a6d0 con 0x7f1e44104f10 2024-02-26T10:45:41.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.002+0000 7f1e42000640 1 -- 172.21.15.60:0/1016305251 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f1e3c004020 con 0x7f1e44104f10 2024-02-26T10:45:42.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.005+0000 7f1e42000640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e2c0a1e10 0x7f1e2c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:42.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.005+0000 7f1e42000640 1 -- 172.21.15.60:0/1016305251 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f1e3c0ce780 con 0x7f1e44104f10 2024-02-26T10:45:42.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.005+0000 7f1e49400640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e2c0a1e10 0x7f1e2c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:42.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.005+0000 7f1e49400640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e2c0a1e10 0x7f1e2c0a42d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e3802c170 tx=0x7f1e3802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:42.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.008+0000 7f1e42000640 1 -- 172.21.15.60:0/1016305251 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e3c09b640 con 0x7f1e44104f10 2024-02-26T10:45:42.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:41 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2704116197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:42.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:41 smithi060 ceph-mon[31539]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T10:45:42.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:41 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3689161350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:42.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:41 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:42.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:41 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:42.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:41 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2742506241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:42.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:41 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2704116197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:42.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:41 smithi100 ceph-mon[36051]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T10:45:42.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:41 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3689161350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:42.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:41 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:42.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:41 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:42.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:41 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2742506241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.217+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e44061e60 con 0x7f1e44104f10 2024-02-26T10:45:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.218+0000 7f1e42000640 1 -- 172.21.15.60:0/1016305251 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e3c09af40 con 0x7f1e44104f10 2024-02-26T10:45:42.213 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:45:42.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.220+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e2c0a1e10 msgr2=0x7f1e2c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.220+0000 7f1e4a800640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e2c0a1e10 0x7f1e2c0a42d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e3802c170 tx=0x7f1e3802df90 comp rx=0 tx=0).stop 2024-02-26T10:45:42.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.220+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e44104f10 msgr2=0x7f1e441a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.220+0000 7f1e4a800640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e44104f10 0x7f1e441a0c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00efd0 tx=0x7f1e3c00c6a0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.221+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 shutdown_connections 2024-02-26T10:45:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.221+0000 7f1e4a800640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f1e2c0a1e10 0x7f1e2c0a42d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.221+0000 7f1e4a800640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e44104f10 0x7f1e441a0c30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.221+0000 7f1e4a800640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e44104550 0x7f1e44199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.221+0000 7f1e4a800640 1 --2- 172.21.15.60:0/1016305251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e44103360 0x7f1e44199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.221+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 >> 172.21.15.60:0/1016305251 conn(0x7f1e440fe8d0 msgr2=0x7f1e441078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.221+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 shutdown_connections 2024-02-26T10:45:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.221+0000 7f1e4a800640 1 -- 172.21.15.60:0/1016305251 wait complete. 2024-02-26T10:45:42.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:45:42.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:41 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2704116197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:42.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:41 smithi113 ceph-mon[36126]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T10:45:42.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:41 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3689161350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:42.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:41 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:42.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:41 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:42.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:41 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2742506241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.449+0000 7fc45aa00640 1 -- 172.21.15.60:0/2597849871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc454073c40 msgr2=0x7fc4540740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.449+0000 7fc45aa00640 1 --2- 172.21.15.60:0/2597849871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc454073c40 0x7fc4540740a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009900 tx=0x7fc43c02bcd0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.449+0000 7fc45aa00640 1 -- 172.21.15.60:0/2597849871 shutdown_connections 2024-02-26T10:45:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.449+0000 7fc45aa00640 1 --2- 172.21.15.60:0/2597849871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc454073c40 0x7fc4540740a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.449+0000 7fc45aa00640 1 --2- 172.21.15.60:0/2597849871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc454109120 0x7fc45410b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.449+0000 7fc45aa00640 1 --2- 172.21.15.60:0/2597849871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc454075220 0x7fc454073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.449+0000 7fc45aa00640 1 -- 172.21.15.60:0/2597849871 >> 172.21.15.60:0/2597849871 conn(0x7fc4540fc800 msgr2=0x7fc4540fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.450+0000 7fc45aa00640 1 -- 172.21.15.60:0/2597849871 shutdown_connections 2024-02-26T10:45:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.450+0000 7fc45aa00640 1 -- 172.21.15.60:0/2597849871 wait complete. 2024-02-26T10:45:42.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc45aa00640 1 Processor -- start 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc45aa00640 1 -- start start 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc45aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc454073c40 0x7fc454199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc45aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc454075220 0x7fc454199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc45aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc454109120 0x7fc4541a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc45aa00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45410f160 con 0x7fc454073c40 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc45aa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45410efe0 con 0x7fc454109120 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc45aa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45410f2e0 con 0x7fc454075220 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc453e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc454073c40 0x7fc454199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.451+0000 7fc453e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc454073c40 0x7fc454199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58182/0 (socket says 172.21.15.60:58182) 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.452+0000 7fc453e00640 1 -- 172.21.15.60:0/3654148194 learned_addr learned my addr 172.21.15.60:0/3654148194 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.452+0000 7fc44b400640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc454075220 0x7fc454199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:42.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.452+0000 7fc458a00640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc454109120 0x7fc4541a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:42.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.452+0000 7fc44b400640 1 -- 172.21.15.60:0/3654148194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc454109120 msgr2=0x7fc4541a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.452+0000 7fc44b400640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc454109120 0x7fc4541a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.452+0000 7fc44b400640 1 -- 172.21.15.60:0/3654148194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc454073c40 msgr2=0x7fc454199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.452+0000 7fc44b400640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc454073c40 0x7fc454199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.452+0000 7fc44b400640 1 -- 172.21.15.60:0/3654148194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4541a1370 con 0x7fc454075220 2024-02-26T10:45:42.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.452+0000 7fc44b400640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc454075220 0x7fc454199c90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc43c036040 tx=0x7fc43c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:42.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.453+0000 7fc451600640 1 -- 172.21.15.60:0/3654148194 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c043020 con 0x7fc454075220 2024-02-26T10:45:42.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.453+0000 7fc451600640 1 -- 172.21.15.60:0/3654148194 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc43c0054e0 con 0x7fc454075220 2024-02-26T10:45:42.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.453+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4541a1600 con 0x7fc454075220 2024-02-26T10:45:42.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.453+0000 7fc451600640 1 -- 172.21.15.60:0/3654148194 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c003c70 con 0x7fc454075220 2024-02-26T10:45:42.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.453+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4541a1b20 con 0x7fc454075220 2024-02-26T10:45:42.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.454+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc454107ee0 con 0x7fc454075220 2024-02-26T10:45:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.456+0000 7fc451600640 1 -- 172.21.15.60:0/3654148194 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fc43c046020 con 0x7fc454075220 2024-02-26T10:45:42.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.457+0000 7fc451600640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc4380a1e10 0x7fc4380a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:42.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.457+0000 7fc451600640 1 -- 172.21.15.60:0/3654148194 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fc43c0f02e0 con 0x7fc454075220 2024-02-26T10:45:42.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.458+0000 7fc453e00640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc4380a1e10 0x7fc4380a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:42.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.458+0000 7fc453e00640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc4380a1e10 0x7fc4380a42d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc444011550 tx=0x7fc444009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:42.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.462+0000 7fc451600640 1 -- 172.21.15.60:0/3654148194 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc43c04d020 con 0x7fc454075220 2024-02-26T10:45:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.666+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc45402d080 con 0x7fc454075220 2024-02-26T10:45:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.667+0000 7fc451600640 1 -- 172.21.15.60:0/3654148194 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc43c0bd2b0 con 0x7fc454075220 2024-02-26T10:45:42.663 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc4380a1e10 msgr2=0x7fc4380a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc4380a1e10 0x7fc4380a42d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc444011550 tx=0x7fc444009500 comp rx=0 tx=0).stop 2024-02-26T10:45:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc454075220 msgr2=0x7fc454199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc454075220 0x7fc454199c90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc43c036040 tx=0x7fc43c005d50 comp rx=0 tx=0).stop 2024-02-26T10:45:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 shutdown_connections 2024-02-26T10:45:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fc4380a1e10 0x7fc4380a42d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc454109120 0x7fc4541a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc454075220 0x7fc454199c90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 --2- 172.21.15.60:0/3654148194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc454073c40 0x7fc454199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 >> 172.21.15.60:0/3654148194 conn(0x7fc4540fc800 msgr2=0x7fc4541050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 shutdown_connections 2024-02-26T10:45:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.670+0000 7fc45aa00640 1 -- 172.21.15.60:0/3654148194 wait complete. 2024-02-26T10:45:42.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:45:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.902+0000 7fb643800640 1 -- 172.21.15.60:0/2014418099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c0752a0 msgr2=0x7fb63c0736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.902+0000 7fb643800640 1 --2- 172.21.15.60:0/2014418099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c0752a0 0x7fb63c0736f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009cd0 tx=0x7fb62c02c0f0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.902+0000 7fb643800640 1 -- 172.21.15.60:0/2014418099 shutdown_connections 2024-02-26T10:45:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.902+0000 7fb643800640 1 --2- 172.21.15.60:0/2014418099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63c10d850 0x7fb63c10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.902+0000 7fb643800640 1 --2- 172.21.15.60:0/2014418099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c073cc0 0x7fb63c074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.902+0000 7fb643800640 1 --2- 172.21.15.60:0/2014418099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c0752a0 0x7fb63c0736f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.902+0000 7fb643800640 1 -- 172.21.15.60:0/2014418099 >> 172.21.15.60:0/2014418099 conn(0x7fb63c0fc810 msgr2=0x7fb63c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.903+0000 7fb643800640 1 -- 172.21.15.60:0/2014418099 shutdown_connections 2024-02-26T10:45:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.903+0000 7fb643800640 1 -- 172.21.15.60:0/2014418099 wait complete. 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb643800640 1 Processor -- start 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb643800640 1 -- start start 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb643800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c073cc0 0x7fb63c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb643800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63c0752a0 0x7fb63c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb643800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c10d850 0x7fb63c1a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb643800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c1134d0 con 0x7fb63c073cc0 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb643800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c113350 con 0x7fb63c10d850 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb643800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c113650 con 0x7fb63c0752a0 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb640e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c073cc0 0x7fb63c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.904+0000 7fb641800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c10d850 0x7fb63c1a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb640e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c073cc0 0x7fb63c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58190/0 (socket says 172.21.15.60:58190) 2024-02-26T10:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb641800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c10d850 0x7fb63c1a5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:57038/0 (socket says 172.21.15.60:57038) 2024-02-26T10:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb640e00640 1 -- 172.21.15.60:0/899243311 learned_addr learned my addr 172.21.15.60:0/899243311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb63be00640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63c0752a0 0x7fb63c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb641800640 1 -- 172.21.15.60:0/899243311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63c0752a0 msgr2=0x7fb63c19e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb641800640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63c0752a0 0x7fb63c19e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb641800640 1 -- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c073cc0 msgr2=0x7fb63c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb641800640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c073cc0 0x7fb63c19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb641800640 1 -- 172.21.15.60:0/899243311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c1a57e0 con 0x7fb63c10d850 2024-02-26T10:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb640e00640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c073cc0 0x7fb63c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb641800640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c10d850 0x7fb63c1a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb63400e970 tx=0x7fb63400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.905+0000 7fb639600640 1 -- 172.21.15.60:0/899243311 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400ccc0 con 0x7fb63c10d850 2024-02-26T10:45:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.906+0000 7fb639600640 1 -- 172.21.15.60:0/899243311 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb634005020 con 0x7fb63c10d850 2024-02-26T10:45:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.906+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c1a5ad0 con 0x7fb63c10d850 2024-02-26T10:45:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.906+0000 7fb639600640 1 -- 172.21.15.60:0/899243311 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634004a00 con 0x7fb63c10d850 2024-02-26T10:45:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.906+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c1a5fd0 con 0x7fb63c10d850 2024-02-26T10:45:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.907+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c103a20 con 0x7fb63c10d850 2024-02-26T10:45:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.908+0000 7fb639600640 1 -- 172.21.15.60:0/899243311 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fb6340051d0 con 0x7fb63c10d850 2024-02-26T10:45:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.910+0000 7fb639600640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb6200a1e10 0x7fb6200a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.910+0000 7fb639600640 1 -- 172.21.15.60:0/899243311 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb6340ce260 con 0x7fb63c10d850 2024-02-26T10:45:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.910+0000 7fb640e00640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb6200a1e10 0x7fb6200a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.911+0000 7fb640e00640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb6200a1e10 0x7fb6200a42d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb62c005e30 tx=0x7fb62c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:42.914+0000 7fb639600640 1 -- 172.21.15.60:0/899243311 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb63409b180 con 0x7fb63c10d850 2024-02-26T10:45:43.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:42 smithi060 ceph-mon[31539]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-02-26T10:45:43.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:42 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1016305251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:43.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:42 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3654148194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:43.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.119+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb63c02cfe0 con 0x7fb63c10d850 2024-02-26T10:45:43.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.121+0000 7fb639600640 1 -- 172.21.15.60:0/899243311 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7fb63409aa80 con 0x7fb63c10d850 2024-02-26T10:45:43.117 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:43.117 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:45:43.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb6200a1e10 msgr2=0x7fb6200a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb6200a1e10 0x7fb6200a42d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb62c005e30 tx=0x7fb62c004a40 comp rx=0 tx=0).stop 2024-02-26T10:45:43.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c10d850 msgr2=0x7fb63c1a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c10d850 0x7fb63c1a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb63400e970 tx=0x7fb63400ee30 comp rx=0 tx=0).stop 2024-02-26T10:45:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 shutdown_connections 2024-02-26T10:45:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb6200a1e10 0x7fb6200a42d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63c10d850 0x7fb63c1a5170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb63c0752a0 0x7fb63c19e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 --2- 172.21.15.60:0/899243311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c073cc0 0x7fb63c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 >> 172.21.15.60:0/899243311 conn(0x7fb63c0fc810 msgr2=0x7fb63c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 shutdown_connections 2024-02-26T10:45:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.124+0000 7fb643800640 1 -- 172.21.15.60:0/899243311 wait complete. 2024-02-26T10:45:43.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:42 smithi100 ceph-mon[36051]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-02-26T10:45:43.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:42 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1016305251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:43.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:42 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3654148194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:43.141 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:45:43.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:45:43.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:42 smithi113 ceph-mon[36126]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-02-26T10:45:43.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:42 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1016305251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:43.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:42 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3654148194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.377+0000 7fba4e200640 1 -- 172.21.15.60:0/2797662544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 msgr2=0x7fba48107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.377+0000 7fba4e200640 1 --2- 172.21.15.60:0/2797662544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 0x7fba48107ca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fba38009900 tx=0x7fba3802bc40 comp rx=0 tx=0).stop 2024-02-26T10:45:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.377+0000 7fba4e200640 1 -- 172.21.15.60:0/2797662544 shutdown_connections 2024-02-26T10:45:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.377+0000 7fba4e200640 1 --2- 172.21.15.60:0/2797662544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba48109430 0x7fba4810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.377+0000 7fba4e200640 1 --2- 172.21.15.60:0/2797662544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba48108a70 0x7fba48108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.377+0000 7fba4e200640 1 --2- 172.21.15.60:0/2797662544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 0x7fba48107ca0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.377+0000 7fba4e200640 1 -- 172.21.15.60:0/2797662544 >> 172.21.15.60:0/2797662544 conn(0x7fba48076820 msgr2=0x7fba48078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.377+0000 7fba4e200640 1 -- 172.21.15.60:0/2797662544 shutdown_connections 2024-02-26T10:45:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.377+0000 7fba4e200640 1 -- 172.21.15.60:0/2797662544 wait complete. 2024-02-26T10:45:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.378+0000 7fba4e200640 1 Processor -- start 2024-02-26T10:45:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.378+0000 7fba4e200640 1 -- start start 2024-02-26T10:45:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba4e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 0x7fba4819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba4e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba48108a70 0x7fba4819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba4e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba48109430 0x7fba481a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba4e200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba481134f0 con 0x7fba48107880 2024-02-26T10:45:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba4e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48113370 con 0x7fba48108a70 2024-02-26T10:45:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba4e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48113670 con 0x7fba48109430 2024-02-26T10:45:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba47400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 0x7fba4819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba47400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 0x7fba4819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58218/0 (socket says 172.21.15.60:58218) 2024-02-26T10:45:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba47400640 1 -- 172.21.15.60:0/802346745 learned_addr learned my addr 172.21.15.60:0/802346745 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba46a00640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba48108a70 0x7fba4819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba47e00640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba48109430 0x7fba481a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba47e00640 1 -- 172.21.15.60:0/802346745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba48108a70 msgr2=0x7fba4819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba47e00640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba48108a70 0x7fba4819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba47e00640 1 -- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 msgr2=0x7fba4819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.379+0000 7fba47e00640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 0x7fba4819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.380+0000 7fba47e00640 1 -- 172.21.15.60:0/802346745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba481a57e0 con 0x7fba48109430 2024-02-26T10:45:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.380+0000 7fba47400640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 0x7fba4819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.380+0000 7fba47e00640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba48109430 0x7fba481a5170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba40009fd0 tx=0x7fba4000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.380+0000 7fba3fe00640 1 -- 172.21.15.60:0/802346745 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000ce20 con 0x7fba48109430 2024-02-26T10:45:43.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.380+0000 7fba3fe00640 1 -- 172.21.15.60:0/802346745 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba40005020 con 0x7fba48109430 2024-02-26T10:45:43.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.380+0000 7fba3fe00640 1 -- 172.21.15.60:0/802346745 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40010720 con 0x7fba48109430 2024-02-26T10:45:43.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.380+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba481a5ad0 con 0x7fba48109430 2024-02-26T10:45:43.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.380+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba481a5fd0 con 0x7fba48109430 2024-02-26T10:45:43.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.382+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4810ebf0 con 0x7fba48109430 2024-02-26T10:45:43.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.384+0000 7fba3fe00640 1 -- 172.21.15.60:0/802346745 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fba40005540 con 0x7fba48109430 2024-02-26T10:45:43.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.385+0000 7fba3fe00640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fba2c0a1e10 0x7fba2c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:43.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.385+0000 7fba3fe00640 1 -- 172.21.15.60:0/802346745 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fba400cea30 con 0x7fba48109430 2024-02-26T10:45:43.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.385+0000 7fba47400640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fba2c0a1e10 0x7fba2c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.386+0000 7fba47400640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fba2c0a1e10 0x7fba2c0a42d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fba3802c8f0 tx=0x7fba38005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:43.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.389+0000 7fba3fe00640 1 -- 172.21.15.60:0/802346745 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba4009b8f0 con 0x7fba48109430 2024-02-26T10:45:43.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.592+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba4802d080 con 0x7fba48109430 2024-02-26T10:45:43.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.592+0000 7fba3fe00640 1 -- 172.21.15.60:0/802346745 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba4009b1f0 con 0x7fba48109430 2024-02-26T10:45:43.588 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.595+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fba2c0a1e10 msgr2=0x7fba2c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.595+0000 7fba4e200640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fba2c0a1e10 0x7fba2c0a42d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fba3802c8f0 tx=0x7fba38005b50 comp rx=0 tx=0).stop 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.595+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba48109430 msgr2=0x7fba481a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.595+0000 7fba4e200640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba48109430 0x7fba481a5170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba40009fd0 tx=0x7fba4000ef10 comp rx=0 tx=0).stop 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.595+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 shutdown_connections 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.595+0000 7fba4e200640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fba2c0a1e10 0x7fba2c0a42d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.595+0000 7fba4e200640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba48108a70 0x7fba4819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.595+0000 7fba4e200640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba48109430 0x7fba481a5170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.595+0000 7fba4e200640 1 --2- 172.21.15.60:0/802346745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba48107880 0x7fba4819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.596+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 >> 172.21.15.60:0/802346745 conn(0x7fba48076820 msgr2=0x7fba4810bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.596+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 shutdown_connections 2024-02-26T10:45:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.596+0000 7fba4e200640 1 -- 172.21.15.60:0/802346745 wait complete. 2024-02-26T10:45:43.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:45:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.846+0000 7f6267e00640 1 -- 172.21.15.60:0/1066586664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268104570 msgr2=0x7f62681049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.846+0000 7f6267e00640 1 --2- 172.21.15.60:0/1066586664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268104570 0x7f62681049f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6250009900 tx=0x7f625002bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.846+0000 7f6267e00640 1 -- 172.21.15.60:0/1066586664 shutdown_connections 2024-02-26T10:45:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.846+0000 7f6267e00640 1 --2- 172.21.15.60:0/1066586664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268104570 0x7f62681049f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.846+0000 7f6267e00640 1 --2- 172.21.15.60:0/1066586664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6268104f30 0x7f626810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.846+0000 7f6267e00640 1 --2- 172.21.15.60:0/1066586664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6268103380 0x7f62681037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.846+0000 7f6267e00640 1 -- 172.21.15.60:0/1066586664 >> 172.21.15.60:0/1066586664 conn(0x7f62680fe910 msgr2=0x7f6268100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.846+0000 7f6267e00640 1 -- 172.21.15.60:0/1066586664 shutdown_connections 2024-02-26T10:45:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.846+0000 7f6267e00640 1 -- 172.21.15.60:0/1066586664 wait complete. 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.847+0000 7f6267e00640 1 Processor -- start 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6267e00640 1 -- start start 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6267e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6268103380 0x7f6268199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6267e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268104570 0x7f6268199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6267e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6268104f30 0x7f62681a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6267e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626810f060 con 0x7f6268103380 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6267e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626810eee0 con 0x7f6268104570 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6267e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626810f1e0 con 0x7f6268104f30 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6266a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6268103380 0x7f6268199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6266a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6268103380 0x7f6268199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35022/0 (socket says 172.21.15.60:35022) 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6266a00640 1 -- 172.21.15.60:0/215173039 learned_addr learned my addr 172.21.15.60:0/215173039 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f625fe00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268104570 0x7f6268199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.848+0000 7f6267400640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6268104f30 0x7f62681a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.849+0000 7f625fe00640 1 -- 172.21.15.60:0/215173039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6268104f30 msgr2=0x7f62681a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.849+0000 7f625fe00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6268104f30 0x7f62681a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.849+0000 7f625fe00640 1 -- 172.21.15.60:0/215173039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6268103380 msgr2=0x7f6268199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.849+0000 7f625fe00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6268103380 0x7f6268199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.849+0000 7f625fe00640 1 -- 172.21.15.60:0/215173039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62681a1250 con 0x7f6268104570 2024-02-26T10:45:43.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.849+0000 7f625fe00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268104570 0x7f6268199b70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6250036040 tx=0x7f625002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:43.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.850+0000 7f625f400640 1 -- 172.21.15.60:0/215173039 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250043020 con 0x7f6268104570 2024-02-26T10:45:43.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.850+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62681a14e0 con 0x7f6268104570 2024-02-26T10:45:43.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.850+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62681a1a50 con 0x7f6268104570 2024-02-26T10:45:43.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.850+0000 7f625f400640 1 -- 172.21.15.60:0/215173039 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6250004c80 con 0x7f6268104570 2024-02-26T10:45:43.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.850+0000 7f625f400640 1 -- 172.21.15.60:0/215173039 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250003850 con 0x7f6268104570 2024-02-26T10:45:43.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.851+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62440052c0 con 0x7f6268104570 2024-02-26T10:45:43.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.853+0000 7f625f400640 1 -- 172.21.15.60:0/215173039 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f6250046020 con 0x7f6268104570 2024-02-26T10:45:43.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.854+0000 7f625f400640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f624c0a1e10 0x7f624c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:43.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.855+0000 7f625f400640 1 -- 172.21.15.60:0/215173039 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f62500b8cf0 con 0x7f6268104570 2024-02-26T10:45:43.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.855+0000 7f6266a00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f624c0a1e10 0x7f624c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:43.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.855+0000 7f6266a00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f624c0a1e10 0x7f624c0a42d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f625800bc80 tx=0x7f625800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:43.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:43.859+0000 7f625f400640 1 -- 172.21.15.60:0/215173039 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f625004d030 con 0x7f6268104570 2024-02-26T10:45:43.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:45:43.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:45:43.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:43.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:43 smithi060 ceph-mon[31539]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-26T10:45:43.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:43 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/899243311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:43.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:43 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/802346745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.064+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6244005880 con 0x7f6268104570 2024-02-26T10:45:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.064+0000 7f625f400640 1 -- 172.21.15.60:0/215173039 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62500bcd80 con 0x7f6268104570 2024-02-26T10:45:44.060 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f624c0a1e10 msgr2=0x7f624c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f624c0a1e10 0x7f624c0a42d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f625800bc80 tx=0x7f625800b410 comp rx=0 tx=0).stop 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268104570 msgr2=0x7f6268199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268104570 0x7f6268199b70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6250036040 tx=0x7f625002c6c0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 shutdown_connections 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f624c0a1e10 0x7f624c0a42d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6268104570 0x7f6268199b70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6268104f30 0x7f62681a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 --2- 172.21.15.60:0/215173039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6268103380 0x7f6268199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.067+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 >> 172.21.15.60:0/215173039 conn(0x7f62680fe910 msgr2=0x7f6268107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.068+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 shutdown_connections 2024-02-26T10:45:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.068+0000 7f6267e00640 1 -- 172.21.15.60:0/215173039 wait complete. 2024-02-26T10:45:44.084 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:45:44.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:45:44.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:45:44.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:44.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:43 smithi100 ceph-mon[36051]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-26T10:45:44.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:43 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/899243311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:44.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:43 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/802346745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:44.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.316+0000 7f3144200640 1 -- 172.21.15.60:0/2082559645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f313c073cc0 msgr2=0x7f313c074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.316+0000 7f3144200640 1 --2- 172.21.15.60:0/2082559645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f313c073cc0 0x7f313c074140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3128009930 tx=0x7f312802bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:44.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.317+0000 7f3144200640 1 -- 172.21.15.60:0/2082559645 shutdown_connections 2024-02-26T10:45:44.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.317+0000 7f3144200640 1 --2- 172.21.15.60:0/2082559645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c10d840 0x7f313c10fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.317+0000 7f3144200640 1 --2- 172.21.15.60:0/2082559645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f313c073cc0 0x7f313c074140 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.317+0000 7f3144200640 1 --2- 172.21.15.60:0/2082559645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c0752a0 0x7f313c0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.317+0000 7f3144200640 1 -- 172.21.15.60:0/2082559645 >> 172.21.15.60:0/2082559645 conn(0x7f313c0fc800 msgr2=0x7f313c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:44.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.317+0000 7f3144200640 1 -- 172.21.15.60:0/2082559645 shutdown_connections 2024-02-26T10:45:44.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.317+0000 7f3144200640 1 -- 172.21.15.60:0/2082559645 wait complete. 2024-02-26T10:45:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.318+0000 7f3144200640 1 Processor -- start 2024-02-26T10:45:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.318+0000 7f3144200640 1 -- start start 2024-02-26T10:45:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.318+0000 7f3144200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c073cc0 0x7f313c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.318+0000 7f3144200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f313c0752a0 0x7f313c19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.318+0000 7f3144200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c10d840 0x7f313c1a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.318+0000 7f3144200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1134e0 con 0x7f313c0752a0 2024-02-26T10:45:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.318+0000 7f3144200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c113360 con 0x7f313c073cc0 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.318+0000 7f3144200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c113660 con 0x7f313c10d840 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3140e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f313c0752a0 0x7f313c19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3140e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f313c0752a0 0x7f313c19e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35042/0 (socket says 172.21.15.60:35042) 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3140e00640 1 -- 172.21.15.60:0/2612455060 learned_addr learned my addr 172.21.15.60:0/2612455060 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3141800640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c073cc0 0x7f313c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3142200640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c10d840 0x7f313c1a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3141800640 1 -- 172.21.15.60:0/2612455060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c10d840 msgr2=0x7f313c1a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3141800640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c10d840 0x7f313c1a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3141800640 1 -- 172.21.15.60:0/2612455060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f313c0752a0 msgr2=0x7f313c19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3141800640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f313c0752a0 0x7f313c19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3141800640 1 -- 172.21.15.60:0/2612455060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c1a5810 con 0x7f313c073cc0 2024-02-26T10:45:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.319+0000 7f3141800640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c073cc0 0x7f313c19dbf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f313000e9f0 tx=0x7f313000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.320+0000 7f313a000640 1 -- 172.21.15.60:0/2612455060 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313000ccc0 con 0x7f313c073cc0 2024-02-26T10:45:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.320+0000 7f313a000640 1 -- 172.21.15.60:0/2612455060 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3130005020 con 0x7f313c073cc0 2024-02-26T10:45:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.320+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313c1a5b00 con 0x7f313c073cc0 2024-02-26T10:45:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.320+0000 7f313a000640 1 -- 172.21.15.60:0/2612455060 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130004a00 con 0x7f313c073cc0 2024-02-26T10:45:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.320+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313c1a5fd0 con 0x7f313c073cc0 2024-02-26T10:45:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.321+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c103900 con 0x7f313c073cc0 2024-02-26T10:45:44.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.323+0000 7f313a000640 1 -- 172.21.15.60:0/2612455060 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f31300051d0 con 0x7f313c073cc0 2024-02-26T10:45:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.324+0000 7f313a000640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31240a1e10 0x7f31240a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.325+0000 7f313a000640 1 -- 172.21.15.60:0/2612455060 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f31300ce260 con 0x7f313c073cc0 2024-02-26T10:45:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.325+0000 7f3140e00640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31240a1e10 0x7f31240a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.326+0000 7f3140e00640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31240a1e10 0x7f31240a42d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f312802c8e0 tx=0x7f3128005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.327+0000 7f313a000640 1 -- 172.21.15.60:0/2612455060 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f313009b120 con 0x7f313c073cc0 2024-02-26T10:45:44.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:45:44.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:45:44.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:45:44.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:43 smithi113 ceph-mon[36126]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-26T10:45:44.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:43 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/899243311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:44.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:43 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/802346745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:44.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.538+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f313c02d080 con 0x7f313c073cc0 2024-02-26T10:45:44.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.539+0000 7f313a000640 1 -- 172.21.15.60:0/2612455060 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f313009aa20 con 0x7f313c073cc0 2024-02-26T10:45:44.535 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:45:44.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.542+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31240a1e10 msgr2=0x7f31240a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.542+0000 7f3144200640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31240a1e10 0x7f31240a42d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f312802c8e0 tx=0x7f3128005b50 comp rx=0 tx=0).stop 2024-02-26T10:45:44.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.542+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c073cc0 msgr2=0x7f313c19dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.542+0000 7f3144200640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c073cc0 0x7f313c19dbf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f313000e9f0 tx=0x7f313000edf0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.542+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 shutdown_connections 2024-02-26T10:45:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.543+0000 7f3144200640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f31240a1e10 0x7f31240a42d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.543+0000 7f3144200640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c10d840 0x7f313c1a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.543+0000 7f3144200640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f313c0752a0 0x7f313c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.543+0000 7f3144200640 1 --2- 172.21.15.60:0/2612455060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f313c073cc0 0x7f313c19dbf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.543+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 >> 172.21.15.60:0/2612455060 conn(0x7f313c0fc800 msgr2=0x7f313c100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.543+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 shutdown_connections 2024-02-26T10:45:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.543+0000 7f3144200640 1 -- 172.21.15.60:0/2612455060 wait complete. 2024-02-26T10:45:44.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:45:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.789+0000 7f53fd600640 1 -- 172.21.15.60:0/1104654172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102350 msgr2=0x7f53f81027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.789+0000 7f53fd600640 1 --2- 172.21.15.60:0/1104654172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102350 0x7f53f81027d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009900 tx=0x7f53e002bc50 comp rx=0 tx=0).stop 2024-02-26T10:45:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.789+0000 7f53fd600640 1 -- 172.21.15.60:0/1104654172 shutdown_connections 2024-02-26T10:45:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.789+0000 7f53fd600640 1 --2- 172.21.15.60:0/1104654172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53f8102d10 0x7f53f81095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.789+0000 7f53fd600640 1 --2- 172.21.15.60:0/1104654172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102350 0x7f53f81027d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.789+0000 7f53fd600640 1 --2- 172.21.15.60:0/1104654172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53f8101160 0x7f53f8101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.789+0000 7f53fd600640 1 -- 172.21.15.60:0/1104654172 >> 172.21.15.60:0/1104654172 conn(0x7f53f80fc6d0 msgr2=0x7f53f80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.789+0000 7f53fd600640 1 -- 172.21.15.60:0/1104654172 shutdown_connections 2024-02-26T10:45:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.789+0000 7f53fd600640 1 -- 172.21.15.60:0/1104654172 wait complete. 2024-02-26T10:45:44.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.790+0000 7f53fd600640 1 Processor -- start 2024-02-26T10:45:44.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.790+0000 7f53fd600640 1 -- start start 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53f8101160 0x7f53f8075010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53f8102350 0x7f53f8073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fd600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102d10 0x7f53f8073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fd600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f810ce40 con 0x7f53f8102d10 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fd600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f810ccc0 con 0x7f53f8101160 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fd600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f810cfc0 con 0x7f53f8102350 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fcc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102d10 0x7f53f8073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fcc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102d10 0x7f53f8073b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35060/0 (socket says 172.21.15.60:35060) 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fcc00640 1 -- 172.21.15.60:0/31172279 learned_addr learned my addr 172.21.15.60:0/31172279 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53f7400640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53f8102350 0x7f53f8073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53f7e00640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53f8101160 0x7f53f8075010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fcc00640 1 -- 172.21.15.60:0/31172279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53f8102350 msgr2=0x7f53f8073650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fcc00640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53f8102350 0x7f53f8073650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fcc00640 1 -- 172.21.15.60:0/31172279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53f8101160 msgr2=0x7f53f8075010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fcc00640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53f8101160 0x7f53f8075010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.791+0000 7f53fcc00640 1 -- 172.21.15.60:0/31172279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f80740d0 con 0x7f53f8102d10 2024-02-26T10:45:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.792+0000 7f53fcc00640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102d10 0x7f53f8073b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f53f000e9b0 tx=0x7f53f000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.792+0000 7f53f4c00640 1 -- 172.21.15.60:0/31172279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f000cd80 con 0x7f53f8102d10 2024-02-26T10:45:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.792+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f80743c0 con 0x7f53f8102d10 2024-02-26T10:45:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.792+0000 7f53f4c00640 1 -- 172.21.15.60:0/31172279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53f00035f0 con 0x7f53f8102d10 2024-02-26T10:45:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.792+0000 7f53f4c00640 1 -- 172.21.15.60:0/31172279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0010670 con 0x7f53f8102d10 2024-02-26T10:45:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.792+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f81a5e50 con 0x7f53f8102d10 2024-02-26T10:45:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.793+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f8108420 con 0x7f53f8102d10 2024-02-26T10:45:44.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.795+0000 7f53f4c00640 1 -- 172.21.15.60:0/31172279 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f53f00037a0 con 0x7f53f8102d10 2024-02-26T10:45:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.797+0000 7f53f4c00640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f53dc0a1e10 0x7f53dc0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.797+0000 7f53f4c00640 1 -- 172.21.15.60:0/31172279 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f53f00ce600 con 0x7f53f8102d10 2024-02-26T10:45:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.797+0000 7f53f7e00640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f53dc0a1e10 0x7f53dc0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.798+0000 7f53f7e00640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f53dc0a1e10 0x7f53dc0a42d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f53f81021b0 tx=0x7f53e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:44.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:44.801+0000 7f53f4c00640 1 -- 172.21.15.60:0/31172279 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53f009b4c0 con 0x7f53f8102d10 2024-02-26T10:45:44.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.003+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f53f8061e60 con 0x7f53f8102d10 2024-02-26T10:45:44.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.004+0000 7f53f4c00640 1 -- 172.21.15.60:0/31172279 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f53f009adc0 con 0x7f53f8102d10 2024-02-26T10:45:44.999 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:45.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f53dc0a1e10 msgr2=0x7f53dc0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f53dc0a1e10 0x7f53dc0a42d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f53f81021b0 tx=0x7f53e8009500 comp rx=0 tx=0).stop 2024-02-26T10:45:45.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102d10 msgr2=0x7f53f8073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102d10 0x7f53f8073b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f53f000e9b0 tx=0x7f53f000ee70 comp rx=0 tx=0).stop 2024-02-26T10:45:45.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 shutdown_connections 2024-02-26T10:45:45.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f53dc0a1e10 0x7f53dc0a42d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53f8102d10 0x7f53f8073b90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53f8102350 0x7f53f8073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 --2- 172.21.15.60:0/31172279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53f8101160 0x7f53f8075010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 >> 172.21.15.60:0/31172279 conn(0x7f53f80fc6d0 msgr2=0x7f53f81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:45.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 shutdown_connections 2024-02-26T10:45:45.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.007+0000 7f53fd600640 1 -- 172.21.15.60:0/31172279 wait complete. 2024-02-26T10:45:45.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:44 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/215173039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:45.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:44 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2612455060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:45.021 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-02-26T10:45:45.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:45:45.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:44 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/215173039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:45.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:44 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2612455060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:45.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.247+0000 7f8418a00640 1 -- 172.21.15.60:0/286543362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8414102c50 msgr2=0x7f84141094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.247+0000 7f8418a00640 1 --2- 172.21.15.60:0/286543362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8414102c50 0x7f84141094e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f840c009be0 tx=0x7f840c02bef0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.247+0000 7f8418a00640 1 -- 172.21.15.60:0/286543362 shutdown_connections 2024-02-26T10:45:45.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.247+0000 7f8418a00640 1 --2- 172.21.15.60:0/286543362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8414102c50 0x7f84141094e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.247+0000 7f8418a00640 1 --2- 172.21.15.60:0/286543362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8414102290 0x7f8414102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.247+0000 7f8418a00640 1 --2- 172.21.15.60:0/286543362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84141010a0 0x7f84141014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.247+0000 7f8418a00640 1 -- 172.21.15.60:0/286543362 >> 172.21.15.60:0/286543362 conn(0x7f84140fc5f0 msgr2=0x7f84140fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:45.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.247+0000 7f8418a00640 1 -- 172.21.15.60:0/286543362 shutdown_connections 2024-02-26T10:45:45.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.247+0000 7f8418a00640 1 -- 172.21.15.60:0/286543362 wait complete. 2024-02-26T10:45:45.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.248+0000 7f8418a00640 1 Processor -- start 2024-02-26T10:45:45.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f8418a00640 1 -- start start 2024-02-26T10:45:45.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f8418a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84141010a0 0x7f84141995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:45.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f8418a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8414102290 0x7f8414199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:45.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f8418a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8414102c50 0x7f84141a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:45.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f8418a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841410ccf0 con 0x7f84141010a0 2024-02-26T10:45:45.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f8418a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841410cb70 con 0x7f8414102c50 2024-02-26T10:45:45.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f8418a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841410ce70 con 0x7f8414102290 2024-02-26T10:45:45.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f840be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8414102290 0x7f8414199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:45.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f8413400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84141010a0 0x7f84141995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f840be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8414102290 0x7f8414199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49502/0 (socket says 172.21.15.60:49502) 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f840be00640 1 -- 172.21.15.60:0/1216356521 learned_addr learned my addr 172.21.15.60:0/1216356521 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.249+0000 7f8413e00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8414102c50 0x7f84141a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f840be00640 1 -- 172.21.15.60:0/1216356521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8414102c50 msgr2=0x7f84141a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f840be00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8414102c50 0x7f84141a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f840be00640 1 -- 172.21.15.60:0/1216356521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84141010a0 msgr2=0x7f84141995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f840be00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84141010a0 0x7f84141995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f840be00640 1 -- 172.21.15.60:0/1216356521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84141a1160 con 0x7f8414102290 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f840be00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8414102290 0x7f8414199b20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00ea60 tx=0x7f83fc00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f8410c00640 1 -- 172.21.15.60:0/1216356521 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00cd20 con 0x7f8414102290 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f8410c00640 1 -- 172.21.15.60:0/1216356521 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83fc0035f0 con 0x7f8414102290 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84141a1450 con 0x7f8414102290 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f8410c00640 1 -- 172.21.15.60:0/1216356521 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc010690 con 0x7f8414102290 2024-02-26T10:45:45.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.250+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84141a19a0 con 0x7f8414102290 2024-02-26T10:45:45.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.251+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84141a1c60 con 0x7f8414102290 2024-02-26T10:45:45.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.253+0000 7f8410c00640 1 -- 172.21.15.60:0/1216356521 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f83fc0037a0 con 0x7f8414102290 2024-02-26T10:45:45.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.255+0000 7f8410c00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83f80a1e10 0x7f83f80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:45.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.255+0000 7f8410c00640 1 -- 172.21.15.60:0/1216356521 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f83fc0ce610 con 0x7f8414102290 2024-02-26T10:45:45.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.255+0000 7f8413400640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83f80a1e10 0x7f83f80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:45.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.256+0000 7f8413400640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83f80a1e10 0x7f83f80a42d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84141020f0 tx=0x7f8404009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.259+0000 7f8410c00640 1 -- 172.21.15.60:0/1216356521 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83fc09b4d0 con 0x7f8414102290 2024-02-26T10:45:45.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:44 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/215173039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:45.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:44 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2612455060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:45.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.504+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84141a1e30 con 0x7f8414102290 2024-02-26T10:45:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.506+0000 7f8410c00640 1 -- 172.21.15.60:0/1216356521 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11359 (secure 0 0 0) 0x7f83fc0aee20 con 0x7f8414102290 2024-02-26T10:45:45.501 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:45.502 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:45:39.355500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:45:45.502 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2024-02-26T10:45:45.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83f80a1e10 msgr2=0x7f83f80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83f80a1e10 0x7f83f80a42d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84141020f0 tx=0x7f8404009260 comp rx=0 tx=0).stop 2024-02-26T10:45:45.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8414102290 msgr2=0x7f8414199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8414102290 0x7f8414199b20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00ea60 tx=0x7f83fc00ee60 comp rx=0 tx=0).stop 2024-02-26T10:45:45.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 shutdown_connections 2024-02-26T10:45:45.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83f80a1e10 0x7f83f80a42d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8414102c50 0x7f84141a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8414102290 0x7f8414199b20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 --2- 172.21.15.60:0/1216356521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84141010a0 0x7f84141995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 >> 172.21.15.60:0/1216356521 conn(0x7f84140fc5f0 msgr2=0x7f8414073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:45.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 shutdown_connections 2024-02-26T10:45:45.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.509+0000 7f8418a00640 1 -- 172.21.15.60:0/1216356521 wait complete. 2024-02-26T10:45:45.524 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-02-26T10:45:45.524 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-02-26T10:45:45.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:45:45.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.759+0000 7fe0b1800640 1 -- 172.21.15.60:0/102329376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ac101b30 msgr2=0x7fe0ac103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.759+0000 7fe0b1800640 1 --2- 172.21.15.60:0/102329376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ac101b30 0x7fe0ac103f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009860 tx=0x7fe09c02bb70 comp rx=0 tx=0).stop 2024-02-26T10:45:45.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.759+0000 7fe0b1800640 1 -- 172.21.15.60:0/102329376 shutdown_connections 2024-02-26T10:45:45.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.759+0000 7fe0b1800640 1 --2- 172.21.15.60:0/102329376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ac106d90 0x7fe0ac1091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.759+0000 7fe0b1800640 1 --2- 172.21.15.60:0/102329376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0ac104460 0x7fe0ac106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.759+0000 7fe0b1800640 1 --2- 172.21.15.60:0/102329376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ac101b30 0x7fe0ac103f20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.759+0000 7fe0b1800640 1 -- 172.21.15.60:0/102329376 >> 172.21.15.60:0/102329376 conn(0x7fe0ac0fb540 msgr2=0x7fe0ac0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:45.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.759+0000 7fe0b1800640 1 -- 172.21.15.60:0/102329376 shutdown_connections 2024-02-26T10:45:45.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.759+0000 7fe0b1800640 1 -- 172.21.15.60:0/102329376 wait complete. 2024-02-26T10:45:45.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.760+0000 7fe0b1800640 1 Processor -- start 2024-02-26T10:45:45.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.760+0000 7fe0b1800640 1 -- start start 2024-02-26T10:45:45.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0b1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ac101b30 0x7fe0ac1014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:45.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0b1800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0ac104460 0x7fe0ac0ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:45.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0b1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ac106d90 0x7fe0ac100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:45.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0b1800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac10cd40 con 0x7fe0ac104460 2024-02-26T10:45:45.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0b1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac10cbc0 con 0x7fe0ac106d90 2024-02-26T10:45:45.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0b1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac10cec0 con 0x7fe0ac101b30 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0a3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0ac104460 0x7fe0ac0ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0a3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0ac104460 0x7fe0ac0ffae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35086/0 (socket says 172.21.15.60:35086) 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0a3e00640 1 -- 172.21.15.60:0/3412061112 learned_addr learned my addr 172.21.15.60:0/3412061112 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0b0e00640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ac106d90 0x7fe0ac100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0abe00640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ac101b30 0x7fe0ac1014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0a3e00640 1 -- 172.21.15.60:0/3412061112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ac101b30 msgr2=0x7fe0ac1014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0a3e00640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ac101b30 0x7fe0ac1014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0a3e00640 1 -- 172.21.15.60:0/3412061112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ac106d90 msgr2=0x7fe0ac100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0a3e00640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ac106d90 0x7fe0ac100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0a3e00640 1 -- 172.21.15.60:0/3412061112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac1008b0 con 0x7fe0ac104460 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.761+0000 7fe0b0e00640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ac106d90 0x7fe0ac100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.762+0000 7fe0a3e00640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0ac104460 0x7fe0ac0ffae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe09400e9a0 tx=0x7fe09400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.762+0000 7fe0a9600640 1 -- 172.21.15.60:0/3412061112 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09400cd80 con 0x7fe0ac104460 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.762+0000 7fe0a9600640 1 -- 172.21.15.60:0/3412061112 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0940035f0 con 0x7fe0ac104460 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.762+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac1a14b0 con 0x7fe0ac104460 2024-02-26T10:45:45.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.762+0000 7fe0a9600640 1 -- 172.21.15.60:0/3412061112 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094010670 con 0x7fe0ac104460 2024-02-26T10:45:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.762+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac1a1a00 con 0x7fe0ac104460 2024-02-26T10:45:45.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.763+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac0fd030 con 0x7fe0ac104460 2024-02-26T10:45:45.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.765+0000 7fe0a9600640 1 -- 172.21.15.60:0/3412061112 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fe0940037a0 con 0x7fe0ac104460 2024-02-26T10:45:45.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.767+0000 7fe0a9600640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe0900a1e10 0x7fe0900a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:45.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.767+0000 7fe0a9600640 1 -- 172.21.15.60:0/3412061112 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe0940ce600 con 0x7fe0ac104460 2024-02-26T10:45:45.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.767+0000 7fe0abe00640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe0900a1e10 0x7fe0900a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:45.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.768+0000 7fe0abe00640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe0900a1e10 0x7fe0900a42d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe09c02c180 tx=0x7fe09c034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:45.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.771+0000 7fe0a9600640 1 -- 172.21.15.60:0/3412061112 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe09409b4c0 con 0x7fe0ac104460 2024-02-26T10:45:45.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:45.968+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0ac0616a0 con 0x7fe0900a1e10 2024-02-26T10:45:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.003+0000 7fe0a9600640 1 -- 172.21.15.60:0/3412061112 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340796 (secure 0 0 0) 0x7fe0ac0616a0 con 0x7fe0900a1e10 2024-02-26T10:45:46.001 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:46.005 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T10:45:46.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe0900a1e10 msgr2=0x7fe0900a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe0900a1e10 0x7fe0900a42d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe09c02c180 tx=0x7fe09c034ad0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0ac104460 msgr2=0x7fe0ac0ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0ac104460 0x7fe0ac0ffae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe09400e9a0 tx=0x7fe09400ee60 comp rx=0 tx=0).stop 2024-02-26T10:45:46.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 shutdown_connections 2024-02-26T10:45:46.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe0900a1e10 0x7fe0900a42d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0ac106d90 0x7fe0ac100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0ac104460 0x7fe0ac0ffae0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 --2- 172.21.15.60:0/3412061112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ac101b30 0x7fe0ac1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 >> 172.21.15.60:0/3412061112 conn(0x7fe0ac0fb540 msgr2=0x7fe0ac102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:46.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 shutdown_connections 2024-02-26T10:45:46.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.011+0000 7fe0b1800640 1 -- 172.21.15.60:0/3412061112 wait complete. 2024-02-26T10:45:46.027 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-02-26T10:45:44.964702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666603,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1036,"num_read_kb":643,"num_write":364,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1244077,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":794,"ondisk_log_size":794,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374372,"kb_used_data":12708,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567452,"statfs":{"total":1151940427776,"available":1151557070848,"internally_reserved":0,"allocated":13012992,"data_stored":8364119,"data_compressed":443487,"data_compressed_allocated":598016,"data_compressed_original":2732032,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":56,"num_write":10,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004716"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308577+0000","last_change":"2024-02-26T10:42:42.176634+0000","last_active":"2024-02-26T10:44:39.308577+0000","last_peered":"2024-02-26T10:44:39.308577+0000","last_clean":"2024-02-26T10:44:39.308577+0000","last_became_active":"2024-02-26T10:42:42.176386+0000","last_became_peered":"2024-02-26T10:42:42.176386+0000","last_unstale":"2024-02-26T10:44:39.308577+0000","last_undegraded":"2024-02-26T10:44:39.308577+0000","last_fullsized":"2024-02-26T10:44:39.308577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:45.005506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528482+0000","last_change":"2024-02-26T10:42:42.176769+0000","last_active":"2024-02-26T10:44:39.528482+0000","last_peered":"2024-02-26T10:44:39.528482+0000","last_clean":"2024-02-26T10:44:39.528482+0000","last_became_active":"2024-02-26T10:42:42.174868+0000","last_became_peered":"2024-02-26T10:42:42.174868+0000","last_unstale":"2024-02-26T10:44:39.528482+0000","last_undegraded":"2024-02-26T10:44:39.528482+0000","last_fullsized":"2024-02-26T10:44:39.528482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:16.202371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528227+0000","last_change":"2024-02-26T10:42:42.184410+0000","last_active":"2024-02-26T10:44:39.528227+0000","last_peered":"2024-02-26T10:44:39.528227+0000","last_clean":"2024-02-26T10:44:39.528227+0000","last_became_active":"2024-02-26T10:42:42.182986+0000","last_became_peered":"2024-02-26T10:42:42.182986+0000","last_unstale":"2024-02-26T10:44:39.528227+0000","last_undegraded":"2024-02-26T10:44:39.528227+0000","last_fullsized":"2024-02-26T10:44:39.528227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:28:45.445325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409010+0000","last_change":"2024-02-26T10:43:38.462828+0000","last_active":"2024-02-26T10:44:39.409010+0000","last_peered":"2024-02-26T10:44:39.409010+0000","last_clean":"2024-02-26T10:44:39.409010+0000","last_became_active":"2024-02-26T10:43:38.461014+0000","last_became_peered":"2024-02-26T10:43:38.461014+0000","last_unstale":"2024-02-26T10:44:39.409010+0000","last_undegraded":"2024-02-26T10:44:39.409010+0000","last_fullsized":"2024-02-26T10:44:39.409010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:07.857849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234724+0000","last_change":"2024-02-26T10:42:42.192448+0000","last_active":"2024-02-26T10:44:39.234724+0000","last_peered":"2024-02-26T10:44:39.234724+0000","last_clean":"2024-02-26T10:44:39.234724+0000","last_became_active":"2024-02-26T10:42:42.192133+0000","last_became_peered":"2024-02-26T10:42:42.192133+0000","last_unstale":"2024-02-26T10:44:39.234724+0000","last_undegraded":"2024-02-26T10:44:39.234724+0000","last_fullsized":"2024-02-26T10:44:39.234724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:43:49.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.256983+0000","last_change":"2024-02-26T10:42:42.169999+0000","last_active":"2024-02-26T10:44:39.256983+0000","last_peered":"2024-02-26T10:44:39.256983+0000","last_clean":"2024-02-26T10:44:39.256983+0000","last_became_active":"2024-02-26T10:42:42.169678+0000","last_became_peered":"2024-02-26T10:42:42.169678+0000","last_unstale":"2024-02-26T10:44:39.256983+0000","last_undegraded":"2024-02-26T10:44:39.256983+0000","last_fullsized":"2024-02-26T10:44:39.256983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:54.435203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234970+0000","last_change":"2024-02-26T10:42:42.177538+0000","last_active":"2024-02-26T10:44:39.234970+0000","last_peered":"2024-02-26T10:44:39.234970+0000","last_clean":"2024-02-26T10:44:39.234970+0000","last_became_active":"2024-02-26T10:42:42.177204+0000","last_became_peered":"2024-02-26T10:42:42.177204+0000","last_unstale":"2024-02-26T10:44:39.234970+0000","last_undegraded":"2024-02-26T10:44:39.234970+0000","last_fullsized":"2024-02-26T10:44:39.234970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:09.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040510+0000","last_change":"2024-02-26T10:42:42.191342+0000","last_active":"2024-02-26T10:44:40.040510+0000","last_peered":"2024-02-26T10:44:40.040510+0000","last_clean":"2024-02-26T10:44:40.040510+0000","last_became_active":"2024-02-26T10:42:42.190990+0000","last_became_peered":"2024-02-26T10:42:42.190990+0000","last_unstale":"2024-02-26T10:44:40.040510+0000","last_undegraded":"2024-02-26T10:44:40.040510+0000","last_fullsized":"2024-02-26T10:44:40.040510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:20:45.190300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464601+0000","last_change":"2024-02-26T10:42:42.173175+0000","last_active":"2024-02-26T10:44:39.464601+0000","last_peered":"2024-02-26T10:44:39.464601+0000","last_clean":"2024-02-26T10:44:39.464601+0000","last_became_active":"2024-02-26T10:42:42.172828+0000","last_became_peered":"2024-02-26T10:42:42.172828+0000","last_unstale":"2024-02-26T10:44:39.464601+0000","last_undegraded":"2024-02-26T10:44:39.464601+0000","last_fullsized":"2024-02-26T10:44:39.464601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:17:34.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257498+0000","last_change":"2024-02-26T10:42:42.167267+0000","last_active":"2024-02-26T10:44:39.257498+0000","last_peered":"2024-02-26T10:44:39.257498+0000","last_clean":"2024-02-26T10:44:39.257498+0000","last_became_active":"2024-02-26T10:42:42.166948+0000","last_became_peered":"2024-02-26T10:42:42.166948+0000","last_unstale":"2024-02-26T10:44:39.257498+0000","last_undegraded":"2024-02-26T10:44:39.257498+0000","last_fullsized":"2024-02-26T10:44:39.257498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:52.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235590+0000","last_change":"2024-02-26T10:42:42.171299+0000","last_active":"2024-02-26T10:44:39.235590+0000","last_peered":"2024-02-26T10:44:39.235590+0000","last_clean":"2024-02-26T10:44:39.235590+0000","last_became_active":"2024-02-26T10:42:42.170616+0000","last_became_peered":"2024-02-26T10:42:42.170616+0000","last_unstale":"2024-02-26T10:44:39.235590+0000","last_undegraded":"2024-02-26T10:44:39.235590+0000","last_fullsized":"2024-02-26T10:44:39.235590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:50.320859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239431+0000","last_change":"2024-02-26T10:42:42.168408+0000","last_active":"2024-02-26T10:44:39.239431+0000","last_peered":"2024-02-26T10:44:39.239431+0000","last_clean":"2024-02-26T10:44:39.239431+0000","last_became_active":"2024-02-26T10:42:42.167150+0000","last_became_peered":"2024-02-26T10:42:42.167150+0000","last_unstale":"2024-02-26T10:44:39.239431+0000","last_undegraded":"2024-02-26T10:44:39.239431+0000","last_fullsized":"2024-02-26T10:44:39.239431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:34.763331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236385+0000","last_change":"2024-02-26T10:42:42.171720+0000","last_active":"2024-02-26T10:44:39.236385+0000","last_peered":"2024-02-26T10:44:39.236385+0000","last_clean":"2024-02-26T10:44:39.236385+0000","last_became_active":"2024-02-26T10:42:42.171309+0000","last_became_peered":"2024-02-26T10:42:42.171309+0000","last_unstale":"2024-02-26T10:44:39.236385+0000","last_undegraded":"2024-02-26T10:44:39.236385+0000","last_fullsized":"2024-02-26T10:44:39.236385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:20:52.151553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527374+0000","last_change":"2024-02-26T10:42:42.176857+0000","last_active":"2024-02-26T10:44:39.527374+0000","last_peered":"2024-02-26T10:44:39.527374+0000","last_clean":"2024-02-26T10:44:39.527374+0000","last_became_active":"2024-02-26T10:42:42.176385+0000","last_became_peered":"2024-02-26T10:42:42.176385+0000","last_unstale":"2024-02-26T10:44:39.527374+0000","last_undegraded":"2024-02-26T10:44:39.527374+0000","last_fullsized":"2024-02-26T10:44:39.527374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:45:51.852212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527401+0000","last_change":"2024-02-26T10:42:42.174682+0000","last_active":"2024-02-26T10:44:39.527401+0000","last_peered":"2024-02-26T10:44:39.527401+0000","last_clean":"2024-02-26T10:44:39.527401+0000","last_became_active":"2024-02-26T10:42:42.174328+0000","last_became_peered":"2024-02-26T10:42:42.174328+0000","last_unstale":"2024-02-26T10:44:39.527401+0000","last_undegraded":"2024-02-26T10:44:39.527401+0000","last_fullsized":"2024-02-26T10:44:39.527401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:47.161020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463946+0000","last_change":"2024-02-26T10:42:42.185289+0000","last_active":"2024-02-26T10:44:39.463946+0000","last_peered":"2024-02-26T10:44:39.463946+0000","last_clean":"2024-02-26T10:44:39.463946+0000","last_became_active":"2024-02-26T10:42:42.185000+0000","last_became_peered":"2024-02-26T10:42:42.185000+0000","last_unstale":"2024-02-26T10:44:39.463946+0000","last_undegraded":"2024-02-26T10:44:39.463946+0000","last_fullsized":"2024-02-26T10:44:39.463946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:48.552584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308483+0000","last_change":"2024-02-26T10:42:42.176558+0000","last_active":"2024-02-26T10:44:39.308483+0000","last_peered":"2024-02-26T10:44:39.308483+0000","last_clean":"2024-02-26T10:44:39.308483+0000","last_became_active":"2024-02-26T10:42:42.176334+0000","last_became_peered":"2024-02-26T10:42:42.176334+0000","last_unstale":"2024-02-26T10:44:39.308483+0000","last_undegraded":"2024-02-26T10:44:39.308483+0000","last_fullsized":"2024-02-26T10:44:39.308483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:59.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409740+0000","last_change":"2024-02-26T10:42:42.168534+0000","last_active":"2024-02-26T10:44:39.409740+0000","last_peered":"2024-02-26T10:44:39.409740+0000","last_clean":"2024-02-26T10:44:39.409740+0000","last_became_active":"2024-02-26T10:42:42.168179+0000","last_became_peered":"2024-02-26T10:42:42.168179+0000","last_unstale":"2024-02-26T10:44:39.409740+0000","last_undegraded":"2024-02-26T10:44:39.409740+0000","last_fullsized":"2024-02-26T10:44:39.409740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:09.625969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238870+0000","last_change":"2024-02-26T10:42:42.192446+0000","last_active":"2024-02-26T10:44:39.238870+0000","last_peered":"2024-02-26T10:44:39.238870+0000","last_clean":"2024-02-26T10:44:39.238870+0000","last_became_active":"2024-02-26T10:42:42.191863+0000","last_became_peered":"2024-02-26T10:42:42.191863+0000","last_unstale":"2024-02-26T10:44:39.238870+0000","last_undegraded":"2024-02-26T10:44:39.238870+0000","last_fullsized":"2024-02-26T10:44:39.238870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:30:19.477529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238594+0000","last_change":"2024-02-26T10:42:47.257605+0000","last_active":"2024-02-26T10:44:39.238594+0000","last_peered":"2024-02-26T10:44:39.238594+0000","last_clean":"2024-02-26T10:44:39.238594+0000","last_became_active":"2024-02-26T10:42:47.257206+0000","last_became_peered":"2024-02-26T10:42:47.257206+0000","last_unstale":"2024-02-26T10:44:39.238594+0000","last_undegraded":"2024-02-26T10:44:39.238594+0000","last_fullsized":"2024-02-26T10:44:39.238594+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:26:17.681862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409986+0000","last_change":"2024-02-26T10:42:41.613087+0000","last_active":"2024-02-26T10:44:39.409986+0000","last_peered":"2024-02-26T10:44:39.409986+0000","last_clean":"2024-02-26T10:44:39.409986+0000","last_became_active":"2024-02-26T10:42:41.612664+0000","last_became_peered":"2024-02-26T10:42:41.612664+0000","last_unstale":"2024-02-26T10:44:39.409986+0000","last_undegraded":"2024-02-26T10:44:39.409986+0000","last_fullsized":"2024-02-26T10:44:39.409986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:00.352166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042024+0000","last_change":"2024-02-26T10:42:42.191438+0000","last_active":"2024-02-26T10:44:40.042024+0000","last_peered":"2024-02-26T10:44:40.042024+0000","last_clean":"2024-02-26T10:44:40.042024+0000","last_became_active":"2024-02-26T10:42:42.191095+0000","last_became_peered":"2024-02-26T10:42:42.191095+0000","last_unstale":"2024-02-26T10:44:40.042024+0000","last_undegraded":"2024-02-26T10:44:40.042024+0000","last_fullsized":"2024-02-26T10:44:40.042024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:20.918715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238951+0000","last_change":"2024-02-26T10:42:47.247503+0000","last_active":"2024-02-26T10:44:39.238951+0000","last_peered":"2024-02-26T10:44:39.238951+0000","last_clean":"2024-02-26T10:44:39.238951+0000","last_became_active":"2024-02-26T10:42:47.247121+0000","last_became_peered":"2024-02-26T10:42:47.247121+0000","last_unstale":"2024-02-26T10:44:39.238951+0000","last_undegraded":"2024-02-26T10:44:39.238951+0000","last_fullsized":"2024-02-26T10:44:39.238951+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:01:38.733131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042711+0000","last_change":"2024-02-26T10:42:41.626227+0000","last_active":"2024-02-26T10:44:40.042711+0000","last_peered":"2024-02-26T10:44:40.042711+0000","last_clean":"2024-02-26T10:44:40.042711+0000","last_became_active":"2024-02-26T10:42:41.625760+0000","last_became_peered":"2024-02-26T10:42:41.625760+0000","last_unstale":"2024-02-26T10:44:40.042711+0000","last_undegraded":"2024-02-26T10:44:40.042711+0000","last_fullsized":"2024-02-26T10:44:40.042711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:04.600151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528152+0000","last_change":"2024-02-26T10:42:42.182808+0000","last_active":"2024-02-26T10:44:39.528152+0000","last_peered":"2024-02-26T10:44:39.528152+0000","last_clean":"2024-02-26T10:44:39.528152+0000","last_became_active":"2024-02-26T10:42:42.182485+0000","last_became_peered":"2024-02-26T10:42:42.182485+0000","last_unstale":"2024-02-26T10:44:39.528152+0000","last_undegraded":"2024-02-26T10:44:39.528152+0000","last_fullsized":"2024-02-26T10:44:39.528152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:45.205969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239151+0000","last_change":"2024-02-26T10:42:47.250694+0000","last_active":"2024-02-26T10:44:39.239151+0000","last_peered":"2024-02-26T10:44:39.239151+0000","last_clean":"2024-02-26T10:44:39.239151+0000","last_became_active":"2024-02-26T10:42:47.249566+0000","last_became_peered":"2024-02-26T10:42:47.249566+0000","last_unstale":"2024-02-26T10:44:39.239151+0000","last_undegraded":"2024-02-26T10:44:39.239151+0000","last_fullsized":"2024-02-26T10:44:39.239151+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:44.016413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235108+0000","last_change":"2024-02-26T10:42:41.606849+0000","last_active":"2024-02-26T10:44:39.235108+0000","last_peered":"2024-02-26T10:44:39.235108+0000","last_clean":"2024-02-26T10:44:39.235108+0000","last_became_active":"2024-02-26T10:42:41.606494+0000","last_became_peered":"2024-02-26T10:42:41.606494+0000","last_unstale":"2024-02-26T10:44:39.235108+0000","last_undegraded":"2024-02-26T10:44:39.235108+0000","last_fullsized":"2024-02-26T10:44:39.235108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:36.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464300+0000","last_change":"2024-02-26T10:42:42.175926+0000","last_active":"2024-02-26T10:44:39.464300+0000","last_peered":"2024-02-26T10:44:39.464300+0000","last_clean":"2024-02-26T10:44:39.464300+0000","last_became_active":"2024-02-26T10:42:42.175614+0000","last_became_peered":"2024-02-26T10:42:42.175614+0000","last_unstale":"2024-02-26T10:44:39.464300+0000","last_undegraded":"2024-02-26T10:44:39.464300+0000","last_fullsized":"2024-02-26T10:44:39.464300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:20:20.829849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.044113+0000","last_change":"2024-02-26T10:44:40.043633+0000","last_active":"2024-02-26T10:44:40.044113+0000","last_peered":"2024-02-26T10:44:40.044113+0000","last_clean":"2024-02-26T10:44:40.044113+0000","last_became_active":"2024-02-26T10:44:40.043209+0000","last_became_peered":"2024-02-26T10:44:40.043209+0000","last_unstale":"2024-02-26T10:44:40.044113+0000","last_undegraded":"2024-02-26T10:44:40.044113+0000","last_fullsized":"2024-02-26T10:44:40.044113+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:43.466990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234893+0000","last_change":"2024-02-26T10:42:41.606407+0000","last_active":"2024-02-26T10:44:39.234893+0000","last_peered":"2024-02-26T10:44:39.234893+0000","last_clean":"2024-02-26T10:44:39.234893+0000","last_became_active":"2024-02-26T10:42:41.606094+0000","last_became_peered":"2024-02-26T10:42:41.606094+0000","last_unstale":"2024-02-26T10:44:39.234893+0000","last_undegraded":"2024-02-26T10:44:39.234893+0000","last_fullsized":"2024-02-26T10:44:39.234893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:53.673017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557475+0000","last_change":"2024-02-26T10:42:42.172967+0000","last_active":"2024-02-26T10:44:39.557475+0000","last_peered":"2024-02-26T10:44:39.557475+0000","last_clean":"2024-02-26T10:44:39.557475+0000","last_became_active":"2024-02-26T10:42:42.172675+0000","last_became_peered":"2024-02-26T10:42:42.172675+0000","last_unstale":"2024-02-26T10:44:39.557475+0000","last_undegraded":"2024-02-26T10:44:39.557475+0000","last_fullsized":"2024-02-26T10:44:39.557475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238546+0000","last_change":"2024-02-26T10:42:47.229375+0000","last_active":"2024-02-26T10:44:39.238546+0000","last_peered":"2024-02-26T10:44:39.238546+0000","last_clean":"2024-02-26T10:44:39.238546+0000","last_became_active":"2024-02-26T10:42:47.228874+0000","last_became_peered":"2024-02-26T10:42:47.228874+0000","last_unstale":"2024-02-26T10:44:39.238546+0000","last_undegraded":"2024-02-26T10:44:39.238546+0000","last_fullsized":"2024-02-26T10:44:39.238546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:02.188519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":190,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:41.266222+0000","last_change":"2024-02-26T10:42:41.602783+0000","last_active":"2024-02-26T10:45:41.266222+0000","last_peered":"2024-02-26T10:45:41.266222+0000","last_clean":"2024-02-26T10:45:41.266222+0000","last_became_active":"2024-02-26T10:42:41.602491+0000","last_became_peered":"2024-02-26T10:42:41.602491+0000","last_unstale":"2024-02-26T10:45:41.266222+0000","last_undegraded":"2024-02-26T10:45:41.266222+0000","last_fullsized":"2024-02-26T10:45:41.266222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:23:33.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234508+0000","last_change":"2024-02-26T10:43:38.461429+0000","last_active":"2024-02-26T10:44:39.234508+0000","last_peered":"2024-02-26T10:44:39.234508+0000","last_clean":"2024-02-26T10:44:39.234508+0000","last_became_active":"2024-02-26T10:43:38.461019+0000","last_became_peered":"2024-02-26T10:43:38.461019+0000","last_unstale":"2024-02-26T10:44:39.234508+0000","last_undegraded":"2024-02-26T10:44:39.234508+0000","last_fullsized":"2024-02-26T10:44:39.234508+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:20.911179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.559125+0000","last_change":"2024-02-26T10:44:39.558649+0000","last_active":"2024-02-26T10:44:39.559125+0000","last_peered":"2024-02-26T10:44:39.559125+0000","last_clean":"2024-02-26T10:44:39.559125+0000","last_became_active":"2024-02-26T10:44:39.558013+0000","last_became_peered":"2024-02-26T10:44:39.558013+0000","last_unstale":"2024-02-26T10:44:39.559125+0000","last_undegraded":"2024-02-26T10:44:39.559125+0000","last_fullsized":"2024-02-26T10:44:39.559125+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:45.879391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.256942+0000","last_change":"2024-02-26T10:42:41.602887+0000","last_active":"2024-02-26T10:44:39.256942+0000","last_peered":"2024-02-26T10:44:39.256942+0000","last_clean":"2024-02-26T10:44:39.256942+0000","last_became_active":"2024-02-26T10:42:41.602564+0000","last_became_peered":"2024-02-26T10:42:41.602564+0000","last_unstale":"2024-02-26T10:44:39.256942+0000","last_undegraded":"2024-02-26T10:44:39.256942+0000","last_fullsized":"2024-02-26T10:44:39.256942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:22:41.711283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556964+0000","last_change":"2024-02-26T10:42:42.192700+0000","last_active":"2024-02-26T10:44:39.556964+0000","last_peered":"2024-02-26T10:44:39.556964+0000","last_clean":"2024-02-26T10:44:39.556964+0000","last_became_active":"2024-02-26T10:42:42.192387+0000","last_became_peered":"2024-02-26T10:42:42.192387+0000","last_unstale":"2024-02-26T10:44:39.556964+0000","last_undegraded":"2024-02-26T10:44:39.556964+0000","last_fullsized":"2024-02-26T10:44:39.556964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:21:15.104872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307862+0000","last_change":"2024-02-26T10:42:47.247000+0000","last_active":"2024-02-26T10:44:39.307862+0000","last_peered":"2024-02-26T10:44:39.307862+0000","last_clean":"2024-02-26T10:44:39.307862+0000","last_became_active":"2024-02-26T10:42:47.246520+0000","last_became_peered":"2024-02-26T10:42:47.246520+0000","last_unstale":"2024-02-26T10:44:39.307862+0000","last_undegraded":"2024-02-26T10:44:39.307862+0000","last_fullsized":"2024-02-26T10:44:39.307862+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:29.977858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235812+0000","last_change":"2024-02-26T10:42:41.609460+0000","last_active":"2024-02-26T10:44:39.235812+0000","last_peered":"2024-02-26T10:44:39.235812+0000","last_clean":"2024-02-26T10:44:39.235812+0000","last_became_active":"2024-02-26T10:42:41.608981+0000","last_became_peered":"2024-02-26T10:42:41.608981+0000","last_unstale":"2024-02-26T10:44:39.235812+0000","last_undegraded":"2024-02-26T10:44:39.235812+0000","last_fullsized":"2024-02-26T10:44:39.235812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:04.599910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040491+0000","last_change":"2024-02-26T10:42:42.185359+0000","last_active":"2024-02-26T10:44:40.040491+0000","last_peered":"2024-02-26T10:44:40.040491+0000","last_clean":"2024-02-26T10:44:40.040491+0000","last_became_active":"2024-02-26T10:42:42.184221+0000","last_became_peered":"2024-02-26T10:42:42.184221+0000","last_unstale":"2024-02-26T10:44:40.040491+0000","last_undegraded":"2024-02-26T10:44:40.040491+0000","last_fullsized":"2024-02-26T10:44:40.040491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:53.907290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526689+0000","last_change":"2024-02-26T10:42:47.241700+0000","last_active":"2024-02-26T10:44:39.526689+0000","last_peered":"2024-02-26T10:44:39.526689+0000","last_clean":"2024-02-26T10:44:39.526689+0000","last_became_active":"2024-02-26T10:42:47.240667+0000","last_became_peered":"2024-02-26T10:42:47.240667+0000","last_unstale":"2024-02-26T10:44:39.526689+0000","last_undegraded":"2024-02-26T10:44:39.526689+0000","last_fullsized":"2024-02-26T10:44:39.526689+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:19.071244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556576+0000","last_change":"2024-02-26T10:42:41.604519+0000","last_active":"2024-02-26T10:44:39.556576+0000","last_peered":"2024-02-26T10:44:39.556576+0000","last_clean":"2024-02-26T10:44:39.556576+0000","last_became_active":"2024-02-26T10:42:41.603596+0000","last_became_peered":"2024-02-26T10:42:41.603596+0000","last_unstale":"2024-02-26T10:44:39.556576+0000","last_undegraded":"2024-02-26T10:44:39.556576+0000","last_fullsized":"2024-02-26T10:44:39.556576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:39.764730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408161+0000","last_change":"2024-02-26T10:42:42.187532+0000","last_active":"2024-02-26T10:44:39.408161+0000","last_peered":"2024-02-26T10:44:39.408161+0000","last_clean":"2024-02-26T10:44:39.408161+0000","last_became_active":"2024-02-26T10:42:42.185635+0000","last_became_peered":"2024-02-26T10:42:42.185635+0000","last_unstale":"2024-02-26T10:44:39.408161+0000","last_undegraded":"2024-02-26T10:44:39.408161+0000","last_fullsized":"2024-02-26T10:44:39.408161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:16.894747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041446+0000","last_change":"2024-02-26T10:42:47.233507+0000","last_active":"2024-02-26T10:44:40.041446+0000","last_peered":"2024-02-26T10:44:40.041446+0000","last_clean":"2024-02-26T10:44:40.041446+0000","last_became_active":"2024-02-26T10:42:47.229589+0000","last_became_peered":"2024-02-26T10:42:47.229589+0000","last_unstale":"2024-02-26T10:44:40.041446+0000","last_undegraded":"2024-02-26T10:44:40.041446+0000","last_fullsized":"2024-02-26T10:44:40.041446+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:02:42.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":185,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:41.265079+0000","last_change":"2024-02-26T10:42:41.613952+0000","last_active":"2024-02-26T10:45:41.265079+0000","last_peered":"2024-02-26T10:45:41.265079+0000","last_clean":"2024-02-26T10:45:41.265079+0000","last_became_active":"2024-02-26T10:42:41.613656+0000","last_became_peered":"2024-02-26T10:42:41.613656+0000","last_unstale":"2024-02-26T10:45:41.265079+0000","last_undegraded":"2024-02-26T10:45:41.265079+0000","last_fullsized":"2024-02-26T10:45:41.265079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:14.090821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464609+0000","last_change":"2024-02-26T10:42:42.178357+0000","last_active":"2024-02-26T10:44:39.464609+0000","last_peered":"2024-02-26T10:44:39.464609+0000","last_clean":"2024-02-26T10:44:39.464609+0000","last_became_active":"2024-02-26T10:42:42.178051+0000","last_became_peered":"2024-02-26T10:42:42.178051+0000","last_unstale":"2024-02-26T10:44:39.464609+0000","last_undegraded":"2024-02-26T10:44:39.464609+0000","last_fullsized":"2024-02-26T10:44:39.464609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:13:01.326122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406187+0000","last_change":"2024-02-26T10:42:47.246231+0000","last_active":"2024-02-26T10:44:39.406187+0000","last_peered":"2024-02-26T10:44:39.406187+0000","last_clean":"2024-02-26T10:44:39.406187+0000","last_became_active":"2024-02-26T10:42:47.245739+0000","last_became_peered":"2024-02-26T10:42:47.245739+0000","last_unstale":"2024-02-26T10:44:39.406187+0000","last_undegraded":"2024-02-26T10:44:39.406187+0000","last_fullsized":"2024-02-26T10:44:39.406187+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:30.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235526+0000","last_change":"2024-02-26T10:42:41.603126+0000","last_active":"2024-02-26T10:44:39.235526+0000","last_peered":"2024-02-26T10:44:39.235526+0000","last_clean":"2024-02-26T10:44:39.235526+0000","last_became_active":"2024-02-26T10:42:41.602790+0000","last_became_peered":"2024-02-26T10:42:41.602790+0000","last_unstale":"2024-02-26T10:44:39.235526+0000","last_undegraded":"2024-02-26T10:44:39.235526+0000","last_fullsized":"2024-02-26T10:44:39.235526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:31:58.908209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528535+0000","last_change":"2024-02-26T10:43:38.459527+0000","last_active":"2024-02-26T10:44:39.528535+0000","last_peered":"2024-02-26T10:44:39.528535+0000","last_clean":"2024-02-26T10:44:39.528535+0000","last_became_active":"2024-02-26T10:43:38.459150+0000","last_became_peered":"2024-02-26T10:43:38.459150+0000","last_unstale":"2024-02-26T10:44:39.528535+0000","last_undegraded":"2024-02-26T10:44:39.528535+0000","last_fullsized":"2024-02-26T10:44:39.528535+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:21:54.028328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239304+0000","last_change":"2024-02-26T10:42:47.250075+0000","last_active":"2024-02-26T10:44:39.239304+0000","last_peered":"2024-02-26T10:44:39.239304+0000","last_clean":"2024-02-26T10:44:39.239304+0000","last_became_active":"2024-02-26T10:42:47.249490+0000","last_became_peered":"2024-02-26T10:42:47.249490+0000","last_unstale":"2024-02-26T10:44:39.239304+0000","last_undegraded":"2024-02-26T10:44:39.239304+0000","last_fullsized":"2024-02-26T10:44:39.239304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:42.659282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557237+0000","last_change":"2024-02-26T10:42:41.615314+0000","last_active":"2024-02-26T10:44:39.557237+0000","last_peered":"2024-02-26T10:44:39.557237+0000","last_clean":"2024-02-26T10:44:39.557237+0000","last_became_active":"2024-02-26T10:42:41.615091+0000","last_became_peered":"2024-02-26T10:42:41.615091+0000","last_unstale":"2024-02-26T10:44:39.557237+0000","last_undegraded":"2024-02-26T10:44:39.557237+0000","last_fullsized":"2024-02-26T10:44:39.557237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:41:52.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464449+0000","last_change":"2024-02-26T10:42:42.176609+0000","last_active":"2024-02-26T10:44:39.464449+0000","last_peered":"2024-02-26T10:44:39.464449+0000","last_clean":"2024-02-26T10:44:39.464449+0000","last_became_active":"2024-02-26T10:42:42.176311+0000","last_became_peered":"2024-02-26T10:42:42.176311+0000","last_unstale":"2024-02-26T10:44:39.464449+0000","last_undegraded":"2024-02-26T10:44:39.464449+0000","last_fullsized":"2024-02-26T10:44:39.464449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:25:38.415311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406280+0000","last_change":"2024-02-26T10:42:47.250255+0000","last_active":"2024-02-26T10:44:39.406280+0000","last_peered":"2024-02-26T10:44:39.406280+0000","last_clean":"2024-02-26T10:44:39.406280+0000","last_became_active":"2024-02-26T10:42:47.249465+0000","last_became_peered":"2024-02-26T10:42:47.249465+0000","last_unstale":"2024-02-26T10:44:39.406280+0000","last_undegraded":"2024-02-26T10:44:39.406280+0000","last_fullsized":"2024-02-26T10:44:39.406280+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:23.173863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'39","reported_seq":397,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:42.656270+0000","last_change":"2024-02-26T10:43:38.460725+0000","last_active":"2024-02-26T10:45:42.656270+0000","last_peered":"2024-02-26T10:45:42.656270+0000","last_clean":"2024-02-26T10:45:42.656270+0000","last_became_active":"2024-02-26T10:43:38.451900+0000","last_became_peered":"2024-02-26T10:43:38.451900+0000","last_unstale":"2024-02-26T10:45:42.656270+0000","last_undegraded":"2024-02-26T10:45:42.656270+0000","last_fullsized":"2024-02-26T10:45:42.656270+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:35:09.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71468,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":303,"num_read_kb":221,"num_write":40,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":62625,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234956+0000","last_change":"2024-02-26T10:42:42.168174+0000","last_active":"2024-02-26T10:44:39.234956+0000","last_peered":"2024-02-26T10:44:39.234956+0000","last_clean":"2024-02-26T10:44:39.234956+0000","last_became_active":"2024-02-26T10:42:42.167840+0000","last_became_peered":"2024-02-26T10:42:42.167840+0000","last_unstale":"2024-02-26T10:44:39.234956+0000","last_undegraded":"2024-02-26T10:44:39.234956+0000","last_fullsized":"2024-02-26T10:44:39.234956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:43.079139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557658+0000","last_change":"2024-02-26T10:42:47.230048+0000","last_active":"2024-02-26T10:44:39.557658+0000","last_peered":"2024-02-26T10:44:39.557658+0000","last_clean":"2024-02-26T10:44:39.557658+0000","last_became_active":"2024-02-26T10:42:47.229396+0000","last_became_peered":"2024-02-26T10:42:47.229396+0000","last_unstale":"2024-02-26T10:44:39.557658+0000","last_undegraded":"2024-02-26T10:44:39.557658+0000","last_fullsized":"2024-02-26T10:44:39.557658+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:57.804297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238511+0000","last_change":"2024-02-26T10:42:41.614413+0000","last_active":"2024-02-26T10:44:39.238511+0000","last_peered":"2024-02-26T10:44:39.238511+0000","last_clean":"2024-02-26T10:44:39.238511+0000","last_became_active":"2024-02-26T10:42:41.614065+0000","last_became_peered":"2024-02-26T10:42:41.614065+0000","last_unstale":"2024-02-26T10:44:39.238511+0000","last_undegraded":"2024-02-26T10:44:39.238511+0000","last_fullsized":"2024-02-26T10:44:39.238511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:50:20.552340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235996+0000","last_change":"2024-02-26T10:42:42.188140+0000","last_active":"2024-02-26T10:44:39.235996+0000","last_peered":"2024-02-26T10:44:39.235996+0000","last_clean":"2024-02-26T10:44:39.235996+0000","last_became_active":"2024-02-26T10:42:42.187055+0000","last_became_peered":"2024-02-26T10:42:42.187055+0000","last_unstale":"2024-02-26T10:44:39.235996+0000","last_undegraded":"2024-02-26T10:44:39.235996+0000","last_fullsized":"2024-02-26T10:44:39.235996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:49:43.568562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308124+0000","last_change":"2024-02-26T10:43:38.450681+0000","last_active":"2024-02-26T10:44:39.308124+0000","last_peered":"2024-02-26T10:44:39.308124+0000","last_clean":"2024-02-26T10:44:39.308124+0000","last_became_active":"2024-02-26T10:43:38.450120+0000","last_became_peered":"2024-02-26T10:43:38.450120+0000","last_unstale":"2024-02-26T10:44:39.308124+0000","last_undegraded":"2024-02-26T10:44:39.308124+0000","last_fullsized":"2024-02-26T10:44:39.308124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:10.983700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556913+0000","last_change":"2024-02-26T10:42:41.612337+0000","last_active":"2024-02-26T10:44:39.556913+0000","last_peered":"2024-02-26T10:44:39.556913+0000","last_clean":"2024-02-26T10:44:39.556913+0000","last_became_active":"2024-02-26T10:42:41.611920+0000","last_became_peered":"2024-02-26T10:42:41.611920+0000","last_unstale":"2024-02-26T10:44:39.556913+0000","last_undegraded":"2024-02-26T10:44:39.556913+0000","last_fullsized":"2024-02-26T10:44:39.556913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:29:13.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235333+0000","last_change":"2024-02-26T10:43:38.456919+0000","last_active":"2024-02-26T10:44:39.235333+0000","last_peered":"2024-02-26T10:44:39.235333+0000","last_clean":"2024-02-26T10:44:39.235333+0000","last_became_active":"2024-02-26T10:43:38.456461+0000","last_became_peered":"2024-02-26T10:43:38.456461+0000","last_unstale":"2024-02-26T10:44:39.235333+0000","last_undegraded":"2024-02-26T10:44:39.235333+0000","last_fullsized":"2024-02-26T10:44:39.235333+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:44.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.407141+0000","last_change":"2024-02-26T10:42:47.254585+0000","last_active":"2024-02-26T10:44:39.407141+0000","last_peered":"2024-02-26T10:44:39.407141+0000","last_clean":"2024-02-26T10:44:39.407141+0000","last_became_active":"2024-02-26T10:42:47.253870+0000","last_became_peered":"2024-02-26T10:42:47.253870+0000","last_unstale":"2024-02-26T10:44:39.407141+0000","last_undegraded":"2024-02-26T10:44:39.407141+0000","last_fullsized":"2024-02-26T10:44:39.407141+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:49.276906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":190,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:41.338041+0000","last_change":"2024-02-26T10:42:41.600901+0000","last_active":"2024-02-26T10:45:41.338041+0000","last_peered":"2024-02-26T10:45:41.338041+0000","last_clean":"2024-02-26T10:45:41.338041+0000","last_became_active":"2024-02-26T10:42:41.600018+0000","last_became_peered":"2024-02-26T10:42:41.600018+0000","last_unstale":"2024-02-26T10:45:41.338041+0000","last_undegraded":"2024-02-26T10:45:41.338041+0000","last_fullsized":"2024-02-26T10:45:41.338041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:07.876694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235012+0000","last_change":"2024-02-26T10:42:42.168702+0000","last_active":"2024-02-26T10:44:39.235012+0000","last_peered":"2024-02-26T10:44:39.235012+0000","last_clean":"2024-02-26T10:44:39.235012+0000","last_became_active":"2024-02-26T10:42:42.168420+0000","last_became_peered":"2024-02-26T10:42:42.168420+0000","last_unstale":"2024-02-26T10:44:39.235012+0000","last_undegraded":"2024-02-26T10:44:39.235012+0000","last_fullsized":"2024-02-26T10:44:39.235012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:35.111699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236688+0000","last_change":"2024-02-26T10:42:47.250714+0000","last_active":"2024-02-26T10:44:39.236688+0000","last_peered":"2024-02-26T10:44:39.236688+0000","last_clean":"2024-02-26T10:44:39.236688+0000","last_became_active":"2024-02-26T10:42:47.249506+0000","last_became_peered":"2024-02-26T10:42:47.249506+0000","last_unstale":"2024-02-26T10:44:39.236688+0000","last_undegraded":"2024-02-26T10:44:39.236688+0000","last_fullsized":"2024-02-26T10:44:39.236688+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:05.058052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307933+0000","last_change":"2024-02-26T10:43:38.472808+0000","last_active":"2024-02-26T10:44:39.307933+0000","last_peered":"2024-02-26T10:44:39.307933+0000","last_clean":"2024-02-26T10:44:39.307933+0000","last_became_active":"2024-02-26T10:43:38.456509+0000","last_became_peered":"2024-02-26T10:43:38.456509+0000","last_unstale":"2024-02-26T10:44:39.307933+0000","last_undegraded":"2024-02-26T10:44:39.307933+0000","last_fullsized":"2024-02-26T10:44:39.307933+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:03:06.127386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556435+0000","last_change":"2024-02-26T10:42:42.172845+0000","last_active":"2024-02-26T10:44:39.556435+0000","last_peered":"2024-02-26T10:44:39.556435+0000","last_clean":"2024-02-26T10:44:39.556435+0000","last_became_active":"2024-02-26T10:42:42.171263+0000","last_became_peered":"2024-02-26T10:42:42.171263+0000","last_unstale":"2024-02-26T10:44:39.556435+0000","last_undegraded":"2024-02-26T10:44:39.556435+0000","last_fullsized":"2024-02-26T10:44:39.556435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:21.439474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556408+0000","last_change":"2024-02-26T10:42:47.236434+0000","last_active":"2024-02-26T10:44:39.556408+0000","last_peered":"2024-02-26T10:44:39.556408+0000","last_clean":"2024-02-26T10:44:39.556408+0000","last_became_active":"2024-02-26T10:42:47.236040+0000","last_became_peered":"2024-02-26T10:42:47.236040+0000","last_unstale":"2024-02-26T10:44:39.556408+0000","last_undegraded":"2024-02-26T10:44:39.556408+0000","last_fullsized":"2024-02-26T10:44:39.556408+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:20:23.230348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235239+0000","last_change":"2024-02-26T10:42:41.604208+0000","last_active":"2024-02-26T10:44:39.235239+0000","last_peered":"2024-02-26T10:44:39.235239+0000","last_clean":"2024-02-26T10:44:39.235239+0000","last_became_active":"2024-02-26T10:42:41.603894+0000","last_became_peered":"2024-02-26T10:42:41.603894+0000","last_unstale":"2024-02-26T10:44:39.235239+0000","last_undegraded":"2024-02-26T10:44:39.235239+0000","last_fullsized":"2024-02-26T10:44:39.235239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:57:13.775868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308423+0000","last_change":"2024-02-26T10:42:42.169395+0000","last_active":"2024-02-26T10:44:39.308423+0000","last_peered":"2024-02-26T10:44:39.308423+0000","last_clean":"2024-02-26T10:44:39.308423+0000","last_became_active":"2024-02-26T10:42:42.168686+0000","last_became_peered":"2024-02-26T10:42:42.168686+0000","last_unstale":"2024-02-26T10:44:39.308423+0000","last_undegraded":"2024-02-26T10:44:39.308423+0000","last_fullsized":"2024-02-26T10:44:39.308423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:21:35.085188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234671+0000","last_change":"2024-02-26T10:42:47.252119+0000","last_active":"2024-02-26T10:44:39.234671+0000","last_peered":"2024-02-26T10:44:39.234671+0000","last_clean":"2024-02-26T10:44:39.234671+0000","last_became_active":"2024-02-26T10:42:47.251723+0000","last_became_peered":"2024-02-26T10:42:47.251723+0000","last_unstale":"2024-02-26T10:44:39.234671+0000","last_undegraded":"2024-02-26T10:44:39.234671+0000","last_fullsized":"2024-02-26T10:44:39.234671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:17.130806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408624+0000","last_change":"2024-02-26T10:42:41.621393+0000","last_active":"2024-02-26T10:44:39.408624+0000","last_peered":"2024-02-26T10:44:39.408624+0000","last_clean":"2024-02-26T10:44:39.408624+0000","last_became_active":"2024-02-26T10:42:41.621008+0000","last_became_peered":"2024-02-26T10:42:41.621008+0000","last_unstale":"2024-02-26T10:44:39.408624+0000","last_undegraded":"2024-02-26T10:44:39.408624+0000","last_fullsized":"2024-02-26T10:44:39.408624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:46.979854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257087+0000","last_change":"2024-02-26T10:43:38.458926+0000","last_active":"2024-02-26T10:44:39.257087+0000","last_peered":"2024-02-26T10:44:39.257087+0000","last_clean":"2024-02-26T10:44:39.257087+0000","last_became_active":"2024-02-26T10:43:38.458537+0000","last_became_peered":"2024-02-26T10:43:38.458537+0000","last_unstale":"2024-02-26T10:44:39.257087+0000","last_undegraded":"2024-02-26T10:44:39.257087+0000","last_fullsized":"2024-02-26T10:44:39.257087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:08.204646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236653+0000","last_change":"2024-02-26T10:42:47.238520+0000","last_active":"2024-02-26T10:44:39.236653+0000","last_peered":"2024-02-26T10:44:39.236653+0000","last_clean":"2024-02-26T10:44:39.236653+0000","last_became_active":"2024-02-26T10:42:47.237213+0000","last_became_peered":"2024-02-26T10:42:47.237213+0000","last_unstale":"2024-02-26T10:44:39.236653+0000","last_undegraded":"2024-02-26T10:44:39.236653+0000","last_fullsized":"2024-02-26T10:44:39.236653+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:29.283850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235085+0000","last_change":"2024-02-26T10:42:41.609479+0000","last_active":"2024-02-26T10:44:39.235085+0000","last_peered":"2024-02-26T10:44:39.235085+0000","last_clean":"2024-02-26T10:44:39.235085+0000","last_became_active":"2024-02-26T10:42:41.609084+0000","last_became_peered":"2024-02-26T10:42:41.609084+0000","last_unstale":"2024-02-26T10:44:39.235085+0000","last_undegraded":"2024-02-26T10:44:39.235085+0000","last_fullsized":"2024-02-26T10:44:39.235085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:32.412507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257066+0000","last_change":"2024-02-26T10:42:42.174985+0000","last_active":"2024-02-26T10:44:39.257066+0000","last_peered":"2024-02-26T10:44:39.257066+0000","last_clean":"2024-02-26T10:44:39.257066+0000","last_became_active":"2024-02-26T10:42:42.174694+0000","last_became_peered":"2024-02-26T10:42:42.174694+0000","last_unstale":"2024-02-26T10:44:39.257066+0000","last_undegraded":"2024-02-26T10:44:39.257066+0000","last_fullsized":"2024-02-26T10:44:39.257066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:10.000154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237057+0000","last_change":"2024-02-26T10:42:47.242587+0000","last_active":"2024-02-26T10:44:39.237057+0000","last_peered":"2024-02-26T10:44:39.237057+0000","last_clean":"2024-02-26T10:44:39.237057+0000","last_became_active":"2024-02-26T10:42:47.242099+0000","last_became_peered":"2024-02-26T10:42:47.242099+0000","last_unstale":"2024-02-26T10:44:39.237057+0000","last_undegraded":"2024-02-26T10:44:39.237057+0000","last_fullsized":"2024-02-26T10:44:39.237057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:38:42.852080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556815+0000","last_change":"2024-02-26T10:42:42.170613+0000","last_active":"2024-02-26T10:44:39.556815+0000","last_peered":"2024-02-26T10:44:39.556815+0000","last_clean":"2024-02-26T10:44:39.556815+0000","last_became_active":"2024-02-26T10:42:42.170317+0000","last_became_peered":"2024-02-26T10:42:42.170317+0000","last_unstale":"2024-02-26T10:44:39.556815+0000","last_undegraded":"2024-02-26T10:44:39.556815+0000","last_fullsized":"2024-02-26T10:44:39.556815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:29:44.276718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":187,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:42.339228+0000","last_change":"2024-02-26T10:42:41.617447+0000","last_active":"2024-02-26T10:45:42.339228+0000","last_peered":"2024-02-26T10:45:42.339228+0000","last_clean":"2024-02-26T10:45:42.339228+0000","last_became_active":"2024-02-26T10:42:41.617107+0000","last_became_peered":"2024-02-26T10:42:41.617107+0000","last_unstale":"2024-02-26T10:45:42.339228+0000","last_undegraded":"2024-02-26T10:45:42.339228+0000","last_fullsized":"2024-02-26T10:45:42.339228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:06.052364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235432+0000","last_change":"2024-02-26T10:42:47.251971+0000","last_active":"2024-02-26T10:44:39.235432+0000","last_peered":"2024-02-26T10:44:39.235432+0000","last_clean":"2024-02-26T10:44:39.235432+0000","last_became_active":"2024-02-26T10:42:47.251506+0000","last_became_peered":"2024-02-26T10:42:47.251506+0000","last_unstale":"2024-02-26T10:44:39.235432+0000","last_undegraded":"2024-02-26T10:44:39.235432+0000","last_fullsized":"2024-02-26T10:44:39.235432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:37.029348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041442+0000","last_change":"2024-02-26T10:42:42.194414+0000","last_active":"2024-02-26T10:44:40.041442+0000","last_peered":"2024-02-26T10:44:40.041442+0000","last_clean":"2024-02-26T10:44:40.041442+0000","last_became_active":"2024-02-26T10:42:42.193959+0000","last_became_peered":"2024-02-26T10:42:42.193959+0000","last_unstale":"2024-02-26T10:44:40.041442+0000","last_undegraded":"2024-02-26T10:44:40.041442+0000","last_fullsized":"2024-02-26T10:44:40.041442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:50.398674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239392+0000","last_change":"2024-02-26T10:42:41.607947+0000","last_active":"2024-02-26T10:44:39.239392+0000","last_peered":"2024-02-26T10:44:39.239392+0000","last_clean":"2024-02-26T10:44:39.239392+0000","last_became_active":"2024-02-26T10:42:41.607522+0000","last_became_peered":"2024-02-26T10:42:41.607522+0000","last_unstale":"2024-02-26T10:44:39.239392+0000","last_undegraded":"2024-02-26T10:44:39.239392+0000","last_fullsized":"2024-02-26T10:44:39.239392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:32:36.647621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234832+0000","last_change":"2024-02-26T10:41:21.177202+0000","last_active":"2024-02-26T10:44:39.234832+0000","last_peered":"2024-02-26T10:44:39.234832+0000","last_clean":"2024-02-26T10:44:39.234832+0000","last_became_active":"2024-02-26T10:41:21.176762+0000","last_became_peered":"2024-02-26T10:41:21.176762+0000","last_unstale":"2024-02-26T10:44:39.234832+0000","last_undegraded":"2024-02-26T10:44:39.234832+0000","last_fullsized":"2024-02-26T10:44:39.234832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_clean_scrub_stamp":"2024-02-26T10:37:25.424790+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:19.619104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.555996+0000","last_change":"2024-02-26T10:42:47.244346+0000","last_active":"2024-02-26T10:44:39.555996+0000","last_peered":"2024-02-26T10:44:39.555996+0000","last_clean":"2024-02-26T10:44:39.555996+0000","last_became_active":"2024-02-26T10:42:47.243872+0000","last_became_peered":"2024-02-26T10:42:47.243872+0000","last_unstale":"2024-02-26T10:44:39.555996+0000","last_undegraded":"2024-02-26T10:44:39.555996+0000","last_fullsized":"2024-02-26T10:44:39.555996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:36.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235371+0000","last_change":"2024-02-26T10:42:42.166266+0000","last_active":"2024-02-26T10:44:39.235371+0000","last_peered":"2024-02-26T10:44:39.235371+0000","last_clean":"2024-02-26T10:44:39.235371+0000","last_became_active":"2024-02-26T10:42:42.165721+0000","last_became_peered":"2024-02-26T10:42:42.165721+0000","last_unstale":"2024-02-26T10:44:39.235371+0000","last_undegraded":"2024-02-26T10:44:39.235371+0000","last_fullsized":"2024-02-26T10:44:39.235371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:32.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234981+0000","last_change":"2024-02-26T10:42:41.606268+0000","last_active":"2024-02-26T10:44:39.234981+0000","last_peered":"2024-02-26T10:44:39.234981+0000","last_clean":"2024-02-26T10:44:39.234981+0000","last_became_active":"2024-02-26T10:42:41.605950+0000","last_became_peered":"2024-02-26T10:42:41.605950+0000","last_unstale":"2024-02-26T10:44:39.234981+0000","last_undegraded":"2024-02-26T10:44:39.234981+0000","last_fullsized":"2024-02-26T10:44:39.234981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:40:03.616987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307850+0000","last_change":"2024-02-26T10:42:47.233203+0000","last_active":"2024-02-26T10:44:39.307850+0000","last_peered":"2024-02-26T10:44:39.307850+0000","last_clean":"2024-02-26T10:44:39.307850+0000","last_became_active":"2024-02-26T10:42:47.232705+0000","last_became_peered":"2024-02-26T10:42:47.232705+0000","last_unstale":"2024-02-26T10:44:39.307850+0000","last_undegraded":"2024-02-26T10:44:39.307850+0000","last_fullsized":"2024-02-26T10:44:39.307850+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:53.818901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308265+0000","last_change":"2024-02-26T10:43:38.479911+0000","last_active":"2024-02-26T10:44:39.308265+0000","last_peered":"2024-02-26T10:44:39.308265+0000","last_clean":"2024-02-26T10:44:39.308265+0000","last_became_active":"2024-02-26T10:43:38.458874+0000","last_became_peered":"2024-02-26T10:43:38.458874+0000","last_unstale":"2024-02-26T10:44:39.308265+0000","last_undegraded":"2024-02-26T10:44:39.308265+0000","last_fullsized":"2024-02-26T10:44:39.308265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:56.093775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237091+0000","last_change":"2024-02-26T10:42:42.167674+0000","last_active":"2024-02-26T10:44:39.237091+0000","last_peered":"2024-02-26T10:44:39.237091+0000","last_clean":"2024-02-26T10:44:39.237091+0000","last_became_active":"2024-02-26T10:42:42.167224+0000","last_became_peered":"2024-02-26T10:42:42.167224+0000","last_unstale":"2024-02-26T10:44:39.237091+0000","last_undegraded":"2024-02-26T10:44:39.237091+0000","last_fullsized":"2024-02-26T10:44:39.237091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:49:20.433913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.039668+0000","last_change":"2024-02-26T10:42:47.244076+0000","last_active":"2024-02-26T10:44:40.039668+0000","last_peered":"2024-02-26T10:44:40.039668+0000","last_clean":"2024-02-26T10:44:40.039668+0000","last_became_active":"2024-02-26T10:42:47.243569+0000","last_became_peered":"2024-02-26T10:42:47.243569+0000","last_unstale":"2024-02-26T10:44:40.039668+0000","last_undegraded":"2024-02-26T10:44:40.039668+0000","last_fullsized":"2024-02-26T10:44:40.039668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:46:19.855621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'12","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:55.201582+0000","last_change":"2024-02-26T10:42:41.623604+0000","last_active":"2024-02-26T10:44:55.201582+0000","last_peered":"2024-02-26T10:44:55.201582+0000","last_clean":"2024-02-26T10:44:55.201582+0000","last_became_active":"2024-02-26T10:42:41.623272+0000","last_became_peered":"2024-02-26T10:42:41.623272+0000","last_unstale":"2024-02-26T10:44:55.201582+0000","last_undegraded":"2024-02-26T10:44:55.201582+0000","last_fullsized":"2024-02-26T10:44:55.201582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:13:57.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042221+0000","last_change":"2024-02-26T10:42:42.186300+0000","last_active":"2024-02-26T10:44:40.042221+0000","last_peered":"2024-02-26T10:44:40.042221+0000","last_clean":"2024-02-26T10:44:40.042221+0000","last_became_active":"2024-02-26T10:42:42.185965+0000","last_became_peered":"2024-02-26T10:42:42.185965+0000","last_unstale":"2024-02-26T10:44:40.042221+0000","last_undegraded":"2024-02-26T10:44:40.042221+0000","last_fullsized":"2024-02-26T10:44:40.042221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:06:50.352308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238335+0000","last_change":"2024-02-26T10:42:47.227753+0000","last_active":"2024-02-26T10:44:39.238335+0000","last_peered":"2024-02-26T10:44:39.238335+0000","last_clean":"2024-02-26T10:44:39.238335+0000","last_became_active":"2024-02-26T10:42:47.227300+0000","last_became_peered":"2024-02-26T10:42:47.227300+0000","last_unstale":"2024-02-26T10:44:39.238335+0000","last_undegraded":"2024-02-26T10:44:39.238335+0000","last_fullsized":"2024-02-26T10:44:39.238335+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:08.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239810+0000","last_change":"2024-02-26T10:42:41.602960+0000","last_active":"2024-02-26T10:44:39.239810+0000","last_peered":"2024-02-26T10:44:39.239810+0000","last_clean":"2024-02-26T10:44:39.239810+0000","last_became_active":"2024-02-26T10:42:41.602676+0000","last_became_peered":"2024-02-26T10:42:41.602676+0000","last_unstale":"2024-02-26T10:44:39.239810+0000","last_undegraded":"2024-02-26T10:44:39.239810+0000","last_fullsized":"2024-02-26T10:44:39.239810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:57.997612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528737+0000","last_change":"2024-02-26T10:42:42.185451+0000","last_active":"2024-02-26T10:44:39.528737+0000","last_peered":"2024-02-26T10:44:39.528737+0000","last_clean":"2024-02-26T10:44:39.528737+0000","last_became_active":"2024-02-26T10:42:42.185116+0000","last_became_peered":"2024-02-26T10:42:42.185116+0000","last_unstale":"2024-02-26T10:44:39.528737+0000","last_undegraded":"2024-02-26T10:44:39.528737+0000","last_fullsized":"2024-02-26T10:44:39.528737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:43:39.823370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239619+0000","last_change":"2024-02-26T10:42:47.250682+0000","last_active":"2024-02-26T10:44:39.239619+0000","last_peered":"2024-02-26T10:44:39.239619+0000","last_clean":"2024-02-26T10:44:39.239619+0000","last_became_active":"2024-02-26T10:42:47.249524+0000","last_became_peered":"2024-02-26T10:42:47.249524+0000","last_unstale":"2024-02-26T10:44:39.239619+0000","last_undegraded":"2024-02-26T10:44:39.239619+0000","last_fullsized":"2024-02-26T10:44:39.239619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:07:01.307206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463955+0000","last_change":"2024-02-26T10:42:41.620764+0000","last_active":"2024-02-26T10:44:39.463955+0000","last_peered":"2024-02-26T10:44:39.463955+0000","last_clean":"2024-02-26T10:44:39.463955+0000","last_became_active":"2024-02-26T10:42:41.620500+0000","last_became_peered":"2024-02-26T10:42:41.620500+0000","last_unstale":"2024-02-26T10:44:39.463955+0000","last_undegraded":"2024-02-26T10:44:39.463955+0000","last_fullsized":"2024-02-26T10:44:39.463955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:15.542396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234462+0000","last_change":"2024-02-26T10:42:42.165060+0000","last_active":"2024-02-26T10:44:39.234462+0000","last_peered":"2024-02-26T10:44:39.234462+0000","last_clean":"2024-02-26T10:44:39.234462+0000","last_became_active":"2024-02-26T10:42:42.164733+0000","last_became_peered":"2024-02-26T10:42:42.164733+0000","last_unstale":"2024-02-26T10:44:39.234462+0000","last_undegraded":"2024-02-26T10:44:39.234462+0000","last_fullsized":"2024-02-26T10:44:39.234462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:34.190747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.044327+0000","last_change":"2024-02-26T10:44:40.044327+0000","last_active":"2024-02-26T10:44:40.044327+0000","last_peered":"2024-02-26T10:44:40.044327+0000","last_clean":"2024-02-26T10:44:40.044327+0000","last_became_active":"2024-02-26T10:44:40.043776+0000","last_became_peered":"2024-02-26T10:44:40.043776+0000","last_unstale":"2024-02-26T10:44:40.044327+0000","last_undegraded":"2024-02-26T10:44:40.044327+0000","last_fullsized":"2024-02-26T10:44:40.044327+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:50:51.502411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040404+0000","last_change":"2024-02-26T10:42:41.624940+0000","last_active":"2024-02-26T10:44:40.040404+0000","last_peered":"2024-02-26T10:44:40.040404+0000","last_clean":"2024-02-26T10:44:40.040404+0000","last_became_active":"2024-02-26T10:42:41.624538+0000","last_became_peered":"2024-02-26T10:42:41.624538+0000","last_unstale":"2024-02-26T10:44:40.040404+0000","last_undegraded":"2024-02-26T10:44:40.040404+0000","last_fullsized":"2024-02-26T10:44:40.040404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:10.052806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308063+0000","last_change":"2024-02-26T10:42:42.174932+0000","last_active":"2024-02-26T10:44:39.308063+0000","last_peered":"2024-02-26T10:44:39.308063+0000","last_clean":"2024-02-26T10:44:39.308063+0000","last_became_active":"2024-02-26T10:42:42.174566+0000","last_became_peered":"2024-02-26T10:42:42.174566+0000","last_unstale":"2024-02-26T10:44:39.308063+0000","last_undegraded":"2024-02-26T10:44:39.308063+0000","last_fullsized":"2024-02-26T10:44:39.308063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:03.242487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239861+0000","last_change":"2024-02-26T10:42:47.250023+0000","last_active":"2024-02-26T10:44:39.239861+0000","last_peered":"2024-02-26T10:44:39.239861+0000","last_clean":"2024-02-26T10:44:39.239861+0000","last_became_active":"2024-02-26T10:42:47.249571+0000","last_became_peered":"2024-02-26T10:42:47.249571+0000","last_unstale":"2024-02-26T10:44:39.239861+0000","last_undegraded":"2024-02-26T10:44:39.239861+0000","last_fullsized":"2024-02-26T10:44:39.239861+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:53.666184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:53.651538+0000","last_change":"2024-02-26T10:42:41.614403+0000","last_active":"2024-02-26T10:44:53.651538+0000","last_peered":"2024-02-26T10:44:53.651538+0000","last_clean":"2024-02-26T10:44:53.651538+0000","last_became_active":"2024-02-26T10:42:41.614057+0000","last_became_peered":"2024-02-26T10:42:41.614057+0000","last_unstale":"2024-02-26T10:44:53.651538+0000","last_undegraded":"2024-02-26T10:44:53.651538+0000","last_fullsized":"2024-02-26T10:44:53.651538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:16:54.739090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041052+0000","last_change":"2024-02-26T10:42:42.187465+0000","last_active":"2024-02-26T10:44:40.041052+0000","last_peered":"2024-02-26T10:44:40.041052+0000","last_clean":"2024-02-26T10:44:40.041052+0000","last_became_active":"2024-02-26T10:42:42.182310+0000","last_became_peered":"2024-02-26T10:42:42.182310+0000","last_unstale":"2024-02-26T10:44:40.041052+0000","last_undegraded":"2024-02-26T10:44:40.041052+0000","last_fullsized":"2024-02-26T10:44:40.041052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:01:56.645330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239438+0000","last_change":"2024-02-26T10:42:47.241751+0000","last_active":"2024-02-26T10:44:39.239438+0000","last_peered":"2024-02-26T10:44:39.239438+0000","last_clean":"2024-02-26T10:44:39.239438+0000","last_became_active":"2024-02-26T10:42:47.241271+0000","last_became_peered":"2024-02-26T10:42:47.241271+0000","last_unstale":"2024-02-26T10:44:39.239438+0000","last_undegraded":"2024-02-26T10:44:39.239438+0000","last_fullsized":"2024-02-26T10:44:39.239438+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:40:39.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234633+0000","last_change":"2024-02-26T10:42:41.607243+0000","last_active":"2024-02-26T10:44:39.234633+0000","last_peered":"2024-02-26T10:44:39.234633+0000","last_clean":"2024-02-26T10:44:39.234633+0000","last_became_active":"2024-02-26T10:42:41.606921+0000","last_became_peered":"2024-02-26T10:42:41.606921+0000","last_unstale":"2024-02-26T10:44:39.234633+0000","last_undegraded":"2024-02-26T10:44:39.234633+0000","last_fullsized":"2024-02-26T10:44:39.234633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:04.626391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235242+0000","last_change":"2024-02-26T10:42:42.178240+0000","last_active":"2024-02-26T10:44:39.235242+0000","last_peered":"2024-02-26T10:44:39.235242+0000","last_clean":"2024-02-26T10:44:39.235242+0000","last_became_active":"2024-02-26T10:42:42.177790+0000","last_became_peered":"2024-02-26T10:42:42.177790+0000","last_unstale":"2024-02-26T10:44:39.235242+0000","last_undegraded":"2024-02-26T10:44:39.235242+0000","last_fullsized":"2024-02-26T10:44:39.235242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:24.110406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463760+0000","last_change":"2024-02-26T10:42:47.233120+0000","last_active":"2024-02-26T10:44:39.463760+0000","last_peered":"2024-02-26T10:44:39.463760+0000","last_clean":"2024-02-26T10:44:39.463760+0000","last_became_active":"2024-02-26T10:42:47.232630+0000","last_became_peered":"2024-02-26T10:42:47.232630+0000","last_unstale":"2024-02-26T10:44:39.463760+0000","last_undegraded":"2024-02-26T10:44:39.463760+0000","last_fullsized":"2024-02-26T10:44:39.463760+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:05:03.023227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239351+0000","last_change":"2024-02-26T10:43:38.477127+0000","last_active":"2024-02-26T10:44:39.239351+0000","last_peered":"2024-02-26T10:44:39.239351+0000","last_clean":"2024-02-26T10:44:39.239351+0000","last_became_active":"2024-02-26T10:43:38.460135+0000","last_became_peered":"2024-02-26T10:43:38.460135+0000","last_unstale":"2024-02-26T10:44:39.239351+0000","last_undegraded":"2024-02-26T10:44:39.239351+0000","last_fullsized":"2024-02-26T10:44:39.239351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:13:16.521943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235617+0000","last_change":"2024-02-26T10:42:42.171821+0000","last_active":"2024-02-26T10:44:39.235617+0000","last_peered":"2024-02-26T10:44:39.235617+0000","last_clean":"2024-02-26T10:44:39.235617+0000","last_became_active":"2024-02-26T10:42:42.171523+0000","last_became_peered":"2024-02-26T10:42:42.171523+0000","last_unstale":"2024-02-26T10:44:39.235617+0000","last_undegraded":"2024-02-26T10:44:39.235617+0000","last_fullsized":"2024-02-26T10:44:39.235617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:00.877998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526701+0000","last_change":"2024-02-26T10:42:47.250072+0000","last_active":"2024-02-26T10:44:39.526701+0000","last_peered":"2024-02-26T10:44:39.526701+0000","last_clean":"2024-02-26T10:44:39.526701+0000","last_became_active":"2024-02-26T10:42:47.249570+0000","last_became_peered":"2024-02-26T10:42:47.249570+0000","last_unstale":"2024-02-26T10:44:39.526701+0000","last_undegraded":"2024-02-26T10:44:39.526701+0000","last_fullsized":"2024-02-26T10:44:39.526701+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:43:03.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557531+0000","last_change":"2024-02-26T10:42:41.614686+0000","last_active":"2024-02-26T10:44:39.557531+0000","last_peered":"2024-02-26T10:44:39.557531+0000","last_clean":"2024-02-26T10:44:39.557531+0000","last_became_active":"2024-02-26T10:42:41.614019+0000","last_became_peered":"2024-02-26T10:42:41.614019+0000","last_unstale":"2024-02-26T10:44:39.557531+0000","last_undegraded":"2024-02-26T10:44:39.557531+0000","last_fullsized":"2024-02-26T10:44:39.557531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:13:28.127288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235390+0000","last_change":"2024-02-26T10:42:42.181774+0000","last_active":"2024-02-26T10:44:39.235390+0000","last_peered":"2024-02-26T10:44:39.235390+0000","last_clean":"2024-02-26T10:44:39.235390+0000","last_became_active":"2024-02-26T10:42:42.181412+0000","last_became_peered":"2024-02-26T10:42:42.181412+0000","last_unstale":"2024-02-26T10:44:39.235390+0000","last_undegraded":"2024-02-26T10:44:39.235390+0000","last_fullsized":"2024-02-26T10:44:39.235390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:09.159964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556176+0000","last_change":"2024-02-26T10:42:47.228247+0000","last_active":"2024-02-26T10:44:39.556176+0000","last_peered":"2024-02-26T10:44:39.556176+0000","last_clean":"2024-02-26T10:44:39.556176+0000","last_became_active":"2024-02-26T10:42:47.227569+0000","last_became_peered":"2024-02-26T10:42:47.227569+0000","last_unstale":"2024-02-26T10:44:39.556176+0000","last_undegraded":"2024-02-26T10:44:39.556176+0000","last_fullsized":"2024-02-26T10:44:39.556176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:15.001264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236850+0000","last_change":"2024-02-26T10:42:41.626573+0000","last_active":"2024-02-26T10:44:39.236850+0000","last_peered":"2024-02-26T10:44:39.236850+0000","last_clean":"2024-02-26T10:44:39.236850+0000","last_became_active":"2024-02-26T10:42:41.624091+0000","last_became_peered":"2024-02-26T10:42:41.624091+0000","last_unstale":"2024-02-26T10:44:39.236850+0000","last_undegraded":"2024-02-26T10:44:39.236850+0000","last_fullsized":"2024-02-26T10:44:39.236850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:24:14.077234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235668+0000","last_change":"2024-02-26T10:42:42.166426+0000","last_active":"2024-02-26T10:44:39.235668+0000","last_peered":"2024-02-26T10:44:39.235668+0000","last_clean":"2024-02-26T10:44:39.235668+0000","last_became_active":"2024-02-26T10:42:42.166082+0000","last_became_peered":"2024-02-26T10:42:42.166082+0000","last_unstale":"2024-02-26T10:44:39.235668+0000","last_undegraded":"2024-02-26T10:44:39.235668+0000","last_fullsized":"2024-02-26T10:44:39.235668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:17.136568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308191+0000","last_change":"2024-02-26T10:42:42.168227+0000","last_active":"2024-02-26T10:44:39.308191+0000","last_peered":"2024-02-26T10:44:39.308191+0000","last_clean":"2024-02-26T10:44:39.308191+0000","last_became_active":"2024-02-26T10:42:42.167897+0000","last_became_peered":"2024-02-26T10:42:42.167897+0000","last_unstale":"2024-02-26T10:44:39.308191+0000","last_undegraded":"2024-02-26T10:44:39.308191+0000","last_fullsized":"2024-02-26T10:44:39.308191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:52:37.391022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238723+0000","last_change":"2024-02-26T10:42:42.168640+0000","last_active":"2024-02-26T10:44:39.238723+0000","last_peered":"2024-02-26T10:44:39.238723+0000","last_clean":"2024-02-26T10:44:39.238723+0000","last_became_active":"2024-02-26T10:42:42.168237+0000","last_became_peered":"2024-02-26T10:42:42.168237+0000","last_unstale":"2024-02-26T10:44:39.238723+0000","last_undegraded":"2024-02-26T10:44:39.238723+0000","last_fullsized":"2024-02-26T10:44:39.238723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:14.853730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238359+0000","last_change":"2024-02-26T10:42:42.192703+0000","last_active":"2024-02-26T10:44:39.238359+0000","last_peered":"2024-02-26T10:44:39.238359+0000","last_clean":"2024-02-26T10:44:39.238359+0000","last_became_active":"2024-02-26T10:42:42.191899+0000","last_became_peered":"2024-02-26T10:42:42.191899+0000","last_unstale":"2024-02-26T10:44:39.238359+0000","last_undegraded":"2024-02-26T10:44:39.238359+0000","last_fullsized":"2024-02-26T10:44:39.238359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:40.479523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528040+0000","last_change":"2024-02-26T10:42:42.188709+0000","last_active":"2024-02-26T10:44:39.528040+0000","last_peered":"2024-02-26T10:44:39.528040+0000","last_clean":"2024-02-26T10:44:39.528040+0000","last_became_active":"2024-02-26T10:42:42.188418+0000","last_became_peered":"2024-02-26T10:42:42.188418+0000","last_unstale":"2024-02-26T10:44:39.528040+0000","last_undegraded":"2024-02-26T10:44:39.528040+0000","last_fullsized":"2024-02-26T10:44:39.528040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:54:15.697746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236925+0000","last_change":"2024-02-26T10:42:42.174221+0000","last_active":"2024-02-26T10:44:39.236925+0000","last_peered":"2024-02-26T10:44:39.236925+0000","last_clean":"2024-02-26T10:44:39.236925+0000","last_became_active":"2024-02-26T10:42:42.171308+0000","last_became_peered":"2024-02-26T10:42:42.171308+0000","last_unstale":"2024-02-26T10:44:39.236925+0000","last_undegraded":"2024-02-26T10:44:39.236925+0000","last_fullsized":"2024-02-26T10:44:39.236925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:46.765856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308467+0000","last_change":"2024-02-26T10:42:42.170749+0000","last_active":"2024-02-26T10:44:39.308467+0000","last_peered":"2024-02-26T10:44:39.308467+0000","last_clean":"2024-02-26T10:44:39.308467+0000","last_became_active":"2024-02-26T10:42:42.169966+0000","last_became_peered":"2024-02-26T10:42:42.169966+0000","last_unstale":"2024-02-26T10:44:39.308467+0000","last_undegraded":"2024-02-26T10:44:39.308467+0000","last_fullsized":"2024-02-26T10:44:39.308467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:02.633195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238925+0000","last_change":"2024-02-26T10:42:42.166529+0000","last_active":"2024-02-26T10:44:39.238925+0000","last_peered":"2024-02-26T10:44:39.238925+0000","last_clean":"2024-02-26T10:44:39.238925+0000","last_became_active":"2024-02-26T10:42:42.166235+0000","last_became_peered":"2024-02-26T10:42:42.166235+0000","last_unstale":"2024-02-26T10:44:39.238925+0000","last_undegraded":"2024-02-26T10:44:39.238925+0000","last_fullsized":"2024-02-26T10:44:39.238925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:03.068328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042639+0000","last_change":"2024-02-26T10:42:42.191312+0000","last_active":"2024-02-26T10:44:40.042639+0000","last_peered":"2024-02-26T10:44:40.042639+0000","last_clean":"2024-02-26T10:44:40.042639+0000","last_became_active":"2024-02-26T10:42:42.190964+0000","last_became_peered":"2024-02-26T10:42:42.190964+0000","last_unstale":"2024-02-26T10:44:40.042639+0000","last_undegraded":"2024-02-26T10:44:40.042639+0000","last_fullsized":"2024-02-26T10:44:40.042639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:24:04.697129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556790+0000","last_change":"2024-02-26T10:42:42.182882+0000","last_active":"2024-02-26T10:44:39.556790+0000","last_peered":"2024-02-26T10:44:39.556790+0000","last_clean":"2024-02-26T10:44:39.556790+0000","last_became_active":"2024-02-26T10:42:42.182551+0000","last_became_peered":"2024-02-26T10:42:42.182551+0000","last_unstale":"2024-02-26T10:44:39.556790+0000","last_undegraded":"2024-02-26T10:44:39.556790+0000","last_fullsized":"2024-02-26T10:44:39.556790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:11:58.578832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040743+0000","last_change":"2024-02-26T10:42:42.193353+0000","last_active":"2024-02-26T10:44:40.040743+0000","last_peered":"2024-02-26T10:44:40.040743+0000","last_clean":"2024-02-26T10:44:40.040743+0000","last_became_active":"2024-02-26T10:42:42.190935+0000","last_became_peered":"2024-02-26T10:42:42.190935+0000","last_unstale":"2024-02-26T10:44:40.040743+0000","last_undegraded":"2024-02-26T10:44:40.040743+0000","last_fullsized":"2024-02-26T10:44:40.040743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:03:14.319769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464179+0000","last_change":"2024-02-26T10:42:42.194529+0000","last_active":"2024-02-26T10:44:39.464179+0000","last_peered":"2024-02-26T10:44:39.464179+0000","last_clean":"2024-02-26T10:44:39.464179+0000","last_became_active":"2024-02-26T10:42:42.194212+0000","last_became_peered":"2024-02-26T10:42:42.194212+0000","last_unstale":"2024-02-26T10:44:39.464179+0000","last_undegraded":"2024-02-26T10:44:39.464179+0000","last_fullsized":"2024-02-26T10:44:39.464179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:36.876672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237256+0000","last_change":"2024-02-26T10:42:42.178119+0000","last_active":"2024-02-26T10:44:39.237256+0000","last_peered":"2024-02-26T10:44:39.237256+0000","last_clean":"2024-02-26T10:44:39.237256+0000","last_became_active":"2024-02-26T10:42:42.177798+0000","last_became_peered":"2024-02-26T10:42:42.177798+0000","last_unstale":"2024-02-26T10:44:39.237256+0000","last_undegraded":"2024-02-26T10:44:39.237256+0000","last_fullsized":"2024-02-26T10:44:39.237256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:52:29.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308152+0000","last_change":"2024-02-26T10:42:42.167380+0000","last_active":"2024-02-26T10:44:39.308152+0000","last_peered":"2024-02-26T10:44:39.308152+0000","last_clean":"2024-02-26T10:44:39.308152+0000","last_became_active":"2024-02-26T10:42:42.167079+0000","last_became_peered":"2024-02-26T10:42:42.167079+0000","last_unstale":"2024-02-26T10:44:39.308152+0000","last_undegraded":"2024-02-26T10:44:39.308152+0000","last_fullsized":"2024-02-26T10:44:39.308152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:14:25.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250986,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76001,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":910,"num_read_kb":534,"num_write":181,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":63341,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":228003,"data_compressed":14031,"data_compressed_allocated":20480,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":532499,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612664,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1196032,"data_stored":1191633,"data_compressed":64863,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776189,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":536697,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037829,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":532387,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299465,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":1128962,"data_compressed":63031,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462991,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":527777,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":599155,"data_compressed":28539,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":529165,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":528846,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":527971,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":605009,"data_compressed":37074,"data_compressed_allocated":49152,"data_compressed_original":135168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724630,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1124018,"data_compressed":63031,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":71468,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":73226,"data_compressed":10367,"data_compressed_allocated":12288,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73578,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:45:46.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:45:46.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:45 smithi060 ceph-mon[31539]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T10:45:46.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:45 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/31172279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:46.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:45 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1216356521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:46.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:45 smithi100 ceph-mon[36051]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T10:45:46.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:45 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/31172279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:46.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:45 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1216356521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.299+0000 7faf3c600640 1 -- 172.21.15.60:0/841770721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf34105050 msgr2=0x7faf3410b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.299+0000 7faf3c600640 1 --2- 172.21.15.60:0/841770721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf34105050 0x7faf3410b8e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009be0 tx=0x7faf2c02bef0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.299+0000 7faf3c600640 1 -- 172.21.15.60:0/841770721 shutdown_connections 2024-02-26T10:45:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.299+0000 7faf3c600640 1 --2- 172.21.15.60:0/841770721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf34104690 0x7faf34104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.299+0000 7faf3c600640 1 --2- 172.21.15.60:0/841770721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf34105050 0x7faf3410b8e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.299+0000 7faf3c600640 1 --2- 172.21.15.60:0/841770721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf341034a0 0x7faf341038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.299+0000 7faf3c600640 1 -- 172.21.15.60:0/841770721 >> 172.21.15.60:0/841770721 conn(0x7faf340fea10 msgr2=0x7faf34100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.299+0000 7faf3c600640 1 -- 172.21.15.60:0/841770721 shutdown_connections 2024-02-26T10:45:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.299+0000 7faf3c600640 1 -- 172.21.15.60:0/841770721 wait complete. 2024-02-26T10:45:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.300+0000 7faf3c600640 1 Processor -- start 2024-02-26T10:45:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.300+0000 7faf3c600640 1 -- start start 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf3c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf341034a0 0x7faf34199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf3c600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf34104690 0x7faf34199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf3c600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf34105050 0x7faf341a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf3c600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3410f150 con 0x7faf34104690 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf3c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3410efd0 con 0x7faf341034a0 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf3c600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3410f2d0 con 0x7faf34105050 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf39200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf34104690 0x7faf34199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf3a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf34105050 0x7faf341a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf39200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf34104690 0x7faf34199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35098/0 (socket says 172.21.15.60:35098) 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf39c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf341034a0 0x7faf34199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.301+0000 7faf39200640 1 -- 172.21.15.60:0/2838798345 learned_addr learned my addr 172.21.15.60:0/2838798345 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:46.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf3a600640 1 -- 172.21.15.60:0/2838798345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf341034a0 msgr2=0x7faf34199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf3a600640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf341034a0 0x7faf34199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf3a600640 1 -- 172.21.15.60:0/2838798345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf34104690 msgr2=0x7faf34199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf3a600640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf34104690 0x7faf34199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf3a600640 1 -- 172.21.15.60:0/2838798345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf341a1380 con 0x7faf34105050 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf39c00640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf341034a0 0x7faf34199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf3a600640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf34105050 0x7faf341a0d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf2c02c430 tx=0x7faf2c0337b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf32000640 1 -- 172.21.15.60:0/2838798345 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c043020 con 0x7faf34105050 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf32000640 1 -- 172.21.15.60:0/2838798345 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf2c02ca60 con 0x7faf34105050 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf32000640 1 -- 172.21.15.60:0/2838798345 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c034690 con 0x7faf34105050 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf341a1610 con 0x7faf34105050 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.302+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf341a1bb0 con 0x7faf34105050 2024-02-26T10:45:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.304+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3410a7c0 con 0x7faf34105050 2024-02-26T10:45:46.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.305+0000 7faf32000640 1 -- 172.21.15.60:0/2838798345 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7faf2c046020 con 0x7faf34105050 2024-02-26T10:45:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.307+0000 7faf32000640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faf1c0a1e90 0x7faf1c0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.307+0000 7faf32000640 1 -- 172.21.15.60:0/2838798345 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7faf2c0f0710 con 0x7faf34105050 2024-02-26T10:45:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.307+0000 7faf39c00640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faf1c0a1e90 0x7faf1c0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.308+0000 7faf39c00640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faf1c0a1e90 0x7faf1c0a4350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faf341044f0 tx=0x7faf28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:46.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.311+0000 7faf32000640 1 -- 172.21.15.60:0/2838798345 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf2c04d020 con 0x7faf34105050 2024-02-26T10:45:46.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:45 smithi113 ceph-mon[36126]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T10:45:46.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:45 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/31172279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:46.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:45 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1216356521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.509+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf340616a0 con 0x7faf1c0a1e90 2024-02-26T10:45:46.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.525+0000 7faf32000640 1 -- 172.21.15.60:0/2838798345 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340796 (secure 0 0 0) 0x7faf340616a0 con 0x7faf1c0a1e90 2024-02-26T10:45:46.523 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:46.527 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T10:45:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faf1c0a1e90 msgr2=0x7faf1c0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faf1c0a1e90 0x7faf1c0a4350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faf341044f0 tx=0x7faf28005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf34105050 msgr2=0x7faf341a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf34105050 0x7faf341a0d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf2c02c430 tx=0x7faf2c0337b0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 shutdown_connections 2024-02-26T10:45:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faf1c0a1e90 0x7faf1c0a4350 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf34105050 0x7faf341a0d10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf34104690 0x7faf34199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 --2- 172.21.15.60:0/2838798345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf341034a0 0x7faf34199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 >> 172.21.15.60:0/2838798345 conn(0x7faf340fea10 msgr2=0x7faf34107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 shutdown_connections 2024-02-26T10:45:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.533+0000 7faf3c600640 1 -- 172.21.15.60:0/2838798345 wait complete. 2024-02-26T10:45:46.547 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-02-26T10:45:44.964702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666603,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1036,"num_read_kb":643,"num_write":364,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1244077,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":794,"ondisk_log_size":794,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374372,"kb_used_data":12708,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567452,"statfs":{"total":1151940427776,"available":1151557070848,"internally_reserved":0,"allocated":13012992,"data_stored":8364119,"data_compressed":443487,"data_compressed_allocated":598016,"data_compressed_original":2732032,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":56,"num_write":10,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004716"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308577+0000","last_change":"2024-02-26T10:42:42.176634+0000","last_active":"2024-02-26T10:44:39.308577+0000","last_peered":"2024-02-26T10:44:39.308577+0000","last_clean":"2024-02-26T10:44:39.308577+0000","last_became_active":"2024-02-26T10:42:42.176386+0000","last_became_peered":"2024-02-26T10:42:42.176386+0000","last_unstale":"2024-02-26T10:44:39.308577+0000","last_undegraded":"2024-02-26T10:44:39.308577+0000","last_fullsized":"2024-02-26T10:44:39.308577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:45.005506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528482+0000","last_change":"2024-02-26T10:42:42.176769+0000","last_active":"2024-02-26T10:44:39.528482+0000","last_peered":"2024-02-26T10:44:39.528482+0000","last_clean":"2024-02-26T10:44:39.528482+0000","last_became_active":"2024-02-26T10:42:42.174868+0000","last_became_peered":"2024-02-26T10:42:42.174868+0000","last_unstale":"2024-02-26T10:44:39.528482+0000","last_undegraded":"2024-02-26T10:44:39.528482+0000","last_fullsized":"2024-02-26T10:44:39.528482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:16.202371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528227+0000","last_change":"2024-02-26T10:42:42.184410+0000","last_active":"2024-02-26T10:44:39.528227+0000","last_peered":"2024-02-26T10:44:39.528227+0000","last_clean":"2024-02-26T10:44:39.528227+0000","last_became_active":"2024-02-26T10:42:42.182986+0000","last_became_peered":"2024-02-26T10:42:42.182986+0000","last_unstale":"2024-02-26T10:44:39.528227+0000","last_undegraded":"2024-02-26T10:44:39.528227+0000","last_fullsized":"2024-02-26T10:44:39.528227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:28:45.445325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409010+0000","last_change":"2024-02-26T10:43:38.462828+0000","last_active":"2024-02-26T10:44:39.409010+0000","last_peered":"2024-02-26T10:44:39.409010+0000","last_clean":"2024-02-26T10:44:39.409010+0000","last_became_active":"2024-02-26T10:43:38.461014+0000","last_became_peered":"2024-02-26T10:43:38.461014+0000","last_unstale":"2024-02-26T10:44:39.409010+0000","last_undegraded":"2024-02-26T10:44:39.409010+0000","last_fullsized":"2024-02-26T10:44:39.409010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:07.857849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234724+0000","last_change":"2024-02-26T10:42:42.192448+0000","last_active":"2024-02-26T10:44:39.234724+0000","last_peered":"2024-02-26T10:44:39.234724+0000","last_clean":"2024-02-26T10:44:39.234724+0000","last_became_active":"2024-02-26T10:42:42.192133+0000","last_became_peered":"2024-02-26T10:42:42.192133+0000","last_unstale":"2024-02-26T10:44:39.234724+0000","last_undegraded":"2024-02-26T10:44:39.234724+0000","last_fullsized":"2024-02-26T10:44:39.234724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:43:49.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.256983+0000","last_change":"2024-02-26T10:42:42.169999+0000","last_active":"2024-02-26T10:44:39.256983+0000","last_peered":"2024-02-26T10:44:39.256983+0000","last_clean":"2024-02-26T10:44:39.256983+0000","last_became_active":"2024-02-26T10:42:42.169678+0000","last_became_peered":"2024-02-26T10:42:42.169678+0000","last_unstale":"2024-02-26T10:44:39.256983+0000","last_undegraded":"2024-02-26T10:44:39.256983+0000","last_fullsized":"2024-02-26T10:44:39.256983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:54.435203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234970+0000","last_change":"2024-02-26T10:42:42.177538+0000","last_active":"2024-02-26T10:44:39.234970+0000","last_peered":"2024-02-26T10:44:39.234970+0000","last_clean":"2024-02-26T10:44:39.234970+0000","last_became_active":"2024-02-26T10:42:42.177204+0000","last_became_peered":"2024-02-26T10:42:42.177204+0000","last_unstale":"2024-02-26T10:44:39.234970+0000","last_undegraded":"2024-02-26T10:44:39.234970+0000","last_fullsized":"2024-02-26T10:44:39.234970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:09.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040510+0000","last_change":"2024-02-26T10:42:42.191342+0000","last_active":"2024-02-26T10:44:40.040510+0000","last_peered":"2024-02-26T10:44:40.040510+0000","last_clean":"2024-02-26T10:44:40.040510+0000","last_became_active":"2024-02-26T10:42:42.190990+0000","last_became_peered":"2024-02-26T10:42:42.190990+0000","last_unstale":"2024-02-26T10:44:40.040510+0000","last_undegraded":"2024-02-26T10:44:40.040510+0000","last_fullsized":"2024-02-26T10:44:40.040510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:20:45.190300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464601+0000","last_change":"2024-02-26T10:42:42.173175+0000","last_active":"2024-02-26T10:44:39.464601+0000","last_peered":"2024-02-26T10:44:39.464601+0000","last_clean":"2024-02-26T10:44:39.464601+0000","last_became_active":"2024-02-26T10:42:42.172828+0000","last_became_peered":"2024-02-26T10:42:42.172828+0000","last_unstale":"2024-02-26T10:44:39.464601+0000","last_undegraded":"2024-02-26T10:44:39.464601+0000","last_fullsized":"2024-02-26T10:44:39.464601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:17:34.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257498+0000","last_change":"2024-02-26T10:42:42.167267+0000","last_active":"2024-02-26T10:44:39.257498+0000","last_peered":"2024-02-26T10:44:39.257498+0000","last_clean":"2024-02-26T10:44:39.257498+0000","last_became_active":"2024-02-26T10:42:42.166948+0000","last_became_peered":"2024-02-26T10:42:42.166948+0000","last_unstale":"2024-02-26T10:44:39.257498+0000","last_undegraded":"2024-02-26T10:44:39.257498+0000","last_fullsized":"2024-02-26T10:44:39.257498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:52.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235590+0000","last_change":"2024-02-26T10:42:42.171299+0000","last_active":"2024-02-26T10:44:39.235590+0000","last_peered":"2024-02-26T10:44:39.235590+0000","last_clean":"2024-02-26T10:44:39.235590+0000","last_became_active":"2024-02-26T10:42:42.170616+0000","last_became_peered":"2024-02-26T10:42:42.170616+0000","last_unstale":"2024-02-26T10:44:39.235590+0000","last_undegraded":"2024-02-26T10:44:39.235590+0000","last_fullsized":"2024-02-26T10:44:39.235590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:50.320859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239431+0000","last_change":"2024-02-26T10:42:42.168408+0000","last_active":"2024-02-26T10:44:39.239431+0000","last_peered":"2024-02-26T10:44:39.239431+0000","last_clean":"2024-02-26T10:44:39.239431+0000","last_became_active":"2024-02-26T10:42:42.167150+0000","last_became_peered":"2024-02-26T10:42:42.167150+0000","last_unstale":"2024-02-26T10:44:39.239431+0000","last_undegraded":"2024-02-26T10:44:39.239431+0000","last_fullsized":"2024-02-26T10:44:39.239431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:34.763331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236385+0000","last_change":"2024-02-26T10:42:42.171720+0000","last_active":"2024-02-26T10:44:39.236385+0000","last_peered":"2024-02-26T10:44:39.236385+0000","last_clean":"2024-02-26T10:44:39.236385+0000","last_became_active":"2024-02-26T10:42:42.171309+0000","last_became_peered":"2024-02-26T10:42:42.171309+0000","last_unstale":"2024-02-26T10:44:39.236385+0000","last_undegraded":"2024-02-26T10:44:39.236385+0000","last_fullsized":"2024-02-26T10:44:39.236385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:20:52.151553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527374+0000","last_change":"2024-02-26T10:42:42.176857+0000","last_active":"2024-02-26T10:44:39.527374+0000","last_peered":"2024-02-26T10:44:39.527374+0000","last_clean":"2024-02-26T10:44:39.527374+0000","last_became_active":"2024-02-26T10:42:42.176385+0000","last_became_peered":"2024-02-26T10:42:42.176385+0000","last_unstale":"2024-02-26T10:44:39.527374+0000","last_undegraded":"2024-02-26T10:44:39.527374+0000","last_fullsized":"2024-02-26T10:44:39.527374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:45:51.852212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527401+0000","last_change":"2024-02-26T10:42:42.174682+0000","last_active":"2024-02-26T10:44:39.527401+0000","last_peered":"2024-02-26T10:44:39.527401+0000","last_clean":"2024-02-26T10:44:39.527401+0000","last_became_active":"2024-02-26T10:42:42.174328+0000","last_became_peered":"2024-02-26T10:42:42.174328+0000","last_unstale":"2024-02-26T10:44:39.527401+0000","last_undegraded":"2024-02-26T10:44:39.527401+0000","last_fullsized":"2024-02-26T10:44:39.527401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:47.161020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463946+0000","last_change":"2024-02-26T10:42:42.185289+0000","last_active":"2024-02-26T10:44:39.463946+0000","last_peered":"2024-02-26T10:44:39.463946+0000","last_clean":"2024-02-26T10:44:39.463946+0000","last_became_active":"2024-02-26T10:42:42.185000+0000","last_became_peered":"2024-02-26T10:42:42.185000+0000","last_unstale":"2024-02-26T10:44:39.463946+0000","last_undegraded":"2024-02-26T10:44:39.463946+0000","last_fullsized":"2024-02-26T10:44:39.463946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:48.552584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308483+0000","last_change":"2024-02-26T10:42:42.176558+0000","last_active":"2024-02-26T10:44:39.308483+0000","last_peered":"2024-02-26T10:44:39.308483+0000","last_clean":"2024-02-26T10:44:39.308483+0000","last_became_active":"2024-02-26T10:42:42.176334+0000","last_became_peered":"2024-02-26T10:42:42.176334+0000","last_unstale":"2024-02-26T10:44:39.308483+0000","last_undegraded":"2024-02-26T10:44:39.308483+0000","last_fullsized":"2024-02-26T10:44:39.308483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:59.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409740+0000","last_change":"2024-02-26T10:42:42.168534+0000","last_active":"2024-02-26T10:44:39.409740+0000","last_peered":"2024-02-26T10:44:39.409740+0000","last_clean":"2024-02-26T10:44:39.409740+0000","last_became_active":"2024-02-26T10:42:42.168179+0000","last_became_peered":"2024-02-26T10:42:42.168179+0000","last_unstale":"2024-02-26T10:44:39.409740+0000","last_undegraded":"2024-02-26T10:44:39.409740+0000","last_fullsized":"2024-02-26T10:44:39.409740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:09.625969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238870+0000","last_change":"2024-02-26T10:42:42.192446+0000","last_active":"2024-02-26T10:44:39.238870+0000","last_peered":"2024-02-26T10:44:39.238870+0000","last_clean":"2024-02-26T10:44:39.238870+0000","last_became_active":"2024-02-26T10:42:42.191863+0000","last_became_peered":"2024-02-26T10:42:42.191863+0000","last_unstale":"2024-02-26T10:44:39.238870+0000","last_undegraded":"2024-02-26T10:44:39.238870+0000","last_fullsized":"2024-02-26T10:44:39.238870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:30:19.477529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238594+0000","last_change":"2024-02-26T10:42:47.257605+0000","last_active":"2024-02-26T10:44:39.238594+0000","last_peered":"2024-02-26T10:44:39.238594+0000","last_clean":"2024-02-26T10:44:39.238594+0000","last_became_active":"2024-02-26T10:42:47.257206+0000","last_became_peered":"2024-02-26T10:42:47.257206+0000","last_unstale":"2024-02-26T10:44:39.238594+0000","last_undegraded":"2024-02-26T10:44:39.238594+0000","last_fullsized":"2024-02-26T10:44:39.238594+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:26:17.681862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409986+0000","last_change":"2024-02-26T10:42:41.613087+0000","last_active":"2024-02-26T10:44:39.409986+0000","last_peered":"2024-02-26T10:44:39.409986+0000","last_clean":"2024-02-26T10:44:39.409986+0000","last_became_active":"2024-02-26T10:42:41.612664+0000","last_became_peered":"2024-02-26T10:42:41.612664+0000","last_unstale":"2024-02-26T10:44:39.409986+0000","last_undegraded":"2024-02-26T10:44:39.409986+0000","last_fullsized":"2024-02-26T10:44:39.409986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:00.352166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042024+0000","last_change":"2024-02-26T10:42:42.191438+0000","last_active":"2024-02-26T10:44:40.042024+0000","last_peered":"2024-02-26T10:44:40.042024+0000","last_clean":"2024-02-26T10:44:40.042024+0000","last_became_active":"2024-02-26T10:42:42.191095+0000","last_became_peered":"2024-02-26T10:42:42.191095+0000","last_unstale":"2024-02-26T10:44:40.042024+0000","last_undegraded":"2024-02-26T10:44:40.042024+0000","last_fullsized":"2024-02-26T10:44:40.042024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:20.918715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238951+0000","last_change":"2024-02-26T10:42:47.247503+0000","last_active":"2024-02-26T10:44:39.238951+0000","last_peered":"2024-02-26T10:44:39.238951+0000","last_clean":"2024-02-26T10:44:39.238951+0000","last_became_active":"2024-02-26T10:42:47.247121+0000","last_became_peered":"2024-02-26T10:42:47.247121+0000","last_unstale":"2024-02-26T10:44:39.238951+0000","last_undegraded":"2024-02-26T10:44:39.238951+0000","last_fullsized":"2024-02-26T10:44:39.238951+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:01:38.733131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042711+0000","last_change":"2024-02-26T10:42:41.626227+0000","last_active":"2024-02-26T10:44:40.042711+0000","last_peered":"2024-02-26T10:44:40.042711+0000","last_clean":"2024-02-26T10:44:40.042711+0000","last_became_active":"2024-02-26T10:42:41.625760+0000","last_became_peered":"2024-02-26T10:42:41.625760+0000","last_unstale":"2024-02-26T10:44:40.042711+0000","last_undegraded":"2024-02-26T10:44:40.042711+0000","last_fullsized":"2024-02-26T10:44:40.042711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:04.600151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528152+0000","last_change":"2024-02-26T10:42:42.182808+0000","last_active":"2024-02-26T10:44:39.528152+0000","last_peered":"2024-02-26T10:44:39.528152+0000","last_clean":"2024-02-26T10:44:39.528152+0000","last_became_active":"2024-02-26T10:42:42.182485+0000","last_became_peered":"2024-02-26T10:42:42.182485+0000","last_unstale":"2024-02-26T10:44:39.528152+0000","last_undegraded":"2024-02-26T10:44:39.528152+0000","last_fullsized":"2024-02-26T10:44:39.528152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:45.205969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239151+0000","last_change":"2024-02-26T10:42:47.250694+0000","last_active":"2024-02-26T10:44:39.239151+0000","last_peered":"2024-02-26T10:44:39.239151+0000","last_clean":"2024-02-26T10:44:39.239151+0000","last_became_active":"2024-02-26T10:42:47.249566+0000","last_became_peered":"2024-02-26T10:42:47.249566+0000","last_unstale":"2024-02-26T10:44:39.239151+0000","last_undegraded":"2024-02-26T10:44:39.239151+0000","last_fullsized":"2024-02-26T10:44:39.239151+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:44.016413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235108+0000","last_change":"2024-02-26T10:42:41.606849+0000","last_active":"2024-02-26T10:44:39.235108+0000","last_peered":"2024-02-26T10:44:39.235108+0000","last_clean":"2024-02-26T10:44:39.235108+0000","last_became_active":"2024-02-26T10:42:41.606494+0000","last_became_peered":"2024-02-26T10:42:41.606494+0000","last_unstale":"2024-02-26T10:44:39.235108+0000","last_undegraded":"2024-02-26T10:44:39.235108+0000","last_fullsized":"2024-02-26T10:44:39.235108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:36.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464300+0000","last_change":"2024-02-26T10:42:42.175926+0000","last_active":"2024-02-26T10:44:39.464300+0000","last_peered":"2024-02-26T10:44:39.464300+0000","last_clean":"2024-02-26T10:44:39.464300+0000","last_became_active":"2024-02-26T10:42:42.175614+0000","last_became_peered":"2024-02-26T10:42:42.175614+0000","last_unstale":"2024-02-26T10:44:39.464300+0000","last_undegraded":"2024-02-26T10:44:39.464300+0000","last_fullsized":"2024-02-26T10:44:39.464300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:20:20.829849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.044113+0000","last_change":"2024-02-26T10:44:40.043633+0000","last_active":"2024-02-26T10:44:40.044113+0000","last_peered":"2024-02-26T10:44:40.044113+0000","last_clean":"2024-02-26T10:44:40.044113+0000","last_became_active":"2024-02-26T10:44:40.043209+0000","last_became_peered":"2024-02-26T10:44:40.043209+0000","last_unstale":"2024-02-26T10:44:40.044113+0000","last_undegraded":"2024-02-26T10:44:40.044113+0000","last_fullsized":"2024-02-26T10:44:40.044113+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:43.466990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234893+0000","last_change":"2024-02-26T10:42:41.606407+0000","last_active":"2024-02-26T10:44:39.234893+0000","last_peered":"2024-02-26T10:44:39.234893+0000","last_clean":"2024-02-26T10:44:39.234893+0000","last_became_active":"2024-02-26T10:42:41.606094+0000","last_became_peered":"2024-02-26T10:42:41.606094+0000","last_unstale":"2024-02-26T10:44:39.234893+0000","last_undegraded":"2024-02-26T10:44:39.234893+0000","last_fullsized":"2024-02-26T10:44:39.234893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:53.673017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557475+0000","last_change":"2024-02-26T10:42:42.172967+0000","last_active":"2024-02-26T10:44:39.557475+0000","last_peered":"2024-02-26T10:44:39.557475+0000","last_clean":"2024-02-26T10:44:39.557475+0000","last_became_active":"2024-02-26T10:42:42.172675+0000","last_became_peered":"2024-02-26T10:42:42.172675+0000","last_unstale":"2024-02-26T10:44:39.557475+0000","last_undegraded":"2024-02-26T10:44:39.557475+0000","last_fullsized":"2024-02-26T10:44:39.557475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238546+0000","last_change":"2024-02-26T10:42:47.229375+0000","last_active":"2024-02-26T10:44:39.238546+0000","last_peered":"2024-02-26T10:44:39.238546+0000","last_clean":"2024-02-26T10:44:39.238546+0000","last_became_active":"2024-02-26T10:42:47.228874+0000","last_became_peered":"2024-02-26T10:42:47.228874+0000","last_unstale":"2024-02-26T10:44:39.238546+0000","last_undegraded":"2024-02-26T10:44:39.238546+0000","last_fullsized":"2024-02-26T10:44:39.238546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:02.188519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":190,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:41.266222+0000","last_change":"2024-02-26T10:42:41.602783+0000","last_active":"2024-02-26T10:45:41.266222+0000","last_peered":"2024-02-26T10:45:41.266222+0000","last_clean":"2024-02-26T10:45:41.266222+0000","last_became_active":"2024-02-26T10:42:41.602491+0000","last_became_peered":"2024-02-26T10:42:41.602491+0000","last_unstale":"2024-02-26T10:45:41.266222+0000","last_undegraded":"2024-02-26T10:45:41.266222+0000","last_fullsized":"2024-02-26T10:45:41.266222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:23:33.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234508+0000","last_change":"2024-02-26T10:43:38.461429+0000","last_active":"2024-02-26T10:44:39.234508+0000","last_peered":"2024-02-26T10:44:39.234508+0000","last_clean":"2024-02-26T10:44:39.234508+0000","last_became_active":"2024-02-26T10:43:38.461019+0000","last_became_peered":"2024-02-26T10:43:38.461019+0000","last_unstale":"2024-02-26T10:44:39.234508+0000","last_undegraded":"2024-02-26T10:44:39.234508+0000","last_fullsized":"2024-02-26T10:44:39.234508+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:20.911179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.559125+0000","last_change":"2024-02-26T10:44:39.558649+0000","last_active":"2024-02-26T10:44:39.559125+0000","last_peered":"2024-02-26T10:44:39.559125+0000","last_clean":"2024-02-26T10:44:39.559125+0000","last_became_active":"2024-02-26T10:44:39.558013+0000","last_became_peered":"2024-02-26T10:44:39.558013+0000","last_unstale":"2024-02-26T10:44:39.559125+0000","last_undegraded":"2024-02-26T10:44:39.559125+0000","last_fullsized":"2024-02-26T10:44:39.559125+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:45.879391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.256942+0000","last_change":"2024-02-26T10:42:41.602887+0000","last_active":"2024-02-26T10:44:39.256942+0000","last_peered":"2024-02-26T10:44:39.256942+0000","last_clean":"2024-02-26T10:44:39.256942+0000","last_became_active":"2024-02-26T10:42:41.602564+0000","last_became_peered":"2024-02-26T10:42:41.602564+0000","last_unstale":"2024-02-26T10:44:39.256942+0000","last_undegraded":"2024-02-26T10:44:39.256942+0000","last_fullsized":"2024-02-26T10:44:39.256942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:22:41.711283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556964+0000","last_change":"2024-02-26T10:42:42.192700+0000","last_active":"2024-02-26T10:44:39.556964+0000","last_peered":"2024-02-26T10:44:39.556964+0000","last_clean":"2024-02-26T10:44:39.556964+0000","last_became_active":"2024-02-26T10:42:42.192387+0000","last_became_peered":"2024-02-26T10:42:42.192387+0000","last_unstale":"2024-02-26T10:44:39.556964+0000","last_undegraded":"2024-02-26T10:44:39.556964+0000","last_fullsized":"2024-02-26T10:44:39.556964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:21:15.104872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307862+0000","last_change":"2024-02-26T10:42:47.247000+0000","last_active":"2024-02-26T10:44:39.307862+0000","last_peered":"2024-02-26T10:44:39.307862+0000","last_clean":"2024-02-26T10:44:39.307862+0000","last_became_active":"2024-02-26T10:42:47.246520+0000","last_became_peered":"2024-02-26T10:42:47.246520+0000","last_unstale":"2024-02-26T10:44:39.307862+0000","last_undegraded":"2024-02-26T10:44:39.307862+0000","last_fullsized":"2024-02-26T10:44:39.307862+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:29.977858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235812+0000","last_change":"2024-02-26T10:42:41.609460+0000","last_active":"2024-02-26T10:44:39.235812+0000","last_peered":"2024-02-26T10:44:39.235812+0000","last_clean":"2024-02-26T10:44:39.235812+0000","last_became_active":"2024-02-26T10:42:41.608981+0000","last_became_peered":"2024-02-26T10:42:41.608981+0000","last_unstale":"2024-02-26T10:44:39.235812+0000","last_undegraded":"2024-02-26T10:44:39.235812+0000","last_fullsized":"2024-02-26T10:44:39.235812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:04.599910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040491+0000","last_change":"2024-02-26T10:42:42.185359+0000","last_active":"2024-02-26T10:44:40.040491+0000","last_peered":"2024-02-26T10:44:40.040491+0000","last_clean":"2024-02-26T10:44:40.040491+0000","last_became_active":"2024-02-26T10:42:42.184221+0000","last_became_peered":"2024-02-26T10:42:42.184221+0000","last_unstale":"2024-02-26T10:44:40.040491+0000","last_undegraded":"2024-02-26T10:44:40.040491+0000","last_fullsized":"2024-02-26T10:44:40.040491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:53.907290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526689+0000","last_change":"2024-02-26T10:42:47.241700+0000","last_active":"2024-02-26T10:44:39.526689+0000","last_peered":"2024-02-26T10:44:39.526689+0000","last_clean":"2024-02-26T10:44:39.526689+0000","last_became_active":"2024-02-26T10:42:47.240667+0000","last_became_peered":"2024-02-26T10:42:47.240667+0000","last_unstale":"2024-02-26T10:44:39.526689+0000","last_undegraded":"2024-02-26T10:44:39.526689+0000","last_fullsized":"2024-02-26T10:44:39.526689+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:19.071244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556576+0000","last_change":"2024-02-26T10:42:41.604519+0000","last_active":"2024-02-26T10:44:39.556576+0000","last_peered":"2024-02-26T10:44:39.556576+0000","last_clean":"2024-02-26T10:44:39.556576+0000","last_became_active":"2024-02-26T10:42:41.603596+0000","last_became_peered":"2024-02-26T10:42:41.603596+0000","last_unstale":"2024-02-26T10:44:39.556576+0000","last_undegraded":"2024-02-26T10:44:39.556576+0000","last_fullsized":"2024-02-26T10:44:39.556576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:39.764730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408161+0000","last_change":"2024-02-26T10:42:42.187532+0000","last_active":"2024-02-26T10:44:39.408161+0000","last_peered":"2024-02-26T10:44:39.408161+0000","last_clean":"2024-02-26T10:44:39.408161+0000","last_became_active":"2024-02-26T10:42:42.185635+0000","last_became_peered":"2024-02-26T10:42:42.185635+0000","last_unstale":"2024-02-26T10:44:39.408161+0000","last_undegraded":"2024-02-26T10:44:39.408161+0000","last_fullsized":"2024-02-26T10:44:39.408161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:16.894747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041446+0000","last_change":"2024-02-26T10:42:47.233507+0000","last_active":"2024-02-26T10:44:40.041446+0000","last_peered":"2024-02-26T10:44:40.041446+0000","last_clean":"2024-02-26T10:44:40.041446+0000","last_became_active":"2024-02-26T10:42:47.229589+0000","last_became_peered":"2024-02-26T10:42:47.229589+0000","last_unstale":"2024-02-26T10:44:40.041446+0000","last_undegraded":"2024-02-26T10:44:40.041446+0000","last_fullsized":"2024-02-26T10:44:40.041446+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:02:42.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":185,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:41.265079+0000","last_change":"2024-02-26T10:42:41.613952+0000","last_active":"2024-02-26T10:45:41.265079+0000","last_peered":"2024-02-26T10:45:41.265079+0000","last_clean":"2024-02-26T10:45:41.265079+0000","last_became_active":"2024-02-26T10:42:41.613656+0000","last_became_peered":"2024-02-26T10:42:41.613656+0000","last_unstale":"2024-02-26T10:45:41.265079+0000","last_undegraded":"2024-02-26T10:45:41.265079+0000","last_fullsized":"2024-02-26T10:45:41.265079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:14.090821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464609+0000","last_change":"2024-02-26T10:42:42.178357+0000","last_active":"2024-02-26T10:44:39.464609+0000","last_peered":"2024-02-26T10:44:39.464609+0000","last_clean":"2024-02-26T10:44:39.464609+0000","last_became_active":"2024-02-26T10:42:42.178051+0000","last_became_peered":"2024-02-26T10:42:42.178051+0000","last_unstale":"2024-02-26T10:44:39.464609+0000","last_undegraded":"2024-02-26T10:44:39.464609+0000","last_fullsized":"2024-02-26T10:44:39.464609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:13:01.326122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406187+0000","last_change":"2024-02-26T10:42:47.246231+0000","last_active":"2024-02-26T10:44:39.406187+0000","last_peered":"2024-02-26T10:44:39.406187+0000","last_clean":"2024-02-26T10:44:39.406187+0000","last_became_active":"2024-02-26T10:42:47.245739+0000","last_became_peered":"2024-02-26T10:42:47.245739+0000","last_unstale":"2024-02-26T10:44:39.406187+0000","last_undegraded":"2024-02-26T10:44:39.406187+0000","last_fullsized":"2024-02-26T10:44:39.406187+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:30.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235526+0000","last_change":"2024-02-26T10:42:41.603126+0000","last_active":"2024-02-26T10:44:39.235526+0000","last_peered":"2024-02-26T10:44:39.235526+0000","last_clean":"2024-02-26T10:44:39.235526+0000","last_became_active":"2024-02-26T10:42:41.602790+0000","last_became_peered":"2024-02-26T10:42:41.602790+0000","last_unstale":"2024-02-26T10:44:39.235526+0000","last_undegraded":"2024-02-26T10:44:39.235526+0000","last_fullsized":"2024-02-26T10:44:39.235526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:31:58.908209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528535+0000","last_change":"2024-02-26T10:43:38.459527+0000","last_active":"2024-02-26T10:44:39.528535+0000","last_peered":"2024-02-26T10:44:39.528535+0000","last_clean":"2024-02-26T10:44:39.528535+0000","last_became_active":"2024-02-26T10:43:38.459150+0000","last_became_peered":"2024-02-26T10:43:38.459150+0000","last_unstale":"2024-02-26T10:44:39.528535+0000","last_undegraded":"2024-02-26T10:44:39.528535+0000","last_fullsized":"2024-02-26T10:44:39.528535+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:21:54.028328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239304+0000","last_change":"2024-02-26T10:42:47.250075+0000","last_active":"2024-02-26T10:44:39.239304+0000","last_peered":"2024-02-26T10:44:39.239304+0000","last_clean":"2024-02-26T10:44:39.239304+0000","last_became_active":"2024-02-26T10:42:47.249490+0000","last_became_peered":"2024-02-26T10:42:47.249490+0000","last_unstale":"2024-02-26T10:44:39.239304+0000","last_undegraded":"2024-02-26T10:44:39.239304+0000","last_fullsized":"2024-02-26T10:44:39.239304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:42.659282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557237+0000","last_change":"2024-02-26T10:42:41.615314+0000","last_active":"2024-02-26T10:44:39.557237+0000","last_peered":"2024-02-26T10:44:39.557237+0000","last_clean":"2024-02-26T10:44:39.557237+0000","last_became_active":"2024-02-26T10:42:41.615091+0000","last_became_peered":"2024-02-26T10:42:41.615091+0000","last_unstale":"2024-02-26T10:44:39.557237+0000","last_undegraded":"2024-02-26T10:44:39.557237+0000","last_fullsized":"2024-02-26T10:44:39.557237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:41:52.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464449+0000","last_change":"2024-02-26T10:42:42.176609+0000","last_active":"2024-02-26T10:44:39.464449+0000","last_peered":"2024-02-26T10:44:39.464449+0000","last_clean":"2024-02-26T10:44:39.464449+0000","last_became_active":"2024-02-26T10:42:42.176311+0000","last_became_peered":"2024-02-26T10:42:42.176311+0000","last_unstale":"2024-02-26T10:44:39.464449+0000","last_undegraded":"2024-02-26T10:44:39.464449+0000","last_fullsized":"2024-02-26T10:44:39.464449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:25:38.415311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406280+0000","last_change":"2024-02-26T10:42:47.250255+0000","last_active":"2024-02-26T10:44:39.406280+0000","last_peered":"2024-02-26T10:44:39.406280+0000","last_clean":"2024-02-26T10:44:39.406280+0000","last_became_active":"2024-02-26T10:42:47.249465+0000","last_became_peered":"2024-02-26T10:42:47.249465+0000","last_unstale":"2024-02-26T10:44:39.406280+0000","last_undegraded":"2024-02-26T10:44:39.406280+0000","last_fullsized":"2024-02-26T10:44:39.406280+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:23.173863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'39","reported_seq":397,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:42.656270+0000","last_change":"2024-02-26T10:43:38.460725+0000","last_active":"2024-02-26T10:45:42.656270+0000","last_peered":"2024-02-26T10:45:42.656270+0000","last_clean":"2024-02-26T10:45:42.656270+0000","last_became_active":"2024-02-26T10:43:38.451900+0000","last_became_peered":"2024-02-26T10:43:38.451900+0000","last_unstale":"2024-02-26T10:45:42.656270+0000","last_undegraded":"2024-02-26T10:45:42.656270+0000","last_fullsized":"2024-02-26T10:45:42.656270+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:35:09.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71468,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":303,"num_read_kb":221,"num_write":40,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":62625,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234956+0000","last_change":"2024-02-26T10:42:42.168174+0000","last_active":"2024-02-26T10:44:39.234956+0000","last_peered":"2024-02-26T10:44:39.234956+0000","last_clean":"2024-02-26T10:44:39.234956+0000","last_became_active":"2024-02-26T10:42:42.167840+0000","last_became_peered":"2024-02-26T10:42:42.167840+0000","last_unstale":"2024-02-26T10:44:39.234956+0000","last_undegraded":"2024-02-26T10:44:39.234956+0000","last_fullsized":"2024-02-26T10:44:39.234956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:43.079139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557658+0000","last_change":"2024-02-26T10:42:47.230048+0000","last_active":"2024-02-26T10:44:39.557658+0000","last_peered":"2024-02-26T10:44:39.557658+0000","last_clean":"2024-02-26T10:44:39.557658+0000","last_became_active":"2024-02-26T10:42:47.229396+0000","last_became_peered":"2024-02-26T10:42:47.229396+0000","last_unstale":"2024-02-26T10:44:39.557658+0000","last_undegraded":"2024-02-26T10:44:39.557658+0000","last_fullsized":"2024-02-26T10:44:39.557658+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:57.804297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238511+0000","last_change":"2024-02-26T10:42:41.614413+0000","last_active":"2024-02-26T10:44:39.238511+0000","last_peered":"2024-02-26T10:44:39.238511+0000","last_clean":"2024-02-26T10:44:39.238511+0000","last_became_active":"2024-02-26T10:42:41.614065+0000","last_became_peered":"2024-02-26T10:42:41.614065+0000","last_unstale":"2024-02-26T10:44:39.238511+0000","last_undegraded":"2024-02-26T10:44:39.238511+0000","last_fullsized":"2024-02-26T10:44:39.238511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:50:20.552340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235996+0000","last_change":"2024-02-26T10:42:42.188140+0000","last_active":"2024-02-26T10:44:39.235996+0000","last_peered":"2024-02-26T10:44:39.235996+0000","last_clean":"2024-02-26T10:44:39.235996+0000","last_became_active":"2024-02-26T10:42:42.187055+0000","last_became_peered":"2024-02-26T10:42:42.187055+0000","last_unstale":"2024-02-26T10:44:39.235996+0000","last_undegraded":"2024-02-26T10:44:39.235996+0000","last_fullsized":"2024-02-26T10:44:39.235996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:49:43.568562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308124+0000","last_change":"2024-02-26T10:43:38.450681+0000","last_active":"2024-02-26T10:44:39.308124+0000","last_peered":"2024-02-26T10:44:39.308124+0000","last_clean":"2024-02-26T10:44:39.308124+0000","last_became_active":"2024-02-26T10:43:38.450120+0000","last_became_peered":"2024-02-26T10:43:38.450120+0000","last_unstale":"2024-02-26T10:44:39.308124+0000","last_undegraded":"2024-02-26T10:44:39.308124+0000","last_fullsized":"2024-02-26T10:44:39.308124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:10.983700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556913+0000","last_change":"2024-02-26T10:42:41.612337+0000","last_active":"2024-02-26T10:44:39.556913+0000","last_peered":"2024-02-26T10:44:39.556913+0000","last_clean":"2024-02-26T10:44:39.556913+0000","last_became_active":"2024-02-26T10:42:41.611920+0000","last_became_peered":"2024-02-26T10:42:41.611920+0000","last_unstale":"2024-02-26T10:44:39.556913+0000","last_undegraded":"2024-02-26T10:44:39.556913+0000","last_fullsized":"2024-02-26T10:44:39.556913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:29:13.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235333+0000","last_change":"2024-02-26T10:43:38.456919+0000","last_active":"2024-02-26T10:44:39.235333+0000","last_peered":"2024-02-26T10:44:39.235333+0000","last_clean":"2024-02-26T10:44:39.235333+0000","last_became_active":"2024-02-26T10:43:38.456461+0000","last_became_peered":"2024-02-26T10:43:38.456461+0000","last_unstale":"2024-02-26T10:44:39.235333+0000","last_undegraded":"2024-02-26T10:44:39.235333+0000","last_fullsized":"2024-02-26T10:44:39.235333+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:44.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.407141+0000","last_change":"2024-02-26T10:42:47.254585+0000","last_active":"2024-02-26T10:44:39.407141+0000","last_peered":"2024-02-26T10:44:39.407141+0000","last_clean":"2024-02-26T10:44:39.407141+0000","last_became_active":"2024-02-26T10:42:47.253870+0000","last_became_peered":"2024-02-26T10:42:47.253870+0000","last_unstale":"2024-02-26T10:44:39.407141+0000","last_undegraded":"2024-02-26T10:44:39.407141+0000","last_fullsized":"2024-02-26T10:44:39.407141+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:49.276906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":190,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:41.338041+0000","last_change":"2024-02-26T10:42:41.600901+0000","last_active":"2024-02-26T10:45:41.338041+0000","last_peered":"2024-02-26T10:45:41.338041+0000","last_clean":"2024-02-26T10:45:41.338041+0000","last_became_active":"2024-02-26T10:42:41.600018+0000","last_became_peered":"2024-02-26T10:42:41.600018+0000","last_unstale":"2024-02-26T10:45:41.338041+0000","last_undegraded":"2024-02-26T10:45:41.338041+0000","last_fullsized":"2024-02-26T10:45:41.338041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:07.876694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235012+0000","last_change":"2024-02-26T10:42:42.168702+0000","last_active":"2024-02-26T10:44:39.235012+0000","last_peered":"2024-02-26T10:44:39.235012+0000","last_clean":"2024-02-26T10:44:39.235012+0000","last_became_active":"2024-02-26T10:42:42.168420+0000","last_became_peered":"2024-02-26T10:42:42.168420+0000","last_unstale":"2024-02-26T10:44:39.235012+0000","last_undegraded":"2024-02-26T10:44:39.235012+0000","last_fullsized":"2024-02-26T10:44:39.235012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:35.111699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236688+0000","last_change":"2024-02-26T10:42:47.250714+0000","last_active":"2024-02-26T10:44:39.236688+0000","last_peered":"2024-02-26T10:44:39.236688+0000","last_clean":"2024-02-26T10:44:39.236688+0000","last_became_active":"2024-02-26T10:42:47.249506+0000","last_became_peered":"2024-02-26T10:42:47.249506+0000","last_unstale":"2024-02-26T10:44:39.236688+0000","last_undegraded":"2024-02-26T10:44:39.236688+0000","last_fullsized":"2024-02-26T10:44:39.236688+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:05.058052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307933+0000","last_change":"2024-02-26T10:43:38.472808+0000","last_active":"2024-02-26T10:44:39.307933+0000","last_peered":"2024-02-26T10:44:39.307933+0000","last_clean":"2024-02-26T10:44:39.307933+0000","last_became_active":"2024-02-26T10:43:38.456509+0000","last_became_peered":"2024-02-26T10:43:38.456509+0000","last_unstale":"2024-02-26T10:44:39.307933+0000","last_undegraded":"2024-02-26T10:44:39.307933+0000","last_fullsized":"2024-02-26T10:44:39.307933+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:03:06.127386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556435+0000","last_change":"2024-02-26T10:42:42.172845+0000","last_active":"2024-02-26T10:44:39.556435+0000","last_peered":"2024-02-26T10:44:39.556435+0000","last_clean":"2024-02-26T10:44:39.556435+0000","last_became_active":"2024-02-26T10:42:42.171263+0000","last_became_peered":"2024-02-26T10:42:42.171263+0000","last_unstale":"2024-02-26T10:44:39.556435+0000","last_undegraded":"2024-02-26T10:44:39.556435+0000","last_fullsized":"2024-02-26T10:44:39.556435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:21.439474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556408+0000","last_change":"2024-02-26T10:42:47.236434+0000","last_active":"2024-02-26T10:44:39.556408+0000","last_peered":"2024-02-26T10:44:39.556408+0000","last_clean":"2024-02-26T10:44:39.556408+0000","last_became_active":"2024-02-26T10:42:47.236040+0000","last_became_peered":"2024-02-26T10:42:47.236040+0000","last_unstale":"2024-02-26T10:44:39.556408+0000","last_undegraded":"2024-02-26T10:44:39.556408+0000","last_fullsized":"2024-02-26T10:44:39.556408+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:20:23.230348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235239+0000","last_change":"2024-02-26T10:42:41.604208+0000","last_active":"2024-02-26T10:44:39.235239+0000","last_peered":"2024-02-26T10:44:39.235239+0000","last_clean":"2024-02-26T10:44:39.235239+0000","last_became_active":"2024-02-26T10:42:41.603894+0000","last_became_peered":"2024-02-26T10:42:41.603894+0000","last_unstale":"2024-02-26T10:44:39.235239+0000","last_undegraded":"2024-02-26T10:44:39.235239+0000","last_fullsized":"2024-02-26T10:44:39.235239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:57:13.775868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308423+0000","last_change":"2024-02-26T10:42:42.169395+0000","last_active":"2024-02-26T10:44:39.308423+0000","last_peered":"2024-02-26T10:44:39.308423+0000","last_clean":"2024-02-26T10:44:39.308423+0000","last_became_active":"2024-02-26T10:42:42.168686+0000","last_became_peered":"2024-02-26T10:42:42.168686+0000","last_unstale":"2024-02-26T10:44:39.308423+0000","last_undegraded":"2024-02-26T10:44:39.308423+0000","last_fullsized":"2024-02-26T10:44:39.308423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:21:35.085188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234671+0000","last_change":"2024-02-26T10:42:47.252119+0000","last_active":"2024-02-26T10:44:39.234671+0000","last_peered":"2024-02-26T10:44:39.234671+0000","last_clean":"2024-02-26T10:44:39.234671+0000","last_became_active":"2024-02-26T10:42:47.251723+0000","last_became_peered":"2024-02-26T10:42:47.251723+0000","last_unstale":"2024-02-26T10:44:39.234671+0000","last_undegraded":"2024-02-26T10:44:39.234671+0000","last_fullsized":"2024-02-26T10:44:39.234671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:17.130806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408624+0000","last_change":"2024-02-26T10:42:41.621393+0000","last_active":"2024-02-26T10:44:39.408624+0000","last_peered":"2024-02-26T10:44:39.408624+0000","last_clean":"2024-02-26T10:44:39.408624+0000","last_became_active":"2024-02-26T10:42:41.621008+0000","last_became_peered":"2024-02-26T10:42:41.621008+0000","last_unstale":"2024-02-26T10:44:39.408624+0000","last_undegraded":"2024-02-26T10:44:39.408624+0000","last_fullsized":"2024-02-26T10:44:39.408624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:46.979854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257087+0000","last_change":"2024-02-26T10:43:38.458926+0000","last_active":"2024-02-26T10:44:39.257087+0000","last_peered":"2024-02-26T10:44:39.257087+0000","last_clean":"2024-02-26T10:44:39.257087+0000","last_became_active":"2024-02-26T10:43:38.458537+0000","last_became_peered":"2024-02-26T10:43:38.458537+0000","last_unstale":"2024-02-26T10:44:39.257087+0000","last_undegraded":"2024-02-26T10:44:39.257087+0000","last_fullsized":"2024-02-26T10:44:39.257087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:08.204646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236653+0000","last_change":"2024-02-26T10:42:47.238520+0000","last_active":"2024-02-26T10:44:39.236653+0000","last_peered":"2024-02-26T10:44:39.236653+0000","last_clean":"2024-02-26T10:44:39.236653+0000","last_became_active":"2024-02-26T10:42:47.237213+0000","last_became_peered":"2024-02-26T10:42:47.237213+0000","last_unstale":"2024-02-26T10:44:39.236653+0000","last_undegraded":"2024-02-26T10:44:39.236653+0000","last_fullsized":"2024-02-26T10:44:39.236653+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:29.283850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235085+0000","last_change":"2024-02-26T10:42:41.609479+0000","last_active":"2024-02-26T10:44:39.235085+0000","last_peered":"2024-02-26T10:44:39.235085+0000","last_clean":"2024-02-26T10:44:39.235085+0000","last_became_active":"2024-02-26T10:42:41.609084+0000","last_became_peered":"2024-02-26T10:42:41.609084+0000","last_unstale":"2024-02-26T10:44:39.235085+0000","last_undegraded":"2024-02-26T10:44:39.235085+0000","last_fullsized":"2024-02-26T10:44:39.235085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:32.412507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257066+0000","last_change":"2024-02-26T10:42:42.174985+0000","last_active":"2024-02-26T10:44:39.257066+0000","last_peered":"2024-02-26T10:44:39.257066+0000","last_clean":"2024-02-26T10:44:39.257066+0000","last_became_active":"2024-02-26T10:42:42.174694+0000","last_became_peered":"2024-02-26T10:42:42.174694+0000","last_unstale":"2024-02-26T10:44:39.257066+0000","last_undegraded":"2024-02-26T10:44:39.257066+0000","last_fullsized":"2024-02-26T10:44:39.257066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:10.000154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237057+0000","last_change":"2024-02-26T10:42:47.242587+0000","last_active":"2024-02-26T10:44:39.237057+0000","last_peered":"2024-02-26T10:44:39.237057+0000","last_clean":"2024-02-26T10:44:39.237057+0000","last_became_active":"2024-02-26T10:42:47.242099+0000","last_became_peered":"2024-02-26T10:42:47.242099+0000","last_unstale":"2024-02-26T10:44:39.237057+0000","last_undegraded":"2024-02-26T10:44:39.237057+0000","last_fullsized":"2024-02-26T10:44:39.237057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:38:42.852080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556815+0000","last_change":"2024-02-26T10:42:42.170613+0000","last_active":"2024-02-26T10:44:39.556815+0000","last_peered":"2024-02-26T10:44:39.556815+0000","last_clean":"2024-02-26T10:44:39.556815+0000","last_became_active":"2024-02-26T10:42:42.170317+0000","last_became_peered":"2024-02-26T10:42:42.170317+0000","last_unstale":"2024-02-26T10:44:39.556815+0000","last_undegraded":"2024-02-26T10:44:39.556815+0000","last_fullsized":"2024-02-26T10:44:39.556815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:29:44.276718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":187,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:42.339228+0000","last_change":"2024-02-26T10:42:41.617447+0000","last_active":"2024-02-26T10:45:42.339228+0000","last_peered":"2024-02-26T10:45:42.339228+0000","last_clean":"2024-02-26T10:45:42.339228+0000","last_became_active":"2024-02-26T10:42:41.617107+0000","last_became_peered":"2024-02-26T10:42:41.617107+0000","last_unstale":"2024-02-26T10:45:42.339228+0000","last_undegraded":"2024-02-26T10:45:42.339228+0000","last_fullsized":"2024-02-26T10:45:42.339228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:06.052364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235432+0000","last_change":"2024-02-26T10:42:47.251971+0000","last_active":"2024-02-26T10:44:39.235432+0000","last_peered":"2024-02-26T10:44:39.235432+0000","last_clean":"2024-02-26T10:44:39.235432+0000","last_became_active":"2024-02-26T10:42:47.251506+0000","last_became_peered":"2024-02-26T10:42:47.251506+0000","last_unstale":"2024-02-26T10:44:39.235432+0000","last_undegraded":"2024-02-26T10:44:39.235432+0000","last_fullsized":"2024-02-26T10:44:39.235432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:37.029348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041442+0000","last_change":"2024-02-26T10:42:42.194414+0000","last_active":"2024-02-26T10:44:40.041442+0000","last_peered":"2024-02-26T10:44:40.041442+0000","last_clean":"2024-02-26T10:44:40.041442+0000","last_became_active":"2024-02-26T10:42:42.193959+0000","last_became_peered":"2024-02-26T10:42:42.193959+0000","last_unstale":"2024-02-26T10:44:40.041442+0000","last_undegraded":"2024-02-26T10:44:40.041442+0000","last_fullsized":"2024-02-26T10:44:40.041442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:50.398674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239392+0000","last_change":"2024-02-26T10:42:41.607947+0000","last_active":"2024-02-26T10:44:39.239392+0000","last_peered":"2024-02-26T10:44:39.239392+0000","last_clean":"2024-02-26T10:44:39.239392+0000","last_became_active":"2024-02-26T10:42:41.607522+0000","last_became_peered":"2024-02-26T10:42:41.607522+0000","last_unstale":"2024-02-26T10:44:39.239392+0000","last_undegraded":"2024-02-26T10:44:39.239392+0000","last_fullsized":"2024-02-26T10:44:39.239392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:32:36.647621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234832+0000","last_change":"2024-02-26T10:41:21.177202+0000","last_active":"2024-02-26T10:44:39.234832+0000","last_peered":"2024-02-26T10:44:39.234832+0000","last_clean":"2024-02-26T10:44:39.234832+0000","last_became_active":"2024-02-26T10:41:21.176762+0000","last_became_peered":"2024-02-26T10:41:21.176762+0000","last_unstale":"2024-02-26T10:44:39.234832+0000","last_undegraded":"2024-02-26T10:44:39.234832+0000","last_fullsized":"2024-02-26T10:44:39.234832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_clean_scrub_stamp":"2024-02-26T10:37:25.424790+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:19.619104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.555996+0000","last_change":"2024-02-26T10:42:47.244346+0000","last_active":"2024-02-26T10:44:39.555996+0000","last_peered":"2024-02-26T10:44:39.555996+0000","last_clean":"2024-02-26T10:44:39.555996+0000","last_became_active":"2024-02-26T10:42:47.243872+0000","last_became_peered":"2024-02-26T10:42:47.243872+0000","last_unstale":"2024-02-26T10:44:39.555996+0000","last_undegraded":"2024-02-26T10:44:39.555996+0000","last_fullsized":"2024-02-26T10:44:39.555996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:36.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235371+0000","last_change":"2024-02-26T10:42:42.166266+0000","last_active":"2024-02-26T10:44:39.235371+0000","last_peered":"2024-02-26T10:44:39.235371+0000","last_clean":"2024-02-26T10:44:39.235371+0000","last_became_active":"2024-02-26T10:42:42.165721+0000","last_became_peered":"2024-02-26T10:42:42.165721+0000","last_unstale":"2024-02-26T10:44:39.235371+0000","last_undegraded":"2024-02-26T10:44:39.235371+0000","last_fullsized":"2024-02-26T10:44:39.235371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:32.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234981+0000","last_change":"2024-02-26T10:42:41.606268+0000","last_active":"2024-02-26T10:44:39.234981+0000","last_peered":"2024-02-26T10:44:39.234981+0000","last_clean":"2024-02-26T10:44:39.234981+0000","last_became_active":"2024-02-26T10:42:41.605950+0000","last_became_peered":"2024-02-26T10:42:41.605950+0000","last_unstale":"2024-02-26T10:44:39.234981+0000","last_undegraded":"2024-02-26T10:44:39.234981+0000","last_fullsized":"2024-02-26T10:44:39.234981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:40:03.616987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307850+0000","last_change":"2024-02-26T10:42:47.233203+0000","last_active":"2024-02-26T10:44:39.307850+0000","last_peered":"2024-02-26T10:44:39.307850+0000","last_clean":"2024-02-26T10:44:39.307850+0000","last_became_active":"2024-02-26T10:42:47.232705+0000","last_became_peered":"2024-02-26T10:42:47.232705+0000","last_unstale":"2024-02-26T10:44:39.307850+0000","last_undegraded":"2024-02-26T10:44:39.307850+0000","last_fullsized":"2024-02-26T10:44:39.307850+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:53.818901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308265+0000","last_change":"2024-02-26T10:43:38.479911+0000","last_active":"2024-02-26T10:44:39.308265+0000","last_peered":"2024-02-26T10:44:39.308265+0000","last_clean":"2024-02-26T10:44:39.308265+0000","last_became_active":"2024-02-26T10:43:38.458874+0000","last_became_peered":"2024-02-26T10:43:38.458874+0000","last_unstale":"2024-02-26T10:44:39.308265+0000","last_undegraded":"2024-02-26T10:44:39.308265+0000","last_fullsized":"2024-02-26T10:44:39.308265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:56.093775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237091+0000","last_change":"2024-02-26T10:42:42.167674+0000","last_active":"2024-02-26T10:44:39.237091+0000","last_peered":"2024-02-26T10:44:39.237091+0000","last_clean":"2024-02-26T10:44:39.237091+0000","last_became_active":"2024-02-26T10:42:42.167224+0000","last_became_peered":"2024-02-26T10:42:42.167224+0000","last_unstale":"2024-02-26T10:44:39.237091+0000","last_undegraded":"2024-02-26T10:44:39.237091+0000","last_fullsized":"2024-02-26T10:44:39.237091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:49:20.433913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.039668+0000","last_change":"2024-02-26T10:42:47.244076+0000","last_active":"2024-02-26T10:44:40.039668+0000","last_peered":"2024-02-26T10:44:40.039668+0000","last_clean":"2024-02-26T10:44:40.039668+0000","last_became_active":"2024-02-26T10:42:47.243569+0000","last_became_peered":"2024-02-26T10:42:47.243569+0000","last_unstale":"2024-02-26T10:44:40.039668+0000","last_undegraded":"2024-02-26T10:44:40.039668+0000","last_fullsized":"2024-02-26T10:44:40.039668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:46:19.855621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'12","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:55.201582+0000","last_change":"2024-02-26T10:42:41.623604+0000","last_active":"2024-02-26T10:44:55.201582+0000","last_peered":"2024-02-26T10:44:55.201582+0000","last_clean":"2024-02-26T10:44:55.201582+0000","last_became_active":"2024-02-26T10:42:41.623272+0000","last_became_peered":"2024-02-26T10:42:41.623272+0000","last_unstale":"2024-02-26T10:44:55.201582+0000","last_undegraded":"2024-02-26T10:44:55.201582+0000","last_fullsized":"2024-02-26T10:44:55.201582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:13:57.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042221+0000","last_change":"2024-02-26T10:42:42.186300+0000","last_active":"2024-02-26T10:44:40.042221+0000","last_peered":"2024-02-26T10:44:40.042221+0000","last_clean":"2024-02-26T10:44:40.042221+0000","last_became_active":"2024-02-26T10:42:42.185965+0000","last_became_peered":"2024-02-26T10:42:42.185965+0000","last_unstale":"2024-02-26T10:44:40.042221+0000","last_undegraded":"2024-02-26T10:44:40.042221+0000","last_fullsized":"2024-02-26T10:44:40.042221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:06:50.352308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238335+0000","last_change":"2024-02-26T10:42:47.227753+0000","last_active":"2024-02-26T10:44:39.238335+0000","last_peered":"2024-02-26T10:44:39.238335+0000","last_clean":"2024-02-26T10:44:39.238335+0000","last_became_active":"2024-02-26T10:42:47.227300+0000","last_became_peered":"2024-02-26T10:42:47.227300+0000","last_unstale":"2024-02-26T10:44:39.238335+0000","last_undegraded":"2024-02-26T10:44:39.238335+0000","last_fullsized":"2024-02-26T10:44:39.238335+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:08.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239810+0000","last_change":"2024-02-26T10:42:41.602960+0000","last_active":"2024-02-26T10:44:39.239810+0000","last_peered":"2024-02-26T10:44:39.239810+0000","last_clean":"2024-02-26T10:44:39.239810+0000","last_became_active":"2024-02-26T10:42:41.602676+0000","last_became_peered":"2024-02-26T10:42:41.602676+0000","last_unstale":"2024-02-26T10:44:39.239810+0000","last_undegraded":"2024-02-26T10:44:39.239810+0000","last_fullsized":"2024-02-26T10:44:39.239810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:57.997612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528737+0000","last_change":"2024-02-26T10:42:42.185451+0000","last_active":"2024-02-26T10:44:39.528737+0000","last_peered":"2024-02-26T10:44:39.528737+0000","last_clean":"2024-02-26T10:44:39.528737+0000","last_became_active":"2024-02-26T10:42:42.185116+0000","last_became_peered":"2024-02-26T10:42:42.185116+0000","last_unstale":"2024-02-26T10:44:39.528737+0000","last_undegraded":"2024-02-26T10:44:39.528737+0000","last_fullsized":"2024-02-26T10:44:39.528737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:43:39.823370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239619+0000","last_change":"2024-02-26T10:42:47.250682+0000","last_active":"2024-02-26T10:44:39.239619+0000","last_peered":"2024-02-26T10:44:39.239619+0000","last_clean":"2024-02-26T10:44:39.239619+0000","last_became_active":"2024-02-26T10:42:47.249524+0000","last_became_peered":"2024-02-26T10:42:47.249524+0000","last_unstale":"2024-02-26T10:44:39.239619+0000","last_undegraded":"2024-02-26T10:44:39.239619+0000","last_fullsized":"2024-02-26T10:44:39.239619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:07:01.307206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463955+0000","last_change":"2024-02-26T10:42:41.620764+0000","last_active":"2024-02-26T10:44:39.463955+0000","last_peered":"2024-02-26T10:44:39.463955+0000","last_clean":"2024-02-26T10:44:39.463955+0000","last_became_active":"2024-02-26T10:42:41.620500+0000","last_became_peered":"2024-02-26T10:42:41.620500+0000","last_unstale":"2024-02-26T10:44:39.463955+0000","last_undegraded":"2024-02-26T10:44:39.463955+0000","last_fullsized":"2024-02-26T10:44:39.463955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:15.542396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234462+0000","last_change":"2024-02-26T10:42:42.165060+0000","last_active":"2024-02-26T10:44:39.234462+0000","last_peered":"2024-02-26T10:44:39.234462+0000","last_clean":"2024-02-26T10:44:39.234462+0000","last_became_active":"2024-02-26T10:42:42.164733+0000","last_became_peered":"2024-02-26T10:42:42.164733+0000","last_unstale":"2024-02-26T10:44:39.234462+0000","last_undegraded":"2024-02-26T10:44:39.234462+0000","last_fullsized":"2024-02-26T10:44:39.234462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:34.190747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.044327+0000","last_change":"2024-02-26T10:44:40.044327+0000","last_active":"2024-02-26T10:44:40.044327+0000","last_peered":"2024-02-26T10:44:40.044327+0000","last_clean":"2024-02-26T10:44:40.044327+0000","last_became_active":"2024-02-26T10:44:40.043776+0000","last_became_peered":"2024-02-26T10:44:40.043776+0000","last_unstale":"2024-02-26T10:44:40.044327+0000","last_undegraded":"2024-02-26T10:44:40.044327+0000","last_fullsized":"2024-02-26T10:44:40.044327+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:50:51.502411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040404+0000","last_change":"2024-02-26T10:42:41.624940+0000","last_active":"2024-02-26T10:44:40.040404+0000","last_peered":"2024-02-26T10:44:40.040404+0000","last_clean":"2024-02-26T10:44:40.040404+0000","last_became_active":"2024-02-26T10:42:41.624538+0000","last_became_peered":"2024-02-26T10:42:41.624538+0000","last_unstale":"2024-02-26T10:44:40.040404+0000","last_undegraded":"2024-02-26T10:44:40.040404+0000","last_fullsized":"2024-02-26T10:44:40.040404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:10.052806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308063+0000","last_change":"2024-02-26T10:42:42.174932+0000","last_active":"2024-02-26T10:44:39.308063+0000","last_peered":"2024-02-26T10:44:39.308063+0000","last_clean":"2024-02-26T10:44:39.308063+0000","last_became_active":"2024-02-26T10:42:42.174566+0000","last_became_peered":"2024-02-26T10:42:42.174566+0000","last_unstale":"2024-02-26T10:44:39.308063+0000","last_undegraded":"2024-02-26T10:44:39.308063+0000","last_fullsized":"2024-02-26T10:44:39.308063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:03.242487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239861+0000","last_change":"2024-02-26T10:42:47.250023+0000","last_active":"2024-02-26T10:44:39.239861+0000","last_peered":"2024-02-26T10:44:39.239861+0000","last_clean":"2024-02-26T10:44:39.239861+0000","last_became_active":"2024-02-26T10:42:47.249571+0000","last_became_peered":"2024-02-26T10:42:47.249571+0000","last_unstale":"2024-02-26T10:44:39.239861+0000","last_undegraded":"2024-02-26T10:44:39.239861+0000","last_fullsized":"2024-02-26T10:44:39.239861+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:53.666184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:53.651538+0000","last_change":"2024-02-26T10:42:41.614403+0000","last_active":"2024-02-26T10:44:53.651538+0000","last_peered":"2024-02-26T10:44:53.651538+0000","last_clean":"2024-02-26T10:44:53.651538+0000","last_became_active":"2024-02-26T10:42:41.614057+0000","last_became_peered":"2024-02-26T10:42:41.614057+0000","last_unstale":"2024-02-26T10:44:53.651538+0000","last_undegraded":"2024-02-26T10:44:53.651538+0000","last_fullsized":"2024-02-26T10:44:53.651538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:16:54.739090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041052+0000","last_change":"2024-02-26T10:42:42.187465+0000","last_active":"2024-02-26T10:44:40.041052+0000","last_peered":"2024-02-26T10:44:40.041052+0000","last_clean":"2024-02-26T10:44:40.041052+0000","last_became_active":"2024-02-26T10:42:42.182310+0000","last_became_peered":"2024-02-26T10:42:42.182310+0000","last_unstale":"2024-02-26T10:44:40.041052+0000","last_undegraded":"2024-02-26T10:44:40.041052+0000","last_fullsized":"2024-02-26T10:44:40.041052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:01:56.645330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239438+0000","last_change":"2024-02-26T10:42:47.241751+0000","last_active":"2024-02-26T10:44:39.239438+0000","last_peered":"2024-02-26T10:44:39.239438+0000","last_clean":"2024-02-26T10:44:39.239438+0000","last_became_active":"2024-02-26T10:42:47.241271+0000","last_became_peered":"2024-02-26T10:42:47.241271+0000","last_unstale":"2024-02-26T10:44:39.239438+0000","last_undegraded":"2024-02-26T10:44:39.239438+0000","last_fullsized":"2024-02-26T10:44:39.239438+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:40:39.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234633+0000","last_change":"2024-02-26T10:42:41.607243+0000","last_active":"2024-02-26T10:44:39.234633+0000","last_peered":"2024-02-26T10:44:39.234633+0000","last_clean":"2024-02-26T10:44:39.234633+0000","last_became_active":"2024-02-26T10:42:41.606921+0000","last_became_peered":"2024-02-26T10:42:41.606921+0000","last_unstale":"2024-02-26T10:44:39.234633+0000","last_undegraded":"2024-02-26T10:44:39.234633+0000","last_fullsized":"2024-02-26T10:44:39.234633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:04.626391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235242+0000","last_change":"2024-02-26T10:42:42.178240+0000","last_active":"2024-02-26T10:44:39.235242+0000","last_peered":"2024-02-26T10:44:39.235242+0000","last_clean":"2024-02-26T10:44:39.235242+0000","last_became_active":"2024-02-26T10:42:42.177790+0000","last_became_peered":"2024-02-26T10:42:42.177790+0000","last_unstale":"2024-02-26T10:44:39.235242+0000","last_undegraded":"2024-02-26T10:44:39.235242+0000","last_fullsized":"2024-02-26T10:44:39.235242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:24.110406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463760+0000","last_change":"2024-02-26T10:42:47.233120+0000","last_active":"2024-02-26T10:44:39.463760+0000","last_peered":"2024-02-26T10:44:39.463760+0000","last_clean":"2024-02-26T10:44:39.463760+0000","last_became_active":"2024-02-26T10:42:47.232630+0000","last_became_peered":"2024-02-26T10:42:47.232630+0000","last_unstale":"2024-02-26T10:44:39.463760+0000","last_undegraded":"2024-02-26T10:44:39.463760+0000","last_fullsized":"2024-02-26T10:44:39.463760+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:05:03.023227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239351+0000","last_change":"2024-02-26T10:43:38.477127+0000","last_active":"2024-02-26T10:44:39.239351+0000","last_peered":"2024-02-26T10:44:39.239351+0000","last_clean":"2024-02-26T10:44:39.239351+0000","last_became_active":"2024-02-26T10:43:38.460135+0000","last_became_peered":"2024-02-26T10:43:38.460135+0000","last_unstale":"2024-02-26T10:44:39.239351+0000","last_undegraded":"2024-02-26T10:44:39.239351+0000","last_fullsized":"2024-02-26T10:44:39.239351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:13:16.521943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235617+0000","last_change":"2024-02-26T10:42:42.171821+0000","last_active":"2024-02-26T10:44:39.235617+0000","last_peered":"2024-02-26T10:44:39.235617+0000","last_clean":"2024-02-26T10:44:39.235617+0000","last_became_active":"2024-02-26T10:42:42.171523+0000","last_became_peered":"2024-02-26T10:42:42.171523+0000","last_unstale":"2024-02-26T10:44:39.235617+0000","last_undegraded":"2024-02-26T10:44:39.235617+0000","last_fullsized":"2024-02-26T10:44:39.235617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:00.877998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526701+0000","last_change":"2024-02-26T10:42:47.250072+0000","last_active":"2024-02-26T10:44:39.526701+0000","last_peered":"2024-02-26T10:44:39.526701+0000","last_clean":"2024-02-26T10:44:39.526701+0000","last_became_active":"2024-02-26T10:42:47.249570+0000","last_became_peered":"2024-02-26T10:42:47.249570+0000","last_unstale":"2024-02-26T10:44:39.526701+0000","last_undegraded":"2024-02-26T10:44:39.526701+0000","last_fullsized":"2024-02-26T10:44:39.526701+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:43:03.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557531+0000","last_change":"2024-02-26T10:42:41.614686+0000","last_active":"2024-02-26T10:44:39.557531+0000","last_peered":"2024-02-26T10:44:39.557531+0000","last_clean":"2024-02-26T10:44:39.557531+0000","last_became_active":"2024-02-26T10:42:41.614019+0000","last_became_peered":"2024-02-26T10:42:41.614019+0000","last_unstale":"2024-02-26T10:44:39.557531+0000","last_undegraded":"2024-02-26T10:44:39.557531+0000","last_fullsized":"2024-02-26T10:44:39.557531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:13:28.127288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235390+0000","last_change":"2024-02-26T10:42:42.181774+0000","last_active":"2024-02-26T10:44:39.235390+0000","last_peered":"2024-02-26T10:44:39.235390+0000","last_clean":"2024-02-26T10:44:39.235390+0000","last_became_active":"2024-02-26T10:42:42.181412+0000","last_became_peered":"2024-02-26T10:42:42.181412+0000","last_unstale":"2024-02-26T10:44:39.235390+0000","last_undegraded":"2024-02-26T10:44:39.235390+0000","last_fullsized":"2024-02-26T10:44:39.235390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:09.159964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556176+0000","last_change":"2024-02-26T10:42:47.228247+0000","last_active":"2024-02-26T10:44:39.556176+0000","last_peered":"2024-02-26T10:44:39.556176+0000","last_clean":"2024-02-26T10:44:39.556176+0000","last_became_active":"2024-02-26T10:42:47.227569+0000","last_became_peered":"2024-02-26T10:42:47.227569+0000","last_unstale":"2024-02-26T10:44:39.556176+0000","last_undegraded":"2024-02-26T10:44:39.556176+0000","last_fullsized":"2024-02-26T10:44:39.556176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:15.001264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236850+0000","last_change":"2024-02-26T10:42:41.626573+0000","last_active":"2024-02-26T10:44:39.236850+0000","last_peered":"2024-02-26T10:44:39.236850+0000","last_clean":"2024-02-26T10:44:39.236850+0000","last_became_active":"2024-02-26T10:42:41.624091+0000","last_became_peered":"2024-02-26T10:42:41.624091+0000","last_unstale":"2024-02-26T10:44:39.236850+0000","last_undegraded":"2024-02-26T10:44:39.236850+0000","last_fullsized":"2024-02-26T10:44:39.236850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:24:14.077234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235668+0000","last_change":"2024-02-26T10:42:42.166426+0000","last_active":"2024-02-26T10:44:39.235668+0000","last_peered":"2024-02-26T10:44:39.235668+0000","last_clean":"2024-02-26T10:44:39.235668+0000","last_became_active":"2024-02-26T10:42:42.166082+0000","last_became_peered":"2024-02-26T10:42:42.166082+0000","last_unstale":"2024-02-26T10:44:39.235668+0000","last_undegraded":"2024-02-26T10:44:39.235668+0000","last_fullsized":"2024-02-26T10:44:39.235668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:17.136568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308191+0000","last_change":"2024-02-26T10:42:42.168227+0000","last_active":"2024-02-26T10:44:39.308191+0000","last_peered":"2024-02-26T10:44:39.308191+0000","last_clean":"2024-02-26T10:44:39.308191+0000","last_became_active":"2024-02-26T10:42:42.167897+0000","last_became_peered":"2024-02-26T10:42:42.167897+0000","last_unstale":"2024-02-26T10:44:39.308191+0000","last_undegraded":"2024-02-26T10:44:39.308191+0000","last_fullsized":"2024-02-26T10:44:39.308191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:52:37.391022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238723+0000","last_change":"2024-02-26T10:42:42.168640+0000","last_active":"2024-02-26T10:44:39.238723+0000","last_peered":"2024-02-26T10:44:39.238723+0000","last_clean":"2024-02-26T10:44:39.238723+0000","last_became_active":"2024-02-26T10:42:42.168237+0000","last_became_peered":"2024-02-26T10:42:42.168237+0000","last_unstale":"2024-02-26T10:44:39.238723+0000","last_undegraded":"2024-02-26T10:44:39.238723+0000","last_fullsized":"2024-02-26T10:44:39.238723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:14.853730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238359+0000","last_change":"2024-02-26T10:42:42.192703+0000","last_active":"2024-02-26T10:44:39.238359+0000","last_peered":"2024-02-26T10:44:39.238359+0000","last_clean":"2024-02-26T10:44:39.238359+0000","last_became_active":"2024-02-26T10:42:42.191899+0000","last_became_peered":"2024-02-26T10:42:42.191899+0000","last_unstale":"2024-02-26T10:44:39.238359+0000","last_undegraded":"2024-02-26T10:44:39.238359+0000","last_fullsized":"2024-02-26T10:44:39.238359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:40.479523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528040+0000","last_change":"2024-02-26T10:42:42.188709+0000","last_active":"2024-02-26T10:44:39.528040+0000","last_peered":"2024-02-26T10:44:39.528040+0000","last_clean":"2024-02-26T10:44:39.528040+0000","last_became_active":"2024-02-26T10:42:42.188418+0000","last_became_peered":"2024-02-26T10:42:42.188418+0000","last_unstale":"2024-02-26T10:44:39.528040+0000","last_undegraded":"2024-02-26T10:44:39.528040+0000","last_fullsized":"2024-02-26T10:44:39.528040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:54:15.697746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236925+0000","last_change":"2024-02-26T10:42:42.174221+0000","last_active":"2024-02-26T10:44:39.236925+0000","last_peered":"2024-02-26T10:44:39.236925+0000","last_clean":"2024-02-26T10:44:39.236925+0000","last_became_active":"2024-02-26T10:42:42.171308+0000","last_became_peered":"2024-02-26T10:42:42.171308+0000","last_unstale":"2024-02-26T10:44:39.236925+0000","last_undegraded":"2024-02-26T10:44:39.236925+0000","last_fullsized":"2024-02-26T10:44:39.236925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:46.765856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308467+0000","last_change":"2024-02-26T10:42:42.170749+0000","last_active":"2024-02-26T10:44:39.308467+0000","last_peered":"2024-02-26T10:44:39.308467+0000","last_clean":"2024-02-26T10:44:39.308467+0000","last_became_active":"2024-02-26T10:42:42.169966+0000","last_became_peered":"2024-02-26T10:42:42.169966+0000","last_unstale":"2024-02-26T10:44:39.308467+0000","last_undegraded":"2024-02-26T10:44:39.308467+0000","last_fullsized":"2024-02-26T10:44:39.308467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:02.633195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238925+0000","last_change":"2024-02-26T10:42:42.166529+0000","last_active":"2024-02-26T10:44:39.238925+0000","last_peered":"2024-02-26T10:44:39.238925+0000","last_clean":"2024-02-26T10:44:39.238925+0000","last_became_active":"2024-02-26T10:42:42.166235+0000","last_became_peered":"2024-02-26T10:42:42.166235+0000","last_unstale":"2024-02-26T10:44:39.238925+0000","last_undegraded":"2024-02-26T10:44:39.238925+0000","last_fullsized":"2024-02-26T10:44:39.238925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:03.068328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042639+0000","last_change":"2024-02-26T10:42:42.191312+0000","last_active":"2024-02-26T10:44:40.042639+0000","last_peered":"2024-02-26T10:44:40.042639+0000","last_clean":"2024-02-26T10:44:40.042639+0000","last_became_active":"2024-02-26T10:42:42.190964+0000","last_became_peered":"2024-02-26T10:42:42.190964+0000","last_unstale":"2024-02-26T10:44:40.042639+0000","last_undegraded":"2024-02-26T10:44:40.042639+0000","last_fullsized":"2024-02-26T10:44:40.042639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:24:04.697129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556790+0000","last_change":"2024-02-26T10:42:42.182882+0000","last_active":"2024-02-26T10:44:39.556790+0000","last_peered":"2024-02-26T10:44:39.556790+0000","last_clean":"2024-02-26T10:44:39.556790+0000","last_became_active":"2024-02-26T10:42:42.182551+0000","last_became_peered":"2024-02-26T10:42:42.182551+0000","last_unstale":"2024-02-26T10:44:39.556790+0000","last_undegraded":"2024-02-26T10:44:39.556790+0000","last_fullsized":"2024-02-26T10:44:39.556790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:11:58.578832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040743+0000","last_change":"2024-02-26T10:42:42.193353+0000","last_active":"2024-02-26T10:44:40.040743+0000","last_peered":"2024-02-26T10:44:40.040743+0000","last_clean":"2024-02-26T10:44:40.040743+0000","last_became_active":"2024-02-26T10:42:42.190935+0000","last_became_peered":"2024-02-26T10:42:42.190935+0000","last_unstale":"2024-02-26T10:44:40.040743+0000","last_undegraded":"2024-02-26T10:44:40.040743+0000","last_fullsized":"2024-02-26T10:44:40.040743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:03:14.319769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464179+0000","last_change":"2024-02-26T10:42:42.194529+0000","last_active":"2024-02-26T10:44:39.464179+0000","last_peered":"2024-02-26T10:44:39.464179+0000","last_clean":"2024-02-26T10:44:39.464179+0000","last_became_active":"2024-02-26T10:42:42.194212+0000","last_became_peered":"2024-02-26T10:42:42.194212+0000","last_unstale":"2024-02-26T10:44:39.464179+0000","last_undegraded":"2024-02-26T10:44:39.464179+0000","last_fullsized":"2024-02-26T10:44:39.464179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:36.876672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237256+0000","last_change":"2024-02-26T10:42:42.178119+0000","last_active":"2024-02-26T10:44:39.237256+0000","last_peered":"2024-02-26T10:44:39.237256+0000","last_clean":"2024-02-26T10:44:39.237256+0000","last_became_active":"2024-02-26T10:42:42.177798+0000","last_became_peered":"2024-02-26T10:42:42.177798+0000","last_unstale":"2024-02-26T10:44:39.237256+0000","last_undegraded":"2024-02-26T10:44:39.237256+0000","last_fullsized":"2024-02-26T10:44:39.237256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:52:29.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308152+0000","last_change":"2024-02-26T10:42:42.167380+0000","last_active":"2024-02-26T10:44:39.308152+0000","last_peered":"2024-02-26T10:44:39.308152+0000","last_clean":"2024-02-26T10:44:39.308152+0000","last_became_active":"2024-02-26T10:42:42.167079+0000","last_became_peered":"2024-02-26T10:42:42.167079+0000","last_unstale":"2024-02-26T10:44:39.308152+0000","last_undegraded":"2024-02-26T10:44:39.308152+0000","last_fullsized":"2024-02-26T10:44:39.308152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:14:25.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250986,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76001,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":910,"num_read_kb":534,"num_write":181,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":63341,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":228003,"data_compressed":14031,"data_compressed_allocated":20480,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":532499,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612664,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1196032,"data_stored":1191633,"data_compressed":64863,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776189,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":536697,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037829,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":532387,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299465,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":1128962,"data_compressed":63031,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462991,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":527777,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":599155,"data_compressed":28539,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":529165,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":528846,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":527971,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":605009,"data_compressed":37074,"data_compressed_allocated":49152,"data_compressed_original":135168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724630,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1124018,"data_compressed":63031,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":71468,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":73226,"data_compressed":10367,"data_compressed_allocated":12288,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73578,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:45:46.551 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-02-26T10:45:46.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:45:46.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.813+0000 7fcc8be00640 1 -- 172.21.15.60:0/2265196110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 msgr2=0x7fcc8c10b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.813+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2265196110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 0x7fcc8c10b6f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009ca0 tx=0x7fcc8402c0f0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.813+0000 7fcc8be00640 1 -- 172.21.15.60:0/2265196110 shutdown_connections 2024-02-26T10:45:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.813+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2265196110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c1044a0 0x7fcc8c104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.813+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2265196110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 0x7fcc8c10b6f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.813+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2265196110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc8c1032b0 0x7fcc8c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.813+0000 7fcc8be00640 1 -- 172.21.15.60:0/2265196110 >> 172.21.15.60:0/2265196110 conn(0x7fcc8c0fe840 msgr2=0x7fcc8c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.813+0000 7fcc8be00640 1 -- 172.21.15.60:0/2265196110 shutdown_connections 2024-02-26T10:45:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.813+0000 7fcc8be00640 1 -- 172.21.15.60:0/2265196110 wait complete. 2024-02-26T10:45:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.814+0000 7fcc8be00640 1 Processor -- start 2024-02-26T10:45:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.814+0000 7fcc8be00640 1 -- start start 2024-02-26T10:45:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc8c1032b0 0x7fcc8c199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c1044a0 0x7fcc8c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 0x7fcc8c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8be00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c10ef10 con 0x7fcc8c104e60 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c10ed90 con 0x7fcc8c1044a0 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c10f090 con 0x7fcc8c1032b0 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 0x7fcc8c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc8c1032b0 0x7fcc8c199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 0x7fcc8c1a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35120/0 (socket says 172.21.15.60:35120) 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8b400640 1 -- 172.21.15.60:0/2094725798 learned_addr learned my addr 172.21.15.60:0/2094725798 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.815+0000 7fcc8a000640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c1044a0 0x7fcc8c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc8b400640 1 -- 172.21.15.60:0/2094725798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc8c1032b0 msgr2=0x7fcc8c199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc8b400640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc8c1032b0 0x7fcc8c199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc8b400640 1 -- 172.21.15.60:0/2094725798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c1044a0 msgr2=0x7fcc8c199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc8b400640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c1044a0 0x7fcc8c199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc8b400640 1 -- 172.21.15.60:0/2094725798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c1a1140 con 0x7fcc8c104e60 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc8aa00640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc8c1032b0 0x7fcc8c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc8a000640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c1044a0 0x7fcc8c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc8b400640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 0x7fcc8c1a0ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc8402c630 tx=0x7fcc84032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc83400640 1 -- 172.21.15.60:0/2094725798 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84041020 con 0x7fcc8c104e60 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8c1a13d0 con 0x7fcc8c104e60 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc83400640 1 -- 172.21.15.60:0/2094725798 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc84034cd0 con 0x7fcc8c104e60 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.816+0000 7fcc83400640 1 -- 172.21.15.60:0/2094725798 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84039040 con 0x7fcc8c104e60 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.817+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8c1a1910 con 0x7fcc8c104e60 2024-02-26T10:45:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.818+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8c10a570 con 0x7fcc8c104e60 2024-02-26T10:45:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.819+0000 7fcc83400640 1 -- 172.21.15.60:0/2094725798 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fcc84044020 con 0x7fcc8c104e60 2024-02-26T10:45:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.821+0000 7fcc83400640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcc700a1e90 0x7fcc700a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.821+0000 7fcc83400640 1 -- 172.21.15.60:0/2094725798 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fcc840effc0 con 0x7fcc8c104e60 2024-02-26T10:45:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.821+0000 7fcc8aa00640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcc700a1e90 0x7fcc700a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.822+0000 7fcc8aa00640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcc700a1e90 0x7fcc700a4350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c104300 tx=0x7fcc7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:46.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:46.825+0000 7fcc83400640 1 -- 172.21.15.60:0/2094725798 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc8404b020 con 0x7fcc8c104e60 2024-02-26T10:45:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.025+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 --> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc8c0616a0 con 0x7fcc700a1e90 2024-02-26T10:45:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.041+0000 7fcc83400640 1 -- 172.21.15.60:0/2094725798 <== mgr.14152 v2:172.21.15.60:6800/3493132094 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340796 (secure 0 0 0) 0x7fcc8c0616a0 con 0x7fcc700a1e90 2024-02-26T10:45:47.041 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:47.043 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.049+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcc700a1e90 msgr2=0x7fcc700a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.049+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcc700a1e90 0x7fcc700a4350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c104300 tx=0x7fcc7c005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.049+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 msgr2=0x7fcc8c1a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.049+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 0x7fcc8c1a0ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc8402c630 tx=0x7fcc84032ea0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.050+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 shutdown_connections 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.050+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcc700a1e90 0x7fcc700a4350 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.050+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c1044a0 0x7fcc8c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.050+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc8c104e60 0x7fcc8c1a0ad0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.050+0000 7fcc8be00640 1 --2- 172.21.15.60:0/2094725798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc8c1032b0 0x7fcc8c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.050+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 >> 172.21.15.60:0/2094725798 conn(0x7fcc8c0fe840 msgr2=0x7fcc8c107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.050+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 shutdown_connections 2024-02-26T10:45:47.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.050+0000 7fcc8be00640 1 -- 172.21.15.60:0/2094725798 wait complete. 2024-02-26T10:45:47.066 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-02-26T10:45:46.965596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666603,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1036,"num_read_kb":643,"num_write":366,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1244077,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":795,"ondisk_log_size":795,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374368,"kb_used_data":12708,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567456,"statfs":{"total":1151940427776,"available":1151557074944,"internally_reserved":0,"allocated":13012992,"data_stored":8364119,"data_compressed":443487,"data_compressed_allocated":598016,"data_compressed_original":2732032,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":41,"num_write":12,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004717"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308577+0000","last_change":"2024-02-26T10:42:42.176634+0000","last_active":"2024-02-26T10:44:39.308577+0000","last_peered":"2024-02-26T10:44:39.308577+0000","last_clean":"2024-02-26T10:44:39.308577+0000","last_became_active":"2024-02-26T10:42:42.176386+0000","last_became_peered":"2024-02-26T10:42:42.176386+0000","last_unstale":"2024-02-26T10:44:39.308577+0000","last_undegraded":"2024-02-26T10:44:39.308577+0000","last_fullsized":"2024-02-26T10:44:39.308577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:45.005506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528482+0000","last_change":"2024-02-26T10:42:42.176769+0000","last_active":"2024-02-26T10:44:39.528482+0000","last_peered":"2024-02-26T10:44:39.528482+0000","last_clean":"2024-02-26T10:44:39.528482+0000","last_became_active":"2024-02-26T10:42:42.174868+0000","last_became_peered":"2024-02-26T10:42:42.174868+0000","last_unstale":"2024-02-26T10:44:39.528482+0000","last_undegraded":"2024-02-26T10:44:39.528482+0000","last_fullsized":"2024-02-26T10:44:39.528482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:16.202371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528227+0000","last_change":"2024-02-26T10:42:42.184410+0000","last_active":"2024-02-26T10:44:39.528227+0000","last_peered":"2024-02-26T10:44:39.528227+0000","last_clean":"2024-02-26T10:44:39.528227+0000","last_became_active":"2024-02-26T10:42:42.182986+0000","last_became_peered":"2024-02-26T10:42:42.182986+0000","last_unstale":"2024-02-26T10:44:39.528227+0000","last_undegraded":"2024-02-26T10:44:39.528227+0000","last_fullsized":"2024-02-26T10:44:39.528227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:28:45.445325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409010+0000","last_change":"2024-02-26T10:43:38.462828+0000","last_active":"2024-02-26T10:44:39.409010+0000","last_peered":"2024-02-26T10:44:39.409010+0000","last_clean":"2024-02-26T10:44:39.409010+0000","last_became_active":"2024-02-26T10:43:38.461014+0000","last_became_peered":"2024-02-26T10:43:38.461014+0000","last_unstale":"2024-02-26T10:44:39.409010+0000","last_undegraded":"2024-02-26T10:44:39.409010+0000","last_fullsized":"2024-02-26T10:44:39.409010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:07.857849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234724+0000","last_change":"2024-02-26T10:42:42.192448+0000","last_active":"2024-02-26T10:44:39.234724+0000","last_peered":"2024-02-26T10:44:39.234724+0000","last_clean":"2024-02-26T10:44:39.234724+0000","last_became_active":"2024-02-26T10:42:42.192133+0000","last_became_peered":"2024-02-26T10:42:42.192133+0000","last_unstale":"2024-02-26T10:44:39.234724+0000","last_undegraded":"2024-02-26T10:44:39.234724+0000","last_fullsized":"2024-02-26T10:44:39.234724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:43:49.057174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.256983+0000","last_change":"2024-02-26T10:42:42.169999+0000","last_active":"2024-02-26T10:44:39.256983+0000","last_peered":"2024-02-26T10:44:39.256983+0000","last_clean":"2024-02-26T10:44:39.256983+0000","last_became_active":"2024-02-26T10:42:42.169678+0000","last_became_peered":"2024-02-26T10:42:42.169678+0000","last_unstale":"2024-02-26T10:44:39.256983+0000","last_undegraded":"2024-02-26T10:44:39.256983+0000","last_fullsized":"2024-02-26T10:44:39.256983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:54.435203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234970+0000","last_change":"2024-02-26T10:42:42.177538+0000","last_active":"2024-02-26T10:44:39.234970+0000","last_peered":"2024-02-26T10:44:39.234970+0000","last_clean":"2024-02-26T10:44:39.234970+0000","last_became_active":"2024-02-26T10:42:42.177204+0000","last_became_peered":"2024-02-26T10:42:42.177204+0000","last_unstale":"2024-02-26T10:44:39.234970+0000","last_undegraded":"2024-02-26T10:44:39.234970+0000","last_fullsized":"2024-02-26T10:44:39.234970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:49:09.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040510+0000","last_change":"2024-02-26T10:42:42.191342+0000","last_active":"2024-02-26T10:44:40.040510+0000","last_peered":"2024-02-26T10:44:40.040510+0000","last_clean":"2024-02-26T10:44:40.040510+0000","last_became_active":"2024-02-26T10:42:42.190990+0000","last_became_peered":"2024-02-26T10:42:42.190990+0000","last_unstale":"2024-02-26T10:44:40.040510+0000","last_undegraded":"2024-02-26T10:44:40.040510+0000","last_fullsized":"2024-02-26T10:44:40.040510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:20:45.190300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464601+0000","last_change":"2024-02-26T10:42:42.173175+0000","last_active":"2024-02-26T10:44:39.464601+0000","last_peered":"2024-02-26T10:44:39.464601+0000","last_clean":"2024-02-26T10:44:39.464601+0000","last_became_active":"2024-02-26T10:42:42.172828+0000","last_became_peered":"2024-02-26T10:42:42.172828+0000","last_unstale":"2024-02-26T10:44:39.464601+0000","last_undegraded":"2024-02-26T10:44:39.464601+0000","last_fullsized":"2024-02-26T10:44:39.464601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:17:34.156363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257498+0000","last_change":"2024-02-26T10:42:42.167267+0000","last_active":"2024-02-26T10:44:39.257498+0000","last_peered":"2024-02-26T10:44:39.257498+0000","last_clean":"2024-02-26T10:44:39.257498+0000","last_became_active":"2024-02-26T10:42:42.166948+0000","last_became_peered":"2024-02-26T10:42:42.166948+0000","last_unstale":"2024-02-26T10:44:39.257498+0000","last_undegraded":"2024-02-26T10:44:39.257498+0000","last_fullsized":"2024-02-26T10:44:39.257498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:52.211522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235590+0000","last_change":"2024-02-26T10:42:42.171299+0000","last_active":"2024-02-26T10:44:39.235590+0000","last_peered":"2024-02-26T10:44:39.235590+0000","last_clean":"2024-02-26T10:44:39.235590+0000","last_became_active":"2024-02-26T10:42:42.170616+0000","last_became_peered":"2024-02-26T10:42:42.170616+0000","last_unstale":"2024-02-26T10:44:39.235590+0000","last_undegraded":"2024-02-26T10:44:39.235590+0000","last_fullsized":"2024-02-26T10:44:39.235590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:50.320859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239431+0000","last_change":"2024-02-26T10:42:42.168408+0000","last_active":"2024-02-26T10:44:39.239431+0000","last_peered":"2024-02-26T10:44:39.239431+0000","last_clean":"2024-02-26T10:44:39.239431+0000","last_became_active":"2024-02-26T10:42:42.167150+0000","last_became_peered":"2024-02-26T10:42:42.167150+0000","last_unstale":"2024-02-26T10:44:39.239431+0000","last_undegraded":"2024-02-26T10:44:39.239431+0000","last_fullsized":"2024-02-26T10:44:39.239431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:34.763331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236385+0000","last_change":"2024-02-26T10:42:42.171720+0000","last_active":"2024-02-26T10:44:39.236385+0000","last_peered":"2024-02-26T10:44:39.236385+0000","last_clean":"2024-02-26T10:44:39.236385+0000","last_became_active":"2024-02-26T10:42:42.171309+0000","last_became_peered":"2024-02-26T10:42:42.171309+0000","last_unstale":"2024-02-26T10:44:39.236385+0000","last_undegraded":"2024-02-26T10:44:39.236385+0000","last_fullsized":"2024-02-26T10:44:39.236385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:20:52.151553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527374+0000","last_change":"2024-02-26T10:42:42.176857+0000","last_active":"2024-02-26T10:44:39.527374+0000","last_peered":"2024-02-26T10:44:39.527374+0000","last_clean":"2024-02-26T10:44:39.527374+0000","last_became_active":"2024-02-26T10:42:42.176385+0000","last_became_peered":"2024-02-26T10:42:42.176385+0000","last_unstale":"2024-02-26T10:44:39.527374+0000","last_undegraded":"2024-02-26T10:44:39.527374+0000","last_fullsized":"2024-02-26T10:44:39.527374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:45:51.852212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.527401+0000","last_change":"2024-02-26T10:42:42.174682+0000","last_active":"2024-02-26T10:44:39.527401+0000","last_peered":"2024-02-26T10:44:39.527401+0000","last_clean":"2024-02-26T10:44:39.527401+0000","last_became_active":"2024-02-26T10:42:42.174328+0000","last_became_peered":"2024-02-26T10:42:42.174328+0000","last_unstale":"2024-02-26T10:44:39.527401+0000","last_undegraded":"2024-02-26T10:44:39.527401+0000","last_fullsized":"2024-02-26T10:44:39.527401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:47.161020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463946+0000","last_change":"2024-02-26T10:42:42.185289+0000","last_active":"2024-02-26T10:44:39.463946+0000","last_peered":"2024-02-26T10:44:39.463946+0000","last_clean":"2024-02-26T10:44:39.463946+0000","last_became_active":"2024-02-26T10:42:42.185000+0000","last_became_peered":"2024-02-26T10:42:42.185000+0000","last_unstale":"2024-02-26T10:44:39.463946+0000","last_undegraded":"2024-02-26T10:44:39.463946+0000","last_fullsized":"2024-02-26T10:44:39.463946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:48.552584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308483+0000","last_change":"2024-02-26T10:42:42.176558+0000","last_active":"2024-02-26T10:44:39.308483+0000","last_peered":"2024-02-26T10:44:39.308483+0000","last_clean":"2024-02-26T10:44:39.308483+0000","last_became_active":"2024-02-26T10:42:42.176334+0000","last_became_peered":"2024-02-26T10:42:42.176334+0000","last_unstale":"2024-02-26T10:44:39.308483+0000","last_undegraded":"2024-02-26T10:44:39.308483+0000","last_fullsized":"2024-02-26T10:44:39.308483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:59.237975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409740+0000","last_change":"2024-02-26T10:42:42.168534+0000","last_active":"2024-02-26T10:44:39.409740+0000","last_peered":"2024-02-26T10:44:39.409740+0000","last_clean":"2024-02-26T10:44:39.409740+0000","last_became_active":"2024-02-26T10:42:42.168179+0000","last_became_peered":"2024-02-26T10:42:42.168179+0000","last_unstale":"2024-02-26T10:44:39.409740+0000","last_undegraded":"2024-02-26T10:44:39.409740+0000","last_fullsized":"2024-02-26T10:44:39.409740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:09.625969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238870+0000","last_change":"2024-02-26T10:42:42.192446+0000","last_active":"2024-02-26T10:44:39.238870+0000","last_peered":"2024-02-26T10:44:39.238870+0000","last_clean":"2024-02-26T10:44:39.238870+0000","last_became_active":"2024-02-26T10:42:42.191863+0000","last_became_peered":"2024-02-26T10:42:42.191863+0000","last_unstale":"2024-02-26T10:44:39.238870+0000","last_undegraded":"2024-02-26T10:44:39.238870+0000","last_fullsized":"2024-02-26T10:44:39.238870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:30:19.477529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238594+0000","last_change":"2024-02-26T10:42:47.257605+0000","last_active":"2024-02-26T10:44:39.238594+0000","last_peered":"2024-02-26T10:44:39.238594+0000","last_clean":"2024-02-26T10:44:39.238594+0000","last_became_active":"2024-02-26T10:42:47.257206+0000","last_became_peered":"2024-02-26T10:42:47.257206+0000","last_unstale":"2024-02-26T10:44:39.238594+0000","last_undegraded":"2024-02-26T10:44:39.238594+0000","last_fullsized":"2024-02-26T10:44:39.238594+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:26:17.681862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.409986+0000","last_change":"2024-02-26T10:42:41.613087+0000","last_active":"2024-02-26T10:44:39.409986+0000","last_peered":"2024-02-26T10:44:39.409986+0000","last_clean":"2024-02-26T10:44:39.409986+0000","last_became_active":"2024-02-26T10:42:41.612664+0000","last_became_peered":"2024-02-26T10:42:41.612664+0000","last_unstale":"2024-02-26T10:44:39.409986+0000","last_undegraded":"2024-02-26T10:44:39.409986+0000","last_fullsized":"2024-02-26T10:44:39.409986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:00.352166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042024+0000","last_change":"2024-02-26T10:42:42.191438+0000","last_active":"2024-02-26T10:44:40.042024+0000","last_peered":"2024-02-26T10:44:40.042024+0000","last_clean":"2024-02-26T10:44:40.042024+0000","last_became_active":"2024-02-26T10:42:42.191095+0000","last_became_peered":"2024-02-26T10:42:42.191095+0000","last_unstale":"2024-02-26T10:44:40.042024+0000","last_undegraded":"2024-02-26T10:44:40.042024+0000","last_fullsized":"2024-02-26T10:44:40.042024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:15:20.918715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238951+0000","last_change":"2024-02-26T10:42:47.247503+0000","last_active":"2024-02-26T10:44:39.238951+0000","last_peered":"2024-02-26T10:44:39.238951+0000","last_clean":"2024-02-26T10:44:39.238951+0000","last_became_active":"2024-02-26T10:42:47.247121+0000","last_became_peered":"2024-02-26T10:42:47.247121+0000","last_unstale":"2024-02-26T10:44:39.238951+0000","last_undegraded":"2024-02-26T10:44:39.238951+0000","last_fullsized":"2024-02-26T10:44:39.238951+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:01:38.733131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042711+0000","last_change":"2024-02-26T10:42:41.626227+0000","last_active":"2024-02-26T10:44:40.042711+0000","last_peered":"2024-02-26T10:44:40.042711+0000","last_clean":"2024-02-26T10:44:40.042711+0000","last_became_active":"2024-02-26T10:42:41.625760+0000","last_became_peered":"2024-02-26T10:42:41.625760+0000","last_unstale":"2024-02-26T10:44:40.042711+0000","last_undegraded":"2024-02-26T10:44:40.042711+0000","last_fullsized":"2024-02-26T10:44:40.042711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:04.600151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528152+0000","last_change":"2024-02-26T10:42:42.182808+0000","last_active":"2024-02-26T10:44:39.528152+0000","last_peered":"2024-02-26T10:44:39.528152+0000","last_clean":"2024-02-26T10:44:39.528152+0000","last_became_active":"2024-02-26T10:42:42.182485+0000","last_became_peered":"2024-02-26T10:42:42.182485+0000","last_unstale":"2024-02-26T10:44:39.528152+0000","last_undegraded":"2024-02-26T10:44:39.528152+0000","last_fullsized":"2024-02-26T10:44:39.528152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:45.205969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239151+0000","last_change":"2024-02-26T10:42:47.250694+0000","last_active":"2024-02-26T10:44:39.239151+0000","last_peered":"2024-02-26T10:44:39.239151+0000","last_clean":"2024-02-26T10:44:39.239151+0000","last_became_active":"2024-02-26T10:42:47.249566+0000","last_became_peered":"2024-02-26T10:42:47.249566+0000","last_unstale":"2024-02-26T10:44:39.239151+0000","last_undegraded":"2024-02-26T10:44:39.239151+0000","last_fullsized":"2024-02-26T10:44:39.239151+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:44.016413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235108+0000","last_change":"2024-02-26T10:42:41.606849+0000","last_active":"2024-02-26T10:44:39.235108+0000","last_peered":"2024-02-26T10:44:39.235108+0000","last_clean":"2024-02-26T10:44:39.235108+0000","last_became_active":"2024-02-26T10:42:41.606494+0000","last_became_peered":"2024-02-26T10:42:41.606494+0000","last_unstale":"2024-02-26T10:44:39.235108+0000","last_undegraded":"2024-02-26T10:44:39.235108+0000","last_fullsized":"2024-02-26T10:44:39.235108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:36.685594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464300+0000","last_change":"2024-02-26T10:42:42.175926+0000","last_active":"2024-02-26T10:44:39.464300+0000","last_peered":"2024-02-26T10:44:39.464300+0000","last_clean":"2024-02-26T10:44:39.464300+0000","last_became_active":"2024-02-26T10:42:42.175614+0000","last_became_peered":"2024-02-26T10:42:42.175614+0000","last_unstale":"2024-02-26T10:44:39.464300+0000","last_undegraded":"2024-02-26T10:44:39.464300+0000","last_fullsized":"2024-02-26T10:44:39.464300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:20:20.829849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.044113+0000","last_change":"2024-02-26T10:44:40.043633+0000","last_active":"2024-02-26T10:44:40.044113+0000","last_peered":"2024-02-26T10:44:40.044113+0000","last_clean":"2024-02-26T10:44:40.044113+0000","last_became_active":"2024-02-26T10:44:40.043209+0000","last_became_peered":"2024-02-26T10:44:40.043209+0000","last_unstale":"2024-02-26T10:44:40.044113+0000","last_undegraded":"2024-02-26T10:44:40.044113+0000","last_fullsized":"2024-02-26T10:44:40.044113+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:43.466990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234893+0000","last_change":"2024-02-26T10:42:41.606407+0000","last_active":"2024-02-26T10:44:39.234893+0000","last_peered":"2024-02-26T10:44:39.234893+0000","last_clean":"2024-02-26T10:44:39.234893+0000","last_became_active":"2024-02-26T10:42:41.606094+0000","last_became_peered":"2024-02-26T10:42:41.606094+0000","last_unstale":"2024-02-26T10:44:39.234893+0000","last_undegraded":"2024-02-26T10:44:39.234893+0000","last_fullsized":"2024-02-26T10:44:39.234893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:19:53.673017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557475+0000","last_change":"2024-02-26T10:42:42.172967+0000","last_active":"2024-02-26T10:44:39.557475+0000","last_peered":"2024-02-26T10:44:39.557475+0000","last_clean":"2024-02-26T10:44:39.557475+0000","last_became_active":"2024-02-26T10:42:42.172675+0000","last_became_peered":"2024-02-26T10:42:42.172675+0000","last_unstale":"2024-02-26T10:44:39.557475+0000","last_undegraded":"2024-02-26T10:44:39.557475+0000","last_fullsized":"2024-02-26T10:44:39.557475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238546+0000","last_change":"2024-02-26T10:42:47.229375+0000","last_active":"2024-02-26T10:44:39.238546+0000","last_peered":"2024-02-26T10:44:39.238546+0000","last_clean":"2024-02-26T10:44:39.238546+0000","last_became_active":"2024-02-26T10:42:47.228874+0000","last_became_peered":"2024-02-26T10:42:47.228874+0000","last_unstale":"2024-02-26T10:44:39.238546+0000","last_undegraded":"2024-02-26T10:44:39.238546+0000","last_fullsized":"2024-02-26T10:44:39.238546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:02.188519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":190,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:41.266222+0000","last_change":"2024-02-26T10:42:41.602783+0000","last_active":"2024-02-26T10:45:41.266222+0000","last_peered":"2024-02-26T10:45:41.266222+0000","last_clean":"2024-02-26T10:45:41.266222+0000","last_became_active":"2024-02-26T10:42:41.602491+0000","last_became_peered":"2024-02-26T10:42:41.602491+0000","last_unstale":"2024-02-26T10:45:41.266222+0000","last_undegraded":"2024-02-26T10:45:41.266222+0000","last_fullsized":"2024-02-26T10:45:41.266222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:23:33.070543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234508+0000","last_change":"2024-02-26T10:43:38.461429+0000","last_active":"2024-02-26T10:44:39.234508+0000","last_peered":"2024-02-26T10:44:39.234508+0000","last_clean":"2024-02-26T10:44:39.234508+0000","last_became_active":"2024-02-26T10:43:38.461019+0000","last_became_peered":"2024-02-26T10:43:38.461019+0000","last_unstale":"2024-02-26T10:44:39.234508+0000","last_undegraded":"2024-02-26T10:44:39.234508+0000","last_fullsized":"2024-02-26T10:44:39.234508+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:20.911179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.559125+0000","last_change":"2024-02-26T10:44:39.558649+0000","last_active":"2024-02-26T10:44:39.559125+0000","last_peered":"2024-02-26T10:44:39.559125+0000","last_clean":"2024-02-26T10:44:39.559125+0000","last_became_active":"2024-02-26T10:44:39.558013+0000","last_became_peered":"2024-02-26T10:44:39.558013+0000","last_unstale":"2024-02-26T10:44:39.559125+0000","last_undegraded":"2024-02-26T10:44:39.559125+0000","last_fullsized":"2024-02-26T10:44:39.559125+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:45.879391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.256942+0000","last_change":"2024-02-26T10:42:41.602887+0000","last_active":"2024-02-26T10:44:39.256942+0000","last_peered":"2024-02-26T10:44:39.256942+0000","last_clean":"2024-02-26T10:44:39.256942+0000","last_became_active":"2024-02-26T10:42:41.602564+0000","last_became_peered":"2024-02-26T10:42:41.602564+0000","last_unstale":"2024-02-26T10:44:39.256942+0000","last_undegraded":"2024-02-26T10:44:39.256942+0000","last_fullsized":"2024-02-26T10:44:39.256942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:22:41.711283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556964+0000","last_change":"2024-02-26T10:42:42.192700+0000","last_active":"2024-02-26T10:44:39.556964+0000","last_peered":"2024-02-26T10:44:39.556964+0000","last_clean":"2024-02-26T10:44:39.556964+0000","last_became_active":"2024-02-26T10:42:42.192387+0000","last_became_peered":"2024-02-26T10:42:42.192387+0000","last_unstale":"2024-02-26T10:44:39.556964+0000","last_undegraded":"2024-02-26T10:44:39.556964+0000","last_fullsized":"2024-02-26T10:44:39.556964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:21:15.104872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307862+0000","last_change":"2024-02-26T10:42:47.247000+0000","last_active":"2024-02-26T10:44:39.307862+0000","last_peered":"2024-02-26T10:44:39.307862+0000","last_clean":"2024-02-26T10:44:39.307862+0000","last_became_active":"2024-02-26T10:42:47.246520+0000","last_became_peered":"2024-02-26T10:42:47.246520+0000","last_unstale":"2024-02-26T10:44:39.307862+0000","last_undegraded":"2024-02-26T10:44:39.307862+0000","last_fullsized":"2024-02-26T10:44:39.307862+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:29.977858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235812+0000","last_change":"2024-02-26T10:42:41.609460+0000","last_active":"2024-02-26T10:44:39.235812+0000","last_peered":"2024-02-26T10:44:39.235812+0000","last_clean":"2024-02-26T10:44:39.235812+0000","last_became_active":"2024-02-26T10:42:41.608981+0000","last_became_peered":"2024-02-26T10:42:41.608981+0000","last_unstale":"2024-02-26T10:44:39.235812+0000","last_undegraded":"2024-02-26T10:44:39.235812+0000","last_fullsized":"2024-02-26T10:44:39.235812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:33:04.599910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040491+0000","last_change":"2024-02-26T10:42:42.185359+0000","last_active":"2024-02-26T10:44:40.040491+0000","last_peered":"2024-02-26T10:44:40.040491+0000","last_clean":"2024-02-26T10:44:40.040491+0000","last_became_active":"2024-02-26T10:42:42.184221+0000","last_became_peered":"2024-02-26T10:42:42.184221+0000","last_unstale":"2024-02-26T10:44:40.040491+0000","last_undegraded":"2024-02-26T10:44:40.040491+0000","last_fullsized":"2024-02-26T10:44:40.040491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:53.907290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526689+0000","last_change":"2024-02-26T10:42:47.241700+0000","last_active":"2024-02-26T10:44:39.526689+0000","last_peered":"2024-02-26T10:44:39.526689+0000","last_clean":"2024-02-26T10:44:39.526689+0000","last_became_active":"2024-02-26T10:42:47.240667+0000","last_became_peered":"2024-02-26T10:42:47.240667+0000","last_unstale":"2024-02-26T10:44:39.526689+0000","last_undegraded":"2024-02-26T10:44:39.526689+0000","last_fullsized":"2024-02-26T10:44:39.526689+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:19.071244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556576+0000","last_change":"2024-02-26T10:42:41.604519+0000","last_active":"2024-02-26T10:44:39.556576+0000","last_peered":"2024-02-26T10:44:39.556576+0000","last_clean":"2024-02-26T10:44:39.556576+0000","last_became_active":"2024-02-26T10:42:41.603596+0000","last_became_peered":"2024-02-26T10:42:41.603596+0000","last_unstale":"2024-02-26T10:44:39.556576+0000","last_undegraded":"2024-02-26T10:44:39.556576+0000","last_fullsized":"2024-02-26T10:44:39.556576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:23:39.764730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408161+0000","last_change":"2024-02-26T10:42:42.187532+0000","last_active":"2024-02-26T10:44:39.408161+0000","last_peered":"2024-02-26T10:44:39.408161+0000","last_clean":"2024-02-26T10:44:39.408161+0000","last_became_active":"2024-02-26T10:42:42.185635+0000","last_became_peered":"2024-02-26T10:42:42.185635+0000","last_unstale":"2024-02-26T10:44:39.408161+0000","last_undegraded":"2024-02-26T10:44:39.408161+0000","last_fullsized":"2024-02-26T10:44:39.408161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:16.894747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041446+0000","last_change":"2024-02-26T10:42:47.233507+0000","last_active":"2024-02-26T10:44:40.041446+0000","last_peered":"2024-02-26T10:44:40.041446+0000","last_clean":"2024-02-26T10:44:40.041446+0000","last_became_active":"2024-02-26T10:42:47.229589+0000","last_became_peered":"2024-02-26T10:42:47.229589+0000","last_unstale":"2024-02-26T10:44:40.041446+0000","last_undegraded":"2024-02-26T10:44:40.041446+0000","last_fullsized":"2024-02-26T10:44:40.041446+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:02:42.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":185,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:41.265079+0000","last_change":"2024-02-26T10:42:41.613952+0000","last_active":"2024-02-26T10:45:41.265079+0000","last_peered":"2024-02-26T10:45:41.265079+0000","last_clean":"2024-02-26T10:45:41.265079+0000","last_became_active":"2024-02-26T10:42:41.613656+0000","last_became_peered":"2024-02-26T10:42:41.613656+0000","last_unstale":"2024-02-26T10:45:41.265079+0000","last_undegraded":"2024-02-26T10:45:41.265079+0000","last_fullsized":"2024-02-26T10:45:41.265079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:14.090821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464609+0000","last_change":"2024-02-26T10:42:42.178357+0000","last_active":"2024-02-26T10:44:39.464609+0000","last_peered":"2024-02-26T10:44:39.464609+0000","last_clean":"2024-02-26T10:44:39.464609+0000","last_became_active":"2024-02-26T10:42:42.178051+0000","last_became_peered":"2024-02-26T10:42:42.178051+0000","last_unstale":"2024-02-26T10:44:39.464609+0000","last_undegraded":"2024-02-26T10:44:39.464609+0000","last_fullsized":"2024-02-26T10:44:39.464609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:13:01.326122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406187+0000","last_change":"2024-02-26T10:42:47.246231+0000","last_active":"2024-02-26T10:44:39.406187+0000","last_peered":"2024-02-26T10:44:39.406187+0000","last_clean":"2024-02-26T10:44:39.406187+0000","last_became_active":"2024-02-26T10:42:47.245739+0000","last_became_peered":"2024-02-26T10:42:47.245739+0000","last_unstale":"2024-02-26T10:44:39.406187+0000","last_undegraded":"2024-02-26T10:44:39.406187+0000","last_fullsized":"2024-02-26T10:44:39.406187+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:30.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235526+0000","last_change":"2024-02-26T10:42:41.603126+0000","last_active":"2024-02-26T10:44:39.235526+0000","last_peered":"2024-02-26T10:44:39.235526+0000","last_clean":"2024-02-26T10:44:39.235526+0000","last_became_active":"2024-02-26T10:42:41.602790+0000","last_became_peered":"2024-02-26T10:42:41.602790+0000","last_unstale":"2024-02-26T10:44:39.235526+0000","last_undegraded":"2024-02-26T10:44:39.235526+0000","last_fullsized":"2024-02-26T10:44:39.235526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:31:58.908209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528535+0000","last_change":"2024-02-26T10:43:38.459527+0000","last_active":"2024-02-26T10:44:39.528535+0000","last_peered":"2024-02-26T10:44:39.528535+0000","last_clean":"2024-02-26T10:44:39.528535+0000","last_became_active":"2024-02-26T10:43:38.459150+0000","last_became_peered":"2024-02-26T10:43:38.459150+0000","last_unstale":"2024-02-26T10:44:39.528535+0000","last_undegraded":"2024-02-26T10:44:39.528535+0000","last_fullsized":"2024-02-26T10:44:39.528535+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:21:54.028328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239304+0000","last_change":"2024-02-26T10:42:47.250075+0000","last_active":"2024-02-26T10:44:39.239304+0000","last_peered":"2024-02-26T10:44:39.239304+0000","last_clean":"2024-02-26T10:44:39.239304+0000","last_became_active":"2024-02-26T10:42:47.249490+0000","last_became_peered":"2024-02-26T10:42:47.249490+0000","last_unstale":"2024-02-26T10:44:39.239304+0000","last_undegraded":"2024-02-26T10:44:39.239304+0000","last_fullsized":"2024-02-26T10:44:39.239304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:42.659282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557237+0000","last_change":"2024-02-26T10:42:41.615314+0000","last_active":"2024-02-26T10:44:39.557237+0000","last_peered":"2024-02-26T10:44:39.557237+0000","last_clean":"2024-02-26T10:44:39.557237+0000","last_became_active":"2024-02-26T10:42:41.615091+0000","last_became_peered":"2024-02-26T10:42:41.615091+0000","last_unstale":"2024-02-26T10:44:39.557237+0000","last_undegraded":"2024-02-26T10:44:39.557237+0000","last_fullsized":"2024-02-26T10:44:39.557237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:41:52.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464449+0000","last_change":"2024-02-26T10:42:42.176609+0000","last_active":"2024-02-26T10:44:39.464449+0000","last_peered":"2024-02-26T10:44:39.464449+0000","last_clean":"2024-02-26T10:44:39.464449+0000","last_became_active":"2024-02-26T10:42:42.176311+0000","last_became_peered":"2024-02-26T10:42:42.176311+0000","last_unstale":"2024-02-26T10:44:39.464449+0000","last_undegraded":"2024-02-26T10:44:39.464449+0000","last_fullsized":"2024-02-26T10:44:39.464449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:25:38.415311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.406280+0000","last_change":"2024-02-26T10:42:47.250255+0000","last_active":"2024-02-26T10:44:39.406280+0000","last_peered":"2024-02-26T10:44:39.406280+0000","last_clean":"2024-02-26T10:44:39.406280+0000","last_became_active":"2024-02-26T10:42:47.249465+0000","last_became_peered":"2024-02-26T10:42:47.249465+0000","last_unstale":"2024-02-26T10:44:39.406280+0000","last_undegraded":"2024-02-26T10:44:39.406280+0000","last_fullsized":"2024-02-26T10:44:39.406280+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:27:23.173863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'39","reported_seq":397,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:42.656270+0000","last_change":"2024-02-26T10:43:38.460725+0000","last_active":"2024-02-26T10:45:42.656270+0000","last_peered":"2024-02-26T10:45:42.656270+0000","last_clean":"2024-02-26T10:45:42.656270+0000","last_became_active":"2024-02-26T10:43:38.451900+0000","last_became_peered":"2024-02-26T10:43:38.451900+0000","last_unstale":"2024-02-26T10:45:42.656270+0000","last_undegraded":"2024-02-26T10:45:42.656270+0000","last_fullsized":"2024-02-26T10:45:42.656270+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:35:09.905452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71468,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":303,"num_read_kb":221,"num_write":40,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":62625,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234956+0000","last_change":"2024-02-26T10:42:42.168174+0000","last_active":"2024-02-26T10:44:39.234956+0000","last_peered":"2024-02-26T10:44:39.234956+0000","last_clean":"2024-02-26T10:44:39.234956+0000","last_became_active":"2024-02-26T10:42:42.167840+0000","last_became_peered":"2024-02-26T10:42:42.167840+0000","last_unstale":"2024-02-26T10:44:39.234956+0000","last_undegraded":"2024-02-26T10:44:39.234956+0000","last_fullsized":"2024-02-26T10:44:39.234956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:43.079139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557658+0000","last_change":"2024-02-26T10:42:47.230048+0000","last_active":"2024-02-26T10:44:39.557658+0000","last_peered":"2024-02-26T10:44:39.557658+0000","last_clean":"2024-02-26T10:44:39.557658+0000","last_became_active":"2024-02-26T10:42:47.229396+0000","last_became_peered":"2024-02-26T10:42:47.229396+0000","last_unstale":"2024-02-26T10:44:39.557658+0000","last_undegraded":"2024-02-26T10:44:39.557658+0000","last_fullsized":"2024-02-26T10:44:39.557658+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:57.804297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238511+0000","last_change":"2024-02-26T10:42:41.614413+0000","last_active":"2024-02-26T10:44:39.238511+0000","last_peered":"2024-02-26T10:44:39.238511+0000","last_clean":"2024-02-26T10:44:39.238511+0000","last_became_active":"2024-02-26T10:42:41.614065+0000","last_became_peered":"2024-02-26T10:42:41.614065+0000","last_unstale":"2024-02-26T10:44:39.238511+0000","last_undegraded":"2024-02-26T10:44:39.238511+0000","last_fullsized":"2024-02-26T10:44:39.238511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:50:20.552340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235996+0000","last_change":"2024-02-26T10:42:42.188140+0000","last_active":"2024-02-26T10:44:39.235996+0000","last_peered":"2024-02-26T10:44:39.235996+0000","last_clean":"2024-02-26T10:44:39.235996+0000","last_became_active":"2024-02-26T10:42:42.187055+0000","last_became_peered":"2024-02-26T10:42:42.187055+0000","last_unstale":"2024-02-26T10:44:39.235996+0000","last_undegraded":"2024-02-26T10:44:39.235996+0000","last_fullsized":"2024-02-26T10:44:39.235996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:49:43.568562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308124+0000","last_change":"2024-02-26T10:43:38.450681+0000","last_active":"2024-02-26T10:44:39.308124+0000","last_peered":"2024-02-26T10:44:39.308124+0000","last_clean":"2024-02-26T10:44:39.308124+0000","last_became_active":"2024-02-26T10:43:38.450120+0000","last_became_peered":"2024-02-26T10:43:38.450120+0000","last_unstale":"2024-02-26T10:44:39.308124+0000","last_undegraded":"2024-02-26T10:44:39.308124+0000","last_fullsized":"2024-02-26T10:44:39.308124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:10.983700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556913+0000","last_change":"2024-02-26T10:42:41.612337+0000","last_active":"2024-02-26T10:44:39.556913+0000","last_peered":"2024-02-26T10:44:39.556913+0000","last_clean":"2024-02-26T10:44:39.556913+0000","last_became_active":"2024-02-26T10:42:41.611920+0000","last_became_peered":"2024-02-26T10:42:41.611920+0000","last_unstale":"2024-02-26T10:44:39.556913+0000","last_undegraded":"2024-02-26T10:44:39.556913+0000","last_fullsized":"2024-02-26T10:44:39.556913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:29:13.612583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235333+0000","last_change":"2024-02-26T10:43:38.456919+0000","last_active":"2024-02-26T10:44:39.235333+0000","last_peered":"2024-02-26T10:44:39.235333+0000","last_clean":"2024-02-26T10:44:39.235333+0000","last_became_active":"2024-02-26T10:43:38.456461+0000","last_became_peered":"2024-02-26T10:43:38.456461+0000","last_unstale":"2024-02-26T10:44:39.235333+0000","last_undegraded":"2024-02-26T10:44:39.235333+0000","last_fullsized":"2024-02-26T10:44:39.235333+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:44.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.407141+0000","last_change":"2024-02-26T10:42:47.254585+0000","last_active":"2024-02-26T10:44:39.407141+0000","last_peered":"2024-02-26T10:44:39.407141+0000","last_clean":"2024-02-26T10:44:39.407141+0000","last_became_active":"2024-02-26T10:42:47.253870+0000","last_became_peered":"2024-02-26T10:42:47.253870+0000","last_unstale":"2024-02-26T10:44:39.407141+0000","last_undegraded":"2024-02-26T10:44:39.407141+0000","last_fullsized":"2024-02-26T10:44:39.407141+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:08:49.276906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":190,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:41.338041+0000","last_change":"2024-02-26T10:42:41.600901+0000","last_active":"2024-02-26T10:45:41.338041+0000","last_peered":"2024-02-26T10:45:41.338041+0000","last_clean":"2024-02-26T10:45:41.338041+0000","last_became_active":"2024-02-26T10:42:41.600018+0000","last_became_peered":"2024-02-26T10:42:41.600018+0000","last_unstale":"2024-02-26T10:45:41.338041+0000","last_undegraded":"2024-02-26T10:45:41.338041+0000","last_fullsized":"2024-02-26T10:45:41.338041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:07.876694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235012+0000","last_change":"2024-02-26T10:42:42.168702+0000","last_active":"2024-02-26T10:44:39.235012+0000","last_peered":"2024-02-26T10:44:39.235012+0000","last_clean":"2024-02-26T10:44:39.235012+0000","last_became_active":"2024-02-26T10:42:42.168420+0000","last_became_peered":"2024-02-26T10:42:42.168420+0000","last_unstale":"2024-02-26T10:44:39.235012+0000","last_undegraded":"2024-02-26T10:44:39.235012+0000","last_fullsized":"2024-02-26T10:44:39.235012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:34:35.111699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236688+0000","last_change":"2024-02-26T10:42:47.250714+0000","last_active":"2024-02-26T10:44:39.236688+0000","last_peered":"2024-02-26T10:44:39.236688+0000","last_clean":"2024-02-26T10:44:39.236688+0000","last_became_active":"2024-02-26T10:42:47.249506+0000","last_became_peered":"2024-02-26T10:42:47.249506+0000","last_unstale":"2024-02-26T10:44:39.236688+0000","last_undegraded":"2024-02-26T10:44:39.236688+0000","last_fullsized":"2024-02-26T10:44:39.236688+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:05.058052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307933+0000","last_change":"2024-02-26T10:43:38.472808+0000","last_active":"2024-02-26T10:44:39.307933+0000","last_peered":"2024-02-26T10:44:39.307933+0000","last_clean":"2024-02-26T10:44:39.307933+0000","last_became_active":"2024-02-26T10:43:38.456509+0000","last_became_peered":"2024-02-26T10:43:38.456509+0000","last_unstale":"2024-02-26T10:44:39.307933+0000","last_undegraded":"2024-02-26T10:44:39.307933+0000","last_fullsized":"2024-02-26T10:44:39.307933+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:03:06.127386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556435+0000","last_change":"2024-02-26T10:42:42.172845+0000","last_active":"2024-02-26T10:44:39.556435+0000","last_peered":"2024-02-26T10:44:39.556435+0000","last_clean":"2024-02-26T10:44:39.556435+0000","last_became_active":"2024-02-26T10:42:42.171263+0000","last_became_peered":"2024-02-26T10:42:42.171263+0000","last_unstale":"2024-02-26T10:44:39.556435+0000","last_undegraded":"2024-02-26T10:44:39.556435+0000","last_fullsized":"2024-02-26T10:44:39.556435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:21.439474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556408+0000","last_change":"2024-02-26T10:42:47.236434+0000","last_active":"2024-02-26T10:44:39.556408+0000","last_peered":"2024-02-26T10:44:39.556408+0000","last_clean":"2024-02-26T10:44:39.556408+0000","last_became_active":"2024-02-26T10:42:47.236040+0000","last_became_peered":"2024-02-26T10:42:47.236040+0000","last_unstale":"2024-02-26T10:44:39.556408+0000","last_undegraded":"2024-02-26T10:44:39.556408+0000","last_fullsized":"2024-02-26T10:44:39.556408+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:20:23.230348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235239+0000","last_change":"2024-02-26T10:42:41.604208+0000","last_active":"2024-02-26T10:44:39.235239+0000","last_peered":"2024-02-26T10:44:39.235239+0000","last_clean":"2024-02-26T10:44:39.235239+0000","last_became_active":"2024-02-26T10:42:41.603894+0000","last_became_peered":"2024-02-26T10:42:41.603894+0000","last_unstale":"2024-02-26T10:44:39.235239+0000","last_undegraded":"2024-02-26T10:44:39.235239+0000","last_fullsized":"2024-02-26T10:44:39.235239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:57:13.775868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308423+0000","last_change":"2024-02-26T10:42:42.169395+0000","last_active":"2024-02-26T10:44:39.308423+0000","last_peered":"2024-02-26T10:44:39.308423+0000","last_clean":"2024-02-26T10:44:39.308423+0000","last_became_active":"2024-02-26T10:42:42.168686+0000","last_became_peered":"2024-02-26T10:42:42.168686+0000","last_unstale":"2024-02-26T10:44:39.308423+0000","last_undegraded":"2024-02-26T10:44:39.308423+0000","last_fullsized":"2024-02-26T10:44:39.308423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:21:35.085188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234671+0000","last_change":"2024-02-26T10:42:47.252119+0000","last_active":"2024-02-26T10:44:39.234671+0000","last_peered":"2024-02-26T10:44:39.234671+0000","last_clean":"2024-02-26T10:44:39.234671+0000","last_became_active":"2024-02-26T10:42:47.251723+0000","last_became_peered":"2024-02-26T10:42:47.251723+0000","last_unstale":"2024-02-26T10:44:39.234671+0000","last_undegraded":"2024-02-26T10:44:39.234671+0000","last_fullsized":"2024-02-26T10:44:39.234671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:17.130806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.408624+0000","last_change":"2024-02-26T10:42:41.621393+0000","last_active":"2024-02-26T10:44:39.408624+0000","last_peered":"2024-02-26T10:44:39.408624+0000","last_clean":"2024-02-26T10:44:39.408624+0000","last_became_active":"2024-02-26T10:42:41.621008+0000","last_became_peered":"2024-02-26T10:42:41.621008+0000","last_unstale":"2024-02-26T10:44:39.408624+0000","last_undegraded":"2024-02-26T10:44:39.408624+0000","last_fullsized":"2024-02-26T10:44:39.408624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:46.979854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257087+0000","last_change":"2024-02-26T10:43:38.458926+0000","last_active":"2024-02-26T10:44:39.257087+0000","last_peered":"2024-02-26T10:44:39.257087+0000","last_clean":"2024-02-26T10:44:39.257087+0000","last_became_active":"2024-02-26T10:43:38.458537+0000","last_became_peered":"2024-02-26T10:43:38.458537+0000","last_unstale":"2024-02-26T10:44:39.257087+0000","last_undegraded":"2024-02-26T10:44:39.257087+0000","last_fullsized":"2024-02-26T10:44:39.257087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:46:08.204646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236653+0000","last_change":"2024-02-26T10:42:47.238520+0000","last_active":"2024-02-26T10:44:39.236653+0000","last_peered":"2024-02-26T10:44:39.236653+0000","last_clean":"2024-02-26T10:44:39.236653+0000","last_became_active":"2024-02-26T10:42:47.237213+0000","last_became_peered":"2024-02-26T10:42:47.237213+0000","last_unstale":"2024-02-26T10:44:39.236653+0000","last_undegraded":"2024-02-26T10:44:39.236653+0000","last_fullsized":"2024-02-26T10:44:39.236653+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:29.283850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235085+0000","last_change":"2024-02-26T10:42:41.609479+0000","last_active":"2024-02-26T10:44:39.235085+0000","last_peered":"2024-02-26T10:44:39.235085+0000","last_clean":"2024-02-26T10:44:39.235085+0000","last_became_active":"2024-02-26T10:42:41.609084+0000","last_became_peered":"2024-02-26T10:42:41.609084+0000","last_unstale":"2024-02-26T10:44:39.235085+0000","last_undegraded":"2024-02-26T10:44:39.235085+0000","last_fullsized":"2024-02-26T10:44:39.235085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:32.412507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.257066+0000","last_change":"2024-02-26T10:42:42.174985+0000","last_active":"2024-02-26T10:44:39.257066+0000","last_peered":"2024-02-26T10:44:39.257066+0000","last_clean":"2024-02-26T10:44:39.257066+0000","last_became_active":"2024-02-26T10:42:42.174694+0000","last_became_peered":"2024-02-26T10:42:42.174694+0000","last_unstale":"2024-02-26T10:44:39.257066+0000","last_undegraded":"2024-02-26T10:44:39.257066+0000","last_fullsized":"2024-02-26T10:44:39.257066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:01:10.000154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237057+0000","last_change":"2024-02-26T10:42:47.242587+0000","last_active":"2024-02-26T10:44:39.237057+0000","last_peered":"2024-02-26T10:44:39.237057+0000","last_clean":"2024-02-26T10:44:39.237057+0000","last_became_active":"2024-02-26T10:42:47.242099+0000","last_became_peered":"2024-02-26T10:42:47.242099+0000","last_unstale":"2024-02-26T10:44:39.237057+0000","last_undegraded":"2024-02-26T10:44:39.237057+0000","last_fullsized":"2024-02-26T10:44:39.237057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:38:42.852080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556815+0000","last_change":"2024-02-26T10:42:42.170613+0000","last_active":"2024-02-26T10:44:39.556815+0000","last_peered":"2024-02-26T10:44:39.556815+0000","last_clean":"2024-02-26T10:44:39.556815+0000","last_became_active":"2024-02-26T10:42:42.170317+0000","last_became_peered":"2024-02-26T10:42:42.170317+0000","last_unstale":"2024-02-26T10:44:39.556815+0000","last_undegraded":"2024-02-26T10:44:39.556815+0000","last_fullsized":"2024-02-26T10:44:39.556815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:29:44.276718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":187,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:42.339228+0000","last_change":"2024-02-26T10:42:41.617447+0000","last_active":"2024-02-26T10:45:42.339228+0000","last_peered":"2024-02-26T10:45:42.339228+0000","last_clean":"2024-02-26T10:45:42.339228+0000","last_became_active":"2024-02-26T10:42:41.617107+0000","last_became_peered":"2024-02-26T10:42:41.617107+0000","last_unstale":"2024-02-26T10:45:42.339228+0000","last_undegraded":"2024-02-26T10:45:42.339228+0000","last_fullsized":"2024-02-26T10:45:42.339228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:06.052364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235432+0000","last_change":"2024-02-26T10:42:47.251971+0000","last_active":"2024-02-26T10:44:39.235432+0000","last_peered":"2024-02-26T10:44:39.235432+0000","last_clean":"2024-02-26T10:44:39.235432+0000","last_became_active":"2024-02-26T10:42:47.251506+0000","last_became_peered":"2024-02-26T10:42:47.251506+0000","last_unstale":"2024-02-26T10:44:39.235432+0000","last_undegraded":"2024-02-26T10:44:39.235432+0000","last_fullsized":"2024-02-26T10:44:39.235432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:51:37.029348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041442+0000","last_change":"2024-02-26T10:42:42.194414+0000","last_active":"2024-02-26T10:44:40.041442+0000","last_peered":"2024-02-26T10:44:40.041442+0000","last_clean":"2024-02-26T10:44:40.041442+0000","last_became_active":"2024-02-26T10:42:42.193959+0000","last_became_peered":"2024-02-26T10:42:42.193959+0000","last_unstale":"2024-02-26T10:44:40.041442+0000","last_undegraded":"2024-02-26T10:44:40.041442+0000","last_fullsized":"2024-02-26T10:44:40.041442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:50.398674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239392+0000","last_change":"2024-02-26T10:42:41.607947+0000","last_active":"2024-02-26T10:44:39.239392+0000","last_peered":"2024-02-26T10:44:39.239392+0000","last_clean":"2024-02-26T10:44:39.239392+0000","last_became_active":"2024-02-26T10:42:41.607522+0000","last_became_peered":"2024-02-26T10:42:41.607522+0000","last_unstale":"2024-02-26T10:44:39.239392+0000","last_undegraded":"2024-02-26T10:44:39.239392+0000","last_fullsized":"2024-02-26T10:44:39.239392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:32:36.647621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234832+0000","last_change":"2024-02-26T10:41:21.177202+0000","last_active":"2024-02-26T10:44:39.234832+0000","last_peered":"2024-02-26T10:44:39.234832+0000","last_clean":"2024-02-26T10:44:39.234832+0000","last_became_active":"2024-02-26T10:41:21.176762+0000","last_became_peered":"2024-02-26T10:41:21.176762+0000","last_unstale":"2024-02-26T10:44:39.234832+0000","last_undegraded":"2024-02-26T10:44:39.234832+0000","last_fullsized":"2024-02-26T10:44:39.234832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:37:25.424790+0000","last_clean_scrub_stamp":"2024-02-26T10:37:25.424790+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:19.619104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.555996+0000","last_change":"2024-02-26T10:42:47.244346+0000","last_active":"2024-02-26T10:44:39.555996+0000","last_peered":"2024-02-26T10:44:39.555996+0000","last_clean":"2024-02-26T10:44:39.555996+0000","last_became_active":"2024-02-26T10:42:47.243872+0000","last_became_peered":"2024-02-26T10:42:47.243872+0000","last_unstale":"2024-02-26T10:44:39.555996+0000","last_undegraded":"2024-02-26T10:44:39.555996+0000","last_fullsized":"2024-02-26T10:44:39.555996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:36.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235371+0000","last_change":"2024-02-26T10:42:42.166266+0000","last_active":"2024-02-26T10:44:39.235371+0000","last_peered":"2024-02-26T10:44:39.235371+0000","last_clean":"2024-02-26T10:44:39.235371+0000","last_became_active":"2024-02-26T10:42:42.165721+0000","last_became_peered":"2024-02-26T10:42:42.165721+0000","last_unstale":"2024-02-26T10:44:39.235371+0000","last_undegraded":"2024-02-26T10:44:39.235371+0000","last_fullsized":"2024-02-26T10:44:39.235371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:42:32.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234981+0000","last_change":"2024-02-26T10:42:41.606268+0000","last_active":"2024-02-26T10:44:39.234981+0000","last_peered":"2024-02-26T10:44:39.234981+0000","last_clean":"2024-02-26T10:44:39.234981+0000","last_became_active":"2024-02-26T10:42:41.605950+0000","last_became_peered":"2024-02-26T10:42:41.605950+0000","last_unstale":"2024-02-26T10:44:39.234981+0000","last_undegraded":"2024-02-26T10:44:39.234981+0000","last_fullsized":"2024-02-26T10:44:39.234981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:40:03.616987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.307850+0000","last_change":"2024-02-26T10:42:47.233203+0000","last_active":"2024-02-26T10:44:39.307850+0000","last_peered":"2024-02-26T10:44:39.307850+0000","last_clean":"2024-02-26T10:44:39.307850+0000","last_became_active":"2024-02-26T10:42:47.232705+0000","last_became_peered":"2024-02-26T10:42:47.232705+0000","last_unstale":"2024-02-26T10:44:39.307850+0000","last_undegraded":"2024-02-26T10:44:39.307850+0000","last_fullsized":"2024-02-26T10:44:39.307850+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:12:53.818901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308265+0000","last_change":"2024-02-26T10:43:38.479911+0000","last_active":"2024-02-26T10:44:39.308265+0000","last_peered":"2024-02-26T10:44:39.308265+0000","last_clean":"2024-02-26T10:44:39.308265+0000","last_became_active":"2024-02-26T10:43:38.458874+0000","last_became_peered":"2024-02-26T10:43:38.458874+0000","last_unstale":"2024-02-26T10:44:39.308265+0000","last_undegraded":"2024-02-26T10:44:39.308265+0000","last_fullsized":"2024-02-26T10:44:39.308265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:24:56.093775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237091+0000","last_change":"2024-02-26T10:42:42.167674+0000","last_active":"2024-02-26T10:44:39.237091+0000","last_peered":"2024-02-26T10:44:39.237091+0000","last_clean":"2024-02-26T10:44:39.237091+0000","last_became_active":"2024-02-26T10:42:42.167224+0000","last_became_peered":"2024-02-26T10:42:42.167224+0000","last_unstale":"2024-02-26T10:44:39.237091+0000","last_undegraded":"2024-02-26T10:44:39.237091+0000","last_fullsized":"2024-02-26T10:44:39.237091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:49:20.433913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.039668+0000","last_change":"2024-02-26T10:42:47.244076+0000","last_active":"2024-02-26T10:44:40.039668+0000","last_peered":"2024-02-26T10:44:40.039668+0000","last_clean":"2024-02-26T10:44:40.039668+0000","last_became_active":"2024-02-26T10:42:47.243569+0000","last_became_peered":"2024-02-26T10:42:47.243569+0000","last_unstale":"2024-02-26T10:44:40.039668+0000","last_undegraded":"2024-02-26T10:44:40.039668+0000","last_fullsized":"2024-02-26T10:44:40.039668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:46:19.855621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'13","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:45:39.094956+0000","last_change":"2024-02-26T10:42:41.623604+0000","last_active":"2024-02-26T10:45:39.094956+0000","last_peered":"2024-02-26T10:45:39.094956+0000","last_clean":"2024-02-26T10:45:39.094956+0000","last_became_active":"2024-02-26T10:42:41.623272+0000","last_became_peered":"2024-02-26T10:42:41.623272+0000","last_unstale":"2024-02-26T10:45:39.094956+0000","last_undegraded":"2024-02-26T10:45:39.094956+0000","last_fullsized":"2024-02-26T10:45:39.094956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:13:57.430453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042221+0000","last_change":"2024-02-26T10:42:42.186300+0000","last_active":"2024-02-26T10:44:40.042221+0000","last_peered":"2024-02-26T10:44:40.042221+0000","last_clean":"2024-02-26T10:44:40.042221+0000","last_became_active":"2024-02-26T10:42:42.185965+0000","last_became_peered":"2024-02-26T10:42:42.185965+0000","last_unstale":"2024-02-26T10:44:40.042221+0000","last_undegraded":"2024-02-26T10:44:40.042221+0000","last_fullsized":"2024-02-26T10:44:40.042221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:06:50.352308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238335+0000","last_change":"2024-02-26T10:42:47.227753+0000","last_active":"2024-02-26T10:44:39.238335+0000","last_peered":"2024-02-26T10:44:39.238335+0000","last_clean":"2024-02-26T10:44:39.238335+0000","last_became_active":"2024-02-26T10:42:47.227300+0000","last_became_peered":"2024-02-26T10:42:47.227300+0000","last_unstale":"2024-02-26T10:44:39.238335+0000","last_undegraded":"2024-02-26T10:44:39.238335+0000","last_fullsized":"2024-02-26T10:44:39.238335+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:08.204910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239810+0000","last_change":"2024-02-26T10:42:41.602960+0000","last_active":"2024-02-26T10:44:39.239810+0000","last_peered":"2024-02-26T10:44:39.239810+0000","last_clean":"2024-02-26T10:44:39.239810+0000","last_became_active":"2024-02-26T10:42:41.602676+0000","last_became_peered":"2024-02-26T10:42:41.602676+0000","last_unstale":"2024-02-26T10:44:39.239810+0000","last_undegraded":"2024-02-26T10:44:39.239810+0000","last_fullsized":"2024-02-26T10:44:39.239810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:57.997612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528737+0000","last_change":"2024-02-26T10:42:42.185451+0000","last_active":"2024-02-26T10:44:39.528737+0000","last_peered":"2024-02-26T10:44:39.528737+0000","last_clean":"2024-02-26T10:44:39.528737+0000","last_became_active":"2024-02-26T10:42:42.185116+0000","last_became_peered":"2024-02-26T10:42:42.185116+0000","last_unstale":"2024-02-26T10:44:39.528737+0000","last_undegraded":"2024-02-26T10:44:39.528737+0000","last_fullsized":"2024-02-26T10:44:39.528737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:43:39.823370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239619+0000","last_change":"2024-02-26T10:42:47.250682+0000","last_active":"2024-02-26T10:44:39.239619+0000","last_peered":"2024-02-26T10:44:39.239619+0000","last_clean":"2024-02-26T10:44:39.239619+0000","last_became_active":"2024-02-26T10:42:47.249524+0000","last_became_peered":"2024-02-26T10:42:47.249524+0000","last_unstale":"2024-02-26T10:44:39.239619+0000","last_undegraded":"2024-02-26T10:44:39.239619+0000","last_fullsized":"2024-02-26T10:44:39.239619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:07:01.307206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463955+0000","last_change":"2024-02-26T10:42:41.620764+0000","last_active":"2024-02-26T10:44:39.463955+0000","last_peered":"2024-02-26T10:44:39.463955+0000","last_clean":"2024-02-26T10:44:39.463955+0000","last_became_active":"2024-02-26T10:42:41.620500+0000","last_became_peered":"2024-02-26T10:42:41.620500+0000","last_unstale":"2024-02-26T10:44:39.463955+0000","last_undegraded":"2024-02-26T10:44:39.463955+0000","last_fullsized":"2024-02-26T10:44:39.463955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:59:15.542396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234462+0000","last_change":"2024-02-26T10:42:42.165060+0000","last_active":"2024-02-26T10:44:39.234462+0000","last_peered":"2024-02-26T10:44:39.234462+0000","last_clean":"2024-02-26T10:44:39.234462+0000","last_became_active":"2024-02-26T10:42:42.164733+0000","last_became_peered":"2024-02-26T10:42:42.164733+0000","last_unstale":"2024-02-26T10:44:39.234462+0000","last_undegraded":"2024-02-26T10:44:39.234462+0000","last_fullsized":"2024-02-26T10:44:39.234462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:34.190747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.044327+0000","last_change":"2024-02-26T10:44:40.044327+0000","last_active":"2024-02-26T10:44:40.044327+0000","last_peered":"2024-02-26T10:44:40.044327+0000","last_clean":"2024-02-26T10:44:40.044327+0000","last_became_active":"2024-02-26T10:44:40.043776+0000","last_became_peered":"2024-02-26T10:44:40.043776+0000","last_unstale":"2024-02-26T10:44:40.044327+0000","last_undegraded":"2024-02-26T10:44:40.044327+0000","last_fullsized":"2024-02-26T10:44:40.044327+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:50:51.502411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040404+0000","last_change":"2024-02-26T10:42:41.624940+0000","last_active":"2024-02-26T10:44:40.040404+0000","last_peered":"2024-02-26T10:44:40.040404+0000","last_clean":"2024-02-26T10:44:40.040404+0000","last_became_active":"2024-02-26T10:42:41.624538+0000","last_became_peered":"2024-02-26T10:42:41.624538+0000","last_unstale":"2024-02-26T10:44:40.040404+0000","last_undegraded":"2024-02-26T10:44:40.040404+0000","last_fullsized":"2024-02-26T10:44:40.040404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:10.052806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308063+0000","last_change":"2024-02-26T10:42:42.174932+0000","last_active":"2024-02-26T10:44:39.308063+0000","last_peered":"2024-02-26T10:44:39.308063+0000","last_clean":"2024-02-26T10:44:39.308063+0000","last_became_active":"2024-02-26T10:42:42.174566+0000","last_became_peered":"2024-02-26T10:42:42.174566+0000","last_unstale":"2024-02-26T10:44:39.308063+0000","last_undegraded":"2024-02-26T10:44:39.308063+0000","last_fullsized":"2024-02-26T10:44:39.308063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:03.242487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239861+0000","last_change":"2024-02-26T10:42:47.250023+0000","last_active":"2024-02-26T10:44:39.239861+0000","last_peered":"2024-02-26T10:44:39.239861+0000","last_clean":"2024-02-26T10:44:39.239861+0000","last_became_active":"2024-02-26T10:42:47.249571+0000","last_became_peered":"2024-02-26T10:42:47.249571+0000","last_unstale":"2024-02-26T10:44:39.239861+0000","last_undegraded":"2024-02-26T10:44:39.239861+0000","last_fullsized":"2024-02-26T10:44:39.239861+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:53.666184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:53.651538+0000","last_change":"2024-02-26T10:42:41.614403+0000","last_active":"2024-02-26T10:44:53.651538+0000","last_peered":"2024-02-26T10:44:53.651538+0000","last_clean":"2024-02-26T10:44:53.651538+0000","last_became_active":"2024-02-26T10:42:41.614057+0000","last_became_peered":"2024-02-26T10:42:41.614057+0000","last_unstale":"2024-02-26T10:44:53.651538+0000","last_undegraded":"2024-02-26T10:44:53.651538+0000","last_fullsized":"2024-02-26T10:44:53.651538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:16:54.739090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.041052+0000","last_change":"2024-02-26T10:42:42.187465+0000","last_active":"2024-02-26T10:44:40.041052+0000","last_peered":"2024-02-26T10:44:40.041052+0000","last_clean":"2024-02-26T10:44:40.041052+0000","last_became_active":"2024-02-26T10:42:42.182310+0000","last_became_peered":"2024-02-26T10:42:42.182310+0000","last_unstale":"2024-02-26T10:44:40.041052+0000","last_undegraded":"2024-02-26T10:44:40.041052+0000","last_fullsized":"2024-02-26T10:44:40.041052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:01:56.645330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239438+0000","last_change":"2024-02-26T10:42:47.241751+0000","last_active":"2024-02-26T10:44:39.239438+0000","last_peered":"2024-02-26T10:44:39.239438+0000","last_clean":"2024-02-26T10:44:39.239438+0000","last_became_active":"2024-02-26T10:42:47.241271+0000","last_became_peered":"2024-02-26T10:42:47.241271+0000","last_unstale":"2024-02-26T10:44:39.239438+0000","last_undegraded":"2024-02-26T10:44:39.239438+0000","last_fullsized":"2024-02-26T10:44:39.239438+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:40:39.470379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.234633+0000","last_change":"2024-02-26T10:42:41.607243+0000","last_active":"2024-02-26T10:44:39.234633+0000","last_peered":"2024-02-26T10:44:39.234633+0000","last_clean":"2024-02-26T10:44:39.234633+0000","last_became_active":"2024-02-26T10:42:41.606921+0000","last_became_peered":"2024-02-26T10:42:41.606921+0000","last_unstale":"2024-02-26T10:44:39.234633+0000","last_undegraded":"2024-02-26T10:44:39.234633+0000","last_fullsized":"2024-02-26T10:44:39.234633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:36:04.626391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235242+0000","last_change":"2024-02-26T10:42:42.178240+0000","last_active":"2024-02-26T10:44:39.235242+0000","last_peered":"2024-02-26T10:44:39.235242+0000","last_clean":"2024-02-26T10:44:39.235242+0000","last_became_active":"2024-02-26T10:42:42.177790+0000","last_became_peered":"2024-02-26T10:42:42.177790+0000","last_unstale":"2024-02-26T10:44:39.235242+0000","last_undegraded":"2024-02-26T10:44:39.235242+0000","last_fullsized":"2024-02-26T10:44:39.235242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:24.110406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.463760+0000","last_change":"2024-02-26T10:42:47.233120+0000","last_active":"2024-02-26T10:44:39.463760+0000","last_peered":"2024-02-26T10:44:39.463760+0000","last_clean":"2024-02-26T10:44:39.463760+0000","last_became_active":"2024-02-26T10:42:47.232630+0000","last_became_peered":"2024-02-26T10:42:47.232630+0000","last_unstale":"2024-02-26T10:44:39.463760+0000","last_undegraded":"2024-02-26T10:44:39.463760+0000","last_fullsized":"2024-02-26T10:44:39.463760+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:05:03.023227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.239351+0000","last_change":"2024-02-26T10:43:38.477127+0000","last_active":"2024-02-26T10:44:39.239351+0000","last_peered":"2024-02-26T10:44:39.239351+0000","last_clean":"2024-02-26T10:44:39.239351+0000","last_became_active":"2024-02-26T10:43:38.460135+0000","last_became_peered":"2024-02-26T10:43:38.460135+0000","last_unstale":"2024-02-26T10:44:39.239351+0000","last_undegraded":"2024-02-26T10:44:39.239351+0000","last_fullsized":"2024-02-26T10:44:39.239351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:13:16.521943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235617+0000","last_change":"2024-02-26T10:42:42.171821+0000","last_active":"2024-02-26T10:44:39.235617+0000","last_peered":"2024-02-26T10:44:39.235617+0000","last_clean":"2024-02-26T10:44:39.235617+0000","last_became_active":"2024-02-26T10:42:42.171523+0000","last_became_peered":"2024-02-26T10:42:42.171523+0000","last_unstale":"2024-02-26T10:44:39.235617+0000","last_undegraded":"2024-02-26T10:44:39.235617+0000","last_fullsized":"2024-02-26T10:44:39.235617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:00.877998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.526701+0000","last_change":"2024-02-26T10:42:47.250072+0000","last_active":"2024-02-26T10:44:39.526701+0000","last_peered":"2024-02-26T10:44:39.526701+0000","last_clean":"2024-02-26T10:44:39.526701+0000","last_became_active":"2024-02-26T10:42:47.249570+0000","last_became_peered":"2024-02-26T10:42:47.249570+0000","last_unstale":"2024-02-26T10:44:39.526701+0000","last_undegraded":"2024-02-26T10:44:39.526701+0000","last_fullsized":"2024-02-26T10:44:39.526701+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:43:03.518243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.557531+0000","last_change":"2024-02-26T10:42:41.614686+0000","last_active":"2024-02-26T10:44:39.557531+0000","last_peered":"2024-02-26T10:44:39.557531+0000","last_clean":"2024-02-26T10:44:39.557531+0000","last_became_active":"2024-02-26T10:42:41.614019+0000","last_became_peered":"2024-02-26T10:42:41.614019+0000","last_unstale":"2024-02-26T10:44:39.557531+0000","last_undegraded":"2024-02-26T10:44:39.557531+0000","last_fullsized":"2024-02-26T10:44:39.557531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:13:28.127288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235390+0000","last_change":"2024-02-26T10:42:42.181774+0000","last_active":"2024-02-26T10:44:39.235390+0000","last_peered":"2024-02-26T10:44:39.235390+0000","last_clean":"2024-02-26T10:44:39.235390+0000","last_became_active":"2024-02-26T10:42:42.181412+0000","last_became_peered":"2024-02-26T10:42:42.181412+0000","last_unstale":"2024-02-26T10:44:39.235390+0000","last_undegraded":"2024-02-26T10:44:39.235390+0000","last_fullsized":"2024-02-26T10:44:39.235390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:26:09.159964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556176+0000","last_change":"2024-02-26T10:42:47.228247+0000","last_active":"2024-02-26T10:44:39.556176+0000","last_peered":"2024-02-26T10:44:39.556176+0000","last_clean":"2024-02-26T10:44:39.556176+0000","last_became_active":"2024-02-26T10:42:47.227569+0000","last_became_peered":"2024-02-26T10:42:47.227569+0000","last_unstale":"2024-02-26T10:44:39.556176+0000","last_undegraded":"2024-02-26T10:44:39.556176+0000","last_fullsized":"2024-02-26T10:44:39.556176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:46.177569+0000","last_clean_scrub_stamp":"2024-02-26T10:42:46.177569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:15.001264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236850+0000","last_change":"2024-02-26T10:42:41.626573+0000","last_active":"2024-02-26T10:44:39.236850+0000","last_peered":"2024-02-26T10:44:39.236850+0000","last_clean":"2024-02-26T10:44:39.236850+0000","last_became_active":"2024-02-26T10:42:41.624091+0000","last_became_peered":"2024-02-26T10:42:41.624091+0000","last_unstale":"2024-02-26T10:44:39.236850+0000","last_undegraded":"2024-02-26T10:44:39.236850+0000","last_fullsized":"2024-02-26T10:44:39.236850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:40.563768+0000","last_clean_scrub_stamp":"2024-02-26T10:42:40.563768+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:24:14.077234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.235668+0000","last_change":"2024-02-26T10:42:42.166426+0000","last_active":"2024-02-26T10:44:39.235668+0000","last_peered":"2024-02-26T10:44:39.235668+0000","last_clean":"2024-02-26T10:44:39.235668+0000","last_became_active":"2024-02-26T10:42:42.166082+0000","last_became_peered":"2024-02-26T10:42:42.166082+0000","last_unstale":"2024-02-26T10:44:39.235668+0000","last_undegraded":"2024-02-26T10:44:39.235668+0000","last_fullsized":"2024-02-26T10:44:39.235668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:04:17.136568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308191+0000","last_change":"2024-02-26T10:42:42.168227+0000","last_active":"2024-02-26T10:44:39.308191+0000","last_peered":"2024-02-26T10:44:39.308191+0000","last_clean":"2024-02-26T10:44:39.308191+0000","last_became_active":"2024-02-26T10:42:42.167897+0000","last_became_peered":"2024-02-26T10:42:42.167897+0000","last_unstale":"2024-02-26T10:44:39.308191+0000","last_undegraded":"2024-02-26T10:44:39.308191+0000","last_fullsized":"2024-02-26T10:44:39.308191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:52:37.391022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238723+0000","last_change":"2024-02-26T10:42:42.168640+0000","last_active":"2024-02-26T10:44:39.238723+0000","last_peered":"2024-02-26T10:44:39.238723+0000","last_clean":"2024-02-26T10:44:39.238723+0000","last_became_active":"2024-02-26T10:42:42.168237+0000","last_became_peered":"2024-02-26T10:42:42.168237+0000","last_unstale":"2024-02-26T10:44:39.238723+0000","last_undegraded":"2024-02-26T10:44:39.238723+0000","last_fullsized":"2024-02-26T10:44:39.238723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:14.853730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238359+0000","last_change":"2024-02-26T10:42:42.192703+0000","last_active":"2024-02-26T10:44:39.238359+0000","last_peered":"2024-02-26T10:44:39.238359+0000","last_clean":"2024-02-26T10:44:39.238359+0000","last_became_active":"2024-02-26T10:42:42.191899+0000","last_became_peered":"2024-02-26T10:42:42.191899+0000","last_unstale":"2024-02-26T10:44:39.238359+0000","last_undegraded":"2024-02-26T10:44:39.238359+0000","last_fullsized":"2024-02-26T10:44:39.238359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:40.479523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.528040+0000","last_change":"2024-02-26T10:42:42.188709+0000","last_active":"2024-02-26T10:44:39.528040+0000","last_peered":"2024-02-26T10:44:39.528040+0000","last_clean":"2024-02-26T10:44:39.528040+0000","last_became_active":"2024-02-26T10:42:42.188418+0000","last_became_peered":"2024-02-26T10:42:42.188418+0000","last_unstale":"2024-02-26T10:44:39.528040+0000","last_undegraded":"2024-02-26T10:44:39.528040+0000","last_fullsized":"2024-02-26T10:44:39.528040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:54:15.697746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.236925+0000","last_change":"2024-02-26T10:42:42.174221+0000","last_active":"2024-02-26T10:44:39.236925+0000","last_peered":"2024-02-26T10:44:39.236925+0000","last_clean":"2024-02-26T10:44:39.236925+0000","last_became_active":"2024-02-26T10:42:42.171308+0000","last_became_peered":"2024-02-26T10:42:42.171308+0000","last_unstale":"2024-02-26T10:44:39.236925+0000","last_undegraded":"2024-02-26T10:44:39.236925+0000","last_fullsized":"2024-02-26T10:44:39.236925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:46.765856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308467+0000","last_change":"2024-02-26T10:42:42.170749+0000","last_active":"2024-02-26T10:44:39.308467+0000","last_peered":"2024-02-26T10:44:39.308467+0000","last_clean":"2024-02-26T10:44:39.308467+0000","last_became_active":"2024-02-26T10:42:42.169966+0000","last_became_peered":"2024-02-26T10:42:42.169966+0000","last_unstale":"2024-02-26T10:44:39.308467+0000","last_undegraded":"2024-02-26T10:44:39.308467+0000","last_fullsized":"2024-02-26T10:44:39.308467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:02.633195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.238925+0000","last_change":"2024-02-26T10:42:42.166529+0000","last_active":"2024-02-26T10:44:39.238925+0000","last_peered":"2024-02-26T10:44:39.238925+0000","last_clean":"2024-02-26T10:44:39.238925+0000","last_became_active":"2024-02-26T10:42:42.166235+0000","last_became_peered":"2024-02-26T10:42:42.166235+0000","last_unstale":"2024-02-26T10:44:39.238925+0000","last_undegraded":"2024-02-26T10:44:39.238925+0000","last_fullsized":"2024-02-26T10:44:39.238925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:03.068328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.042639+0000","last_change":"2024-02-26T10:42:42.191312+0000","last_active":"2024-02-26T10:44:40.042639+0000","last_peered":"2024-02-26T10:44:40.042639+0000","last_clean":"2024-02-26T10:44:40.042639+0000","last_became_active":"2024-02-26T10:42:42.190964+0000","last_became_peered":"2024-02-26T10:42:42.190964+0000","last_unstale":"2024-02-26T10:44:40.042639+0000","last_undegraded":"2024-02-26T10:44:40.042639+0000","last_fullsized":"2024-02-26T10:44:40.042639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:24:04.697129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.556790+0000","last_change":"2024-02-26T10:42:42.182882+0000","last_active":"2024-02-26T10:44:39.556790+0000","last_peered":"2024-02-26T10:44:39.556790+0000","last_clean":"2024-02-26T10:44:39.556790+0000","last_became_active":"2024-02-26T10:42:42.182551+0000","last_became_peered":"2024-02-26T10:42:42.182551+0000","last_unstale":"2024-02-26T10:44:39.556790+0000","last_undegraded":"2024-02-26T10:44:39.556790+0000","last_fullsized":"2024-02-26T10:44:39.556790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:11:58.578832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:40.040743+0000","last_change":"2024-02-26T10:42:42.193353+0000","last_active":"2024-02-26T10:44:40.040743+0000","last_peered":"2024-02-26T10:44:40.040743+0000","last_clean":"2024-02-26T10:44:40.040743+0000","last_became_active":"2024-02-26T10:42:42.190935+0000","last_became_peered":"2024-02-26T10:42:42.190935+0000","last_unstale":"2024-02-26T10:44:40.040743+0000","last_undegraded":"2024-02-26T10:44:40.040743+0000","last_fullsized":"2024-02-26T10:44:40.040743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:03:14.319769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.464179+0000","last_change":"2024-02-26T10:42:42.194529+0000","last_active":"2024-02-26T10:44:39.464179+0000","last_peered":"2024-02-26T10:44:39.464179+0000","last_clean":"2024-02-26T10:44:39.464179+0000","last_became_active":"2024-02-26T10:42:42.194212+0000","last_became_peered":"2024-02-26T10:42:42.194212+0000","last_unstale":"2024-02-26T10:44:39.464179+0000","last_undegraded":"2024-02-26T10:44:39.464179+0000","last_fullsized":"2024-02-26T10:44:39.464179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:36.876672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.237256+0000","last_change":"2024-02-26T10:42:42.178119+0000","last_active":"2024-02-26T10:44:39.237256+0000","last_peered":"2024-02-26T10:44:39.237256+0000","last_clean":"2024-02-26T10:44:39.237256+0000","last_became_active":"2024-02-26T10:42:42.177798+0000","last_became_peered":"2024-02-26T10:42:42.177798+0000","last_unstale":"2024-02-26T10:44:39.237256+0000","last_undegraded":"2024-02-26T10:44:39.237256+0000","last_fullsized":"2024-02-26T10:44:39.237256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:52:29.076075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-02-26T10:44:39.308152+0000","last_change":"2024-02-26T10:42:42.167380+0000","last_active":"2024-02-26T10:44:39.308152+0000","last_peered":"2024-02-26T10:44:39.308152+0000","last_clean":"2024-02-26T10:44:39.308152+0000","last_became_active":"2024-02-26T10:42:42.167079+0000","last_became_peered":"2024-02-26T10:42:42.167079+0000","last_unstale":"2024-02-26T10:44:39.308152+0000","last_undegraded":"2024-02-26T10:44:39.308152+0000","last_fullsized":"2024-02-26T10:44:39.308152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:41.582591+0000","last_clean_scrub_stamp":"2024-02-26T10:42:41.582591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:14:25.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250986,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76001,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":910,"num_read_kb":534,"num_write":183,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":63341,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":228003,"data_compressed":14031,"data_compressed_allocated":20480,"data_compressed_original":77824,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":532499,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612664,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1196032,"data_stored":1191633,"data_compressed":64863,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776190,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":536697,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037829,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":532387,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299465,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":1128962,"data_compressed":63031,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462991,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":527777,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":599155,"data_compressed":28539,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":529165,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":528846,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":527971,"data_compressed":26707,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":605009,"data_compressed":37074,"data_compressed_allocated":49152,"data_compressed_original":135168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724630,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1124018,"data_compressed":63031,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36324,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":71468,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":73226,"data_compressed":10367,"data_compressed_allocated":12288,"data_compressed_original":61440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73578,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:45:47.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:45:47.125 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:46 smithi100 ceph-mon[36051]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:45:47.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:46 smithi100 ceph-mon[36051]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:45:47.197 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:46 smithi060 ceph-mon[31539]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:45:47.198 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:46 smithi060 ceph-mon[31539]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:45:47.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:46 smithi113 ceph-mon[36126]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:45:47.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:46 smithi113 ceph-mon[36126]: from='client.25483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:45:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.344+0000 7fb980000640 1 -- 172.21.15.60:0/1915940346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 msgr2=0x7fb97810b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.344+0000 7fb980000640 1 --2- 172.21.15.60:0/1915940346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 0x7fb97810b5d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb970009ca0 tx=0x7fb97002c100 comp rx=0 tx=0).stop 2024-02-26T10:45:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.344+0000 7fb980000640 1 -- 172.21.15.60:0/1915940346 shutdown_connections 2024-02-26T10:45:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.344+0000 7fb980000640 1 --2- 172.21.15.60:0/1915940346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb978073cc0 0x7fb978074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.344+0000 7fb980000640 1 --2- 172.21.15.60:0/1915940346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 0x7fb97810b5d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.344+0000 7fb980000640 1 --2- 172.21.15.60:0/1915940346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9780752a0 0x7fb9780736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.344+0000 7fb980000640 1 -- 172.21.15.60:0/1915940346 >> 172.21.15.60:0/1915940346 conn(0x7fb9780fc810 msgr2=0x7fb9780fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.344+0000 7fb980000640 1 -- 172.21.15.60:0/1915940346 shutdown_connections 2024-02-26T10:45:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.344+0000 7fb980000640 1 -- 172.21.15.60:0/1915940346 wait complete. 2024-02-26T10:45:47.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.345+0000 7fb980000640 1 Processor -- start 2024-02-26T10:45:47.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.345+0000 7fb980000640 1 -- start start 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.345+0000 7fb980000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb978073cc0 0x7fb978199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb980000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9780752a0 0x7fb978199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb980000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 0x7fb9781a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb980000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97810f100 con 0x7fb978109140 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb980000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97810ef80 con 0x7fb9780752a0 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb980000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97810f280 con 0x7fb978073cc0 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 0x7fb9781a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb978073cc0 0x7fb978199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 0x7fb9781a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35150/0 (socket says 172.21.15.60:35150) 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97e000640 1 -- 172.21.15.60:0/1910702572 learned_addr learned my addr 172.21.15.60:0/1910702572 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97cc00640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9780752a0 0x7fb978199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97e000640 1 -- 172.21.15.60:0/1910702572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb978073cc0 msgr2=0x7fb978199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97e000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb978073cc0 0x7fb978199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97e000640 1 -- 172.21.15.60:0/1910702572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9780752a0 msgr2=0x7fb978199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97e000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9780752a0 0x7fb978199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97e000640 1 -- 172.21.15.60:0/1910702572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9781a1330 con 0x7fb978109140 2024-02-26T10:45:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.346+0000 7fb97cc00640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9780752a0 0x7fb978199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:47.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.347+0000 7fb97e000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 0x7fb9781a0cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb97002c640 tx=0x7fb970035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:47.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.347+0000 7fb976000640 1 -- 172.21.15.60:0/1910702572 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970039040 con 0x7fb978109140 2024-02-26T10:45:47.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.347+0000 7fb976000640 1 -- 172.21.15.60:0/1910702572 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb970044070 con 0x7fb978109140 2024-02-26T10:45:47.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.347+0000 7fb976000640 1 -- 172.21.15.60:0/1910702572 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9700043c0 con 0x7fb978109140 2024-02-26T10:45:47.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.347+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9781a15c0 con 0x7fb978109140 2024-02-26T10:45:47.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.347+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9781a1ab0 con 0x7fb978109140 2024-02-26T10:45:47.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.349+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9780684a0 con 0x7fb978109140 2024-02-26T10:45:47.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.350+0000 7fb976000640 1 -- 172.21.15.60:0/1910702572 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fb970004560 con 0x7fb978109140 2024-02-26T10:45:47.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.352+0000 7fb976000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9600a1e10 0x7fb9600a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:47.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.352+0000 7fb976000640 1 -- 172.21.15.60:0/1910702572 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb9700f07f0 con 0x7fb978109140 2024-02-26T10:45:47.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.352+0000 7fb97d600640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9600a1e10 0x7fb9600a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:47.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.353+0000 7fb97d600640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9600a1e10 0x7fb9600a42d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009bd0 tx=0x7fb96c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:47.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.356+0000 7fb976000640 1 -- 172.21.15.60:0/1910702572 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9700bd7c0 con 0x7fb978109140 2024-02-26T10:45:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.558+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb978061e60 con 0x7fb978109140 2024-02-26T10:45:47.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.561+0000 7fb976000640 1 -- 172.21.15.60:0/1910702572 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7fb9700bd0c0 con 0x7fb978109140 2024-02-26T10:45:47.557 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:47.557 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:45:47.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9600a1e10 msgr2=0x7fb9600a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9600a1e10 0x7fb9600a42d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009bd0 tx=0x7fb96c005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 msgr2=0x7fb9781a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 0x7fb9781a0cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb97002c640 tx=0x7fb970035d10 comp rx=0 tx=0).stop 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 shutdown_connections 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9600a1e10 0x7fb9600a42d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9780752a0 0x7fb978199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb978109140 0x7fb9781a0cc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 --2- 172.21.15.60:0/1910702572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb978073cc0 0x7fb978199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 >> 172.21.15.60:0/1910702572 conn(0x7fb9780fc810 msgr2=0x7fb978105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 shutdown_connections 2024-02-26T10:45:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.564+0000 7fb980000640 1 -- 172.21.15.60:0/1910702572 wait complete. 2024-02-26T10:45:47.579 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:45:47.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:45:47.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.812+0000 7f4511800640 1 -- 172.21.15.60:0/2934845503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450c076e60 msgr2=0x7f450c0772c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.812+0000 7f4511800640 1 --2- 172.21.15.60:0/2934845503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450c076e60 0x7f450c0772c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009860 tx=0x7f44f802bb70 comp rx=0 tx=0).stop 2024-02-26T10:45:47.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.812+0000 7f4511800640 1 -- 172.21.15.60:0/2934845503 shutdown_connections 2024-02-26T10:45:47.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.812+0000 7f4511800640 1 --2- 172.21.15.60:0/2934845503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450c076e60 0x7f450c0772c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.812+0000 7f4511800640 1 --2- 172.21.15.60:0/2934845503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f450c077800 0x7f450c0780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.812+0000 7f4511800640 1 --2- 172.21.15.60:0/2934845503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450c075c20 0x7f450c076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.812+0000 7f4511800640 1 -- 172.21.15.60:0/2934845503 >> 172.21.15.60:0/2934845503 conn(0x7f450c0fea10 msgr2=0x7f450c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:47.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.812+0000 7f4511800640 1 -- 172.21.15.60:0/2934845503 shutdown_connections 2024-02-26T10:45:47.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.812+0000 7f4511800640 1 -- 172.21.15.60:0/2934845503 wait complete. 2024-02-26T10:45:47.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.813+0000 7f4511800640 1 Processor -- start 2024-02-26T10:45:47.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.813+0000 7f4511800640 1 -- start start 2024-02-26T10:45:47.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f4511800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450c075c20 0x7f450c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f4511800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f450c076e60 0x7f450c19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f4511800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450c077800 0x7f450c1a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f4511800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c113530 con 0x7f450c076e60 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f4511800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c1133b0 con 0x7f450c077800 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f4511800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c1136b0 con 0x7f450c075c20 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450c075c20 0x7f450c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450c075c20 0x7f450c19dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49596/0 (socket says 172.21.15.60:49596) 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450aa00640 1 -- 172.21.15.60:0/2954397871 learned_addr learned my addr 172.21.15.60:0/2954397871 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450a000640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f450c076e60 0x7f450c19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450b400640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450c077800 0x7f450c1a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450aa00640 1 -- 172.21.15.60:0/2954397871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450c077800 msgr2=0x7f450c1a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450aa00640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450c077800 0x7f450c1a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450aa00640 1 -- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f450c076e60 msgr2=0x7f450c19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450aa00640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f450c076e60 0x7f450c19e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.814+0000 7f450aa00640 1 -- 172.21.15.60:0/2954397871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c1a57e0 con 0x7f450c075c20 2024-02-26T10:45:47.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.815+0000 7f450aa00640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450c075c20 0x7f450c19dc10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00e9f0 tx=0x7f44fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:47.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.815+0000 7f4503400640 1 -- 172.21.15.60:0/2954397871 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc00ccc0 con 0x7f450c075c20 2024-02-26T10:45:47.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.815+0000 7f4503400640 1 -- 172.21.15.60:0/2954397871 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44fc005500 con 0x7f450c075c20 2024-02-26T10:45:47.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.815+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c1a5ad0 con 0x7f450c075c20 2024-02-26T10:45:47.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.815+0000 7f4503400640 1 -- 172.21.15.60:0/2954397871 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc010690 con 0x7f450c075c20 2024-02-26T10:45:47.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.815+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c1a5fa0 con 0x7f450c075c20 2024-02-26T10:45:47.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.816+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450c10fe10 con 0x7f450c075c20 2024-02-26T10:45:47.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.818+0000 7f4503400640 1 -- 172.21.15.60:0/2954397871 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f44fc0046c0 con 0x7f450c075c20 2024-02-26T10:45:47.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.820+0000 7f4503400640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44f00a1e10 0x7f44f00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:47.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.820+0000 7f4503400640 1 -- 172.21.15.60:0/2954397871 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f44fc0ce690 con 0x7f450c075c20 2024-02-26T10:45:47.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.820+0000 7f450a000640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44f00a1e10 0x7f44f00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:47.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.821+0000 7f450a000640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44f00a1e10 0x7f44f00a42d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f44f802c180 tx=0x7f44f8034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:47.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:47.824+0000 7f4503400640 1 -- 172.21.15.60:0/2954397871 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f44fc09b550 con 0x7f450c075c20 2024-02-26T10:45:48.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.027+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f450c02d0a0 con 0x7f450c075c20 2024-02-26T10:45:48.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.028+0000 7f4503400640 1 -- 172.21.15.60:0/2954397871 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44fc09ae50 con 0x7f450c075c20 2024-02-26T10:45:48.023 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:45:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44f00a1e10 msgr2=0x7f44f00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44f00a1e10 0x7f44f00a42d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f44f802c180 tx=0x7f44f8034ad0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450c075c20 msgr2=0x7f450c19dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450c075c20 0x7f450c19dc10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00e9f0 tx=0x7f44fc00eeb0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 shutdown_connections 2024-02-26T10:45:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f44f00a1e10 0x7f44f00a42d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450c077800 0x7f450c1a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f450c076e60 0x7f450c19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 --2- 172.21.15.60:0/2954397871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450c075c20 0x7f450c19dc10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 >> 172.21.15.60:0/2954397871 conn(0x7f450c0fea10 msgr2=0x7f450c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 shutdown_connections 2024-02-26T10:45:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.031+0000 7f4511800640 1 -- 172.21.15.60:0/2954397871 wait complete. 2024-02-26T10:45:48.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:45:48.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:47 smithi060 ceph-mon[31539]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:45:48.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:47 smithi060 ceph-mon[31539]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:45:48.129 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:47 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1910702572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:48.234 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:47 smithi113 ceph-mon[36126]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:45:48.234 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:47 smithi113 ceph-mon[36126]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:45:48.234 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:47 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1910702572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.281+0000 7f0f5e400640 1 -- 172.21.15.60:0/3470186464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f58104690 msgr2=0x7f0f58104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.281+0000 7f0f5e400640 1 --2- 172.21.15.60:0/3470186464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f58104690 0x7f0f58104b10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009900 tx=0x7f0f4402bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.282+0000 7f0f5e400640 1 -- 172.21.15.60:0/3470186464 shutdown_connections 2024-02-26T10:45:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.282+0000 7f0f5e400640 1 --2- 172.21.15.60:0/3470186464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f58105050 0x7f0f5810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.282+0000 7f0f5e400640 1 --2- 172.21.15.60:0/3470186464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f58104690 0x7f0f58104b10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.282+0000 7f0f5e400640 1 --2- 172.21.15.60:0/3470186464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f581034a0 0x7f0f581038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.282+0000 7f0f5e400640 1 -- 172.21.15.60:0/3470186464 >> 172.21.15.60:0/3470186464 conn(0x7f0f580fea10 msgr2=0x7f0f58100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.282+0000 7f0f5e400640 1 -- 172.21.15.60:0/3470186464 shutdown_connections 2024-02-26T10:45:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.282+0000 7f0f5e400640 1 -- 172.21.15.60:0/3470186464 wait complete. 2024-02-26T10:45:48.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f5e400640 1 Processor -- start 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f5e400640 1 -- start start 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f5e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f581034a0 0x7f0f58199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f5e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f58104690 0x7f0f58199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f5e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f58105050 0x7f0f581a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f5e400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5810f180 con 0x7f0f581034a0 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f5e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5810f000 con 0x7f0f58105050 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f5e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5810f300 con 0x7f0f58104690 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f57400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f581034a0 0x7f0f58199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f57400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f581034a0 0x7f0f58199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35202/0 (socket says 172.21.15.60:35202) 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f57400640 1 -- 172.21.15.60:0/1923940331 learned_addr learned my addr 172.21.15.60:0/1923940331 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f56a00640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f58104690 0x7f0f58199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:48.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.283+0000 7f0f57e00640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f58105050 0x7f0f581a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.284+0000 7f0f57e00640 1 -- 172.21.15.60:0/1923940331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f58104690 msgr2=0x7f0f58199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.284+0000 7f0f57e00640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f58104690 0x7f0f58199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.284+0000 7f0f57e00640 1 -- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f581034a0 msgr2=0x7f0f58199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.284+0000 7f0f57e00640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f581034a0 0x7f0f58199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.284+0000 7f0f57e00640 1 -- 172.21.15.60:0/1923940331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f581a1370 con 0x7f0f58105050 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.284+0000 7f0f57400640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f581034a0 0x7f0f58199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.284+0000 7f0f57e00640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f58105050 0x7f0f581a0d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000e970 tx=0x7f0f5000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.284+0000 7f0f4fe00640 1 -- 172.21.15.60:0/1923940331 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000ccc0 con 0x7f0f58105050 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.285+0000 7f0f4fe00640 1 -- 172.21.15.60:0/1923940331 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f50005020 con 0x7f0f58105050 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.285+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581a1660 con 0x7f0f58105050 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.285+0000 7f0f4fe00640 1 -- 172.21.15.60:0/1923940331 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50004a00 con 0x7f0f58105050 2024-02-26T10:45:48.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.285+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f581a1b30 con 0x7f0f58105050 2024-02-26T10:45:48.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.286+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5810a7c0 con 0x7f0f58105050 2024-02-26T10:45:48.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.288+0000 7f0f4fe00640 1 -- 172.21.15.60:0/1923940331 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f0f500051d0 con 0x7f0f58105050 2024-02-26T10:45:48.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.289+0000 7f0f4fe00640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0f3c0a1e10 0x7f0f3c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:48.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.289+0000 7f0f4fe00640 1 -- 172.21.15.60:0/1923940331 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f0f500ce2c0 con 0x7f0f58105050 2024-02-26T10:45:48.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.289+0000 7f0f57400640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0f3c0a1e10 0x7f0f3c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:48.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.290+0000 7f0f57400640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0f3c0a1e10 0x7f0f3c0a42d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800f7b0 tx=0x7f0f48009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.293+0000 7f0f4fe00640 1 -- 172.21.15.60:0/1923940331 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f5009b180 con 0x7f0f58105050 2024-02-26T10:45:48.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:47 smithi100 ceph-mon[36051]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:45:48.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:47 smithi100 ceph-mon[36051]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T10:45:48.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:47 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1910702572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.495+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f58061e60 con 0x7f0f58105050 2024-02-26T10:45:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.496+0000 7f0f4fe00640 1 -- 172.21.15.60:0/1923940331 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f5009aa80 con 0x7f0f58105050 2024-02-26T10:45:48.492 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0f3c0a1e10 msgr2=0x7f0f3c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0f3c0a1e10 0x7f0f3c0a42d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800f7b0 tx=0x7f0f48009500 comp rx=0 tx=0).stop 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f58105050 msgr2=0x7f0f581a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f58105050 0x7f0f581a0d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000e970 tx=0x7f0f5000ee30 comp rx=0 tx=0).stop 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 shutdown_connections 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0f3c0a1e10 0x7f0f3c0a42d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f58105050 0x7f0f581a0d00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f58104690 0x7f0f58199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 --2- 172.21.15.60:0/1923940331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f581034a0 0x7f0f58199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.499+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 >> 172.21.15.60:0/1923940331 conn(0x7f0f580fea10 msgr2=0x7f0f58107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.500+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 shutdown_connections 2024-02-26T10:45:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.500+0000 7f0f5e400640 1 -- 172.21.15.60:0/1923940331 wait complete. 2024-02-26T10:45:48.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:45:48.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.739+0000 7ff49f200640 1 -- 172.21.15.60:0/2559342281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a0109020 msgr2=0x7ff4a010b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.739+0000 7ff49f200640 1 --2- 172.21.15.60:0/2559342281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a0109020 0x7ff4a010b4b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff498009900 tx=0x7ff49802bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:48.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.739+0000 7ff49f200640 1 -- 172.21.15.60:0/2559342281 shutdown_connections 2024-02-26T10:45:48.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.739+0000 7ff49f200640 1 --2- 172.21.15.60:0/2559342281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a0109020 0x7ff4a010b4b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.739+0000 7ff49f200640 1 --2- 172.21.15.60:0/2559342281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a0073cc0 0x7ff4a0074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.739+0000 7ff49f200640 1 --2- 172.21.15.60:0/2559342281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a00752a0 0x7ff4a00736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.739+0000 7ff49f200640 1 -- 172.21.15.60:0/2559342281 >> 172.21.15.60:0/2559342281 conn(0x7ff4a00fc6d0 msgr2=0x7ff4a00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:48.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.739+0000 7ff49f200640 1 -- 172.21.15.60:0/2559342281 shutdown_connections 2024-02-26T10:45:48.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.739+0000 7ff49f200640 1 -- 172.21.15.60:0/2559342281 wait complete. 2024-02-26T10:45:48.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.740+0000 7ff49f200640 1 Processor -- start 2024-02-26T10:45:48.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.740+0000 7ff49f200640 1 -- start start 2024-02-26T10:45:48.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff49f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0073cc0 0x7ff4a0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:48.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff49f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a00752a0 0x7ff4a0199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:48.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff49f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a0109020 0x7ff4a01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:48.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff49f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a010eff0 con 0x7ff4a00752a0 2024-02-26T10:45:48.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff49f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a010ee70 con 0x7ff4a0073cc0 2024-02-26T10:45:48.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff49f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a010f170 con 0x7ff4a0109020 2024-02-26T10:45:48.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff495400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a00752a0 0x7ff4a0199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:48.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff49de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0073cc0 0x7ff4a0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:48.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff495400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a00752a0 0x7ff4a0199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35208/0 (socket says 172.21.15.60:35208) 2024-02-26T10:45:48.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff49e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a0109020 0x7ff4a01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:48.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.741+0000 7ff495400640 1 -- 172.21.15.60:0/1091558280 learned_addr learned my addr 172.21.15.60:0/1091558280 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff49de00640 1 -- 172.21.15.60:0/1091558280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a0109020 msgr2=0x7ff4a01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff49de00640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a0109020 0x7ff4a01a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff49de00640 1 -- 172.21.15.60:0/1091558280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a00752a0 msgr2=0x7ff4a0199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff49de00640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a00752a0 0x7ff4a0199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff49de00640 1 -- 172.21.15.60:0/1091558280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a01a1220 con 0x7ff4a0073cc0 2024-02-26T10:45:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff49de00640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0073cc0 0x7ff4a0199600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff49000e9f0 tx=0x7ff49000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff497400640 1 -- 172.21.15.60:0/1091558280 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49000cd80 con 0x7ff4a0073cc0 2024-02-26T10:45:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a01a1510 con 0x7ff4a0073cc0 2024-02-26T10:45:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff497400640 1 -- 172.21.15.60:0/1091558280 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4900035f0 con 0x7ff4a0073cc0 2024-02-26T10:45:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff497400640 1 -- 172.21.15.60:0/1091558280 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490010670 con 0x7ff4a0073cc0 2024-02-26T10:45:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.742+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a01a1a60 con 0x7ff4a0073cc0 2024-02-26T10:45:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.743+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a0107d50 con 0x7ff4a0073cc0 2024-02-26T10:45:48.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.746+0000 7ff497400640 1 -- 172.21.15.60:0/1091558280 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff4900037a0 con 0x7ff4a0073cc0 2024-02-26T10:45:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.747+0000 7ff497400640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4840a1e10 0x7ff4840a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.747+0000 7ff497400640 1 -- 172.21.15.60:0/1091558280 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7ff4900ce5f0 con 0x7ff4a0073cc0 2024-02-26T10:45:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.747+0000 7ff495400640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4840a1e10 0x7ff4840a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:48.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.748+0000 7ff495400640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4840a1e10 0x7ff4840a42d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff4a019ab10 tx=0x7ff488009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:48.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.751+0000 7ff497400640 1 -- 172.21.15.60:0/1091558280 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff49009b4b0 con 0x7ff4a0073cc0 2024-02-26T10:45:48.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.951+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4a0061e60 con 0x7ff4a0073cc0 2024-02-26T10:45:48.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.951+0000 7ff497400640 1 -- 172.21.15.60:0/1091558280 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff49009adb0 con 0x7ff4a0073cc0 2024-02-26T10:45:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:45:48.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4840a1e10 msgr2=0x7ff4840a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4840a1e10 0x7ff4840a42d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff4a019ab10 tx=0x7ff488009500 comp rx=0 tx=0).stop 2024-02-26T10:45:48.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0073cc0 msgr2=0x7ff4a0199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0073cc0 0x7ff4a0199600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff49000e9f0 tx=0x7ff49000eeb0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 shutdown_connections 2024-02-26T10:45:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff4840a1e10 0x7ff4840a42d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a0109020 0x7ff4a01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a00752a0 0x7ff4a0199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 --2- 172.21.15.60:0/1091558280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0073cc0 0x7ff4a0199600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.954+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 >> 172.21.15.60:0/1091558280 conn(0x7ff4a00fc6d0 msgr2=0x7ff4a0104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.955+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 shutdown_connections 2024-02-26T10:45:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:48.955+0000 7ff49f200640 1 -- 172.21.15.60:0/1091558280 wait complete. 2024-02-26T10:45:48.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:45:49.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.199+0000 7f1004e00640 1 -- 172.21.15.60:0/2619530913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1000101c20 msgr2=0x7f1000104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.199+0000 7f1004e00640 1 --2- 172.21.15.60:0/2619530913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1000101c20 0x7f1000104010 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009ca0 tx=0x7f0ff802c100 comp rx=0 tx=0).stop 2024-02-26T10:45:49.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.200+0000 7f1004e00640 1 -- 172.21.15.60:0/2619530913 shutdown_connections 2024-02-26T10:45:49.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.200+0000 7f1004e00640 1 --2- 172.21.15.60:0/2619530913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1000104550 0x7f1000106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.200+0000 7f1004e00640 1 --2- 172.21.15.60:0/2619530913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000106e80 0x7f1000109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.200+0000 7f1004e00640 1 --2- 172.21.15.60:0/2619530913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1000101c20 0x7f1000104010 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.200+0000 7f1004e00640 1 -- 172.21.15.60:0/2619530913 >> 172.21.15.60:0/2619530913 conn(0x7f10000fb5d0 msgr2=0x7f10000fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:49.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.200+0000 7f1004e00640 1 -- 172.21.15.60:0/2619530913 shutdown_connections 2024-02-26T10:45:49.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.200+0000 7f1004e00640 1 -- 172.21.15.60:0/2619530913 wait complete. 2024-02-26T10:45:49.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.201+0000 7f1004e00640 1 Processor -- start 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.201+0000 7f1004e00640 1 -- start start 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.201+0000 7f1004e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000101c20 0x7f1000199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.201+0000 7f1004e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1000104550 0x7f1000199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.201+0000 7f1004e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1000106e80 0x7f10001a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.201+0000 7f1004e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100010cdd0 con 0x7f1000101c20 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.201+0000 7f1004e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100010cc50 con 0x7f1000104550 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.201+0000 7f1004e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100010cf50 con 0x7f1000106e80 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0fff400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000101c20 0x7f1000199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0ffea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1000104550 0x7f1000199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0ffea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1000104550 0x7f1000199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56392/0 (socket says 172.21.15.60:56392) 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0ffea00640 1 -- 172.21.15.60:0/4163412544 learned_addr learned my addr 172.21.15.60:0/4163412544 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0fffe00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1000106e80 0x7f10001a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0fff400640 1 -- 172.21.15.60:0/4163412544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1000106e80 msgr2=0x7f10001a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0fff400640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1000106e80 0x7f10001a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0fff400640 1 -- 172.21.15.60:0/4163412544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1000104550 msgr2=0x7f1000199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0fff400640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1000104550 0x7f1000199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0fff400640 1 -- 172.21.15.60:0/4163412544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10001a12a0 con 0x7f1000101c20 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0fff400640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000101c20 0x7f1000199680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009bf0 tx=0x7f0ff8035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.202+0000 7f0ff7e00640 1 -- 172.21.15.60:0/4163412544 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8039040 con 0x7f1000101c20 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.203+0000 7f0ff7e00640 1 -- 172.21.15.60:0/4163412544 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ff8003970 con 0x7f1000101c20 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.203+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10001a1530 con 0x7f1000101c20 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.203+0000 7f0ff7e00640 1 -- 172.21.15.60:0/4163412544 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8004360 con 0x7f1000101c20 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.203+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10001a19f0 con 0x7f1000101c20 2024-02-26T10:45:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.204+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1000108970 con 0x7f1000101c20 2024-02-26T10:45:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.206+0000 7f0ff7e00640 1 -- 172.21.15.60:0/4163412544 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f0ff8046030 con 0x7f1000101c20 2024-02-26T10:45:49.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.207+0000 7f0ff7e00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0fe40a1e10 0x7f0fe40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:49.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.207+0000 7f0ff7e00640 1 -- 172.21.15.60:0/4163412544 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f0ff80efef0 con 0x7f1000101c20 2024-02-26T10:45:49.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.207+0000 7f0ffea00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0fe40a1e10 0x7f0fe40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:49.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.208+0000 7f0ffea00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0fe40a1e10 0x7f0fe40a42d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f100019ab90 tx=0x7f0fe800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:49.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.211+0000 7f0ff7e00640 1 -- 172.21.15.60:0/4163412544 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ff804d060 con 0x7f1000101c20 2024-02-26T10:45:49.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:48 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2954397871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:49.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:48 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1923940331' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:49.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:48 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2954397871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:49.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:48 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1923940331' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:49.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:48 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2954397871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:45:49.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:48 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1923940331' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:45:49.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.418+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f100002d050 con 0x7f1000101c20 2024-02-26T10:45:49.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.419+0000 7f0ff7e00640 1 -- 172.21.15.60:0/4163412544 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0ff80bcf50 con 0x7f1000101c20 2024-02-26T10:45:49.414 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0fe40a1e10 msgr2=0x7f0fe40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0fe40a1e10 0x7f0fe40a42d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f100019ab90 tx=0x7f0fe800b410 comp rx=0 tx=0).stop 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000101c20 msgr2=0x7f1000199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000101c20 0x7f1000199680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009bf0 tx=0x7f0ff8035d10 comp rx=0 tx=0).stop 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 shutdown_connections 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0fe40a1e10 0x7f0fe40a42d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1000104550 0x7f1000199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1000106e80 0x7f10001a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 --2- 172.21.15.60:0/4163412544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000101c20 0x7f1000199680 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 >> 172.21.15.60:0/4163412544 conn(0x7f10000fb5d0 msgr2=0x7f1000101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:49.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 shutdown_connections 2024-02-26T10:45:49.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.422+0000 7f1004e00640 1 -- 172.21.15.60:0/4163412544 wait complete. 2024-02-26T10:45:49.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:45:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.665+0000 7f1307a00640 1 -- 172.21.15.60:0/16605066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13001006a0 msgr2=0x7f1300100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.665+0000 7f1307a00640 1 --2- 172.21.15.60:0/16605066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13001006a0 0x7f1300100b20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009900 tx=0x7f12e802bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.665+0000 7f1307a00640 1 -- 172.21.15.60:0/16605066 shutdown_connections 2024-02-26T10:45:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.665+0000 7f1307a00640 1 --2- 172.21.15.60:0/16605066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1300101060 0x7f13001078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.665+0000 7f1307a00640 1 --2- 172.21.15.60:0/16605066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13001006a0 0x7f1300100b20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.665+0000 7f1307a00640 1 --2- 172.21.15.60:0/16605066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13000ff4b0 0x7f13000ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.665+0000 7f1307a00640 1 -- 172.21.15.60:0/16605066 >> 172.21.15.60:0/16605066 conn(0x7f13000faa60 msgr2=0x7f13000fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.665+0000 7f1307a00640 1 -- 172.21.15.60:0/16605066 shutdown_connections 2024-02-26T10:45:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.665+0000 7f1307a00640 1 -- 172.21.15.60:0/16605066 wait complete. 2024-02-26T10:45:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.666+0000 7f1307a00640 1 Processor -- start 2024-02-26T10:45:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f1307a00640 1 -- start start 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f1307a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13000ff4b0 0x7f1300103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f1307a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13001006a0 0x7f1300101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f1307a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1300101060 0x7f1300102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f1307a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130010ad00 con 0x7f13001006a0 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f1307a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130010ab80 con 0x7f1300101060 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f1307a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130010ae80 con 0x7f13000ff4b0 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f12f7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13001006a0 0x7f1300101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f1305a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1300101060 0x7f1300102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f12f7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13001006a0 0x7f1300101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35236/0 (socket says 172.21.15.60:35236) 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f1305000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13000ff4b0 0x7f1300103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.667+0000 7f12f7e00640 1 -- 172.21.15.60:0/1527650854 learned_addr learned my addr 172.21.15.60:0/1527650854 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.668+0000 7f1305000640 1 -- 172.21.15.60:0/1527650854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1300101060 msgr2=0x7f1300102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.668+0000 7f1305000640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1300101060 0x7f1300102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.668+0000 7f1305000640 1 -- 172.21.15.60:0/1527650854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13001006a0 msgr2=0x7f1300101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.668+0000 7f1305000640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13001006a0 0x7f1300101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.668+0000 7f1305000640 1 -- 172.21.15.60:0/1527650854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300102920 con 0x7f13000ff4b0 2024-02-26T10:45:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.668+0000 7f1305000640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13000ff4b0 0x7f1300103510 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12f00097d0 tx=0x7f12f0009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.668+0000 7f12fe000640 1 -- 172.21.15.60:0/1527650854 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0004230 con 0x7f13000ff4b0 2024-02-26T10:45:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.669+0000 7f12fe000640 1 -- 172.21.15.60:0/1527650854 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12f00043d0 con 0x7f13000ff4b0 2024-02-26T10:45:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.669+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13001a1690 con 0x7f13000ff4b0 2024-02-26T10:45:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.669+0000 7f12fe000640 1 -- 172.21.15.60:0/1527650854 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0003d30 con 0x7f13000ff4b0 2024-02-26T10:45:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.669+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13001a1bb0 con 0x7f13000ff4b0 2024-02-26T10:45:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.670+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13000684a0 con 0x7f13000ff4b0 2024-02-26T10:45:49.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.672+0000 7f12fe000640 1 -- 172.21.15.60:0/1527650854 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f12f000e6b0 con 0x7f13000ff4b0 2024-02-26T10:45:49.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.673+0000 7f12fe000640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f12e40a1e90 0x7f12e40a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:49.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.673+0000 7f12fe000640 1 -- 172.21.15.60:0/1527650854 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f12f00ce9b0 con 0x7f13000ff4b0 2024-02-26T10:45:49.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.674+0000 7f12f7e00640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f12e40a1e90 0x7f12e40a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:49.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.674+0000 7f12f7e00640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f12e40a1e90 0x7f12e40a4350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f12e8000c00 tx=0x7f12e802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:49.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.676+0000 7f12fe000640 1 -- 172.21.15.60:0/1527650854 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12f0011070 con 0x7f13000ff4b0 2024-02-26T10:45:49.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.918+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13001a1ed0 con 0x7f13000ff4b0 2024-02-26T10:45:49.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.919+0000 7f12fe000640 1 -- 172.21.15.60:0/1527650854 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11359 (secure 0 0 0) 0x7f12f009b870 con 0x7f13000ff4b0 2024-02-26T10:45:49.914 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:49.915 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:45:39.355500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:45:49.915 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2024-02-26T10:45:49.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f12e40a1e90 msgr2=0x7f12e40a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f12e40a1e90 0x7f12e40a4350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f12e8000c00 tx=0x7f12e802c880 comp rx=0 tx=0).stop 2024-02-26T10:45:49.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13000ff4b0 msgr2=0x7f1300103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:49.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13000ff4b0 0x7f1300103510 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12f00097d0 tx=0x7f12f0009c90 comp rx=0 tx=0).stop 2024-02-26T10:45:49.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 shutdown_connections 2024-02-26T10:45:49.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f12e40a1e90 0x7f12e40a4350 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1300101060 0x7f1300102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13001006a0 0x7f1300101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 --2- 172.21.15.60:0/1527650854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13000ff4b0 0x7f1300103510 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 >> 172.21.15.60:0/1527650854 conn(0x7f13000faa60 msgr2=0x7f1300105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 shutdown_connections 2024-02-26T10:45:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:49.922+0000 7f1307a00640 1 -- 172.21.15.60:0/1527650854 wait complete. 2024-02-26T10:45:49.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:45:50.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.169+0000 7fa137e00640 1 -- 172.21.15.60:0/3796587372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138108f60 msgr2=0x7fa13810b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.169+0000 7fa137e00640 1 --2- 172.21.15.60:0/3796587372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138108f60 0x7fa13810b3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa130009ca0 tx=0x7fa13002c100 comp rx=0 tx=0).stop 2024-02-26T10:45:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.169+0000 7fa137e00640 1 -- 172.21.15.60:0/3796587372 shutdown_connections 2024-02-26T10:45:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.169+0000 7fa137e00640 1 --2- 172.21.15.60:0/3796587372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa138073cc0 0x7fa138074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.169+0000 7fa137e00640 1 --2- 172.21.15.60:0/3796587372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138108f60 0x7fa13810b3f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.169+0000 7fa137e00640 1 --2- 172.21.15.60:0/3796587372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1380752a0 0x7fa1380736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.169+0000 7fa137e00640 1 -- 172.21.15.60:0/3796587372 >> 172.21.15.60:0/3796587372 conn(0x7fa1380fc610 msgr2=0x7fa1380fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.169+0000 7fa137e00640 1 -- 172.21.15.60:0/3796587372 shutdown_connections 2024-02-26T10:45:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.169+0000 7fa137e00640 1 -- 172.21.15.60:0/3796587372 wait complete. 2024-02-26T10:45:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.171+0000 7fa137e00640 1 Processor -- start 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.171+0000 7fa137e00640 1 -- start start 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.171+0000 7fa137e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138073cc0 0x7fa138199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.171+0000 7fa137e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1380752a0 0x7fa138199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa137e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa138108f60 0x7fa1381a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa137e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13810ef20 con 0x7fa1380752a0 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa137e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13810eda0 con 0x7fa138108f60 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa137e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13810f0a0 con 0x7fa138073cc0 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1380752a0 0x7fa138199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1380752a0 0x7fa138199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35256/0 (socket says 172.21.15.60:35256) 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136000640 1 -- 172.21.15.60:0/273964459 learned_addr learned my addr 172.21.15.60:0/273964459 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136a00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138073cc0 0x7fa138199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa137400640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa138108f60 0x7fa1381a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136000640 1 -- 172.21.15.60:0/273964459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138073cc0 msgr2=0x7fa138199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136000640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138073cc0 0x7fa138199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136000640 1 -- 172.21.15.60:0/273964459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa138108f60 msgr2=0x7fa1381a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136000640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa138108f60 0x7fa1381a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136000640 1 -- 172.21.15.60:0/273964459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1381a1190 con 0x7fa1380752a0 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa136a00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138073cc0 0x7fa138199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.172+0000 7fa137400640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa138108f60 0x7fa1381a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.173+0000 7fa136000640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1380752a0 0x7fa138199ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa12400e9f0 tx=0x7fa12400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.173+0000 7fa12f400640 1 -- 172.21.15.60:0/273964459 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12400cd80 con 0x7fa1380752a0 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.173+0000 7fa12f400640 1 -- 172.21.15.60:0/273964459 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa124005500 con 0x7fa1380752a0 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.173+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1381a1480 con 0x7fa1380752a0 2024-02-26T10:45:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.173+0000 7fa12f400640 1 -- 172.21.15.60:0/273964459 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124010690 con 0x7fa1380752a0 2024-02-26T10:45:50.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.173+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1381a19d0 con 0x7fa1380752a0 2024-02-26T10:45:50.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.175+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa138107c90 con 0x7fa1380752a0 2024-02-26T10:45:50.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.177+0000 7fa12f400640 1 -- 172.21.15.60:0/273964459 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa1240046c0 con 0x7fa1380752a0 2024-02-26T10:45:50.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.178+0000 7fa12f400640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa11c0a1e90 0x7fa11c0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.178+0000 7fa12f400640 1 -- 172.21.15.60:0/273964459 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa1240ce6e0 con 0x7fa1380752a0 2024-02-26T10:45:50.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.179+0000 7fa136a00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa11c0a1e90 0x7fa11c0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.179+0000 7fa136a00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa11c0a1e90 0x7fa11c0a4350 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa128005d00 tx=0x7fa12800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:50.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.182+0000 7fa12f400640 1 -- 172.21.15.60:0/273964459 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa12409b5a0 con 0x7fa1380752a0 2024-02-26T10:45:50.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:49 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1091558280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:50.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:49 smithi113 ceph-mon[36126]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-02-26T10:45:50.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:49 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/4163412544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:50.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:49 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1091558280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:50.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:49 smithi060 ceph-mon[31539]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-02-26T10:45:50.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:49 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4163412544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:50.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:49 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1091558280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:45:50.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:49 smithi100 ceph-mon[36051]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-02-26T10:45:50.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:49 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/4163412544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:45:50.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.385+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa13802d080 con 0x7fa1380752a0 2024-02-26T10:45:50.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.390+0000 7fa12f400640 1 -- 172.21.15.60:0/273964459 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7fa12409aea0 con 0x7fa1380752a0 2024-02-26T10:45:50.385 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:45:50.385 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.392+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa11c0a1e90 msgr2=0x7fa11c0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa11c0a1e90 0x7fa11c0a4350 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa128005d00 tx=0x7fa12800b410 comp rx=0 tx=0).stop 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1380752a0 msgr2=0x7fa138199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1380752a0 0x7fa138199ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa12400e9f0 tx=0x7fa12400edf0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 shutdown_connections 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa11c0a1e90 0x7fa11c0a4350 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa138108f60 0x7fa1381a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1380752a0 0x7fa138199ab0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 --2- 172.21.15.60:0/273964459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa138073cc0 0x7fa138199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 >> 172.21.15.60:0/273964459 conn(0x7fa1380fc610 msgr2=0x7fa138104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:50.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 shutdown_connections 2024-02-26T10:45:50.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.393+0000 7fa137e00640 1 -- 172.21.15.60:0/273964459 wait complete. 2024-02-26T10:45:50.409 DEBUG:tasks.fwd_scrub:Yielding 2024-02-26T10:45:50.410 INFO:teuthology.run_tasks:Running task background_exec... 2024-02-26T10:45:50.420 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T10:45:50.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-02-26T10:45:50.420 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-02-26T10:45:50.420 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T10:45:50.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:45:50.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T10:45:50.428 INFO:teuthology.run_tasks:Running task workunit... 2024-02-26T10:45:50.446 INFO:tasks.workunit:Pulling workunits from ref 6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:45:50.446 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-26T10:45:50.446 INFO:tasks.workunit:timeout=6h 2024-02-26T10:45:50.447 INFO:tasks.workunit:cleanup=True 2024-02-26T10:45:50.447 DEBUG:teuthology.orchestra.run.smithi060:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-26T10:45:50.490 INFO:teuthology.orchestra.run.smithi060.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-26T10:45:50.490 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-26T10:45:50.490 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-02-26T10:45:50.490 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:45:50.490 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-02-26T10:45:50.491 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 10:43:18.126216761 +0000 2024-02-26T10:45:50.491 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 10:43:18.126216761 +0000 2024-02-26T10:45:50.491 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 10:45:38.548927747 +0000 2024-02-26T10:45:50.491 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 10:43:18.126216761 +0000 2024-02-26T10:45:50.491 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-26T10:45:50.491 DEBUG:teuthology.orchestra.run.smithi060:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-26T10:45:50.624 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:45:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.636+0000 7f3648a00640 1 -- 172.21.15.60:0/142677521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3644075220 msgr2=0x7f3644073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.636+0000 7f3648a00640 1 --2- 172.21.15.60:0/142677521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3644075220 0x7f3644073670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3634009900 tx=0x7f363402bcc0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.637+0000 7f3648a00640 1 -- 172.21.15.60:0/142677521 shutdown_connections 2024-02-26T10:45:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.637+0000 7f3648a00640 1 --2- 172.21.15.60:0/142677521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3644109010 0x7f364410b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.637+0000 7f3648a00640 1 --2- 172.21.15.60:0/142677521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3644073c40 0x7f36440740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.637+0000 7f3648a00640 1 --2- 172.21.15.60:0/142677521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3644075220 0x7f3644073670 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.637+0000 7f3648a00640 1 -- 172.21.15.60:0/142677521 >> 172.21.15.60:0/142677521 conn(0x7f36440fc6d0 msgr2=0x7f36440feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.637+0000 7f3648a00640 1 -- 172.21.15.60:0/142677521 shutdown_connections 2024-02-26T10:45:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.637+0000 7f3648a00640 1 -- 172.21.15.60:0/142677521 wait complete. 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.638+0000 7f3648a00640 1 Processor -- start 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.638+0000 7f3648a00640 1 -- start start 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.638+0000 7f3648a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3644073c40 0x7f3644199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.638+0000 7f3648a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3644075220 0x7f3644199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.638+0000 7f3648a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3644109010 0x7f36441a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.638+0000 7f3648a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364410f050 con 0x7f3644073c40 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.638+0000 7f3648a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364410eed0 con 0x7f3644109010 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.638+0000 7f3648a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364410f1d0 con 0x7f3644075220 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3642800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3644073c40 0x7f3644199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3642800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3644073c40 0x7f3644199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35274/0 (socket says 172.21.15.60:35274) 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3642800640 1 -- 172.21.15.60:0/3536302364 learned_addr learned my addr 172.21.15.60:0/3536302364 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3639e00640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3644075220 0x7f3644199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3643200640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3644109010 0x7f36441a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3642800640 1 -- 172.21.15.60:0/3536302364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3644075220 msgr2=0x7f3644199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3642800640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3644075220 0x7f3644199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3642800640 1 -- 172.21.15.60:0/3536302364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3644109010 msgr2=0x7f36441a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3642800640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3644109010 0x7f36441a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3642800640 1 -- 172.21.15.60:0/3536302364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36441a1260 con 0x7f3644073c40 2024-02-26T10:45:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3639e00640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3644075220 0x7f3644199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.639+0000 7f3643200640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3644109010 0x7f36441a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.640+0000 7f3642800640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3644073c40 0x7f3644199640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3634000c00 tx=0x7f363402c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.640+0000 7f363be00640 1 -- 172.21.15.60:0/3536302364 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634043020 con 0x7f3644073c40 2024-02-26T10:45:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.640+0000 7f363be00640 1 -- 172.21.15.60:0/3536302364 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36340054e0 con 0x7f3644073c40 2024-02-26T10:45:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.640+0000 7f3648a00640 1 -- 172.21.15.60:0/3536302364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36441a14f0 con 0x7f3644073c40 2024-02-26T10:45:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.640+0000 7f363be00640 1 -- 172.21.15.60:0/3536302364 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634003c20 con 0x7f3644073c40 2024-02-26T10:45:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.640+0000 7f3648a00640 1 -- 172.21.15.60:0/3536302364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36441a1a90 con 0x7f3644073c40 2024-02-26T10:45:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.643+0000 7f363be00640 1 -- 172.21.15.60:0/3536302364 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f3634046020 con 0x7f3644073c40 2024-02-26T10:45:50.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.644+0000 7f364c40f740 1 Processor -- start 2024-02-26T10:45:50.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.644+0000 7f364c40f740 1 -- start start 2024-02-26T10:45:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.644+0000 7f364c40f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715a280 0x5605e715a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.644+0000 7f364c40f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e715f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.644+0000 7f364c40f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7162270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.644+0000 7f364c40f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e706ddf0 con 0x5605e715fde0 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.644+0000 7f364c40f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e70bf9f0 con 0x5605e715a280 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.644+0000 7f364c40f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e7068ad0 con 0x5605e715ac70 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3639e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3642800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e715f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3643200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715a280 0x5605e715a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3639e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7162270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35276/0 (socket says 172.21.15.60:35276) 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3639e00640 1 -- 172.21.15.60:0/3483916479 learned_addr learned my addr 172.21.15.60:0/3483916479 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3639e00640 1 -- 172.21.15.60:0/3483916479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 msgr2=0x5605e715f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3639e00640 1 --2- 172.21.15.60:0/3483916479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e715f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3639e00640 1 -- 172.21.15.60:0/3483916479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715a280 msgr2=0x5605e715a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3639e00640 1 --2- 172.21.15.60:0/3483916479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715a280 0x5605e715a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3639e00640 1 -- 172.21.15.60:0/3483916479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e707dde0 con 0x5605e715fde0 2024-02-26T10:45:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3643200640 1 --2- 172.21.15.60:0/3483916479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715a280 0x5605e715a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f3642800640 1 --2- 172.21.15.60:0/3483916479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e715f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.645+0000 7f363be00640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36240a1e90 0x7f36240a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.646+0000 7f3639e00640 1 --2- 172.21.15.60:0/3483916479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7162270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f362c0114f0 tx=0x7f362c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ec75084c3de78b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.646+0000 7f363be00640 1 -- 172.21.15.60:0/3536302364 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f36340f01d0 con 0x7f3644073c40 2024-02-26T10:45:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.646+0000 7f3643200640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36240a1e90 0x7f36240a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.646+0000 7f3633400640 1 -- 172.21.15.60:0/3483916479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c01e070 con 0x5605e715fde0 2024-02-26T10:45:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.646+0000 7f3633400640 1 -- 172.21.15.60:0/3483916479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f362c00faa0 con 0x5605e715fde0 2024-02-26T10:45:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.646+0000 7f3633400640 1 -- 172.21.15.60:0/3483916479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c019be0 con 0x5605e715fde0 2024-02-26T10:45:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f364c40f740 1 -- 172.21.15.60:0/3483916479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 msgr2=0x5605e7162270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f364c40f740 1 --2- 172.21.15.60:0/3483916479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7162270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f362c0114f0 tx=0x7f362c0119b0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f3643200640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36240a1e90 0x7f36240a4350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5605e715b4d0 tx=0x7f363c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f364c40f740 1 -- 172.21.15.60:0/3483916479 shutdown_connections 2024-02-26T10:45:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f364c40f740 1 --2- 172.21.15.60:0/3483916479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7162270 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f364c40f740 1 --2- 172.21.15.60:0/3483916479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e715f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f364c40f740 1 --2- 172.21.15.60:0/3483916479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715a280 0x5605e715a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f364c40f740 1 -- 172.21.15.60:0/3483916479 >> 172.21.15.60:0/3483916479 conn(0x5605e71557f0 msgr2=0x5605e7155bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f364c40f740 1 -- 172.21.15.60:0/3483916479 shutdown_connections 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.647+0000 7f364c40f740 1 -- 172.21.15.60:0/3483916479 wait complete. 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.648+0000 7f364c40f740 1 -- start start 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.648+0000 7f364c40f740 1 Processor -- start 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.648+0000 7f364c40f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.648+0000 7f364c40f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e7164b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.648+0000 7f364c40f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715fde0 0x5605e716bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.648+0000 7f364c40f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e707bf10 con 0x5605e715a280 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.648+0000 7f364c40f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e70bba90 con 0x5605e715fde0 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.648+0000 7f364c40f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e7162de0 con 0x5605e715ac70 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f364c40f740 10 client.0 _ll_register_callbacks cb 0x5605e70baa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:50.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3642800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3642800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35290/0 (socket says 172.21.15.60:35290) 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3642800640 1 -- 172.21.15.60:0/2847871394 learned_addr learned my addr 172.21.15.60:0/2847871394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3643200640 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715fde0 0x5605e716bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3639e00640 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e7164b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3642800640 1 -- 172.21.15.60:0/2847871394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 msgr2=0x5605e7164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3642800640 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e7164b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3642800640 1 -- 172.21.15.60:0/2847871394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715fde0 msgr2=0x5605e716bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3642800640 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715fde0 0x5605e716bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3642800640 1 -- 172.21.15.60:0/2847871394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605e6e9c7e0 con 0x5605e715a280 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3639e00640 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e7164b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.649+0000 7f3642800640 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71645d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3634005ab0 tx=0x7f3634005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf4297a4fce0c43 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.650+0000 7f362be00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:45:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.650+0000 7f362be00640 1 -- 172.21.15.60:0/2847871394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340b8da0 con 0x5605e715a280 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.650+0000 7f362be00640 1 -- 172.21.15.60:0/2847871394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f3634004250 con 0x5605e715a280 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.650+0000 7f364c40f740 1 -- 172.21.15.60:0/2847871394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f362c00faa0 con 0x5605e715a280 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.650+0000 7f362be00640 1 -- 172.21.15.60:0/2847871394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36340bc460 con 0x5605e715a280 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.650+0000 7f364c40f740 10 client.15423 fetch_fsmap learned FSMap version 24 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.650+0000 7f364c40f740 1 -- 172.21.15.60:0/2847871394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5605e6efcf70 con 0x5605e715a280 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.650+0000 7f362be00640 1 -- 172.21.15.60:0/2847871394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f362c00faa0 con 0x5605e715a280 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.651+0000 7f362be00640 1 -- 172.21.15.60:0/2847871394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f36340c4e10 con 0x5605e715a280 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.651+0000 7f364c40f740 10 client.15423 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.651+0000 7f364c40f740 10 client.15423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.651+0000 7f364c40f740 20 client.15423 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.651+0000 7f364c40f740 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5605e71703d0 0x5605e7172840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.651+0000 7f364c40f740 4 client.15423 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.651+0000 7f364c40f740 1 -- 172.21.15.60:0/2847871394 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605e70ccf30 con 0x5605e71703d0 2024-02-26T10:45:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.651+0000 7f3639e00640 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5605e71703d0 0x5605e7172840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.652+0000 7f3639e00640 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5605e71703d0 0x5605e7172840 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c835158a89617763 server_cookie=321d91f59e3d882b in_seq=0 out_seq=0 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.652+0000 7f362be00640 10 client.15423 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f362be00640 1 -- 172.21.15.60:0/2847871394 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605e70ccf30 con 0x5605e71703d0 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f362be00640 10 client.15423 handle_command_reply: tid=0 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 client.15423 shutdown 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 -- 172.21.15.60:0/2847871394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 msgr2=0x5605e71645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71645d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3634005ab0 tx=0x7f3634005e50 comp rx=0 tx=0).stop 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 -- 172.21.15.60:0/2847871394 shutdown_connections 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5605e71703d0 0x5605e7172840 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715fde0 0x5605e716bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715ac70 0x5605e7164b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 --2- 172.21.15.60:0/2847871394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71645d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f362be00640 0 client.15423 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 -- 172.21.15.60:0/2847871394 >> 172.21.15.60:0/2847871394 conn(0x5605e71557f0 msgr2=0x5605e7154a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 -- 172.21.15.60:0/2847871394 shutdown_connections 2024-02-26T10:45:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 1 -- 172.21.15.60:0/2847871394 wait complete. 2024-02-26T10:45:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.653+0000 7f364c40f740 20 client.15423 trim_cache size 0 max 16384 2024-02-26T10:45:50.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.670+0000 7f364c40f740 1 Processor -- start 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.670+0000 7f364c40f740 1 -- start start 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.670+0000 7f364c40f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f364c40f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f364c40f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715fde0 0x5605e7164720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f364c40f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441a9e70 con 0x5605e715a280 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f364c40f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441a7a10 con 0x5605e715ac70 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f364c40f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441a5ab0 con 0x5605e715fde0 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3642800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3642800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35296/0 (socket says 172.21.15.60:35296) 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3639e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3642800640 1 -- 172.21.15.60:0/4175032837 learned_addr learned my addr 172.21.15.60:0/4175032837 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3643200640 1 --2- 172.21.15.60:0/4175032837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715fde0 0x5605e7164720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3639e00640 1 -- 172.21.15.60:0/4175032837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715fde0 msgr2=0x5605e7164720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3639e00640 1 --2- 172.21.15.60:0/4175032837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715fde0 0x5605e7164720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3639e00640 1 -- 172.21.15.60:0/4175032837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 msgr2=0x5605e71641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3639e00640 1 --2- 172.21.15.60:0/4175032837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.671+0000 7f3639e00640 1 -- 172.21.15.60:0/4175032837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e6d94170 con 0x5605e715ac70 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.672+0000 7f3639e00640 1 --2- 172.21.15.60:0/4175032837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716bb10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f362c01c040 tx=0x7f362c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d0860826dcf6f20 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.672+0000 7f3632a00640 1 -- 172.21.15.60:0/4175032837 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c01e040 con 0x5605e715ac70 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.672+0000 7f3632a00640 1 -- 172.21.15.60:0/4175032837 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f362c0050b0 con 0x5605e715ac70 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.672+0000 7f3632a00640 1 -- 172.21.15.60:0/4175032837 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0294f0 con 0x5605e715ac70 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 -- 172.21.15.60:0/4175032837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 msgr2=0x5605e716bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 --2- 172.21.15.60:0/4175032837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716bb10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f362c01c040 tx=0x7f362c009770 comp rx=0 tx=0).stop 2024-02-26T10:45:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 -- 172.21.15.60:0/4175032837 shutdown_connections 2024-02-26T10:45:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 --2- 172.21.15.60:0/4175032837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716bb10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 --2- 172.21.15.60:0/4175032837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715fde0 0x5605e7164720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 --2- 172.21.15.60:0/4175032837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715a280 0x5605e71641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 -- 172.21.15.60:0/4175032837 >> 172.21.15.60:0/4175032837 conn(0x5605e71557f0 msgr2=0x5605e715c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 -- 172.21.15.60:0/4175032837 shutdown_connections 2024-02-26T10:45:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 -- 172.21.15.60:0/4175032837 wait complete. 2024-02-26T10:45:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.673+0000 7f364c40f740 1 -- start start 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f364c40f740 1 Processor -- start 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f364c40f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715a280 0x5605e716b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f364c40f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f364c40f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7169c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f364c40f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441a60b0 con 0x5605e715fde0 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f364c40f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441a73b0 con 0x5605e715ac70 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f364c40f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441a66f0 con 0x5605e715a280 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f364c40f740 10 client.0 _ll_register_callbacks cb 0x7f362c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f3639e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f3639e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56484/0 (socket says 172.21.15.60:56484) 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f3643200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f3639e00640 1 -- 172.21.15.60:0/1427153420 learned_addr learned my addr 172.21.15.60:0/1427153420 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.674+0000 7f3642800640 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715a280 0x5605e716b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f3639e00640 1 -- 172.21.15.60:0/1427153420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715a280 msgr2=0x5605e716b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f3639e00640 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715a280 0x5605e716b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f3639e00640 1 -- 172.21.15.60:0/1427153420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 msgr2=0x5605e7169c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f3639e00640 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7169c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f3639e00640 1 -- 172.21.15.60:0/1427153420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605e716a480 con 0x5605e715ac70 2024-02-26T10:45:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f3642800640 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715a280 0x5605e716b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:50.672 INFO:tasks.workunit.client.0.smithi060.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f3639e00640 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716ba90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f362c009250 tx=0x7f362c009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d7b8eb15fe79a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f362be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f362be00640 1 -- 172.21.15.60:0/1427153420 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c01e040 con 0x5605e715ac70 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f364c40f740 1 -- 172.21.15.60:0/1427153420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f362c0050b0 con 0x5605e715ac70 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.675+0000 7f362be00640 1 -- 172.21.15.60:0/1427153420 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f362c00f920 con 0x5605e715ac70 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f362be00640 1 -- 172.21.15.60:0/1427153420 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f362c005260 con 0x5605e715ac70 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f364c40f740 10 client.25544 fetch_fsmap learned FSMap version 24 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f364c40f740 1 -- 172.21.15.60:0/1427153420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5605e71c74f0 con 0x5605e715ac70 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f362be00640 1 -- 172.21.15.60:0/1427153420 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f362c0050b0 con 0x5605e715ac70 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f362be00640 1 -- 172.21.15.60:0/1427153420 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f362c003850 con 0x5605e715ac70 2024-02-26T10:45:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f364c40f740 10 client.25544 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f364c40f740 10 client.25544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f364c40f740 20 client.25544 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f364c40f740 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5605e71c7d30 0x5605e71ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f364c40f740 4 client.25544 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.676+0000 7f364c40f740 1 -- 172.21.15.60:0/1427153420 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5605e70c7280 con 0x5605e71c7d30 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.677+0000 7f3642800640 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5605e71c7d30 0x5605e71ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.677+0000 7f3642800640 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5605e71c7d30 0x5605e71ca150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90463d6025b83d48 server_cookie=4635ef5ce514d4da in_seq=0 out_seq=0 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.677+0000 7f362be00640 10 client.25544 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.678+0000 7f362be00640 1 -- 172.21.15.60:0/1427153420 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5605e70c7280 con 0x5605e71c7d30 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.678+0000 7f362be00640 10 client.25544 handle_command_reply: tid=0 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.678+0000 7f364c40f740 1 client.25544 shutdown 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 -- 172.21.15.60:0/1427153420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 msgr2=0x5605e716ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716ba90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f362c009250 tx=0x7f362c009c80 comp rx=0 tx=0).stop 2024-02-26T10:45:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 -- 172.21.15.60:0/1427153420 shutdown_connections 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5605e71c7d30 0x5605e71ca150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605e715ac70 0x5605e716ba90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605e715fde0 0x5605e7169c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 --2- 172.21.15.60:0/1427153420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5605e715a280 0x5605e716b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f362be00640 0 client.25544 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 -- 172.21.15.60:0/1427153420 >> 172.21.15.60:0/1427153420 conn(0x5605e71557f0 msgr2=0x5605e71641e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 -- 172.21.15.60:0/1427153420 shutdown_connections 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 1 -- 172.21.15.60:0/1427153420 wait complete. 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.679+0000 7f364c40f740 20 client.25544 trim_cache size 0 max 16384 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:45:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-02-26T10:45:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "48ba58c7-fc0b-43e9-b9a6-f0e351e3647a", 2024-02-26T10:45:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-02-26T10:45:50.678 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:45:50.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 -- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36240a1e90 msgr2=0x7f36240a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36240a1e90 0x7f36240a4350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5605e715b4d0 tx=0x7f363c009500 comp rx=0 tx=0).stop 2024-02-26T10:45:50.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 -- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3644073c40 msgr2=0x7f3644199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3644073c40 0x7f3644199640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3634000c00 tx=0x7f363402c6e0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 -- 172.21.15.60:0/3536302364 shutdown_connections 2024-02-26T10:45:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36240a1e90 0x7f36240a4350 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3644109010 0x7f36441a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3644075220 0x7f3644199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 --2- 172.21.15.60:0/3536302364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3644073c40 0x7f3644199640 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 -- 172.21.15.60:0/3536302364 >> 172.21.15.60:0/3536302364 conn(0x7f36440fc6d0 msgr2=0x7f3644104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 -- 172.21.15.60:0/3536302364 shutdown_connections 2024-02-26T10:45:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.680+0000 7f3648a00640 1 -- 172.21.15.60:0/3536302364 wait complete. 2024-02-26T10:45:50.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T10:45:50.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.999+0000 7f4570200640 1 -- 172.21.15.60:0/936946896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 msgr2=0x7f4568104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.999+0000 7f4570200640 1 --2- 172.21.15.60:0/936946896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 0x7f4568104af0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4554009900 tx=0x7f455402bc30 comp rx=0 tx=0).stop 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.999+0000 7f4570200640 1 -- 172.21.15.60:0/936946896 shutdown_connections 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.999+0000 7f4570200640 1 --2- 172.21.15.60:0/936946896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 0x7f4568104af0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.999+0000 7f4570200640 1 --2- 172.21.15.60:0/936946896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4568105030 0x7f456810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.999+0000 7f4570200640 1 --2- 172.21.15.60:0/936946896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4568103480 0x7f45681038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.999+0000 7f4570200640 1 -- 172.21.15.60:0/936946896 >> 172.21.15.60:0/936946896 conn(0x7f45680fea10 msgr2=0x7f4568100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.999+0000 7f4570200640 1 -- 172.21.15.60:0/936946896 shutdown_connections 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:50.999+0000 7f4570200640 1 -- 172.21.15.60:0/936946896 wait complete. 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.000+0000 7f4570200640 1 Processor -- start 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.000+0000 7f4570200640 1 -- start start 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.000+0000 7f4570200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4568103480 0x7f4568199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f4570200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 0x7f4568199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f4570200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4568105030 0x7f45681a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f4570200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456810f160 con 0x7f4568105030 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f4570200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456810efe0 con 0x7f4568104670 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f4570200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456810f2e0 con 0x7f4568103480 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 0x7f4568199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4568103480 0x7f4568199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4568105030 0x7f45681a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 0x7f4568199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56492/0 (socket says 172.21.15.60:56492) 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456ce00640 1 -- 172.21.15.60:0/1547274091 learned_addr learned my addr 172.21.15.60:0/1547274091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456ce00640 1 -- 172.21.15.60:0/1547274091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4568103480 msgr2=0x7f4568199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456ce00640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4568103480 0x7f4568199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456ce00640 1 -- 172.21.15.60:0/1547274091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4568105030 msgr2=0x7f45681a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456ce00640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4568105030 0x7f45681a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456ce00640 1 -- 172.21.15.60:0/1547274091 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45681a13a0 con 0x7f4568104670 2024-02-26T10:45:50.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.001+0000 7f456d800640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4568103480 0x7f4568199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:51.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.002+0000 7f456ce00640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 0x7f4568199cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4554000c00 tx=0x7f455402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:51.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.002+0000 7f4566000640 1 -- 172.21.15.60:0/1547274091 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554043020 con 0x7f4568104670 2024-02-26T10:45:51.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.002+0000 7f4566000640 1 -- 172.21.15.60:0/1547274091 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45540054e0 con 0x7f4568104670 2024-02-26T10:45:51.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.002+0000 7f4570200640 1 -- 172.21.15.60:0/1547274091 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45681a1630 con 0x7f4568104670 2024-02-26T10:45:51.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.002+0000 7f4566000640 1 -- 172.21.15.60:0/1547274091 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554003c40 con 0x7f4568104670 2024-02-26T10:45:51.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.002+0000 7f4570200640 1 -- 172.21.15.60:0/1547274091 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45681a1b80 con 0x7f4568104670 2024-02-26T10:45:51.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.005+0000 7f4566000640 1 -- 172.21.15.60:0/1547274091 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f4554046020 con 0x7f4568104670 2024-02-26T10:45:51.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.006+0000 7f4566000640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45500a1e10 0x7f45500a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.006+0000 7f4566000640 1 -- 172.21.15.60:0/1547274091 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f45540f0270 con 0x7f4568104670 2024-02-26T10:45:51.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.007+0000 7f456d800640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45500a1e10 0x7f45500a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.007+0000 7f4573198740 1 Processor -- start 2024-02-26T10:45:51.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.007+0000 7f4573198740 1 -- start start 2024-02-26T10:45:51.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.007+0000 7f456d800640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45500a1e10 0x7f45500a42d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f45681044d0 tx=0x7f455c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:51.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f4573198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f4573198740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f4573198740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 0x55ab89d1fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f4573198740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab89c45060 con 0x55ab89d18510 2024-02-26T10:45:51.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f4573198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab89c7cd00 con 0x55ab89d17b20 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f4573198740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab89c25a00 con 0x55ab89d1d680 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d1d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35326/0 (socket says 172.21.15.60:35326) 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456ce00640 1 -- 172.21.15.60:0/3087018235 learned_addr learned my addr 172.21.15.60:0/3087018235 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456d800640 1 --2- 172.21.15.60:0/3087018235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456e200640 1 --2- 172.21.15.60:0/3087018235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 0x55ab89d1fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456ce00640 1 -- 172.21.15.60:0/3087018235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 msgr2=0x55ab89d1fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456ce00640 1 --2- 172.21.15.60:0/3087018235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 0x55ab89d1fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456ce00640 1 -- 172.21.15.60:0/3087018235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 msgr2=0x55ab89d17f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456ce00640 1 --2- 172.21.15.60:0/3087018235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d17f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.008+0000 7f456ce00640 1 -- 172.21.15.60:0/3087018235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab89a5b7d0 con 0x55ab89d18510 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.009+0000 7f456ce00640 1 --2- 172.21.15.60:0/3087018235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d1d010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f45540b9510 tx=0x7f45540bc7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73119b7ec5aaadfd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.009+0000 7f455aa00640 1 -- 172.21.15.60:0/3087018235 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45540d5070 con 0x55ab89d18510 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.009+0000 7f455aa00640 1 -- 172.21.15.60:0/3087018235 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4554005690 con 0x55ab89d18510 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.009+0000 7f455aa00640 1 -- 172.21.15.60:0/3087018235 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45540bae40 con 0x55ab89d18510 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.010+0000 7f4573198740 1 -- 172.21.15.60:0/3087018235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 msgr2=0x55ab89d1d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.010+0000 7f4573198740 1 --2- 172.21.15.60:0/3087018235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d1d010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f45540b9510 tx=0x7f45540bc7b0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.010+0000 7f4573198740 1 -- 172.21.15.60:0/3087018235 shutdown_connections 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.010+0000 7f4573198740 1 --2- 172.21.15.60:0/3087018235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 0x55ab89d1fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.010+0000 7f4573198740 1 --2- 172.21.15.60:0/3087018235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d1d010 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.010+0000 7f4573198740 1 --2- 172.21.15.60:0/3087018235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d17f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.010+0000 7f4573198740 1 -- 172.21.15.60:0/3087018235 >> 172.21.15.60:0/3087018235 conn(0x55ab89d13090 msgr2=0x55ab89d13490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.010+0000 7f4573198740 1 -- 172.21.15.60:0/3087018235 shutdown_connections 2024-02-26T10:45:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.010+0000 7f4573198740 1 -- 172.21.15.60:0/3087018235 wait complete. 2024-02-26T10:45:51.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.011+0000 7f4573198740 1 -- start start 2024-02-26T10:45:51.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.011+0000 7f4573198740 1 Processor -- start 2024-02-26T10:45:51.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.011+0000 7f4573198740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d17b20 0x55ab89d21be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.011+0000 7f4573198740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f4573198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d1d680 0x55ab89d29160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f4573198740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab89c1b090 con 0x55ab89d17b20 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f4573198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab89c78da0 con 0x55ab89d1d680 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f4573198740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab89d203f0 con 0x55ab89d18510 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f4573198740 10 client.0 _ll_register_callbacks cb 0x55ab89c38830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d17b20 0x55ab89d21be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d17b20 0x55ab89d21be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35330/0 (socket says 172.21.15.60:35330) 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456d800640 1 -- 172.21.15.60:0/1171421987 learned_addr learned my addr 172.21.15.60:0/1171421987 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456ce00640 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456e200640 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d1d680 0x55ab89d29160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456d800640 1 -- 172.21.15.60:0/1171421987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 msgr2=0x55ab89d22120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456d800640 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d22120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456d800640 1 -- 172.21.15.60:0/1171421987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d1d680 msgr2=0x55ab89d29160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:51.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456d800640 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d1d680 0x55ab89d29160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.012+0000 7f456d800640 1 -- 172.21.15.60:0/1171421987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab899913f0 con 0x55ab89d17b20 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.013+0000 7f456d800640 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d17b20 0x55ab89d21be0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f455c003760 tx=0x7f455c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2c4050c7b7d1e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.013+0000 7f454f400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.013+0000 7f454f400640 1 -- 172.21.15.60:0/1171421987 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c02b060 con 0x55ab89d17b20 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.013+0000 7f454f400640 1 -- 172.21.15.60:0/1171421987 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f455c00f690 con 0x55ab89d17b20 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.013+0000 7f4573198740 1 -- 172.21.15.60:0/1171421987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab89c8a240 con 0x55ab89d17b20 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.013+0000 7f454f400640 1 -- 172.21.15.60:0/1171421987 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f455c032050 con 0x55ab89d17b20 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.013+0000 7f4573198740 10 client.15441 fetch_fsmap learned FSMap version 24 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.013+0000 7f4573198740 1 -- 172.21.15.60:0/1171421987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ab899510f0 con 0x55ab89d17b20 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.014+0000 7f454f400640 1 -- 172.21.15.60:0/1171421987 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ab89c8a240 con 0x55ab89d17b20 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.014+0000 7f454f400640 1 -- 172.21.15.60:0/1171421987 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f455c01d940 con 0x55ab89d17b20 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.014+0000 7f4573198740 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.014+0000 7f4573198740 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.014+0000 7f4573198740 20 client.15441 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:51.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.014+0000 7f4573198740 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ab89d2daa0 0x55ab89d2ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.014+0000 7f4573198740 4 client.15441 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:45:51.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.014+0000 7f4573198740 1 -- 172.21.15.60:0/1171421987 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab89c84590 con 0x55ab89d2daa0 2024-02-26T10:45:51.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.014+0000 7f456ce00640 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ab89d2daa0 0x55ab89d2ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.015+0000 7f456ce00640 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ab89d2daa0 0x55ab89d2ff40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e512ecb8ea8bd2 server_cookie=822618891d9383a5 in_seq=0 out_seq=0 2024-02-26T10:45:51.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.015+0000 7f454f400640 10 client.15441 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:51.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.015+0000 7f454f400640 1 -- 172.21.15.60:0/1171421987 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab89c84590 con 0x55ab89d2daa0 2024-02-26T10:45:51.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.015+0000 7f454f400640 10 client.15441 handle_command_reply: tid=0 2024-02-26T10:45:51.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 client.15441 shutdown 2024-02-26T10:45:51.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 -- 172.21.15.60:0/1171421987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d17b20 msgr2=0x55ab89d21be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:51.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d17b20 0x55ab89d21be0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f455c003760 tx=0x7f455c004730 comp rx=0 tx=0).stop 2024-02-26T10:45:51.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 -- 172.21.15.60:0/1171421987 shutdown_connections 2024-02-26T10:45:51.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ab89d2daa0 0x55ab89d2ff40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d1d680 0x55ab89d29160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d22120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 --2- 172.21.15.60:0/1171421987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d17b20 0x55ab89d21be0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f454f400640 0 client.15441 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:51.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 -- 172.21.15.60:0/1171421987 >> 172.21.15.60:0/1171421987 conn(0x55ab89d13090 msgr2=0x55ab89d12330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:51.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 -- 172.21.15.60:0/1171421987 shutdown_connections 2024-02-26T10:45:51.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 1 -- 172.21.15.60:0/1171421987 wait complete. 2024-02-26T10:45:51.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.016+0000 7f4573198740 20 client.15441 trim_cache size 0 max 16384 2024-02-26T10:45:51.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.037+0000 7f4573198740 1 Processor -- start 2024-02-26T10:45:51.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.037+0000 7f4573198740 1 -- start start 2024-02-26T10:45:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f4573198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d11aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f4573198740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f4573198740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d1d680 0x55ab89d11fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f4573198740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681ae480 con 0x55ab89d1d680 2024-02-26T10:45:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f4573198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681ac020 con 0x55ab89d17b20 2024-02-26T10:45:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f4573198740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681aa0c0 con 0x55ab89d18510 2024-02-26T10:45:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f456e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d1d680 0x55ab89d11fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f456ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f456d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d11aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f456e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d1d680 0x55ab89d11fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35334/0 (socket says 172.21.15.60:35334) 2024-02-26T10:45:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f456ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d29120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49786/0 (socket says 172.21.15.60:49786) 2024-02-26T10:45:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f456e200640 1 -- 172.21.15.60:0/4289436236 learned_addr learned my addr 172.21.15.60:0/4289436236 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.038+0000 7f456d800640 1 --2- 172.21.15.60:0/4289436236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d11aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56514/0 (socket says 172.21.15.60:56514) 2024-02-26T10:45:51.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.039+0000 7f456ce00640 1 -- 172.21.15.60:0/4289436236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 msgr2=0x55ab89d11aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.039+0000 7f456ce00640 1 --2- 172.21.15.60:0/4289436236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d11aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.039+0000 7f456ce00640 1 -- 172.21.15.60:0/4289436236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d1d680 msgr2=0x55ab89d11fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.039+0000 7f456ce00640 1 --2- 172.21.15.60:0/4289436236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d1d680 0x55ab89d11fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.039+0000 7f456ce00640 1 -- 172.21.15.60:0/4289436236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab89d13e00 con 0x55ab89d18510 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.039+0000 7f456ce00640 1 --2- 172.21.15.60:0/4289436236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d29120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45540c0e50 tx=0x7f45540f0170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca2f7a2f3d42be77 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.039+0000 7f455a000640 1 -- 172.21.15.60:0/4289436236 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455409f070 con 0x55ab89d18510 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.039+0000 7f455a000640 1 -- 172.21.15.60:0/4289436236 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45540c0c90 con 0x55ab89d18510 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.039+0000 7f455a000640 1 -- 172.21.15.60:0/4289436236 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45540d5040 con 0x55ab89d18510 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.040+0000 7f4573198740 1 -- 172.21.15.60:0/4289436236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 msgr2=0x55ab89d29120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.040+0000 7f4573198740 1 --2- 172.21.15.60:0/4289436236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d29120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45540c0e50 tx=0x7f45540f0170 comp rx=0 tx=0).stop 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.040+0000 7f4573198740 1 -- 172.21.15.60:0/4289436236 shutdown_connections 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.040+0000 7f4573198740 1 --2- 172.21.15.60:0/4289436236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d1d680 0x55ab89d11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.040+0000 7f4573198740 1 --2- 172.21.15.60:0/4289436236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d18510 0x55ab89d29120 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.040+0000 7f4573198740 1 --2- 172.21.15.60:0/4289436236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d11aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.040+0000 7f4573198740 1 -- 172.21.15.60:0/4289436236 >> 172.21.15.60:0/4289436236 conn(0x55ab89d13090 msgr2=0x55ab89d13470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.040+0000 7f4573198740 1 -- 172.21.15.60:0/4289436236 shutdown_connections 2024-02-26T10:45:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.040+0000 7f4573198740 1 -- 172.21.15.60:0/4289436236 wait complete. 2024-02-26T10:45:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.041+0000 7f4573198740 1 -- start start 2024-02-26T10:45:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.041+0000 7f4573198740 1 Processor -- start 2024-02-26T10:45:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.041+0000 7f4573198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d28ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.041+0000 7f4573198740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d29000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.041+0000 7f4573198740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 0x55ab89d27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.041+0000 7f4573198740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681aa6c0 con 0x55ab89d18510 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.041+0000 7f4573198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681ab9c0 con 0x55ab89d17b20 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.041+0000 7f4573198740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681aad00 con 0x55ab89d1d680 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f4573198740 10 client.0 _ll_register_callbacks cb 0x7f455c004100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d29000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d28ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 0x55ab89d27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d29000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35342/0 (socket says 172.21.15.60:35342) 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d28ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56522/0 (socket says 172.21.15.60:56522) 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456ce00640 1 -- 172.21.15.60:0/2299024522 learned_addr learned my addr 172.21.15.60:0/2299024522 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456e200640 1 -- 172.21.15.60:0/2299024522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 msgr2=0x55ab89d28ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456e200640 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d28ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456e200640 1 -- 172.21.15.60:0/2299024522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 msgr2=0x55ab89d29000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456e200640 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d29000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456e200640 1 -- 172.21.15.60:0/2299024522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab89d27880 con 0x55ab89d1d680 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.042+0000 7f456ce00640 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d29000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.043+0000 7f456e200640 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 0x55ab89d27140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55ab89d17900 tx=0x7f4560004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a86bbde4515349da server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.043+0000 7f454f400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.043+0000 7f454f400640 1 -- 172.21.15.60:0/2299024522 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560022060 con 0x55ab89d1d680 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.043+0000 7f4573198740 1 -- 172.21.15.60:0/2299024522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab89c02370 con 0x55ab89d1d680 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.043+0000 7f454f400640 1 -- 172.21.15.60:0/2299024522 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f4560012b80 con 0x55ab89d1d680 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.043+0000 7f454f400640 1 -- 172.21.15.60:0/2299024522 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4560029050 con 0x55ab89d1d680 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.043+0000 7f4573198740 10 client.25567 fetch_fsmap learned FSMap version 24 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.043+0000 7f4573198740 1 -- 172.21.15.60:0/2299024522 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ab89d84bf0 con 0x55ab89d1d680 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.044+0000 7f454f400640 1 -- 172.21.15.60:0/2299024522 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ab89c02370 con 0x55ab89d1d680 2024-02-26T10:45:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.044+0000 7f454f400640 1 -- 172.21.15.60:0/2299024522 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f456001c3a0 con 0x55ab89d1d680 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.044+0000 7f4573198740 10 client.25567 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.044+0000 7f4573198740 10 client.25567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.044+0000 7f4573198740 20 client.25567 populate_metadata read hostname 'smithi060' 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.044+0000 7f4573198740 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ab89d85530 0x55ab89d87950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.044+0000 7f4573198740 4 client.25567 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.044+0000 7f4573198740 1 -- 172.21.15.60:0/2299024522 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab89c4ebb0 con 0x55ab89d85530 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.044+0000 7f456d800640 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ab89d85530 0x55ab89d87950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f456d800640 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ab89d85530 0x55ab89d87950 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa21b8b20f464c29 server_cookie=99009b8f89cc5f0 in_seq=0 out_seq=0 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f454f400640 10 client.25567 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f454f400640 1 -- 172.21.15.60:0/2299024522 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ab89c4ebb0 con 0x55ab89d85530 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f454f400640 10 client.25567 handle_command_reply: tid=0 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 client.25567 shutdown 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 -- 172.21.15.60:0/2299024522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 msgr2=0x55ab89d27140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 0x55ab89d27140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55ab89d17900 tx=0x7f4560004750 comp rx=0 tx=0).stop 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 -- 172.21.15.60:0/2299024522 shutdown_connections 2024-02-26T10:45:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ab89d85530 0x55ab89d87950 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab89d1d680 0x55ab89d27140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab89d18510 0x55ab89d29000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 --2- 172.21.15.60:0/2299024522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab89d17b20 0x55ab89d28ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f454f400640 0 client.25567 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:45:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 -- 172.21.15.60:0/2299024522 >> 172.21.15.60:0/2299024522 conn(0x55ab89d13090 msgr2=0x55ab89d162f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 -- 172.21.15.60:0/2299024522 shutdown_connections 2024-02-26T10:45:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.045+0000 7f4573198740 1 -- 172.21.15.60:0/2299024522 wait complete. 2024-02-26T10:45:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4573198740 20 client.25567 trim_cache size 0 max 16384 2024-02-26T10:45:51.044 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:45:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T10:45:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-02-26T10:45:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: "48ba58c7-fc0b-43e9-b9a6-f0e351e3647a": { 2024-02-26T10:45:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-02-26T10:45:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "48ba58c7-fc0b-43e9-b9a6-f0e351e3647a", 2024-02-26T10:45:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-02-26T10:45:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:45:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:45:51.045 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 -- 172.21.15.60:0/1547274091 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45500a1e10 msgr2=0x7f45500a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45500a1e10 0x7f45500a42d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f45681044d0 tx=0x7f455c005f90 comp rx=0 tx=0).stop 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 -- 172.21.15.60:0/1547274091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 msgr2=0x7f4568199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 0x7f4568199cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4554000c00 tx=0x7f455402c650 comp rx=0 tx=0).stop 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 -- 172.21.15.60:0/1547274091 shutdown_connections 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f45500a1e10 0x7f45500a42d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4568104670 0x7f4568199cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4568105030 0x7f45681a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 --2- 172.21.15.60:0/1547274091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4568103480 0x7f4568199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:45:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.046+0000 7f4570200640 1 -- 172.21.15.60:0/1547274091 >> 172.21.15.60:0/1547274091 conn(0x7f45680fea10 msgr2=0x7f4568107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:45:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.047+0000 7f4570200640 1 -- 172.21.15.60:0/1547274091 shutdown_connections 2024-02-26T10:45:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:45:51.047+0000 7f4570200640 1 -- 172.21.15.60:0/1547274091 wait complete. 2024-02-26T10:45:51.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:50 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1527650854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:51.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:50 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/273964459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:51.060 INFO:tasks.cephfs.filesystem:scrub status for tag:48ba58c7-fc0b-43e9-b9a6-f0e351e3647a - {'path': '/', 'tag': '48ba58c7-fc0b-43e9-b9a6-f0e351e3647a', 'options': 'recursive,force'} 2024-02-26T10:45:51.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:50 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1527650854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:51.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:50 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/273964459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:51.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:50 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1527650854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:45:51.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:50 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/273964459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:45:52.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:51 smithi113 ceph-mon[36126]: scrub queued for path: / 2024-02-26T10:45:52.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:51 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:45:52.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:51 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:45:52.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:51 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:45:52.333 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:51 smithi113 ceph-mon[36126]: pgmap v422: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:45:52.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:51 smithi060 ceph-mon[31539]: scrub queued for path: / 2024-02-26T10:45:52.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:51 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:45:52.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:51 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:45:52.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:51 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:45:52.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:51 smithi060 ceph-mon[31539]: pgmap v422: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:45:52.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:51 smithi100 ceph-mon[36051]: scrub queued for path: / 2024-02-26T10:45:52.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:51 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:45:52.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:51 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:45:52.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:51 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:45:52.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:51 smithi100 ceph-mon[36051]: pgmap v422: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:45:54.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:54 smithi113 ceph-mon[36126]: pgmap v423: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:45:54.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:54 smithi060 ceph-mon[31539]: pgmap v423: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:45:54.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:54 smithi100 ceph-mon[36051]: pgmap v423: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:45:56.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:56 smithi113 ceph-mon[36126]: pgmap v424: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T10:45:56.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:56 smithi060 ceph-mon[31539]: pgmap v424: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T10:45:56.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:56 smithi100 ceph-mon[36051]: pgmap v424: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T10:45:58.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:45:58 smithi060 ceph-mon[31539]: pgmap v425: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-02-26T10:45:58.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:45:58 smithi113 ceph-mon[36126]: pgmap v425: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-02-26T10:45:58.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:45:58 smithi100 ceph-mon[36051]: pgmap v425: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-02-26T10:46:00.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:00 smithi113 ceph-mon[36126]: pgmap v426: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 8 op/s 2024-02-26T10:46:00.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:00 smithi060 ceph-mon[31539]: pgmap v426: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 8 op/s 2024-02-26T10:46:00.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:00 smithi100 ceph-mon[36051]: pgmap v426: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 8 op/s 2024-02-26T10:46:02.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:02 smithi113 ceph-mon[36126]: pgmap v427: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-26T10:46:02.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:02 smithi113 ceph-mon[36126]: scrub summary: idle 2024-02-26T10:46:02.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:02 smithi060 ceph-mon[31539]: pgmap v427: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-26T10:46:02.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:02 smithi060 ceph-mon[31539]: scrub summary: idle 2024-02-26T10:46:02.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:02 smithi100 ceph-mon[36051]: pgmap v427: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-26T10:46:02.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:02 smithi100 ceph-mon[36051]: scrub summary: idle 2024-02-26T10:46:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.680+0000 7f15b2e00640 1 -- 172.21.15.60:0/2648188574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac1078c0 msgr2=0x7f15ac107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.680+0000 7f15b2e00640 1 --2- 172.21.15.60:0/2648188574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac1078c0 0x7f15ac107ce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f159c009900 tx=0x7f159c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:46:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.680+0000 7f15b2e00640 1 -- 172.21.15.60:0/2648188574 shutdown_connections 2024-02-26T10:46:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.680+0000 7f15b2e00640 1 --2- 172.21.15.60:0/2648188574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ac109470 0x7f15ac10fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.680+0000 7f15b2e00640 1 --2- 172.21.15.60:0/2648188574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac108ab0 0x7f15ac108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.680+0000 7f15b2e00640 1 --2- 172.21.15.60:0/2648188574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac1078c0 0x7f15ac107ce0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.680+0000 7f15b2e00640 1 -- 172.21.15.60:0/2648188574 >> 172.21.15.60:0/2648188574 conn(0x7f15ac076870 msgr2=0x7f15ac078c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.680+0000 7f15b2e00640 1 -- 172.21.15.60:0/2648188574 shutdown_connections 2024-02-26T10:46:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.680+0000 7f15b2e00640 1 -- 172.21.15.60:0/2648188574 wait complete. 2024-02-26T10:46:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.681+0000 7f15b2e00640 1 Processor -- start 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.681+0000 7f15b2e00640 1 -- start start 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15b2e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac1078c0 0x7f15ac19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15b2e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ac108ab0 0x7f15ac19e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15b2e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac109470 0x7f15ac1a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15b2e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac113560 con 0x7f15ac109470 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15b2e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1133e0 con 0x7f15ac108ab0 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15b2e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1136e0 con 0x7f15ac1078c0 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15b0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac109470 0x7f15ac1a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15abe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac1078c0 0x7f15ac19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15b0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac109470 0x7f15ac1a50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47504/0 (socket says 172.21.15.60:47504) 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ac108ab0 0x7f15ac19e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.682+0000 7f15b0e00640 1 -- 172.21.15.60:0/3053180687 learned_addr learned my addr 172.21.15.60:0/3053180687 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15b0e00640 1 -- 172.21.15.60:0/3053180687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac1078c0 msgr2=0x7f15ac19db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15b0e00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac1078c0 0x7f15ac19db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15b0e00640 1 -- 172.21.15.60:0/3053180687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ac108ab0 msgr2=0x7f15ac19e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15b0e00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ac108ab0 0x7f15ac19e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15b0e00640 1 -- 172.21.15.60:0/3053180687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac1a5730 con 0x7f15ac109470 2024-02-26T10:46:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15b0e00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac109470 0x7f15ac1a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f15a400efd0 tx=0x7f15a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15a8c00640 1 -- 172.21.15.60:0/3053180687 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4003b70 con 0x7f15ac109470 2024-02-26T10:46:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15a8c00640 1 -- 172.21.15.60:0/3053180687 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15a4003d10 con 0x7f15ac109470 2024-02-26T10:46:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15a8c00640 1 -- 172.21.15.60:0/3053180687 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40109f0 con 0x7f15ac109470 2024-02-26T10:46:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15b2e00640 1 -- 172.21.15.60:0/3053180687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac1a5a20 con 0x7f15ac109470 2024-02-26T10:46:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.683+0000 7f15b2e00640 1 -- 172.21.15.60:0/3053180687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac1a5f20 con 0x7f15ac109470 2024-02-26T10:46:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.688+0000 7f15b5d73740 1 Processor -- start 2024-02-26T10:46:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.688+0000 7f15a8c00640 1 -- 172.21.15.60:0/3053180687 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f15a4004020 con 0x7f15ac109470 2024-02-26T10:46:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.688+0000 7f15b5d73740 1 -- start start 2024-02-26T10:46:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 0x556b94103c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94108d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b9410b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15b5d73740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b94069340 con 0x556b94109380 2024-02-26T10:46:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15b5d73740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b94012520 con 0x556b94104210 2024-02-26T10:46:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15b5d73740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b940653e0 con 0x556b94103820 2024-02-26T10:46:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94108d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15ab400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 0x556b94103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94108d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53438/0 (socket says 172.21.15.60:53438) 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15abe00640 1 -- 172.21.15.60:0/1314695830 learned_addr learned my addr 172.21.15.60:0/1314695830 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15ab400640 1 --2- 172.21.15.60:0/1314695830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b9410b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15abe00640 1 -- 172.21.15.60:0/1314695830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 msgr2=0x556b94103c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15abe00640 1 --2- 172.21.15.60:0/1314695830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 0x556b94103c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15abe00640 1 -- 172.21.15.60:0/1314695830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 msgr2=0x556b9410b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15abe00640 1 --2- 172.21.15.60:0/1314695830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b9410b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15abe00640 1 -- 172.21.15.60:0/1314695830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b94026390 con 0x556b94104210 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.689+0000 7f15ab400640 1 --2- 172.21.15.60:0/1314695830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b9410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.690+0000 7f15abe00640 1 --2- 172.21.15.60:0/1314695830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94108d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f159c000c00 tx=0x7f159c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=792b74f8c36cff83 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.690+0000 7f15a8c00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f15900a1ec0 0x7f15900a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.690+0000 7f15a1600640 1 -- 172.21.15.60:0/1314695830 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c02c980 con 0x556b94104210 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.690+0000 7f15a1600640 1 -- 172.21.15.60:0/1314695830 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f159c02cb20 con 0x556b94104210 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.690+0000 7f15a8c00640 1 -- 172.21.15.60:0/3053180687 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f15a40ce780 con 0x7f15ac109470 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.690+0000 7f15a1600640 1 -- 172.21.15.60:0/1314695830 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c0046c0 con 0x556b94104210 2024-02-26T10:46:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.690+0000 7f15ab400640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f15900a1ec0 0x7f15900a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15b5d73740 1 -- 172.21.15.60:0/1314695830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 msgr2=0x556b94108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15ab400640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f15900a1ec0 0x7f15900a4380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x556b94104a70 tx=0x7f1594013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1314695830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94108d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f159c000c00 tx=0x7f159c009560 comp rx=0 tx=0).stop 2024-02-26T10:46:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15b5d73740 1 -- 172.21.15.60:0/1314695830 shutdown_connections 2024-02-26T10:46:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1314695830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94108d10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1314695830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b9410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1314695830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 0x556b94103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15b5d73740 1 -- 172.21.15.60:0/1314695830 >> 172.21.15.60:0/1314695830 conn(0x556b940fed90 msgr2=0x556b940ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15b5d73740 1 -- 172.21.15.60:0/1314695830 shutdown_connections 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.691+0000 7f15b5d73740 1 -- 172.21.15.60:0/1314695830 wait complete. 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.692+0000 7f15b5d73740 1 -- start start 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.692+0000 7f15b5d73740 1 Processor -- start 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.692+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b9410db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.692+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 0x556b9410e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.692+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 0x556b94114ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.692+0000 7f15b5d73740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9410c2d0 con 0x556b94103820 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.692+0000 7f15b5d73740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b94025980 con 0x556b94109380 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.692+0000 7f15b5d73740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9410c450 con 0x556b94104210 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.692+0000 7f15b5d73740 10 client.0 _ll_register_callbacks cb 0x556b94061310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b9410db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b9410db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47510/0 (socket says 172.21.15.60:47510) 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15abe00640 1 -- 172.21.15.60:0/1416401636 learned_addr learned my addr 172.21.15.60:0/1416401636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15ab400640 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 0x556b9410e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15b0e00640 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 0x556b94114ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15abe00640 1 -- 172.21.15.60:0/1416401636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 msgr2=0x556b9410e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15abe00640 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 0x556b9410e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15abe00640 1 -- 172.21.15.60:0/1416401636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 msgr2=0x556b94114ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15abe00640 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 0x556b94114ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15abe00640 1 -- 172.21.15.60:0/1416401636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b93d7e210 con 0x556b94103820 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.693+0000 7f15abe00640 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b9410db50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f159c009900 tx=0x7f159c03f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34766cd5c3edb0f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.694+0000 7f159a000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.694+0000 7f159a000640 1 -- 172.21.15.60:0/1416401636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c02c980 con 0x556b94103820 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.694+0000 7f159a000640 1 -- 172.21.15.60:0/1416401636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f159c03fdd0 con 0x556b94103820 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.694+0000 7f15b5d73740 1 -- 172.21.15.60:0/1416401636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f159c02cb20 con 0x556b94103820 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.694+0000 7f159a000640 1 -- 172.21.15.60:0/1416401636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f159c05a050 con 0x556b94103820 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.694+0000 7f15b5d73740 10 client.15459 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.694+0000 7f15b5d73740 1 -- 172.21.15.60:0/1416401636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556b93e4a620 con 0x556b94103820 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.694+0000 7f159a000640 1 -- 172.21.15.60:0/1416401636 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f159c02cb20 con 0x556b94103820 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.695+0000 7f159a000640 1 -- 172.21.15.60:0/1416401636 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f159c049590 con 0x556b94103820 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.695+0000 7f15b5d73740 10 client.15459 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.695+0000 7f15b5d73740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.695+0000 7f15b5d73740 20 client.15459 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.695+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556b941194e0 0x556b9411b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.695+0000 7f15b5d73740 4 client.15459 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.695+0000 7f15b5d73740 1 -- 172.21.15.60:0/1416401636 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b94076880 con 0x556b941194e0 2024-02-26T10:46:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.695+0000 7f15ab400640 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556b941194e0 0x556b9411b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.696+0000 7f15ab400640 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556b941194e0 0x556b9411b8d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514a4f300601cddb server_cookie=ac75205b496ca964 in_seq=0 out_seq=0 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.696+0000 7f159a000640 10 client.15459 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.696+0000 7f159a000640 1 -- 172.21.15.60:0/1416401636 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b94076880 con 0x556b941194e0 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.696+0000 7f159a000640 10 client.15459 handle_command_reply: tid=0 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.696+0000 7f15b5d73740 1 client.15459 shutdown 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 -- 172.21.15.60:0/1416401636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 msgr2=0x556b9410db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b9410db50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f159c009900 tx=0x7f159c03f9e0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 -- 172.21.15.60:0/1416401636 shutdown_connections 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556b941194e0 0x556b9411b8d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 0x556b94114ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 0x556b9410e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 --2- 172.21.15.60:0/1416401636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b9410db50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f159a000640 0 client.15459 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 -- 172.21.15.60:0/1416401636 >> 172.21.15.60:0/1416401636 conn(0x556b940fed90 msgr2=0x556b940fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 -- 172.21.15.60:0/1416401636 shutdown_connections 2024-02-26T10:46:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 1 -- 172.21.15.60:0/1416401636 wait complete. 2024-02-26T10:46:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.697+0000 7f15b5d73740 20 client.15459 trim_cache size 0 max 16384 2024-02-26T10:46:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15b5d73740 1 Processor -- start 2024-02-26T10:46:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15b5d73740 1 -- start start 2024-02-26T10:46:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b940fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 0x556b94114f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 0x556b940fdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15b5d73740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1b2710 con 0x556b94103820 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15b5d73740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1b02b0 con 0x556b94109380 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15b5d73740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1ae350 con 0x556b94104210 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b940fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b940fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47522/0 (socket says 172.21.15.60:47522) 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15abe00640 1 -- 172.21.15.60:0/2182147333 learned_addr learned my addr 172.21.15.60:0/2182147333 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15ab400640 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 0x556b94114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.712+0000 7f15b0e00640 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 0x556b940fdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15ab400640 1 -- 172.21.15.60:0/2182147333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 msgr2=0x556b940fdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15ab400640 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 0x556b940fdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15ab400640 1 -- 172.21.15.60:0/2182147333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 msgr2=0x556b940fd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15ab400640 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b940fd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15ab400640 1 -- 172.21.15.60:0/2182147333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b940fa5e0 con 0x556b94104210 2024-02-26T10:46:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15abe00640 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b940fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15ab400640 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 0x556b94114f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x556b9411c0c0 tx=0x7f15940096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f6cb81b37a3ac56 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15a0c00640 1 -- 172.21.15.60:0/2182147333 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15940041c0 con 0x556b94104210 2024-02-26T10:46:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15a0c00640 1 -- 172.21.15.60:0/2182147333 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f159402c890 con 0x556b94104210 2024-02-26T10:46:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.713+0000 7f15a0c00640 1 -- 172.21.15.60:0/2182147333 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159400fd20 con 0x556b94104210 2024-02-26T10:46:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 -- 172.21.15.60:0/2182147333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 msgr2=0x556b94114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 0x556b94114f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x556b9411c0c0 tx=0x7f15940096c0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 -- 172.21.15.60:0/2182147333 shutdown_connections 2024-02-26T10:46:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94109380 0x556b940fdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94104210 0x556b94114f10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2182147333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94103820 0x556b940fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 -- 172.21.15.60:0/2182147333 >> 172.21.15.60:0/2182147333 conn(0x556b940fed90 msgr2=0x556b94105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 -- 172.21.15.60:0/2182147333 shutdown_connections 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 -- 172.21.15.60:0/2182147333 wait complete. 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.714+0000 7f15b5d73740 1 -- start start 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b5d73740 1 Processor -- start 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 0x556b94114b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94115070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b5d73740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b94113390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b5d73740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1ae950 con 0x556b94109380 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b5d73740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1afc50 con 0x556b94104210 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b5d73740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1aef90 con 0x556b94103820 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b5d73740 10 client.0 _ll_register_callbacks cb 0x7f1594005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b94113390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b94113390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47524/0 (socket says 172.21.15.60:47524) 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.715+0000 7f15b0e00640 1 -- 172.21.15.60:0/2779740698 learned_addr learned my addr 172.21.15.60:0/2779740698 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15ab400640 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15abe00640 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 0x556b94114b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15b0e00640 1 -- 172.21.15.60:0/2779740698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 msgr2=0x556b94114b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15b0e00640 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 0x556b94114b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15b0e00640 1 -- 172.21.15.60:0/2779740698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 msgr2=0x556b94115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15b0e00640 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94115070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15b0e00640 1 -- 172.21.15.60:0/2779740698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b94113bc0 con 0x556b94109380 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15ab400640 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15b0e00640 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b94113390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f15a40cbcc0 tx=0x7f15a4009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bce5c76634ab2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f159a000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f159a000640 1 -- 172.21.15.60:0/2779740698 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a409f090 con 0x556b94109380 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f159a000640 1 -- 172.21.15.60:0/2779740698 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f15a4004310 con 0x556b94109380 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.716+0000 7f15b5d73740 1 -- 172.21.15.60:0/2779740698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f159402c890 con 0x556b94109380 2024-02-26T10:46:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f159a000640 1 -- 172.21.15.60:0/2779740698 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15a4081050 con 0x556b94109380 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f15b5d73740 10 client.15465 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f15b5d73740 1 -- 172.21.15.60:0/2779740698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556b94170530 con 0x556b94109380 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f159a000640 1 -- 172.21.15.60:0/2779740698 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f159402c890 con 0x556b94109380 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f159a000640 1 -- 172.21.15.60:0/2779740698 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f15a401f020 con 0x556b94109380 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f15b5d73740 10 client.15465 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f15b5d73740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f15b5d73740 20 client.15465 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556b94170eb0 0x556b941732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f15b5d73740 4 client.15465 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.717+0000 7f15b5d73740 1 -- 172.21.15.60:0/2779740698 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b94070bd0 con 0x556b94170eb0 2024-02-26T10:46:03.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.718+0000 7f15abe00640 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556b94170eb0 0x556b941732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.718+0000 7f15abe00640 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556b94170eb0 0x556b941732d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b232905f660c65 server_cookie=17ae8e852b63dd09 in_seq=0 out_seq=0 2024-02-26T10:46:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.718+0000 7f159a000640 10 client.15465 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f159a000640 1 -- 172.21.15.60:0/2779740698 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b94070bd0 con 0x556b94170eb0 2024-02-26T10:46:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f159a000640 10 client.15465 handle_command_reply: tid=0 2024-02-26T10:46:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 client.15465 shutdown 2024-02-26T10:46:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 -- 172.21.15.60:0/2779740698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 msgr2=0x556b94113390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b94113390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f15a40cbcc0 tx=0x7f15a4009820 comp rx=0 tx=0).stop 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 -- 172.21.15.60:0/2779740698 shutdown_connections 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556b94170eb0 0x556b941732d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b94104210 0x556b94115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b94109380 0x556b94113390 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f159a000640 0 client.15465 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 --2- 172.21.15.60:0/2779740698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556b94103820 0x556b94114b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 -- 172.21.15.60:0/2779740698 >> 172.21.15.60:0/2779740698 conn(0x556b940fed90 msgr2=0x556b94101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 -- 172.21.15.60:0/2779740698 shutdown_connections 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.732+0000 7f15b5d73740 1 -- 172.21.15.60:0/2779740698 wait complete. 2024-02-26T10:46:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b5d73740 20 client.15465 trim_cache size 0 max 16384 2024-02-26T10:46:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:46:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:46:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:46:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:46:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 -- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f15900a1ec0 msgr2=0x7f15900a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f15900a1ec0 0x7f15900a4380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x556b94104a70 tx=0x7f1594013040 comp rx=0 tx=0).stop 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 -- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac109470 msgr2=0x7f15ac1a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac109470 0x7f15ac1a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f15a400efd0 tx=0x7f15a400c6a0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 -- 172.21.15.60:0/3053180687 shutdown_connections 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f15900a1ec0 0x7f15900a4380 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ac108ab0 0x7f15ac19e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ac109470 0x7f15ac1a50c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 --2- 172.21.15.60:0/3053180687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac1078c0 0x7f15ac19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.733+0000 7f15b2e00640 1 -- 172.21.15.60:0/3053180687 >> 172.21.15.60:0/3053180687 conn(0x7f15ac076870 msgr2=0x7f15ac10bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.734+0000 7f15b2e00640 1 -- 172.21.15.60:0/3053180687 shutdown_connections 2024-02-26T10:46:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:03.734+0000 7f15b2e00640 1 -- 172.21.15.60:0/3053180687 wait complete. 2024-02-26T10:46:04.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:04 smithi113 ceph-mon[36126]: pgmap v428: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T10:46:04.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:04 smithi060 ceph-mon[31539]: pgmap v428: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T10:46:04.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:04 smithi100 ceph-mon[36051]: pgmap v428: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T10:46:06.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:06 smithi113 ceph-mon[36126]: pgmap v429: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 8 op/s 2024-02-26T10:46:06.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:06 smithi060 ceph-mon[31539]: pgmap v429: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 8 op/s 2024-02-26T10:46:06.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:06 smithi100 ceph-mon[36051]: pgmap v429: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 8 op/s 2024-02-26T10:46:08.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:08 smithi113 ceph-mon[36126]: pgmap v430: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-26T10:46:08.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:08 smithi060 ceph-mon[31539]: pgmap v430: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-26T10:46:08.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:08 smithi100 ceph-mon[36051]: pgmap v430: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-26T10:46:10.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:10 smithi113 ceph-mon[36126]: pgmap v431: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-02-26T10:46:10.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:10 smithi060 ceph-mon[31539]: pgmap v431: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-02-26T10:46:10.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:10 smithi100 ceph-mon[36051]: pgmap v431: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-02-26T10:46:12.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:12 smithi113 ceph-mon[36126]: pgmap v432: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-02-26T10:46:12.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:12 smithi060 ceph-mon[31539]: pgmap v432: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-02-26T10:46:12.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:12 smithi100 ceph-mon[36051]: pgmap v432: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-02-26T10:46:14.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:14 smithi113 ceph-mon[36126]: pgmap v433: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-26T10:46:14.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:14 smithi060 ceph-mon[31539]: pgmap v433: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-26T10:46:14.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:14 smithi100 ceph-mon[36051]: pgmap v433: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-26T10:46:16.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:16 smithi113 ceph-mon[36126]: pgmap v434: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-02-26T10:46:16.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:16 smithi060 ceph-mon[31539]: pgmap v434: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-02-26T10:46:16.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:16 smithi100 ceph-mon[36051]: pgmap v434: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-02-26T10:46:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.879+0000 7f4e27200640 1 -- 172.21.15.60:0/3296537642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e201046b0 msgr2=0x7f4e20104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.879+0000 7f4e27200640 1 --2- 172.21.15.60:0/3296537642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e201046b0 0x7f4e20104b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009900 tx=0x7f4e0802bc40 comp rx=0 tx=0).stop 2024-02-26T10:46:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.880+0000 7f4e27200640 1 -- 172.21.15.60:0/3296537642 shutdown_connections 2024-02-26T10:46:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.880+0000 7f4e27200640 1 --2- 172.21.15.60:0/3296537642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e20105070 0x7f4e2010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.880+0000 7f4e27200640 1 --2- 172.21.15.60:0/3296537642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e201046b0 0x7f4e20104b30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.880+0000 7f4e27200640 1 --2- 172.21.15.60:0/3296537642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e201034c0 0x7f4e201038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.880+0000 7f4e27200640 1 -- 172.21.15.60:0/3296537642 >> 172.21.15.60:0/3296537642 conn(0x7f4e200fea50 msgr2=0x7f4e20100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.880+0000 7f4e27200640 1 -- 172.21.15.60:0/3296537642 shutdown_connections 2024-02-26T10:46:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.880+0000 7f4e27200640 1 -- 172.21.15.60:0/3296537642 wait complete. 2024-02-26T10:46:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.881+0000 7f4e27200640 1 Processor -- start 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.881+0000 7f4e27200640 1 -- start start 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e27200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e201034c0 0x7f4e201996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e27200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e201046b0 0x7f4e20199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e27200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e20105070 0x7f4e201a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e27200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2010f1a0 con 0x7f4e201034c0 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e27200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2010f020 con 0x7f4e201046b0 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e27200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2010f320 con 0x7f4e20105070 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e1fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e201034c0 0x7f4e201996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e1fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e201034c0 0x7f4e201996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53624/0 (socket says 172.21.15.60:53624) 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e1fe00640 1 -- 172.21.15.60:0/1207010249 learned_addr learned my addr 172.21.15.60:0/1207010249 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e17400640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e201046b0 0x7f4e20199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.882+0000 7f4e25200640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e20105070 0x7f4e201a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e1fe00640 1 -- 172.21.15.60:0/1207010249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e20105070 msgr2=0x7f4e201a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e1fe00640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e20105070 0x7f4e201a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e1fe00640 1 -- 172.21.15.60:0/1207010249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e201046b0 msgr2=0x7f4e20199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e1fe00640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e201046b0 0x7f4e20199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e1fe00640 1 -- 172.21.15.60:0/1207010249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e201a12d0 con 0x7f4e201034c0 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e1fe00640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e201034c0 0x7f4e201996b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000e9c0 tx=0x7f4e1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e1d600640 1 -- 172.21.15.60:0/1207010249 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000cd80 con 0x7f4e201034c0 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e1d600640 1 -- 172.21.15.60:0/1207010249 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e100035f0 con 0x7f4e201034c0 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e1d600640 1 -- 172.21.15.60:0/1207010249 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10010690 con 0x7f4e201034c0 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.883+0000 7f4e27200640 1 -- 172.21.15.60:0/1207010249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e201a15c0 con 0x7f4e201034c0 2024-02-26T10:46:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.884+0000 7f4e27200640 1 -- 172.21.15.60:0/1207010249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e201a1a90 con 0x7f4e201034c0 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.887+0000 7f4e1d600640 1 -- 172.21.15.60:0/1207010249 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f4e100037a0 con 0x7f4e201034c0 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.887+0000 7f4e2a000740 1 Processor -- start 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.887+0000 7f4e2a000740 1 -- start start 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.887+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272573c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.887+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 0x55c272578d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.887+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272579380 0x55c27257b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.887+0000 7f4e2a000740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2724d9340 con 0x55c272579380 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.887+0000 7f4e2a000740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c272482520 con 0x55c272573820 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.887+0000 7f4e2a000740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2724d53e0 con 0x55c272574210 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e25200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272573c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e1fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 0x55c272578d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e17400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272579380 0x55c27257b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e25200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272573c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51148/0 (socket says 172.21.15.60:51148) 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e25200640 1 -- 172.21.15.60:0/2713752749 learned_addr learned my addr 172.21.15.60:0/2713752749 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e25200640 1 -- 172.21.15.60:0/2713752749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 msgr2=0x55c272578d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e25200640 1 --2- 172.21.15.60:0/2713752749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 0x55c272578d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e25200640 1 -- 172.21.15.60:0/2713752749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272579380 msgr2=0x55c27257b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e25200640 1 --2- 172.21.15.60:0/2713752749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272579380 0x55c27257b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e25200640 1 -- 172.21.15.60:0/2713752749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c272496390 con 0x55c272573820 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.888+0000 7f4e1fe00640 1 --2- 172.21.15.60:0/2713752749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 0x55c272578d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.889+0000 7f4e25200640 1 --2- 172.21.15.60:0/2713752749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272573c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009fd0 tx=0x7f4e180118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f346164bafaedfff server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.889+0000 7f4e1d600640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e040a1ec0 0x7f4e040a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.889+0000 7f4e15600640 1 -- 172.21.15.60:0/2713752749 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18004d20 con 0x55c272573820 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.889+0000 7f4e15600640 1 -- 172.21.15.60:0/2713752749 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e18004050 con 0x55c272573820 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.889+0000 7f4e15600640 1 -- 172.21.15.60:0/2713752749 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18005260 con 0x55c272573820 2024-02-26T10:46:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.889+0000 7f4e1d600640 1 -- 172.21.15.60:0/1207010249 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f4e100ce6d0 con 0x7f4e201034c0 2024-02-26T10:46:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.889+0000 7f4e1fe00640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e040a1ec0 0x7f4e040a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 -- 172.21.15.60:0/2713752749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 msgr2=0x55c272573c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 --2- 172.21.15.60:0/2713752749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272573c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009fd0 tx=0x7f4e180118d0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 -- 172.21.15.60:0/2713752749 shutdown_connections 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 --2- 172.21.15.60:0/2713752749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272579380 0x55c27257b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 --2- 172.21.15.60:0/2713752749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 0x55c272578d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 --2- 172.21.15.60:0/2713752749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272573c40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 -- 172.21.15.60:0/2713752749 >> 172.21.15.60:0/2713752749 conn(0x55c27256ed90 msgr2=0x55c27256f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e1fe00640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e040a1ec0 0x7f4e040a4380 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55c272574610 tx=0x7f4e100d4530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 -- 172.21.15.60:0/2713752749 shutdown_connections 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 -- 172.21.15.60:0/2713752749 wait complete. 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 -- start start 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.890+0000 7f4e2a000740 1 Processor -- start 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272573820 0x55c27257daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c27257dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 0x55c272585020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e2a000740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27257c330 con 0x55c272574210 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e2a000740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c272495980 con 0x55c272579380 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e2a000740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27257c4b0 con 0x55c272573820 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e2a000740 10 client.0 _ll_register_callbacks cb 0x55c2724d1310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e17400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c27257dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e25200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272573820 0x55c27257daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e25200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272573820 0x55c27257daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:48972/0 (socket says 172.21.15.60:48972) 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e25200640 1 -- 172.21.15.60:0/1524174189 learned_addr learned my addr 172.21.15.60:0/1524174189 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.891+0000 7f4e25200640 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 0x55c272585020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e25200640 1 -- 172.21.15.60:0/1524174189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 msgr2=0x55c272585020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e25200640 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 0x55c272585020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e25200640 1 -- 172.21.15.60:0/1524174189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 msgr2=0x55c27257dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e25200640 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c27257dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e25200640 1 -- 172.21.15.60:0/1524174189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2721ee210 con 0x55c272573820 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e25200640 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272573820 0x55c27257daa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4e18003a60 tx=0x7f4e18000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ee62be0770712e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e0e000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e0e000640 1 -- 172.21.15.60:0/1524174189 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18004400 con 0x55c272573820 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e2a000740 1 -- 172.21.15.60:0/1524174189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e18004050 con 0x55c272573820 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.892+0000 7f4e0e000640 1 -- 172.21.15.60:0/1524174189 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f4e180045a0 con 0x55c272573820 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e0e000640 1 -- 172.21.15.60:0/1524174189 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e1800f4e0 con 0x55c272573820 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e2a000740 10 client.25597 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e2a000740 1 -- 172.21.15.60:0/1524174189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c2722ba620 con 0x55c272573820 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e0e000640 1 -- 172.21.15.60:0/1524174189 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e18004050 con 0x55c272573820 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e0e000640 1 -- 172.21.15.60:0/1524174189 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4e180287b0 con 0x55c272573820 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e2a000740 10 client.25597 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e2a000740 10 client.25597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e2a000740 20 client.25597 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e2a000740 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55c272589690 0x55c27258ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e2a000740 4 client.25597 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.893+0000 7f4e2a000740 1 -- 172.21.15.60:0/1524174189 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2724e6880 con 0x55c272589690 2024-02-26T10:46:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.894+0000 7f4e17400640 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55c272589690 0x55c27258ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.894+0000 7f4e17400640 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55c272589690 0x55c27258ba80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a6b91cc8eec400 server_cookie=8e67be00c0398b42 in_seq=0 out_seq=0 2024-02-26T10:46:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.894+0000 7f4e0e000640 10 client.25597 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e0e000640 1 -- 172.21.15.60:0/1524174189 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2724e6880 con 0x55c272589690 2024-02-26T10:46:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e0e000640 10 client.25597 handle_command_reply: tid=0 2024-02-26T10:46:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 client.25597 shutdown 2024-02-26T10:46:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 -- 172.21.15.60:0/1524174189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272573820 msgr2=0x55c27257daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272573820 0x55c27257daa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4e18003a60 tx=0x7f4e18000f40 comp rx=0 tx=0).stop 2024-02-26T10:46:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 -- 172.21.15.60:0/1524174189 shutdown_connections 2024-02-26T10:46:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55c272589690 0x55c27258ba80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 0x55c272585020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c27257dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 --2- 172.21.15.60:0/1524174189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272573820 0x55c27257daa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 -- 172.21.15.60:0/1524174189 >> 172.21.15.60:0/1524174189 conn(0x55c27256ed90 msgr2=0x55c27256dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e0e000640 0 client.25597 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.895+0000 7f4e2a000740 1 -- 172.21.15.60:0/1524174189 shutdown_connections 2024-02-26T10:46:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.896+0000 7f4e2a000740 1 -- 172.21.15.60:0/1524174189 wait complete. 2024-02-26T10:46:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.896+0000 7f4e2a000740 20 client.25597 trim_cache size 0 max 16384 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.910+0000 7f4e2a000740 1 Processor -- start 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.910+0000 7f4e2a000740 1 -- start start 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272575490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c2725759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272579380 0x55c272584fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e2a000740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201ae4a0 con 0x55c272574210 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e2a000740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201ac040 con 0x55c272573820 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e2a000740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201aa0e0 con 0x55c272579380 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e17400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c2725759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e17400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c2725759d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53652/0 (socket says 172.21.15.60:53652) 2024-02-26T10:46:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e17400640 1 -- 172.21.15.60:0/3494098790 learned_addr learned my addr 172.21.15.60:0/3494098790 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e25200640 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272575490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e25200640 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272579380 0x55c272584fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.911+0000 7f4e17400640 1 -- 172.21.15.60:0/3494098790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272579380 msgr2=0x55c272584fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.912+0000 7f4e17400640 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272579380 0x55c272584fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.912+0000 7f4e17400640 1 -- 172.21.15.60:0/3494098790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 msgr2=0x55c272575490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.912+0000 7f4e17400640 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272575490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.912+0000 7f4e17400640 1 -- 172.21.15.60:0/3494098790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c27258bdf0 con 0x55c272574210 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.912+0000 7f4e25200640 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272575490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.912+0000 7f4e17400640 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c2725759d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009900 tx=0x7f4e0802cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b468fbafe1ceab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.912+0000 7f4e14c00640 1 -- 172.21.15.60:0/3494098790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0804f070 con 0x55c272574210 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.912+0000 7f4e14c00640 1 -- 172.21.15.60:0/3494098790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e080046c0 con 0x55c272574210 2024-02-26T10:46:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.912+0000 7f4e14c00640 1 -- 172.21.15.60:0/3494098790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08005130 con 0x55c272574210 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.913+0000 7f4e2a000740 1 -- 172.21.15.60:0/3494098790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 msgr2=0x55c2725759d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.913+0000 7f4e2a000740 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c2725759d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009900 tx=0x7f4e0802cf90 comp rx=0 tx=0).stop 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.913+0000 7f4e2a000740 1 -- 172.21.15.60:0/3494098790 shutdown_connections 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.913+0000 7f4e2a000740 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272579380 0x55c272584fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.913+0000 7f4e2a000740 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272574210 0x55c2725759d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.913+0000 7f4e2a000740 1 --2- 172.21.15.60:0/3494098790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272573820 0x55c272575490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.913+0000 7f4e2a000740 1 -- 172.21.15.60:0/3494098790 >> 172.21.15.60:0/3494098790 conn(0x55c27256ed90 msgr2=0x55c2725903b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.913+0000 7f4e2a000740 1 -- 172.21.15.60:0/3494098790 shutdown_connections 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.913+0000 7f4e2a000740 1 -- 172.21.15.60:0/3494098790 wait complete. 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.914+0000 7f4e2a000740 1 -- start start 2024-02-26T10:46:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.914+0000 7f4e2a000740 1 Processor -- start 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.914+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272573820 0x55c272582c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.914+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 0x55c272583140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.914+0000 7f4e2a000740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 0x55c272584150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.914+0000 7f4e2a000740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201aa6e0 con 0x55c272573820 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.914+0000 7f4e2a000740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201ab9e0 con 0x55c272579380 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.914+0000 7f4e2a000740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201aad20 con 0x55c272574210 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.914+0000 7f4e2a000740 10 client.0 _ll_register_callbacks cb 0x7f4e18009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272573820 0x55c272582c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272573820 0x55c272582c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53658/0 (socket says 172.21.15.60:53658) 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 -- 172.21.15.60:0/4148856544 learned_addr learned my addr 172.21.15.60:0/4148856544 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e17400640 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 0x55c272583140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 0x55c272584150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 -- 172.21.15.60:0/4148856544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 msgr2=0x55c272583140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 0x55c272583140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 -- 172.21.15.60:0/4148856544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 msgr2=0x55c272584150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 0x55c272584150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 -- 172.21.15.60:0/4148856544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c272584740 con 0x55c272573820 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e25200640 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272573820 0x55c272582c00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4e18028c20 tx=0x7f4e18004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f1f09dfa13318b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.915+0000 7f4e0e000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.916+0000 7f4e0e000640 1 -- 172.21.15.60:0/4148856544 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18004cb0 con 0x55c272573820 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.916+0000 7f4e0e000640 1 -- 172.21.15.60:0/4148856544 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f4e1800f890 con 0x55c272573820 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.916+0000 7f4e2a000740 1 -- 172.21.15.60:0/4148856544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e080046c0 con 0x55c272573820 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.916+0000 7f4e0e000640 1 -- 172.21.15.60:0/4148856544 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e1800fa90 con 0x55c272573820 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.916+0000 7f4e2a000740 10 client.15483 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.916+0000 7f4e2a000740 1 -- 172.21.15.60:0/4148856544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c2725dffd0 con 0x55c272573820 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.916+0000 7f4e0e000640 1 -- 172.21.15.60:0/4148856544 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e080046c0 con 0x55c272573820 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.916+0000 7f4e0e000640 1 -- 172.21.15.60:0/4148856544 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4e180047e0 con 0x55c272573820 2024-02-26T10:46:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.917+0000 7f4e2a000740 10 client.15483 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.917+0000 7f4e2a000740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.917+0000 7f4e2a000740 20 client.15483 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.917+0000 7f4e2a000740 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55c2725e0810 0x55c2725e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.917+0000 7f4e2a000740 4 client.15483 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:46:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.917+0000 7f4e2a000740 1 -- 172.21.15.60:0/4148856544 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2724e0bd0 con 0x55c2725e0810 2024-02-26T10:46:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.917+0000 7f4e17400640 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55c2725e0810 0x55c2725e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.918+0000 7f4e17400640 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55c2725e0810 0x55c2725e2c30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e1da2da7f00753 server_cookie=87b5973ca6e403bc in_seq=0 out_seq=0 2024-02-26T10:46:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.918+0000 7f4e0e000640 10 client.15483 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e0e000640 1 -- 172.21.15.60:0/4148856544 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2724e0bd0 con 0x55c2725e0810 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e0e000640 10 client.15483 handle_command_reply: tid=0 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 client.15483 shutdown 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 -- 172.21.15.60:0/4148856544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272573820 msgr2=0x55c272582c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272573820 0x55c272582c00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4e18028c20 tx=0x7f4e18004c40 comp rx=0 tx=0).stop 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 -- 172.21.15.60:0/4148856544 shutdown_connections 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55c2725e0810 0x55c2725e2c30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c272579380 0x55c272584150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c272574210 0x55c272583140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e0e000640 0 client.15483 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 --2- 172.21.15.60:0/4148856544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c272573820 0x55c272582c00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 -- 172.21.15.60:0/4148856544 >> 172.21.15.60:0/4148856544 conn(0x55c27256ed90 msgr2=0x55c272575490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 -- 172.21.15.60:0/4148856544 shutdown_connections 2024-02-26T10:46:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 1 -- 172.21.15.60:0/4148856544 wait complete. 2024-02-26T10:46:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.924+0000 7f4e2a000740 20 client.15483 trim_cache size 0 max 16384 2024-02-26T10:46:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:46:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:46:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:46:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:46:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 -- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e040a1ec0 msgr2=0x7f4e040a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e040a1ec0 0x7f4e040a4380 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55c272574610 tx=0x7f4e100d4530 comp rx=0 tx=0).stop 2024-02-26T10:46:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 -- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e201034c0 msgr2=0x7f4e201996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e201034c0 0x7f4e201996b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000e9c0 tx=0x7f4e1000ee80 comp rx=0 tx=0).stop 2024-02-26T10:46:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 -- 172.21.15.60:0/1207010249 shutdown_connections 2024-02-26T10:46:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4e040a1ec0 0x7f4e040a4380 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e201046b0 0x7f4e20199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e20105070 0x7f4e201a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 --2- 172.21.15.60:0/1207010249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e201034c0 0x7f4e201996b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 -- 172.21.15.60:0/1207010249 >> 172.21.15.60:0/1207010249 conn(0x7f4e200fea50 msgr2=0x7f4e201078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 -- 172.21.15.60:0/1207010249 shutdown_connections 2024-02-26T10:46:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:16.925+0000 7f4e27200640 1 -- 172.21.15.60:0/1207010249 wait complete. 2024-02-26T10:46:18.332 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:18 smithi113 ceph-mon[36126]: pgmap v435: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-26T10:46:18.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:18 smithi060 ceph-mon[31539]: pgmap v435: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-26T10:46:18.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:18 smithi100 ceph-mon[36051]: pgmap v435: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-26T10:46:20.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:20 smithi060 ceph-mon[31539]: pgmap v436: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-02-26T10:46:20.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:20 smithi100 ceph-mon[36051]: pgmap v436: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-02-26T10:46:20.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:20 smithi113 ceph-mon[36126]: pgmap v436: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-02-26T10:46:21.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T10:46:21.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T10:46:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.310+0000 7fcf7f200640 1 -- 172.21.15.60:0/1683869443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78109330 msgr2=0x7fcf7810fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.310+0000 7fcf7f200640 1 --2- 172.21.15.60:0/1683869443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78109330 0x7fcf7810fbc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009ca0 tx=0x7fcf7402c100 comp rx=0 tx=0).stop 2024-02-26T10:46:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.311+0000 7fcf7f200640 1 -- 172.21.15.60:0/1683869443 shutdown_connections 2024-02-26T10:46:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.311+0000 7fcf7f200640 1 --2- 172.21.15.60:0/1683869443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78109330 0x7fcf7810fbc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.311+0000 7fcf7f200640 1 --2- 172.21.15.60:0/1683869443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf78108970 0x7fcf78108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.311+0000 7fcf7f200640 1 --2- 172.21.15.60:0/1683869443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf78107780 0x7fcf78107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.311+0000 7fcf7f200640 1 -- 172.21.15.60:0/1683869443 >> 172.21.15.60:0/1683869443 conn(0x7fcf78076730 msgr2=0x7fcf78078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.311+0000 7fcf7f200640 1 -- 172.21.15.60:0/1683869443 shutdown_connections 2024-02-26T10:46:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.311+0000 7fcf7f200640 1 -- 172.21.15.60:0/1683869443 wait complete. 2024-02-26T10:46:21.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.312+0000 7fcf7f200640 1 Processor -- start 2024-02-26T10:46:21.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.312+0000 7fcf7f200640 1 -- start start 2024-02-26T10:46:21.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.312+0000 7fcf7f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78107780 0x7fcf7819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.312+0000 7fcf7f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf78108970 0x7fcf7819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.312+0000 7fcf7f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf78109330 0x7fcf781a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.312+0000 7fcf7f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78113400 con 0x7fcf78107780 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.312+0000 7fcf7f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78113280 con 0x7fcf78109330 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.312+0000 7fcf7f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78113580 con 0x7fcf78108970 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.312+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78107780 0x7fcf7819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf78109330 0x7fcf781a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78107780 0x7fcf7819d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53672/0 (socket says 172.21.15.60:53672) 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7de00640 1 -- 172.21.15.60:0/3604135352 learned_addr learned my addr 172.21.15.60:0/3604135352 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7d400640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf78108970 0x7fcf7819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7e800640 1 -- 172.21.15.60:0/3604135352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf78108970 msgr2=0x7fcf7819df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7e800640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf78108970 0x7fcf7819df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7e800640 1 -- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78107780 msgr2=0x7fcf7819d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7e800640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78107780 0x7fcf7819d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7e800640 1 -- 172.21.15.60:0/3604135352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf781a5600 con 0x7fcf78109330 2024-02-26T10:46:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7d400640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf78108970 0x7fcf7819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7de00640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78107780 0x7fcf7819d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.313+0000 7fcf7e800640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf78109330 0x7fcf781a4f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf74037040 tx=0x7fcf74035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.314+0000 7fcf72a00640 1 -- 172.21.15.60:0/3604135352 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74039040 con 0x7fcf78109330 2024-02-26T10:46:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.314+0000 7fcf72a00640 1 -- 172.21.15.60:0/3604135352 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf74044070 con 0x7fcf78109330 2024-02-26T10:46:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.314+0000 7fcf72a00640 1 -- 172.21.15.60:0/3604135352 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74004350 con 0x7fcf78109330 2024-02-26T10:46:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.314+0000 7fcf7f200640 1 -- 172.21.15.60:0/3604135352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf781a5890 con 0x7fcf78109330 2024-02-26T10:46:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.314+0000 7fcf7f200640 1 -- 172.21.15.60:0/3604135352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf781a5d00 con 0x7fcf78109330 2024-02-26T10:46:21.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.317+0000 7fcf835e8740 1 Processor -- start 2024-02-26T10:46:21.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.317+0000 7fcf72a00640 1 -- 172.21.15.60:0/3604135352 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fcf740044f0 con 0x7fcf78109330 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.317+0000 7fcf835e8740 1 -- start start 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2aace0 0x557ffb2ab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf835e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ffb1d8060 con 0x557ffb2b0840 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf835e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ffb20fd00 con 0x557ffb2aace0 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf835e8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ffb1b8a00 con 0x557ffb2ab6d0 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf7d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49038/0 (socket says 172.21.15.60:49038) 2024-02-26T10:46:21.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf7d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53682/0 (socket says 172.21.15.60:53682) 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf7de00640 1 -- 172.21.15.60:0/3290766380 learned_addr learned my addr 172.21.15.60:0/3290766380 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.318+0000 7fcf7d400640 1 --2- 172.21.15.60:0/3290766380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2aace0 0x557ffb2ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf72a00640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcf600a1e90 0x7fcf600a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf7d400640 1 -- 172.21.15.60:0/3290766380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2ab6d0 msgr2=0x557ffb2b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf7d400640 1 --2- 172.21.15.60:0/3290766380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf7d400640 1 -- 172.21.15.60:0/3290766380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2aace0 msgr2=0x557ffb2ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf7d400640 1 --2- 172.21.15.60:0/3290766380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2aace0 0x557ffb2ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf7d400640 1 -- 172.21.15.60:0/3290766380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ffafee7d0 con 0x557ffb2b0840 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf7de00640 1 --2- 172.21.15.60:0/3290766380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf72a00640 1 -- 172.21.15.60:0/3604135352 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fcf740f0710 con 0x7fcf78109330 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf7e800640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcf600a1e90 0x7fcf600a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf7d400640 1 --2- 172.21.15.60:0/3290766380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2b2cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcf680058c0 tx=0x7fcf68005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937d2c6aafebb91d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:21.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf67000640 1 -- 172.21.15.60:0/3290766380 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68011050 con 0x557ffb2b0840 2024-02-26T10:46:21.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf67000640 1 -- 172.21.15.60:0/3290766380 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf68005500 con 0x557ffb2b0840 2024-02-26T10:46:21.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.319+0000 7fcf67000640 1 -- 172.21.15.60:0/3290766380 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680047a0 con 0x557ffb2b0840 2024-02-26T10:46:21.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf835e8740 1 -- 172.21.15.60:0/3290766380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 msgr2=0x557ffb2b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf835e8740 1 --2- 172.21.15.60:0/3290766380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2b2cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcf680058c0 tx=0x7fcf68005850 comp rx=0 tx=0).stop 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf7e800640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcf600a1e90 0x7fcf600a4350 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcf740bcf70 tx=0x7fcf740095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf835e8740 1 -- 172.21.15.60:0/3290766380 shutdown_connections 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf835e8740 1 --2- 172.21.15.60:0/3290766380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2b2cd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf835e8740 1 --2- 172.21.15.60:0/3290766380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf835e8740 1 --2- 172.21.15.60:0/3290766380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2aace0 0x557ffb2ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf835e8740 1 -- 172.21.15.60:0/3290766380 >> 172.21.15.60:0/3290766380 conn(0x557ffb2a6250 msgr2=0x557ffb2a6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf835e8740 1 -- 172.21.15.60:0/3290766380 shutdown_connections 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.320+0000 7fcf835e8740 1 -- 172.21.15.60:0/3290766380 wait complete. 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.321+0000 7fcf835e8740 1 -- start start 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.321+0000 7fcf835e8740 1 Processor -- start 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2aace0 0x557ffb2b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2b0840 0x557ffb2bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf835e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ffb1ae090 con 0x557ffb2aace0 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf835e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ffb20bda0 con 0x557ffb2ab6d0 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf835e8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ffb2b37f0 con 0x557ffb2b0840 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf835e8740 10 client.0 _ll_register_callbacks cb 0x557ffb1cb830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2aace0 0x557ffb2b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2aace0 0x557ffb2b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53688/0 (socket says 172.21.15.60:53688) 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7d400640 1 -- 172.21.15.60:0/2732522459 learned_addr learned my addr 172.21.15.60:0/2732522459 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7de00640 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7d400640 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2b0840 0x557ffb2bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7d400640 1 -- 172.21.15.60:0/2732522459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2b0840 msgr2=0x557ffb2bc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7d400640 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2b0840 0x557ffb2bc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7d400640 1 -- 172.21.15.60:0/2732522459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 msgr2=0x557ffb2b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7d400640 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7d400640 1 -- 172.21.15.60:0/2732522459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ffaf243f0 con 0x557ffb2aace0 2024-02-26T10:46:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.322+0000 7fcf7de00640 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.323+0000 7fcf7d400640 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2aace0 0x557ffb2b5070 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcf680057b0 tx=0x7fcf68001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47fa997a0b2e657 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.323+0000 7fcf5b400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.323+0000 7fcf5b400640 1 -- 172.21.15.60:0/2732522459 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68011250 con 0x557ffb2aace0 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.323+0000 7fcf5b400640 1 -- 172.21.15.60:0/2732522459 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fcf68004030 con 0x557ffb2aace0 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.323+0000 7fcf835e8740 1 -- 172.21.15.60:0/2732522459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf68005500 con 0x557ffb2aace0 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.323+0000 7fcf5b400640 1 -- 172.21.15.60:0/2732522459 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf68034050 con 0x557ffb2aace0 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.323+0000 7fcf835e8740 10 client.15495 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.323+0000 7fcf835e8740 1 -- 172.21.15.60:0/2732522459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557ffaee40f0 con 0x557ffb2aace0 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.324+0000 7fcf5b400640 1 -- 172.21.15.60:0/2732522459 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf68005500 con 0x557ffb2aace0 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.324+0000 7fcf5b400640 1 -- 172.21.15.60:0/2732522459 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf680035f0 con 0x557ffb2aace0 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.324+0000 7fcf835e8740 10 client.15495 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.324+0000 7fcf835e8740 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.324+0000 7fcf835e8740 20 client.15495 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.324+0000 7fcf835e8740 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557ffb2c07a0 0x557ffb2c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.324+0000 7fcf835e8740 4 client.15495 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:21.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.324+0000 7fcf835e8740 1 -- 172.21.15.60:0/2732522459 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ffb21d240 con 0x557ffb2c07a0 2024-02-26T10:46:21.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.324+0000 7fcf7de00640 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557ffb2c07a0 0x557ffb2c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.325+0000 7fcf7de00640 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557ffb2c07a0 0x557ffb2c2b90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04fd2d86ffc5a2 server_cookie=cfd7238aab638b37 in_seq=0 out_seq=0 2024-02-26T10:46:21.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.325+0000 7fcf5b400640 10 client.15495 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf5b400640 1 -- 172.21.15.60:0/2732522459 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ffb21d240 con 0x557ffb2c07a0 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf5b400640 10 client.15495 handle_command_reply: tid=0 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 client.15495 shutdown 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 -- 172.21.15.60:0/2732522459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2aace0 msgr2=0x557ffb2b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2aace0 0x557ffb2b5070 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcf680057b0 tx=0x7fcf68001e50 comp rx=0 tx=0).stop 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 -- 172.21.15.60:0/2732522459 shutdown_connections 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557ffb2c07a0 0x557ffb2c2b90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2b0840 0x557ffb2bc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 --2- 172.21.15.60:0/2732522459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2aace0 0x557ffb2b5070 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf5b400640 0 client.15495 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 -- 172.21.15.60:0/2732522459 >> 172.21.15.60:0/2732522459 conn(0x557ffb2a6250 msgr2=0x557ffb2a5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 -- 172.21.15.60:0/2732522459 shutdown_connections 2024-02-26T10:46:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 1 -- 172.21.15.60:0/2732522459 wait complete. 2024-02-26T10:46:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.326+0000 7fcf835e8740 20 client.15495 trim_cache size 0 max 16384 2024-02-26T10:46:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.343+0000 7fcf835e8740 1 Processor -- start 2024-02-26T10:46:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.343+0000 7fcf835e8740 1 -- start start 2024-02-26T10:46:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.343+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 0x557ffb2a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.343+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.343+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.343+0000 7fcf835e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf781b2620 con 0x557ffb2b0840 2024-02-26T10:46:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.343+0000 7fcf835e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf781b01c0 con 0x557ffb2ab6d0 2024-02-26T10:46:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.343+0000 7fcf835e8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf781ae260 con 0x557ffb2aace0 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51234/0 (socket says 172.21.15.60:51234) 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7de00640 1 -- 172.21.15.60:0/1655803324 learned_addr learned my addr 172.21.15.60:0/1655803324 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7d400640 1 --2- 172.21.15.60:0/1655803324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 0x557ffb2a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7de00640 1 -- 172.21.15.60:0/1655803324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 msgr2=0x557ffb2a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7de00640 1 --2- 172.21.15.60:0/1655803324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 0x557ffb2a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7de00640 1 -- 172.21.15.60:0/1655803324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 msgr2=0x557ffb2a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7de00640 1 --2- 172.21.15.60:0/1655803324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7de00640 1 -- 172.21.15.60:0/1655803324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ffb2a1cc0 con 0x557ffb2ab6d0 2024-02-26T10:46:21.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.344+0000 7fcf7d400640 1 --2- 172.21.15.60:0/1655803324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:21.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.345+0000 7fcf7de00640 1 --2- 172.21.15.60:0/1655803324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000c00 tx=0x7fcf6c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d028ad957c9913b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:21.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.345+0000 7fcf66600640 1 -- 172.21.15.60:0/1655803324 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0120d0 con 0x557ffb2ab6d0 2024-02-26T10:46:21.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.345+0000 7fcf66600640 1 -- 172.21.15.60:0/1655803324 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c025030 con 0x557ffb2ab6d0 2024-02-26T10:46:21.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.345+0000 7fcf66600640 1 -- 172.21.15.60:0/1655803324 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c004db0 con 0x557ffb2ab6d0 2024-02-26T10:46:21.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.346+0000 7fcf835e8740 1 -- 172.21.15.60:0/1655803324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 msgr2=0x557ffb2bc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.346+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1655803324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000c00 tx=0x7fcf6c005f90 comp rx=0 tx=0).stop 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.346+0000 7fcf835e8740 1 -- 172.21.15.60:0/1655803324 shutdown_connections 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.346+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1655803324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.346+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1655803324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2b0840 0x557ffb2a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.346+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1655803324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 0x557ffb2a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.346+0000 7fcf835e8740 1 -- 172.21.15.60:0/1655803324 >> 172.21.15.60:0/1655803324 conn(0x557ffb2a6250 msgr2=0x557ffb2acb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.346+0000 7fcf835e8740 1 -- 172.21.15.60:0/1655803324 shutdown_connections 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.346+0000 7fcf835e8740 1 -- 172.21.15.60:0/1655803324 wait complete. 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.347+0000 7fcf835e8740 1 -- start start 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.347+0000 7fcf835e8740 1 Processor -- start 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.347+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 0x557ffb2bbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.347+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.347+0000 7fcf835e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2b0840 0x557ffb2ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.347+0000 7fcf835e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf781ae860 con 0x557ffb2ab6d0 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.347+0000 7fcf835e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf781afb60 con 0x557ffb2b0840 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.347+0000 7fcf835e8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf781aeea0 con 0x557ffb2aace0 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.347+0000 7fcf835e8740 10 client.0 _ll_register_callbacks cb 0x7fcf68011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53708/0 (socket says 172.21.15.60:53708) 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7de00640 1 -- 172.21.15.60:0/1264857270 learned_addr learned my addr 172.21.15.60:0/1264857270 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7d400640 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 0x557ffb2bbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7d400640 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2b0840 0x557ffb2ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7de00640 1 -- 172.21.15.60:0/1264857270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 msgr2=0x557ffb2bbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7de00640 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 0x557ffb2bbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7de00640 1 -- 172.21.15.60:0/1264857270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2b0840 msgr2=0x557ffb2ba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7de00640 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2b0840 0x557ffb2ba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7de00640 1 -- 172.21.15.60:0/1264857270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ffb2bac10 con 0x557ffb2ab6d0 2024-02-26T10:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.348+0000 7fcf7d400640 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2b0840 0x557ffb2ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.349+0000 7fcf7de00640 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc3d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c024040 tx=0x7fcf6c004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fd2df5f1b78aca server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.349+0000 7fcf5b400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.349+0000 7fcf5b400640 1 -- 172.21.15.60:0/1264857270 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00fe80 con 0x557ffb2ab6d0 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.349+0000 7fcf5b400640 1 -- 172.21.15.60:0/1264857270 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fcf6c0120d0 con 0x557ffb2ab6d0 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.349+0000 7fcf835e8740 1 -- 172.21.15.60:0/1264857270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf6c025030 con 0x557ffb2ab6d0 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.349+0000 7fcf5b400640 1 -- 172.21.15.60:0/1264857270 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf6c039050 con 0x557ffb2ab6d0 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.349+0000 7fcf835e8740 10 client.15501 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.349+0000 7fcf835e8740 1 -- 172.21.15.60:0/1264857270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557ffb3176d0 con 0x557ffb2ab6d0 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.349+0000 7fcf5b400640 1 -- 172.21.15.60:0/1264857270 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c025030 con 0x557ffb2ab6d0 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.350+0000 7fcf5b400640 1 -- 172.21.15.60:0/1264857270 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf6c02a7f0 con 0x557ffb2ab6d0 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.350+0000 7fcf835e8740 10 client.15501 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.350+0000 7fcf835e8740 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.350+0000 7fcf835e8740 20 client.15501 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.350+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557ffb318060 0x557ffb31a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.350+0000 7fcf835e8740 4 client.15501 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T10:46:21.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.350+0000 7fcf835e8740 1 -- 172.21.15.60:0/1264857270 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557ffb217590 con 0x557ffb318060 2024-02-26T10:46:21.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.350+0000 7fcf7d400640 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557ffb318060 0x557ffb31a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf7d400640 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557ffb318060 0x557ffb31a480 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de0dc4373b7ff60 server_cookie=bd436170305b56f in_seq=0 out_seq=0 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf5b400640 10 client.15501 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf5b400640 1 -- 172.21.15.60:0/1264857270 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557ffb217590 con 0x557ffb318060 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf5b400640 10 client.15501 handle_command_reply: tid=0 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 client.15501 shutdown 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 -- 172.21.15.60:0/1264857270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2ab6d0 msgr2=0x557ffb2bc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc3d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c024040 tx=0x7fcf6c004890 comp rx=0 tx=0).stop 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 -- 172.21.15.60:0/1264857270 shutdown_connections 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x557ffb318060 0x557ffb31a480 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ffb2b0840 0x557ffb2ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557ffb2ab6d0 0x557ffb2bc3d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf5b400640 0 client.15501 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 --2- 172.21.15.60:0/1264857270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ffb2aace0 0x557ffb2bbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 -- 172.21.15.60:0/1264857270 >> 172.21.15.60:0/1264857270 conn(0x557ffb2a6250 msgr2=0x557ffb2a9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 -- 172.21.15.60:0/1264857270 shutdown_connections 2024-02-26T10:46:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.351+0000 7fcf835e8740 1 -- 172.21.15.60:0/1264857270 wait complete. 2024-02-26T10:46:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf835e8740 20 client.15501 trim_cache size 0 max 16384 2024-02-26T10:46:21.350 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:46:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-02-26T10:46:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-02-26T10:46:21.350 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:46:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 -- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcf600a1e90 msgr2=0x7fcf600a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcf600a1e90 0x7fcf600a4350 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcf740bcf70 tx=0x7fcf740095c0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 -- 172.21.15.60:0/3604135352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf78109330 msgr2=0x7fcf781a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf78109330 0x7fcf781a4f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf74037040 tx=0x7fcf74035d10 comp rx=0 tx=0).stop 2024-02-26T10:46:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 -- 172.21.15.60:0/3604135352 shutdown_connections 2024-02-26T10:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fcf600a1e90 0x7fcf600a4350 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf78109330 0x7fcf781a4f90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf78108970 0x7fcf7819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 --2- 172.21.15.60:0/3604135352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf78107780 0x7fcf7819d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 -- 172.21.15.60:0/3604135352 >> 172.21.15.60:0/3604135352 conn(0x7fcf78076730 msgr2=0x7fcf7810bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 -- 172.21.15.60:0/3604135352 shutdown_connections 2024-02-26T10:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.352+0000 7fcf7f200640 1 -- 172.21.15.60:0/3604135352 wait complete. 2024-02-26T10:46:21.364 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T10:46:21.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:46:21.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.587+0000 7fb9c0000640 1 -- 172.21.15.60:0/1238319107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b81078a0 msgr2=0x7fb9b8107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.587+0000 7fb9c0000640 1 --2- 172.21.15.60:0/1238319107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b81078a0 0x7fb9b8107cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009860 tx=0x7fb9ac02bb70 comp rx=0 tx=0).stop 2024-02-26T10:46:21.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.587+0000 7fb9c0000640 1 -- 172.21.15.60:0/1238319107 shutdown_connections 2024-02-26T10:46:21.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.587+0000 7fb9c0000640 1 --2- 172.21.15.60:0/1238319107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b8109450 0x7fb9b810fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.587+0000 7fb9c0000640 1 --2- 172.21.15.60:0/1238319107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b8108a90 0x7fb9b8108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.587+0000 7fb9c0000640 1 --2- 172.21.15.60:0/1238319107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b81078a0 0x7fb9b8107cc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.587+0000 7fb9c0000640 1 -- 172.21.15.60:0/1238319107 >> 172.21.15.60:0/1238319107 conn(0x7fb9b8076870 msgr2=0x7fb9b8078c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:21.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.587+0000 7fb9c0000640 1 -- 172.21.15.60:0/1238319107 shutdown_connections 2024-02-26T10:46:21.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.587+0000 7fb9c0000640 1 -- 172.21.15.60:0/1238319107 wait complete. 2024-02-26T10:46:21.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.588+0000 7fb9c0000640 1 Processor -- start 2024-02-26T10:46:21.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.588+0000 7fb9c0000640 1 -- start start 2024-02-26T10:46:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9c0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b81078a0 0x7fb9b819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9c0000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b8108a90 0x7fb9b819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9c0000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b8109450 0x7fb9b81a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9c0000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b8113510 con 0x7fb9b8109450 2024-02-26T10:46:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9c0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b8113390 con 0x7fb9b81078a0 2024-02-26T10:46:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9c0000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b8113690 con 0x7fb9b8108a90 2024-02-26T10:46:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9be000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b8109450 0x7fb9b81a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9be000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b8109450 0x7fb9b81a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53724/0 (socket says 172.21.15.60:53724) 2024-02-26T10:46:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9be000640 1 -- 172.21.15.60:0/577560590 learned_addr learned my addr 172.21.15.60:0/577560590 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9bd600640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b81078a0 0x7fb9b819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.589+0000 7fb9bcc00640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b8108a90 0x7fb9b819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.590+0000 7fb9bd600640 1 -- 172.21.15.60:0/577560590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b8108a90 msgr2=0x7fb9b819e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.590+0000 7fb9bd600640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b8108a90 0x7fb9b819e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.590+0000 7fb9bd600640 1 -- 172.21.15.60:0/577560590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b8109450 msgr2=0x7fb9b81a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.590+0000 7fb9bd600640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b8109450 0x7fb9b81a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.590+0000 7fb9bd600640 1 -- 172.21.15.60:0/577560590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b81a5810 con 0x7fb9b81078a0 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.590+0000 7fb9bd600640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b81078a0 0x7fb9b819dbf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009780 tx=0x7fb9ac0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.590+0000 7fb9b6000640 1 -- 172.21.15.60:0/577560590 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac041020 con 0x7fb9b81078a0 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.590+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b81a5aa0 con 0x7fb9b81078a0 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.591+0000 7fb9b6000640 1 -- 172.21.15.60:0/577560590 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac0323b0 con 0x7fb9b81078a0 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.591+0000 7fb9b6000640 1 -- 172.21.15.60:0/577560590 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac039040 con 0x7fb9b81078a0 2024-02-26T10:46:21.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.591+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b81a5fb0 con 0x7fb9b81078a0 2024-02-26T10:46:21.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.592+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b810ec40 con 0x7fb9b81078a0 2024-02-26T10:46:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.594+0000 7fb9b6000640 1 -- 172.21.15.60:0/577560590 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fb9ac044020 con 0x7fb9b81078a0 2024-02-26T10:46:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.595+0000 7fb9b6000640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9a00a1e10 0x7fb9a00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.595+0000 7fb9b6000640 1 -- 172.21.15.60:0/577560590 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb9ac0efad0 con 0x7fb9b81078a0 2024-02-26T10:46:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.596+0000 7fb9bcc00640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9a00a1e10 0x7fb9a00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.596+0000 7fb9bcc00640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9a00a1e10 0x7fb9a00a42d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb9b819f100 tx=0x7fb9a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:21.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.598+0000 7fb9b6000640 1 -- 172.21.15.60:0/577560590 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9ac04b020 con 0x7fb9b81078a0 2024-02-26T10:46:21.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.837+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9b802ce30 con 0x7fb9b81078a0 2024-02-26T10:46:21.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.839+0000 7fb9b6000640 1 -- 172.21.15.60:0/577560590 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11359 (secure 0 0 0) 0x7fb9ac0bcaa0 con 0x7fb9b81078a0 2024-02-26T10:46:21.834 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:46:21.834 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:45:39.355500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:46:21.834 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2024-02-26T10:46:21.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.841+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9a00a1e10 msgr2=0x7fb9a00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.841+0000 7fb9c0000640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9a00a1e10 0x7fb9a00a42d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb9b819f100 tx=0x7fb9a4009260 comp rx=0 tx=0).stop 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.841+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b81078a0 msgr2=0x7fb9b819dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.841+0000 7fb9c0000640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b81078a0 0x7fb9b819dbf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009780 tx=0x7fb9ac0042f0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.842+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 shutdown_connections 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.842+0000 7fb9c0000640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb9a00a1e10 0x7fb9a00a42d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.842+0000 7fb9c0000640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b8109450 0x7fb9b81a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.842+0000 7fb9c0000640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b8108a90 0x7fb9b819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.842+0000 7fb9c0000640 1 --2- 172.21.15.60:0/577560590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b81078a0 0x7fb9b819dbf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.842+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 >> 172.21.15.60:0/577560590 conn(0x7fb9b8076870 msgr2=0x7fb9b810be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.842+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 shutdown_connections 2024-02-26T10:46:21.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:21.842+0000 7fb9c0000640 1 -- 172.21.15.60:0/577560590 wait complete. 2024-02-26T10:46:21.857 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:46:21.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T10:46:22.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.090+0000 7f3873e00640 1 -- 172.21.15.60:0/1838858418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3874107780 msgr2=0x7f3874107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.090+0000 7f3873e00640 1 --2- 172.21.15.60:0/1838858418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3874107780 0x7f3874107ba0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3864009900 tx=0x7f386402bc30 comp rx=0 tx=0).stop 2024-02-26T10:46:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.090+0000 7f3873e00640 1 -- 172.21.15.60:0/1838858418 shutdown_connections 2024-02-26T10:46:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.090+0000 7f3873e00640 1 --2- 172.21.15.60:0/1838858418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3874108970 0x7f3874108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.090+0000 7f3873e00640 1 --2- 172.21.15.60:0/1838858418 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3874109330 0x7f387410fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.090+0000 7f3873e00640 1 --2- 172.21.15.60:0/1838858418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3874107780 0x7f3874107ba0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.090+0000 7f3873e00640 1 -- 172.21.15.60:0/1838858418 >> 172.21.15.60:0/1838858418 conn(0x7f3874076730 msgr2=0x7f3874078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.090+0000 7f3873e00640 1 -- 172.21.15.60:0/1838858418 shutdown_connections 2024-02-26T10:46:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.090+0000 7f3873e00640 1 -- 172.21.15.60:0/1838858418 wait complete. 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.091+0000 7f3873e00640 1 Processor -- start 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.091+0000 7f3873e00640 1 -- start start 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3873e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3874107780 0x7f387419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3873e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3874108970 0x7f387419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3873e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3874109330 0x7f38741a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3873e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38741133f0 con 0x7f3874108970 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3873e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3874113270 con 0x7f3874107780 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3873e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3874113570 con 0x7f3874109330 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3872000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3874108970 0x7f387419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3872000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3874108970 0x7f387419e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53738/0 (socket says 172.21.15.60:53738) 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3872000640 1 -- 172.21.15.60:0/1917119744 learned_addr learned my addr 172.21.15.60:0/1917119744 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3873400640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3874109330 0x7f38741a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.092+0000 7f3872a00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3874107780 0x7f387419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.093+0000 7f3872a00640 1 -- 172.21.15.60:0/1917119744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3874109330 msgr2=0x7f38741a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.093+0000 7f3872a00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3874109330 0x7f38741a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.093+0000 7f3872a00640 1 -- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3874108970 msgr2=0x7f387419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.093+0000 7f3872a00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3874108970 0x7f387419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.093+0000 7f3872a00640 1 -- 172.21.15.60:0/1917119744 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38741a56c0 con 0x7f3874107780 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.093+0000 7f3872000640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3874108970 0x7f387419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.093+0000 7f3873400640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3874109330 0x7f38741a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.093+0000 7f3872a00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3874107780 0x7f387419daf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f386402c170 tx=0x7f386402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.093+0000 7f386b400640 1 -- 172.21.15.60:0/1917119744 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864041070 con 0x7f3874107780 2024-02-26T10:46:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.094+0000 7f3873e00640 1 -- 172.21.15.60:0/1917119744 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38741a5950 con 0x7f3874107780 2024-02-26T10:46:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.094+0000 7f386b400640 1 -- 172.21.15.60:0/1917119744 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3864004450 con 0x7f3874107780 2024-02-26T10:46:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.094+0000 7f386b400640 1 -- 172.21.15.60:0/1917119744 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864034440 con 0x7f3874107780 2024-02-26T10:46:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.094+0000 7f3873e00640 1 -- 172.21.15.60:0/1917119744 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38741a5e60 con 0x7f3874107780 2024-02-26T10:46:22.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.097+0000 7f387cbce740 1 Processor -- start 2024-02-26T10:46:22.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.097+0000 7f387cbce740 1 -- start start 2024-02-26T10:46:22.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f387cbce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73760e40 0x556a73761260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f387cbce740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 0x556a73766330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f387cbce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 0x556a73768e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f387cbce740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a736c5d30 con 0x556a73761830 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f387cbce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7366ecc0 con 0x556a737669a0 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f387cbce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a736c1dd0 con 0x556a73760e40 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f3873400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73760e40 0x556a73761260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f3872000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 0x556a73768e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f3873400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73760e40 0x556a73761260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49096/0 (socket says 172.21.15.60:49096) 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f3873400640 1 -- 172.21.15.60:0/4219693725 learned_addr learned my addr 172.21.15.60:0/4219693725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.098+0000 7f386b400640 1 -- 172.21.15.60:0/1917119744 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f3864044070 con 0x7f3874107780 2024-02-26T10:46:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.099+0000 7f3872000640 1 -- 172.21.15.60:0/4219693725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73760e40 msgr2=0x556a73761260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.099+0000 7f3872000640 1 --2- 172.21.15.60:0/4219693725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73760e40 0x556a73761260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.099+0000 7f3872000640 1 -- 172.21.15.60:0/4219693725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 msgr2=0x556a73766330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.099+0000 7f3872000640 1 --2- 172.21.15.60:0/4219693725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 0x556a73766330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.099+0000 7f3872000640 1 -- 172.21.15.60:0/4219693725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a733da3f0 con 0x556a737669a0 2024-02-26T10:46:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.099+0000 7f3872000640 1 --2- 172.21.15.60:0/4219693725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 0x556a73768e30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f387419efb0 tx=0x7f38600119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=412f8fdd2e5ef658 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.099+0000 7f385fe00640 1 -- 172.21.15.60:0/4219693725 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386001e070 con 0x556a737669a0 2024-02-26T10:46:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.099+0000 7f385fe00640 1 -- 172.21.15.60:0/4219693725 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f386000faa0 con 0x556a737669a0 2024-02-26T10:46:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.099+0000 7f385fe00640 1 -- 172.21.15.60:0/4219693725 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860019ca0 con 0x556a737669a0 2024-02-26T10:46:22.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f386b400640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f38580a1e90 0x7f38580a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f387cbce740 1 -- 172.21.15.60:0/4219693725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 msgr2=0x556a73768e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f387cbce740 1 --2- 172.21.15.60:0/4219693725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 0x556a73768e30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f387419efb0 tx=0x7f38600119a0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f387cbce740 1 -- 172.21.15.60:0/4219693725 shutdown_connections 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f386b400640 1 -- 172.21.15.60:0/1917119744 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f38640efcf0 con 0x7f3874107780 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f387cbce740 1 --2- 172.21.15.60:0/4219693725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 0x556a73768e30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f387cbce740 1 --2- 172.21.15.60:0/4219693725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 0x556a73766330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f387cbce740 1 --2- 172.21.15.60:0/4219693725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73760e40 0x556a73761260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f387cbce740 1 -- 172.21.15.60:0/4219693725 >> 172.21.15.60:0/4219693725 conn(0x556a7375c3b0 msgr2=0x556a7375c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f3873400640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f38580a1e90 0x7f38580a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.100+0000 7f387cbce740 1 -- 172.21.15.60:0/4219693725 shutdown_connections 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.101+0000 7f387cbce740 1 -- 172.21.15.60:0/4219693725 wait complete. 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.101+0000 7f387cbce740 1 -- start start 2024-02-26T10:46:22.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.101+0000 7f3873400640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f38580a1e90 0x7f38580a4350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f386c005fd0 tx=0x7f386c0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.101+0000 7f387cbce740 1 Processor -- start 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f387cbce740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a7376b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f387cbce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73761830 0x556a7376b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f387cbce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 0x556a73772900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f387cbce740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a73769950 con 0x556a73760e40 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f387cbce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a736641f0 con 0x556a737669a0 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f387cbce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a73769ad0 con 0x556a73761830 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f387cbce740 10 client.0 _ll_register_callbacks cb 0x556a733d9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f3872a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a7376b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f3872a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a7376b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53756/0 (socket says 172.21.15.60:53756) 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f3872a00640 1 -- 172.21.15.60:0/2578073125 learned_addr learned my addr 172.21.15.60:0/2578073125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f3873400640 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 0x556a73772900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.102+0000 7f3872000640 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73761830 0x556a7376b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f3872a00640 1 -- 172.21.15.60:0/2578073125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73761830 msgr2=0x556a7376b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f3872a00640 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73761830 0x556a7376b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f3872a00640 1 -- 172.21.15.60:0/2578073125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 msgr2=0x556a73772900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f3872a00640 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 0x556a73772900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f3872a00640 1 -- 172.21.15.60:0/2578073125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a734a47d0 con 0x556a73760e40 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f3872000640 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73761830 0x556a7376b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f3872a00640 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a7376b380 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f386402c7b0 tx=0x7f38640bc1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8cf36af54f8583 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f385cc00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f385cc00640 1 -- 172.21.15.60:0/2578073125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386404b040 con 0x556a73760e40 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f385cc00640 1 -- 172.21.15.60:0/2578073125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f38640ed4e0 con 0x556a73760e40 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f387cbce740 1 -- 172.21.15.60:0/2578073125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f386000faa0 con 0x556a73760e40 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f385cc00640 1 -- 172.21.15.60:0/2578073125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38640c0710 con 0x556a73760e40 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f387cbce740 10 client.15516 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.103+0000 7f387cbce740 1 -- 172.21.15.60:0/2578073125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556a73775c70 con 0x556a73760e40 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.104+0000 7f385cc00640 1 -- 172.21.15.60:0/2578073125 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f386000faa0 con 0x556a73760e40 2024-02-26T10:46:22.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.104+0000 7f385cc00640 1 -- 172.21.15.60:0/2578073125 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f38640c48d0 con 0x556a73760e40 2024-02-26T10:46:22.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.104+0000 7f387cbce740 10 client.15516 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:22.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.104+0000 7f387cbce740 10 client.15516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-26T10:46:22.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.104+0000 7f387cbce740 20 client.15516 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:22.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.104+0000 7f387cbce740 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x556a73777280 0x556a737796b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.104+0000 7f387cbce740 4 client.15516 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:22.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.104+0000 7f387cbce740 1 -- 172.21.15.60:0/2578073125 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a736d3270 con 0x556a73777280 2024-02-26T10:46:22.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.105+0000 7f3872000640 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x556a73777280 0x556a737796b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.105+0000 7f3872000640 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x556a73777280 0x556a737796b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f2c3154e00585ab server_cookie=2c3da958d77c2f68 in_seq=0 out_seq=0 2024-02-26T10:46:22.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.105+0000 7f385cc00640 10 client.15516 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:46:22.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f385cc00640 1 -- 172.21.15.60:0/2578073125 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a736d3270 con 0x556a73777280 2024-02-26T10:46:22.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f385cc00640 10 client.15516 handle_command_reply: tid=0 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 client.15516 shutdown 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 -- 172.21.15.60:0/2578073125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 msgr2=0x556a7376b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a7376b380 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f386402c7b0 tx=0x7f38640bc1b0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 -- 172.21.15.60:0/2578073125 shutdown_connections 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x556a73777280 0x556a737796b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a737669a0 0x556a73772900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a73761830 0x556a7376b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 --2- 172.21.15.60:0/2578073125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a7376b380 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f385cc00640 0 client.15516 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 -- 172.21.15.60:0/2578073125 >> 172.21.15.60:0/2578073125 conn(0x556a7375c3b0 msgr2=0x556a7375b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 -- 172.21.15.60:0/2578073125 shutdown_connections 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.106+0000 7f387cbce740 1 -- 172.21.15.60:0/2578073125 wait complete. 2024-02-26T10:46:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.107+0000 7f387cbce740 20 client.15516 trim_cache size 0 max 16384 2024-02-26T10:46:22.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:22 smithi060 ceph-mon[31539]: pgmap v437: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-26T10:46:22.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:22 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/577560590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:46:22.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f387cbce740 1 Processor -- start 2024-02-26T10:46:22.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f387cbce740 1 -- start start 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f387cbce740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a73762d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f387cbce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73761830 0x556a737632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f387cbce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 0x556a73772890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f387cbce740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38741b25a0 con 0x556a73760e40 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f387cbce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38741b0140 con 0x556a73761830 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f387cbce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38741ae1e0 con 0x556a737669a0 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f3872a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a73762d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f3872a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a73762d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53768/0 (socket says 172.21.15.60:53768) 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f3872a00640 1 -- 172.21.15.60:0/1603677505 learned_addr learned my addr 172.21.15.60:0/1603677505 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.120+0000 7f3872000640 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73761830 0x556a737632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f3873400640 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 0x556a73772890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f3872a00640 1 -- 172.21.15.60:0/1603677505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 msgr2=0x556a73772890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f3872a00640 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 0x556a73772890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f3872a00640 1 -- 172.21.15.60:0/1603677505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73761830 msgr2=0x556a737632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f3872a00640 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73761830 0x556a737632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f3872a00640 1 -- 172.21.15.60:0/1603677505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a7375d120 con 0x556a73760e40 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f3872000640 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73761830 0x556a737632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f3872a00640 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a73762d80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f38640c4830 tx=0x7f38640b8250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaae49e28638e53 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f385f400640 1 -- 172.21.15.60:0/1603677505 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386404b070 con 0x556a73760e40 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f385f400640 1 -- 172.21.15.60:0/1603677505 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38640bbdc0 con 0x556a73760e40 2024-02-26T10:46:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.121+0000 7f385f400640 1 -- 172.21.15.60:0/1603677505 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864004ff0 con 0x556a73760e40 2024-02-26T10:46:22.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 -- 172.21.15.60:0/1603677505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 msgr2=0x556a73762d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a73762d80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f38640c4830 tx=0x7f38640b8250 comp rx=0 tx=0).stop 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 -- 172.21.15.60:0/1603677505 shutdown_connections 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73761830 0x556a737632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 0x556a73772890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 --2- 172.21.15.60:0/1603677505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73760e40 0x556a73762d80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 -- 172.21.15.60:0/1603677505 >> 172.21.15.60:0/1603677505 conn(0x556a7375c3b0 msgr2=0x556a7377e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 -- 172.21.15.60:0/1603677505 shutdown_connections 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 -- 172.21.15.60:0/1603677505 wait complete. 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.122+0000 7f387cbce740 1 -- start start 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f387cbce740 1 Processor -- start 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f387cbce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73760e40 0x556a73772410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f387cbce740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 0x556a73772950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f387cbce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 0x556a73770af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f387cbce740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38741ae7e0 con 0x556a73761830 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f387cbce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38741afae0 con 0x556a73760e40 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f387cbce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38741aee20 con 0x556a737669a0 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f387cbce740 10 client.0 _ll_register_callbacks cb 0x7f3860009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f3872000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 0x556a73772950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f3873400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 0x556a73770af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f3872000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 0x556a73772950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53784/0 (socket says 172.21.15.60:53784) 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.123+0000 7f3872000640 1 -- 172.21.15.60:0/2805470035 learned_addr learned my addr 172.21.15.60:0/2805470035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.124+0000 7f3872a00640 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73760e40 0x556a73772410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.124+0000 7f3872000640 1 -- 172.21.15.60:0/2805470035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 msgr2=0x556a73770af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.124+0000 7f3872000640 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 0x556a73770af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.124+0000 7f3872000640 1 -- 172.21.15.60:0/2805470035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73760e40 msgr2=0x556a73772410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.124+0000 7f3872000640 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73760e40 0x556a73772410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.124+0000 7f3872000640 1 -- 172.21.15.60:0/2805470035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a73771320 con 0x556a73761830 2024-02-26T10:46:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.124+0000 7f3872000640 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 0x556a73772950 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f386001c040 tx=0x7f3860019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92b69da5dbac780 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.124+0000 7f385cc00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.124+0000 7f385cc00640 1 -- 172.21.15.60:0/2805470035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386001e040 con 0x556a73761830 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f385cc00640 1 -- 172.21.15.60:0/2805470035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f3860016040 con 0x556a73761830 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f387cbce740 1 -- 172.21.15.60:0/2805470035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38640bbdc0 con 0x556a73761830 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f385cc00640 1 -- 172.21.15.60:0/2805470035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3860037050 con 0x556a73761830 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f387cbce740 10 client.15522 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f387cbce740 1 -- 172.21.15.60:0/2805470035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556a737ce3f0 con 0x556a73761830 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f385cc00640 1 -- 172.21.15.60:0/2805470035 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38640bbdc0 con 0x556a73761830 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f385cc00640 1 -- 172.21.15.60:0/2805470035 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f38600051e0 con 0x556a73761830 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f387cbce740 10 client.15522 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f387cbce740 10 client.15522 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f387cbce740 20 client.15522 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f387cbce740 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x556a737cec00 0x556a737d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f387cbce740 4 client.15522 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:46:22.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.125+0000 7f387cbce740 1 -- 172.21.15.60:0/2805470035 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a736cd5c0 con 0x556a737cec00 2024-02-26T10:46:22.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.126+0000 7f3872a00640 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x556a737cec00 0x556a737d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.127+0000 7f3872a00640 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x556a737cec00 0x556a737d1020 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21c2d1d173af3f57 server_cookie=5ed0c030bab20905 in_seq=0 out_seq=0 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.127+0000 7f385cc00640 10 client.15522 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.127+0000 7f385cc00640 1 -- 172.21.15.60:0/2805470035 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a736cd5c0 con 0x556a737cec00 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.127+0000 7f385cc00640 10 client.15522 handle_command_reply: tid=0 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.127+0000 7f387cbce740 1 client.15522 shutdown 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.127+0000 7f387cbce740 1 -- 172.21.15.60:0/2805470035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 msgr2=0x556a73772950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.127+0000 7f387cbce740 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 0x556a73772950 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f386001c040 tx=0x7f3860019410 comp rx=0 tx=0).stop 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f387cbce740 1 -- 172.21.15.60:0/2805470035 shutdown_connections 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f387cbce740 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x556a737cec00 0x556a737d1020 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f387cbce740 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a737669a0 0x556a73770af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f387cbce740 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556a73761830 0x556a73772950 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f387cbce740 1 --2- 172.21.15.60:0/2805470035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a73760e40 0x556a73772410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f385cc00640 0 client.15522 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:46:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f387cbce740 1 -- 172.21.15.60:0/2805470035 >> 172.21.15.60:0/2805470035 conn(0x556a7375c3b0 msgr2=0x556a73762d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f387cbce740 1 -- 172.21.15.60:0/2805470035 shutdown_connections 2024-02-26T10:46:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f387cbce740 1 -- 172.21.15.60:0/2805470035 wait complete. 2024-02-26T10:46:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f387cbce740 20 client.15522 trim_cache size 0 max 16384 2024-02-26T10:46:22.126 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:46:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f3873e00640 1 -- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f38580a1e90 msgr2=0x7f38580a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.128+0000 7f3873e00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f38580a1e90 0x7f38580a4350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f386c005fd0 tx=0x7f386c0096a0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 -- 172.21.15.60:0/1917119744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3874107780 msgr2=0x7f387419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3874107780 0x7f387419daf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f386402c170 tx=0x7f386402c6c0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 -- 172.21.15.60:0/1917119744 shutdown_connections 2024-02-26T10:46:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f38580a1e90 0x7f38580a4350 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3874109330 0x7f38741a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3874108970 0x7f387419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 --2- 172.21.15.60:0/1917119744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3874107780 0x7f387419daf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 -- 172.21.15.60:0/1917119744 >> 172.21.15.60:0/1917119744 conn(0x7f3874076730 msgr2=0x7f387410bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 -- 172.21.15.60:0/1917119744 shutdown_connections 2024-02-26T10:46:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.129+0000 7f3873e00640 1 -- 172.21.15.60:0/1917119744 wait complete. 2024-02-26T10:46:22.143 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:46:22.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T10:46:22.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.371+0000 7f4772200640 1 -- 172.21.15.60:0/3862044352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c108970 msgr2=0x7f476c108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.371+0000 7f4772200640 1 --2- 172.21.15.60:0/3862044352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c108970 0x7f476c108df0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f475c009900 tx=0x7f475c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:46:22.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.371+0000 7f4772200640 1 -- 172.21.15.60:0/3862044352 shutdown_connections 2024-02-26T10:46:22.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.371+0000 7f4772200640 1 --2- 172.21.15.60:0/3862044352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c109330 0x7f476c10fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.371+0000 7f4772200640 1 --2- 172.21.15.60:0/3862044352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c108970 0x7f476c108df0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.371+0000 7f4772200640 1 --2- 172.21.15.60:0/3862044352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c107780 0x7f476c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.371+0000 7f4772200640 1 -- 172.21.15.60:0/3862044352 >> 172.21.15.60:0/3862044352 conn(0x7f476c076730 msgr2=0x7f476c078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.371+0000 7f4772200640 1 -- 172.21.15.60:0/3862044352 shutdown_connections 2024-02-26T10:46:22.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.371+0000 7f4772200640 1 -- 172.21.15.60:0/3862044352 wait complete. 2024-02-26T10:46:22.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.373+0000 7f4772200640 1 Processor -- start 2024-02-26T10:46:22.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.373+0000 7f4772200640 1 -- start start 2024-02-26T10:46:22.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.373+0000 7f4772200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c107780 0x7f476c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.373+0000 7f4772200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c108970 0x7f476c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.373+0000 7f4772200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c109330 0x7f476c1a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.373+0000 7f4772200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c113450 con 0x7f476c108970 2024-02-26T10:46:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.373+0000 7f4772200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1132d0 con 0x7f476c107780 2024-02-26T10:46:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.373+0000 7f4772200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1135d0 con 0x7f476c109330 2024-02-26T10:46:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f476be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c108970 0x7f476c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f476be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c108970 0x7f476c19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53798/0 (socket says 172.21.15.60:53798) 2024-02-26T10:46:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f476be00640 1 -- 172.21.15.60:0/1888981903 learned_addr learned my addr 172.21.15.60:0/1888981903 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f4771800640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c109330 0x7f476c1a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f4770e00640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c107780 0x7f476c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f476be00640 1 -- 172.21.15.60:0/1888981903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c109330 msgr2=0x7f476c1a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f476be00640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c109330 0x7f476c1a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f476be00640 1 -- 172.21.15.60:0/1888981903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c107780 msgr2=0x7f476c19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f476be00640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c107780 0x7f476c19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.374+0000 7f476be00640 1 -- 172.21.15.60:0/1888981903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c1a5650 con 0x7f476c108970 2024-02-26T10:46:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.375+0000 7f476be00640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c108970 0x7f476c19df70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f475c000c00 tx=0x7f475c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.375+0000 7f4769600640 1 -- 172.21.15.60:0/1888981903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c043020 con 0x7f476c108970 2024-02-26T10:46:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.375+0000 7f4769600640 1 -- 172.21.15.60:0/1888981903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475c0054e0 con 0x7f476c108970 2024-02-26T10:46:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.375+0000 7f4772200640 1 -- 172.21.15.60:0/1888981903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c1a58e0 con 0x7f476c108970 2024-02-26T10:46:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.375+0000 7f4769600640 1 -- 172.21.15.60:0/1888981903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c003c40 con 0x7f476c108970 2024-02-26T10:46:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.375+0000 7f4772200640 1 -- 172.21.15.60:0/1888981903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c1a5e80 con 0x7f476c108970 2024-02-26T10:46:22.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.379+0000 7f477666d740 1 Processor -- start 2024-02-26T10:46:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f477666d740 1 -- start start 2024-02-26T10:46:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f4769600640 1 -- 172.21.15.60:0/1888981903 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f475c046020 con 0x7f476c108970 2024-02-26T10:46:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f477666d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb54a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f477666d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb54f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f477666d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb551e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f477666d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcb4aed30 con 0x562fcb54a830 2024-02-26T10:46:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f477666d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcb457cc0 con 0x562fcb54f9a0 2024-02-26T10:46:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f477666d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcb4aadd0 con 0x562fcb549e40 2024-02-26T10:46:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f476be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb551e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f4771800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb54a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f4770e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb54f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.380+0000 7f4771800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb54a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49122/0 (socket says 172.21.15.60:49122) 2024-02-26T10:46:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.381+0000 7f4771800640 1 -- 172.21.15.60:0/1519731006 learned_addr learned my addr 172.21.15.60:0/1519731006 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:22 smithi100 ceph-mon[36051]: pgmap v437: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-26T10:46:22.377 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:22 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/577560590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.381+0000 7f4771800640 1 -- 172.21.15.60:0/1519731006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 msgr2=0x562fcb551e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.381+0000 7f4771800640 1 --2- 172.21.15.60:0/1519731006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb551e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.381+0000 7f4771800640 1 -- 172.21.15.60:0/1519731006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 msgr2=0x562fcb54f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.381+0000 7f4771800640 1 --2- 172.21.15.60:0/1519731006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb54f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.381+0000 7f4771800640 1 -- 172.21.15.60:0/1519731006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fcb1c33f0 con 0x562fcb549e40 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.381+0000 7f4770e00640 1 --2- 172.21.15.60:0/1519731006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb54f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.381+0000 7f4771800640 1 --2- 172.21.15.60:0/1519731006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb54a260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4764009fd0 tx=0x7f47640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40daaa72a470f15 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.382+0000 7f4769600640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47540a1e90 0x7f47540a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.382+0000 7f475a000640 1 -- 172.21.15.60:0/1519731006 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764004d20 con 0x562fcb549e40 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.382+0000 7f475a000640 1 -- 172.21.15.60:0/1519731006 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4764004050 con 0x562fcb549e40 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.382+0000 7f475a000640 1 -- 172.21.15.60:0/1519731006 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764005260 con 0x562fcb549e40 2024-02-26T10:46:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.382+0000 7f4769600640 1 -- 172.21.15.60:0/1888981903 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f475c0f0190 con 0x7f476c108970 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.382+0000 7f4770e00640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47540a1e90 0x7f47540a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.382+0000 7f477666d740 1 -- 172.21.15.60:0/1519731006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 msgr2=0x562fcb54a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.382+0000 7f477666d740 1 --2- 172.21.15.60:0/1519731006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb54a260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4764009fd0 tx=0x7f47640118d0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.383+0000 7f477666d740 1 -- 172.21.15.60:0/1519731006 shutdown_connections 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.383+0000 7f4770e00640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47540a1e90 0x7f47540a4350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x562fcb54ac30 tx=0x7f4760005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.383+0000 7f477666d740 1 --2- 172.21.15.60:0/1519731006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb551e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.383+0000 7f477666d740 1 --2- 172.21.15.60:0/1519731006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb54f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.383+0000 7f477666d740 1 --2- 172.21.15.60:0/1519731006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb54a260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.383+0000 7f477666d740 1 -- 172.21.15.60:0/1519731006 >> 172.21.15.60:0/1519731006 conn(0x562fcb5453b0 msgr2=0x562fcb5457b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.383+0000 7f477666d740 1 -- 172.21.15.60:0/1519731006 shutdown_connections 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.383+0000 7f477666d740 1 -- 172.21.15.60:0/1519731006 wait complete. 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.383+0000 7f477666d740 1 -- start start 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f477666d740 1 Processor -- start 2024-02-26T10:46:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f477666d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb549e40 0x562fcb554380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f477666d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54a830 0x562fcb5548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f477666d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb54f9a0 0x562fcb55b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f477666d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcb552950 con 0x562fcb549e40 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f477666d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcb44d1f0 con 0x562fcb54a830 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f477666d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcb552ad0 con 0x562fcb54f9a0 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f477666d740 10 client.0 _ll_register_callbacks cb 0x562fcb1c2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f4770e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb549e40 0x562fcb554380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f476be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54a830 0x562fcb5548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f4771800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb54f9a0 0x562fcb55b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f4770e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb549e40 0x562fcb554380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53822/0 (socket says 172.21.15.60:53822) 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.384+0000 7f4770e00640 1 -- 172.21.15.60:0/2371535230 learned_addr learned my addr 172.21.15.60:0/2371535230 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.385+0000 7f4771800640 1 -- 172.21.15.60:0/2371535230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54a830 msgr2=0x562fcb5548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.385+0000 7f4771800640 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54a830 0x562fcb5548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.385+0000 7f4771800640 1 -- 172.21.15.60:0/2371535230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb549e40 msgr2=0x562fcb554380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.385+0000 7f4771800640 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb549e40 0x562fcb554380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.385+0000 7f4771800640 1 -- 172.21.15.60:0/2371535230 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fcb28d7d0 con 0x562fcb54f9a0 2024-02-26T10:46:22.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.385+0000 7f476be00640 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54a830 0x562fcb5548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:22.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.385+0000 7f4771800640 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb54f9a0 0x562fcb55b900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f476401b040 tx=0x7f4764004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3025251c6cbb046c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.385+0000 7f474ea00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:46:22.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.386+0000 7f474ea00640 1 -- 172.21.15.60:0/2371535230 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764028020 con 0x562fcb54f9a0 2024-02-26T10:46:22.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.386+0000 7f474ea00640 1 -- 172.21.15.60:0/2371535230 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f4764004410 con 0x562fcb54f9a0 2024-02-26T10:46:22.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.386+0000 7f477666d740 1 -- 172.21.15.60:0/2371535230 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4764004050 con 0x562fcb54f9a0 2024-02-26T10:46:22.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.386+0000 7f474ea00640 1 -- 172.21.15.60:0/2371535230 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4764033050 con 0x562fcb54f9a0 2024-02-26T10:46:22.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.386+0000 7f477666d740 10 client.25657 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.386+0000 7f477666d740 1 -- 172.21.15.60:0/2371535230 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562fcb55ec70 con 0x562fcb54f9a0 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.386+0000 7f474ea00640 1 -- 172.21.15.60:0/2371535230 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4764004050 con 0x562fcb54f9a0 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.387+0000 7f474ea00640 1 -- 172.21.15.60:0/2371535230 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f476400f4e0 con 0x562fcb54f9a0 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.387+0000 7f477666d740 10 client.25657 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.387+0000 7f477666d740 10 client.25657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.387+0000 7f477666d740 20 client.25657 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.387+0000 7f477666d740 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x562fcb560170 0x562fcb562560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.387+0000 7f477666d740 4 client.25657 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.387+0000 7f477666d740 1 -- 172.21.15.60:0/2371535230 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fcb4bc270 con 0x562fcb560170 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.387+0000 7f4770e00640 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x562fcb560170 0x562fcb562560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.388+0000 7f4770e00640 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x562fcb560170 0x562fcb562560 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43ac9bd763a55d29 server_cookie=e7a1c8f63e9db8d7 in_seq=0 out_seq=0 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.388+0000 7f474ea00640 10 client.25657 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.388+0000 7f474ea00640 1 -- 172.21.15.60:0/2371535230 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562fcb4bc270 con 0x562fcb560170 2024-02-26T10:46:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.388+0000 7f474ea00640 10 client.25657 handle_command_reply: tid=0 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 client.25657 shutdown 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 -- 172.21.15.60:0/2371535230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb54f9a0 msgr2=0x562fcb55b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb54f9a0 0x562fcb55b900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f476401b040 tx=0x7f4764004cf0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 -- 172.21.15.60:0/2371535230 shutdown_connections 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54a830 0x562fcb5548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x562fcb560170 0x562fcb562560 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb54f9a0 0x562fcb55b900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 --2- 172.21.15.60:0/2371535230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb549e40 0x562fcb554380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 -- 172.21.15.60:0/2371535230 >> 172.21.15.60:0/2371535230 conn(0x562fcb5453b0 msgr2=0x562fcb5446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f474ea00640 0 client.25657 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 -- 172.21.15.60:0/2371535230 shutdown_connections 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 1 -- 172.21.15.60:0/2371535230 wait complete. 2024-02-26T10:46:22.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.389+0000 7f477666d740 20 client.25657 trim_cache size 0 max 16384 2024-02-26T10:46:22.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.402+0000 7f477666d740 1 Processor -- start 2024-02-26T10:46:22.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.402+0000 7f477666d740 1 -- start start 2024-02-26T10:46:22.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.402+0000 7f477666d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb554380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.402+0000 7f477666d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.402+0000 7f477666d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb5548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.402+0000 7f477666d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1b2640 con 0x562fcb54a830 2024-02-26T10:46:22.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.402+0000 7f477666d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1b01e0 con 0x562fcb54f9a0 2024-02-26T10:46:22.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.402+0000 7f477666d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1ae280 con 0x562fcb549e40 2024-02-26T10:46:22.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f476be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f476be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53828/0 (socket says 172.21.15.60:53828) 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f476be00640 1 -- 172.21.15.60:0/4003891594 learned_addr learned my addr 172.21.15.60:0/4003891594 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f4771800640 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb5548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f4770e00640 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb554380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f476be00640 1 -- 172.21.15.60:0/4003891594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 msgr2=0x562fcb554380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f476be00640 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb554380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f476be00640 1 -- 172.21.15.60:0/4003891594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 msgr2=0x562fcb5548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f476be00640 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb5548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f476be00640 1 -- 172.21.15.60:0/4003891594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fcb54efd0 con 0x562fcb54a830 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.403+0000 7f4771800640 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb5548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.404+0000 7f476be00640 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55b8c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b9490 tx=0x7f475c0f3ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e93aa8d8718422 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.404+0000 7f4759600640 1 -- 172.21.15.60:0/4003891594 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0b8d60 con 0x562fcb54a830 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.404+0000 7f4759600640 1 -- 172.21.15.60:0/4003891594 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475c004250 con 0x562fcb54a830 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.404+0000 7f4759600640 1 -- 172.21.15.60:0/4003891594 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0bc420 con 0x562fcb54a830 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.404+0000 7f477666d740 1 -- 172.21.15.60:0/4003891594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 msgr2=0x562fcb55b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55b8c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b9490 tx=0x7f475c0f3ce0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 -- 172.21.15.60:0/4003891594 shutdown_connections 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb5548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55b8c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 --2- 172.21.15.60:0/4003891594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb554380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 -- 172.21.15.60:0/4003891594 >> 172.21.15.60:0/4003891594 conn(0x562fcb5453b0 msgr2=0x562fcb54bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 -- 172.21.15.60:0/4003891594 shutdown_connections 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 -- 172.21.15.60:0/4003891594 wait complete. 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 -- start start 2024-02-26T10:46:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.405+0000 7f477666d740 1 Processor -- start 2024-02-26T10:46:22.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f477666d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb55a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f477666d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f477666d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb55abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f477666d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1ae880 con 0x562fcb54a830 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f477666d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1afb80 con 0x562fcb54f9a0 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f477666d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1aeec0 con 0x562fcb549e40 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f477666d740 10 client.0 _ll_register_callbacks cb 0x7f4764009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f476be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f476be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53842/0 (socket says 172.21.15.60:53842) 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f476be00640 1 -- 172.21.15.60:0/3450354735 learned_addr learned my addr 172.21.15.60:0/3450354735 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f4770e00640 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb55a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.406+0000 7f4771800640 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb55abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f476be00640 1 -- 172.21.15.60:0/3450354735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 msgr2=0x562fcb55a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f476be00640 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb55a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f476be00640 1 -- 172.21.15.60:0/3450354735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 msgr2=0x562fcb55abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f476be00640 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb55abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f476be00640 1 -- 172.21.15.60:0/3450354735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fcb5b66c0 con 0x562fcb54a830 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f4770e00640 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb55a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f476be00640 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55a670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b9440 tx=0x7f475c0ed810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edea5dd833c86e9d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f474ea00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f474ea00640 1 -- 172.21.15.60:0/3450354735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0bc600 con 0x562fcb54a830 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f474ea00640 1 -- 172.21.15.60:0/3450354735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f475c004430 con 0x562fcb54a830 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f477666d740 1 -- 172.21.15.60:0/3450354735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fcb4b65c0 con 0x562fcb54a830 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f474ea00640 1 -- 172.21.15.60:0/3450354735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475c0bc890 con 0x562fcb54a830 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f477666d740 10 client.15540 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.407+0000 7f477666d740 1 -- 172.21.15.60:0/3450354735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562fcb5b79c0 con 0x562fcb54a830 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.408+0000 7f474ea00640 1 -- 172.21.15.60:0/3450354735 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562fcb4b65c0 con 0x562fcb54a830 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.408+0000 7f474ea00640 1 -- 172.21.15.60:0/3450354735 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f475c0ed920 con 0x562fcb54a830 2024-02-26T10:46:22.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.408+0000 7f477666d740 10 client.15540 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.408+0000 7f477666d740 10 client.15540 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.408+0000 7f477666d740 20 client.15540 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.408+0000 7f477666d740 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x562fcb5b8270 0x562fcb5ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.408+0000 7f477666d740 4 client.15540 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.408+0000 7f477666d740 1 -- 172.21.15.60:0/3450354735 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562fcb4344d0 con 0x562fcb5b8270 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.408+0000 7f4770e00640 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x562fcb5b8270 0x562fcb5ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f4770e00640 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x562fcb5b8270 0x562fcb5ba690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=329f66d833b4f9a9 server_cookie=75fa900152202632 in_seq=0 out_seq=0 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f474ea00640 10 client.15540 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f474ea00640 1 -- 172.21.15.60:0/3450354735 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562fcb4344d0 con 0x562fcb5b8270 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f474ea00640 10 client.15540 handle_command_reply: tid=0 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f477666d740 1 client.15540 shutdown 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f477666d740 1 -- 172.21.15.60:0/3450354735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 msgr2=0x562fcb55a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f477666d740 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55a670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b9440 tx=0x7f475c0ed810 comp rx=0 tx=0).stop 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f477666d740 1 -- 172.21.15.60:0/3450354735 shutdown_connections 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f477666d740 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcb54f9a0 0x562fcb55abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f477666d740 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x562fcb5b8270 0x562fcb5ba690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f477666d740 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562fcb54a830 0x562fcb55a670 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f477666d740 1 --2- 172.21.15.60:0/3450354735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562fcb549e40 0x562fcb55a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f474ea00640 0 client.15540 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.409+0000 7f477666d740 1 -- 172.21.15.60:0/3450354735 >> 172.21.15.60:0/3450354735 conn(0x562fcb5453b0 msgr2=0x562fcb554380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.410+0000 7f477666d740 1 -- 172.21.15.60:0/3450354735 shutdown_connections 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.410+0000 7f477666d740 1 -- 172.21.15.60:0/3450354735 wait complete. 2024-02-26T10:46:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.410+0000 7f477666d740 20 client.15540 trim_cache size 0 max 16384 2024-02-26T10:46:22.410 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:46:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.410+0000 7f4772200640 1 -- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47540a1e90 msgr2=0x7f47540a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.410+0000 7f4772200640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47540a1e90 0x7f47540a4350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x562fcb54ac30 tx=0x7f4760005f90 comp rx=0 tx=0).stop 2024-02-26T10:46:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.410+0000 7f4772200640 1 -- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c108970 msgr2=0x7f476c19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.410+0000 7f4772200640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c108970 0x7f476c19df70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f475c000c00 tx=0x7f475c02c650 comp rx=0 tx=0).stop 2024-02-26T10:46:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.411+0000 7f4772200640 1 -- 172.21.15.60:0/1888981903 shutdown_connections 2024-02-26T10:46:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.411+0000 7f4772200640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f47540a1e90 0x7f47540a4350 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.411+0000 7f4772200640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c109330 0x7f476c1a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.411+0000 7f4772200640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c108970 0x7f476c19df70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.411+0000 7f4772200640 1 --2- 172.21.15.60:0/1888981903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c107780 0x7f476c19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.411+0000 7f4772200640 1 -- 172.21.15.60:0/1888981903 >> 172.21.15.60:0/1888981903 conn(0x7f476c076730 msgr2=0x7f476c10bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.411+0000 7f4772200640 1 -- 172.21.15.60:0/1888981903 shutdown_connections 2024-02-26T10:46:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.411+0000 7f4772200640 1 -- 172.21.15.60:0/1888981903 wait complete. 2024-02-26T10:46:22.422 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:46:22.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T10:46:22.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:22 smithi113 ceph-mon[36126]: pgmap v437: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-26T10:46:22.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:22 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/577560590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:46:22.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.632+0000 7f2960c00640 1 -- 172.21.15.60:0/3317796036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 msgr2=0x7f295c1027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.632+0000 7f2960c00640 1 --2- 172.21.15.60:0/3317796036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 0x7f295c1027d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2944009860 tx=0x7f294402bb70 comp rx=0 tx=0).stop 2024-02-26T10:46:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.633+0000 7f2960c00640 1 -- 172.21.15.60:0/3317796036 shutdown_connections 2024-02-26T10:46:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.633+0000 7f2960c00640 1 --2- 172.21.15.60:0/3317796036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 0x7f295c1027d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.633+0000 7f2960c00640 1 --2- 172.21.15.60:0/3317796036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f295c102d10 0x7f295c1095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.633+0000 7f2960c00640 1 --2- 172.21.15.60:0/3317796036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295c101160 0x7f295c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.633+0000 7f2960c00640 1 -- 172.21.15.60:0/3317796036 >> 172.21.15.60:0/3317796036 conn(0x7f295c0fc6d0 msgr2=0x7f295c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.633+0000 7f2960c00640 1 -- 172.21.15.60:0/3317796036 shutdown_connections 2024-02-26T10:46:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.633+0000 7f2960c00640 1 -- 172.21.15.60:0/3317796036 wait complete. 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.634+0000 7f2960c00640 1 Processor -- start 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.634+0000 7f2960c00640 1 -- start start 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.634+0000 7f2960c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295c101160 0x7f295c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.634+0000 7f2960c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 0x7f295c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f2960c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f295c102d10 0x7f295c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f2960c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c10cdf0 con 0x7f295c101160 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f2960c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c10cc70 con 0x7f295c102350 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f2960c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c10cf70 con 0x7f295c102d10 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 0x7f295c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295c101160 0x7f295c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 0x7f295c199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51378/0 (socket says 172.21.15.60:51378) 2024-02-26T10:46:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295aa00640 1 -- 172.21.15.60:0/3108537377 learned_addr learned my addr 172.21.15.60:0/3108537377 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295be00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f295c102d10 0x7f295c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295aa00640 1 -- 172.21.15.60:0/3108537377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f295c102d10 msgr2=0x7f295c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295aa00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f295c102d10 0x7f295c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295aa00640 1 -- 172.21.15.60:0/3108537377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295c101160 msgr2=0x7f295c199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295aa00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295c101160 0x7f295c199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.635+0000 7f295aa00640 1 -- 172.21.15.60:0/3108537377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c1a1250 con 0x7f295c102350 2024-02-26T10:46:22.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.636+0000 7f295aa00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 0x7f295c199b70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2944005940 tx=0x7f2944032470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.636+0000 7f2953e00640 1 -- 172.21.15.60:0/3108537377 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944041020 con 0x7f295c102350 2024-02-26T10:46:22.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.636+0000 7f2960c00640 1 -- 172.21.15.60:0/3108537377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c1a14e0 con 0x7f295c102350 2024-02-26T10:46:22.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.636+0000 7f2953e00640 1 -- 172.21.15.60:0/3108537377 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f294402c9f0 con 0x7f295c102350 2024-02-26T10:46:22.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.636+0000 7f2953e00640 1 -- 172.21.15.60:0/3108537377 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944039040 con 0x7f295c102350 2024-02-26T10:46:22.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.636+0000 7f2960c00640 1 -- 172.21.15.60:0/3108537377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c1a1a20 con 0x7f295c102350 2024-02-26T10:46:22.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.640+0000 7f296522e740 1 Processor -- start 2024-02-26T10:46:22.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.640+0000 7f296522e740 1 -- start start 2024-02-26T10:46:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f296522e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b4765a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f296522e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 0x563b4765f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f296522e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b47661e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f296522e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b475bed30 con 0x563b4765f9a0 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f296522e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b47567cc0 con 0x563b4765a830 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f296522e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b475badd0 con 0x563b47659e40 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f2953e00640 1 -- 172.21.15.60:0/3108537377 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f2944044020 con 0x7f295c102350 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f295be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b4765a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f295b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 0x563b4765f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f295be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b4765a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49164/0 (socket says 172.21.15.60:49164) 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.641+0000 7f295be00640 1 -- 172.21.15.60:0/4048092256 learned_addr learned my addr 172.21.15.60:0/4048092256 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f295aa00640 1 --2- 172.21.15.60:0/4048092256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b47661e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f295be00640 1 -- 172.21.15.60:0/4048092256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 msgr2=0x563b4765f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f295be00640 1 --2- 172.21.15.60:0/4048092256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 0x563b4765f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f295be00640 1 -- 172.21.15.60:0/4048092256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 msgr2=0x563b47661e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f295be00640 1 --2- 172.21.15.60:0/4048092256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b47661e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f295be00640 1 -- 172.21.15.60:0/4048092256 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b472d33f0 con 0x563b47659e40 2024-02-26T10:46:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f295aa00640 1 --2- 172.21.15.60:0/4048092256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b47661e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:46:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f295be00640 1 --2- 172.21.15.60:0/4048092256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b4765a260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2954009fd0 tx=0x7f29540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e42ebd576069fc57 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f294be00640 1 -- 172.21.15.60:0/4048092256 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954004d20 con 0x563b47659e40 2024-02-26T10:46:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f294be00640 1 -- 172.21.15.60:0/4048092256 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2954004050 con 0x563b47659e40 2024-02-26T10:46:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.642+0000 7f294be00640 1 -- 172.21.15.60:0/4048092256 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954005220 con 0x563b47659e40 2024-02-26T10:46:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f2953e00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f29400a1e10 0x7f29400a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f296522e740 1 -- 172.21.15.60:0/4048092256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 msgr2=0x563b4765a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f296522e740 1 --2- 172.21.15.60:0/4048092256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b4765a260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2954009fd0 tx=0x7f29540118d0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f2953e00640 1 -- 172.21.15.60:0/3108537377 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f29440efba0 con 0x7f295c102350 2024-02-26T10:46:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f295b400640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f29400a1e10 0x7f29400a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f296522e740 1 -- 172.21.15.60:0/4048092256 shutdown_connections 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f296522e740 1 --2- 172.21.15.60:0/4048092256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 0x563b4765f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f296522e740 1 --2- 172.21.15.60:0/4048092256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b47661e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f296522e740 1 --2- 172.21.15.60:0/4048092256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b4765a260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f296522e740 1 -- 172.21.15.60:0/4048092256 >> 172.21.15.60:0/4048092256 conn(0x563b476553b0 msgr2=0x563b476557b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f296522e740 1 -- 172.21.15.60:0/4048092256 shutdown_connections 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.643+0000 7f296522e740 1 -- 172.21.15.60:0/4048092256 wait complete. 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.644+0000 7f296522e740 1 -- start start 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.644+0000 7f295b400640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f29400a1e10 0x7f29400a42d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x563b47656320 tx=0x7f294c014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.644+0000 7f296522e740 1 Processor -- start 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.644+0000 7f296522e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47664300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f296522e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 0x563b47664840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f296522e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765f9a0 0x563b4766b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f296522e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b47662950 con 0x563b4765a830 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f296522e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4755d1f0 con 0x563b4765f9a0 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f296522e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b47662ad0 con 0x563b47659e40 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f296522e740 10 client.0 _ll_register_callbacks cb 0x563b472d2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 0x563b47664840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47664300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47664300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:49172/0 (socket says 172.21.15.60:49172) 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295b400640 1 -- 172.21.15.60:0/2845966353 learned_addr learned my addr 172.21.15.60:0/2845966353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295be00640 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765f9a0 0x563b4766b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295be00640 1 -- 172.21.15.60:0/2845966353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 msgr2=0x563b47664300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295be00640 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47664300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295be00640 1 -- 172.21.15.60:0/2845966353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 msgr2=0x563b47664840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295be00640 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 0x563b47664840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295be00640 1 -- 172.21.15.60:0/2845966353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b4739d7d0 con 0x563b4765f9a0 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.645+0000 7f295b400640 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47664300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.646+0000 7f295be00640 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765f9a0 0x563b4766b880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f295401b040 tx=0x7f2954004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e626e2f26ec139 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.646+0000 7f2948c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:46:22.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.646+0000 7f2948c00640 1 -- 172.21.15.60:0/2845966353 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954028020 con 0x563b4765f9a0 2024-02-26T10:46:22.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.646+0000 7f2948c00640 1 -- 172.21.15.60:0/2845966353 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2954004410 con 0x563b4765f9a0 2024-02-26T10:46:22.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.646+0000 7f296522e740 1 -- 172.21.15.60:0/2845966353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2954004050 con 0x563b4765f9a0 2024-02-26T10:46:22.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.646+0000 7f2948c00640 1 -- 172.21.15.60:0/2845966353 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2954033050 con 0x563b4765f9a0 2024-02-26T10:46:22.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.646+0000 7f296522e740 10 client.25670 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:22.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.646+0000 7f296522e740 1 -- 172.21.15.60:0/2845966353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563b4766eb60 con 0x563b4765f9a0 2024-02-26T10:46:22.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.647+0000 7f2948c00640 1 -- 172.21.15.60:0/2845966353 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2954004050 con 0x563b4765f9a0 2024-02-26T10:46:22.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.647+0000 7f2948c00640 1 -- 172.21.15.60:0/2845966353 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f295400f4e0 con 0x563b4765f9a0 2024-02-26T10:46:22.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.647+0000 7f296522e740 10 client.25670 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:22.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.647+0000 7f296522e740 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:22.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.647+0000 7f296522e740 20 client.25670 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:22.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.647+0000 7f296522e740 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x563b476700d0 0x563b476724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.647+0000 7f296522e740 4 client.25670 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:22.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.647+0000 7f296522e740 1 -- 172.21.15.60:0/2845966353 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b475cc270 con 0x563b476700d0 2024-02-26T10:46:22.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.648+0000 7f295b400640 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x563b476700d0 0x563b476724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.648+0000 7f295b400640 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x563b476700d0 0x563b476724c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f2115150d5a8ab6 server_cookie=def645d4f8efa3c3 in_seq=0 out_seq=0 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.648+0000 7f2948c00640 10 client.25670 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f2948c00640 1 -- 172.21.15.60:0/2845966353 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b475cc270 con 0x563b476700d0 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f2948c00640 10 client.25670 handle_command_reply: tid=0 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 client.25670 shutdown 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 -- 172.21.15.60:0/2845966353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765f9a0 msgr2=0x563b4766b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765f9a0 0x563b4766b880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f295401b040 tx=0x7f2954004cf0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 -- 172.21.15.60:0/2845966353 shutdown_connections 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x563b476700d0 0x563b476724c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765f9a0 0x563b4766b880 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 0x563b47664840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 --2- 172.21.15.60:0/2845966353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47664300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 -- 172.21.15.60:0/2845966353 >> 172.21.15.60:0/2845966353 conn(0x563b476553b0 msgr2=0x563b476546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f2948c00640 0 client.25670 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 -- 172.21.15.60:0/2845966353 shutdown_connections 2024-02-26T10:46:22.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 1 -- 172.21.15.60:0/2845966353 wait complete. 2024-02-26T10:46:22.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.649+0000 7f296522e740 20 client.25670 trim_cache size 0 max 16384 2024-02-26T10:46:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.662+0000 7f296522e740 1 Processor -- start 2024-02-26T10:46:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f296522e740 1 -- start start 2024-02-26T10:46:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f296522e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47662e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f296522e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 0x563b476633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f296522e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b4766b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f296522e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1ae260 con 0x563b4765f9a0 2024-02-26T10:46:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f296522e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1abe00 con 0x563b4765a830 2024-02-26T10:46:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f296522e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1a9ea0 con 0x563b47659e40 2024-02-26T10:46:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f295be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b4766b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f295be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b4766b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53910/0 (socket says 172.21.15.60:53910) 2024-02-26T10:46:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f295be00640 1 -- 172.21.15.60:0/3855499340 learned_addr learned my addr 172.21.15.60:0/3855499340 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f295b400640 1 --2- 172.21.15.60:0/3855499340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47662e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.663+0000 7f295aa00640 1 --2- 172.21.15.60:0/3855499340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 0x563b476633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.664+0000 7f295be00640 1 -- 172.21.15.60:0/3855499340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 msgr2=0x563b47662e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.664+0000 7f295be00640 1 --2- 172.21.15.60:0/3855499340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47662e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.664+0000 7f295be00640 1 -- 172.21.15.60:0/3855499340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 msgr2=0x563b476633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.664+0000 7f295be00640 1 --2- 172.21.15.60:0/3855499340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 0x563b476633b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.664+0000 7f295be00640 1 -- 172.21.15.60:0/3855499340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b47662830 con 0x563b4765f9a0 2024-02-26T10:46:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.664+0000 7f295be00640 1 --2- 172.21.15.60:0/3855499340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b4766b840 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2954011e50 tx=0x7f29540021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403a202dcf45de21 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.664+0000 7f294b400640 1 -- 172.21.15.60:0/3855499340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954028070 con 0x563b4765f9a0 2024-02-26T10:46:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.664+0000 7f294b400640 1 -- 172.21.15.60:0/3855499340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f295400f4e0 con 0x563b4765f9a0 2024-02-26T10:46:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.664+0000 7f294b400640 1 -- 172.21.15.60:0/3855499340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954019690 con 0x563b4765f9a0 2024-02-26T10:46:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 -- 172.21.15.60:0/3855499340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 msgr2=0x563b4766b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 --2- 172.21.15.60:0/3855499340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b4766b840 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2954011e50 tx=0x7f29540021d0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 -- 172.21.15.60:0/3855499340 shutdown_connections 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 --2- 172.21.15.60:0/3855499340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b4765a830 0x563b476633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 --2- 172.21.15.60:0/3855499340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765f9a0 0x563b4766b840 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 --2- 172.21.15.60:0/3855499340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b47659e40 0x563b47662e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 -- 172.21.15.60:0/3855499340 >> 172.21.15.60:0/3855499340 conn(0x563b476553b0 msgr2=0x563b47676e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 -- 172.21.15.60:0/3855499340 shutdown_connections 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 -- 172.21.15.60:0/3855499340 wait complete. 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.665+0000 7f296522e740 1 -- start start 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f296522e740 1 Processor -- start 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f296522e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b47659e40 0x563b476693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f296522e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 0x563b47669920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f296522e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b4765f9a0 0x563b4766a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f296522e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1aa4a0 con 0x563b4765a830 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f296522e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1ab7a0 con 0x563b47659e40 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f296522e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1aaae0 con 0x563b4765f9a0 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f296522e740 10 client.0 _ll_register_callbacks cb 0x7f2954009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f295b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b47659e40 0x563b476693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f295b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b47659e40 0x563b476693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51420/0 (socket says 172.21.15.60:51420) 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f295b400640 1 -- 172.21.15.60:0/2048922398 learned_addr learned my addr 172.21.15.60:0/2048922398 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f295be00640 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b4765f9a0 0x563b4766a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.666+0000 7f295aa00640 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 0x563b47669920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f295b400640 1 -- 172.21.15.60:0/2048922398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b4765f9a0 msgr2=0x563b4766a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f295b400640 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b4765f9a0 0x563b4766a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f295b400640 1 -- 172.21.15.60:0/2048922398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 msgr2=0x563b47669920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f295b400640 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 0x563b47669920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f295b400640 1 -- 172.21.15.60:0/2048922398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b4766af20 con 0x563b47659e40 2024-02-26T10:46:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f295aa00640 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 0x563b47669920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:22.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f295b400640 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b47659e40 0x563b476693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f294c005dc0 tx=0x7f294c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=706d531e71245de2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:22.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f2948c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:46:22.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f2948c00640 1 -- 172.21.15.60:0/2048922398 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c032070 con 0x563b47659e40 2024-02-26T10:46:22.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f296522e740 1 -- 172.21.15.60:0/2048922398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f295400f4e0 con 0x563b47659e40 2024-02-26T10:46:22.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.667+0000 7f2948c00640 1 -- 172.21.15.60:0/2048922398 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f294c02b690 con 0x563b47659e40 2024-02-26T10:46:22.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f2948c00640 1 -- 172.21.15.60:0/2048922398 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f294c03e050 con 0x563b47659e40 2024-02-26T10:46:22.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f296522e740 10 client.25676 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:22.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f296522e740 1 -- 172.21.15.60:0/2048922398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563b476c6aa0 con 0x563b47659e40 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f2948c00640 1 -- 172.21.15.60:0/2048922398 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f295400f4e0 con 0x563b47659e40 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f2948c00640 1 -- 172.21.15.60:0/2048922398 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f294c02bce0 con 0x563b47659e40 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f296522e740 10 client.25676 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f296522e740 10 client.25676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f296522e740 20 client.25676 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f296522e740 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x563b476c7430 0x563b476c9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f296522e740 4 client.25676 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.668+0000 7f296522e740 1 -- 172.21.15.60:0/2048922398 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563b475c65c0 con 0x563b476c7430 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.669+0000 7f295aa00640 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x563b476c7430 0x563b476c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:22.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.669+0000 7f295aa00640 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x563b476c7430 0x563b476c9850 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96686804676ac4a0 server_cookie=4cfe5358e62bf84d in_seq=0 out_seq=0 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.669+0000 7f2948c00640 10 client.25676 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f2948c00640 1 -- 172.21.15.60:0/2048922398 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563b475c65c0 con 0x563b476c7430 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f2948c00640 10 client.25676 handle_command_reply: tid=0 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 client.25676 shutdown 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 -- 172.21.15.60:0/2048922398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b47659e40 msgr2=0x563b476693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b47659e40 0x563b476693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f294c005dc0 tx=0x7f294c004730 comp rx=0 tx=0).stop 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 -- 172.21.15.60:0/2048922398 shutdown_connections 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x563b476c7430 0x563b476c9850 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b4765f9a0 0x563b4766a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b4765a830 0x563b47669920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 --2- 172.21.15.60:0/2048922398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b47659e40 0x563b476693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f2948c00640 0 client.25676 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 -- 172.21.15.60:0/2048922398 >> 172.21.15.60:0/2048922398 conn(0x563b476553b0 msgr2=0x563b47658610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 -- 172.21.15.60:0/2048922398 shutdown_connections 2024-02-26T10:46:22.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 1 -- 172.21.15.60:0/2048922398 wait complete. 2024-02-26T10:46:22.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.670+0000 7f296522e740 20 client.25676 trim_cache size 0 max 16384 2024-02-26T10:46:22.671 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:46:22.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 -- 172.21.15.60:0/3108537377 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f29400a1e10 msgr2=0x7f29400a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f29400a1e10 0x7f29400a42d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x563b47656320 tx=0x7f294c014040 comp rx=0 tx=0).stop 2024-02-26T10:46:22.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 -- 172.21.15.60:0/3108537377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 msgr2=0x7f295c199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:22.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 0x7f295c199b70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2944005940 tx=0x7f2944032470 comp rx=0 tx=0).stop 2024-02-26T10:46:22.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 -- 172.21.15.60:0/3108537377 shutdown_connections 2024-02-26T10:46:22.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f29400a1e10 0x7f29400a42d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295c102350 0x7f295c199b70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f295c102d10 0x7f295c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 --2- 172.21.15.60:0/3108537377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295c101160 0x7f295c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:22.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 -- 172.21.15.60:0/3108537377 >> 172.21.15.60:0/3108537377 conn(0x7f295c0fc6d0 msgr2=0x7f295c073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:22.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 -- 172.21.15.60:0/3108537377 shutdown_connections 2024-02-26T10:46:22.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:22.671+0000 7f2960c00640 1 -- 172.21.15.60:0/3108537377 wait complete. 2024-02-26T10:46:23.683 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T10:46:23.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:46:23.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T10:46:23.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.928+0000 7f56cbe00640 1 -- 172.21.15.60:0/1863758637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 msgr2=0x7f56cc1037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:23.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.928+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1863758637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 0x7f56cc1037a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009ca0 tx=0x7f56bc02c100 comp rx=0 tx=0).stop 2024-02-26T10:46:23.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.929+0000 7f56cbe00640 1 -- 172.21.15.60:0/1863758637 shutdown_connections 2024-02-26T10:46:23.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.929+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1863758637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc104570 0x7f56cc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.929+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1863758637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc104f30 0x7f56cc10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.929+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1863758637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 0x7f56cc1037a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.929+0000 7f56cbe00640 1 -- 172.21.15.60:0/1863758637 >> 172.21.15.60:0/1863758637 conn(0x7f56cc0fe910 msgr2=0x7f56cc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:23.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.929+0000 7f56cbe00640 1 -- 172.21.15.60:0/1863758637 shutdown_connections 2024-02-26T10:46:23.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.929+0000 7f56cbe00640 1 -- 172.21.15.60:0/1863758637 wait complete. 2024-02-26T10:46:23.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56cbe00640 1 Processor -- start 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56cbe00640 1 -- start start 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 0x7f56cc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc104570 0x7f56cc199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc104f30 0x7f56cc1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56cbe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc10f010 con 0x7f56cc103380 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56cbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc10ee90 con 0x7f56cc104570 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56cbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc10f190 con 0x7f56cc104f30 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56caa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 0x7f56cc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56c2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc104570 0x7f56cc199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56caa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 0x7f56cc1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45718/0 (socket says 172.21.15.60:45718) 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.930+0000 7f56caa00640 1 -- 172.21.15.60:0/1821297534 learned_addr learned my addr 172.21.15.60:0/1821297534 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:23.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.931+0000 7f56cb400640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc104f30 0x7f56cc1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.931+0000 7f56caa00640 1 -- 172.21.15.60:0/1821297534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc104f30 msgr2=0x7f56cc1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.931+0000 7f56caa00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc104f30 0x7f56cc1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.931+0000 7f56caa00640 1 -- 172.21.15.60:0/1821297534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc104570 msgr2=0x7f56cc199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.931+0000 7f56caa00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc104570 0x7f56cc199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.931+0000 7f56caa00640 1 -- 172.21.15.60:0/1821297534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc1a12c0 con 0x7f56cc103380 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.931+0000 7f56c2000640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc104570 0x7f56cc199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.931+0000 7f56caa00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 0x7f56cc1996a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000f80 tx=0x7f56bc035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.932+0000 7f56c3e00640 1 -- 172.21.15.60:0/1821297534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc039040 con 0x7f56cc103380 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.932+0000 7f56c3e00640 1 -- 172.21.15.60:0/1821297534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56bc044070 con 0x7f56cc103380 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.932+0000 7f56cbe00640 1 -- 172.21.15.60:0/1821297534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc1a1550 con 0x7f56cc103380 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.932+0000 7f56c3e00640 1 -- 172.21.15.60:0/1821297534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0044a0 con 0x7f56cc103380 2024-02-26T10:46:23.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.932+0000 7f56cbe00640 1 -- 172.21.15.60:0/1821297534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc1a1a60 con 0x7f56cc103380 2024-02-26T10:46:23.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.935+0000 7f56d4a1c740 1 Processor -- start 2024-02-26T10:46:23.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.935+0000 7f56c3e00640 1 -- 172.21.15.60:0/1821297534 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f56bc004640 con 0x7f56cc103380 2024-02-26T10:46:23.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.935+0000 7f56d4a1c740 1 -- start start 2024-02-26T10:46:23.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 0x5626b0a28770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a2b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b0936df0 con 0x5626b0a28de0 2024-02-26T10:46:23.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b09889f0 con 0x5626b0a23280 2024-02-26T10:46:23.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b0931ad0 con 0x5626b0a23c70 2024-02-26T10:46:23.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56c2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a2b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56c2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a2b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45732/0 (socket says 172.21.15.60:45732) 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56c2000640 1 -- 172.21.15.60:0/951730130 learned_addr learned my addr 172.21.15.60:0/951730130 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56cb400640 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.936+0000 7f56caa00640 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 0x5626b0a28770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56cb400640 1 -- 172.21.15.60:0/951730130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 msgr2=0x5626b0a28770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56cb400640 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 0x5626b0a28770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56cb400640 1 -- 172.21.15.60:0/951730130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 msgr2=0x5626b0a2b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56cb400640 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a2b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56cb400640 1 -- 172.21.15.60:0/951730130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b0946de0 con 0x5626b0a23280 2024-02-26T10:46:23.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56c2000640 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a2b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56cb400640 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a236a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f56c40045b0 tx=0x7f56c400bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8175c9eeb263bac7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56c3e00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f56b00a1e90 0x7f56b00a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56bbe00640 1 -- 172.21.15.60:0/951730130 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4003bd0 con 0x5626b0a23280 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56c3e00640 1 -- 172.21.15.60:0/1821297534 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f56bc0f0520 con 0x7f56cc103380 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56bbe00640 1 -- 172.21.15.60:0/951730130 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56c4015070 con 0x5626b0a23280 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56bbe00640 1 -- 172.21.15.60:0/951730130 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40116d0 con 0x5626b0a23280 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.937+0000 7f56caa00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f56b00a1e90 0x7f56b00a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56d4a1c740 1 -- 172.21.15.60:0/951730130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 msgr2=0x5626b0a236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56caa00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f56b00a1e90 0x7f56b00a4350 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5626b0a246e0 tx=0x7f56bc0f60e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a236a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f56c40045b0 tx=0x7f56c400bb10 comp rx=0 tx=0).stop 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56d4a1c740 1 -- 172.21.15.60:0/951730130 shutdown_connections 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a2b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 0x5626b0a28770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/951730130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a236a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56d4a1c740 1 -- 172.21.15.60:0/951730130 >> 172.21.15.60:0/951730130 conn(0x5626b0a1e7f0 msgr2=0x5626b0a1ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56d4a1c740 1 -- 172.21.15.60:0/951730130 shutdown_connections 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.938+0000 7f56d4a1c740 1 -- 172.21.15.60:0/951730130 wait complete. 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.939+0000 7f56d4a1c740 1 -- start start 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.939+0000 7f56d4a1c740 1 Processor -- start 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.939+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.939+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 0x5626b0a2db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.939+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a34b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.939+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b0944f10 con 0x5626b0a28de0 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.939+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b0984a90 con 0x5626b0a23280 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.939+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b0a2bd90 con 0x5626b0a23c70 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56d4a1c740 10 client.0 _ll_register_callbacks cb 0x5626b0983a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a34b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a34b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45734/0 (socket says 172.21.15.60:45734) 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 -- 172.21.15.60:0/2169302543 learned_addr learned my addr 172.21.15.60:0/2169302543 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56c2000640 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 0x5626b0a2db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 -- 172.21.15.60:0/2169302543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 msgr2=0x5626b0a2db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 0x5626b0a2db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 -- 172.21.15.60:0/2169302543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 msgr2=0x5626b0a2d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 -- 172.21.15.60:0/2169302543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626b07657e0 con 0x5626b0a28de0 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56c2000640 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 0x5626b0a2db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.940+0000 7f56cb400640 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a34b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000c00 tx=0x7f56c400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968e616ae1044fe8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.941+0000 7f56b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.941+0000 7f56b8c00640 1 -- 172.21.15.60:0/2169302543 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4023040 con 0x5626b0a28de0 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.941+0000 7f56b8c00640 1 -- 172.21.15.60:0/2169302543 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f56c401a470 con 0x5626b0a28de0 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.941+0000 7f56d4a1c740 1 -- 172.21.15.60:0/2169302543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56c4015070 con 0x5626b0a28de0 2024-02-26T10:46:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.941+0000 7f56b8c00640 1 -- 172.21.15.60:0/2169302543 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56c4031050 con 0x5626b0a28de0 2024-02-26T10:46:23.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.941+0000 7f56d4a1c740 10 client.15567 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:23.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.941+0000 7f56d4a1c740 1 -- 172.21.15.60:0/2169302543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5626b07c5f70 con 0x5626b0a28de0 2024-02-26T10:46:23.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.941+0000 7f56b8c00640 1 -- 172.21.15.60:0/2169302543 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56c4015070 con 0x5626b0a28de0 2024-02-26T10:46:23.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.942+0000 7f56b8c00640 1 -- 172.21.15.60:0/2169302543 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f56c40050e0 con 0x5626b0a28de0 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.942+0000 7f56d4a1c740 10 client.15567 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.942+0000 7f56d4a1c740 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.942+0000 7f56d4a1c740 20 client.15567 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.942+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5626b0a391a0 0x5626b0a3b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.942+0000 7f56d4a1c740 4 client.15567 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.942+0000 7f56d4a1c740 1 -- 172.21.15.60:0/2169302543 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626b0995f30 con 0x5626b0a391a0 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.942+0000 7f56c2000640 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5626b0a391a0 0x5626b0a3b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.943+0000 7f56c2000640 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5626b0a391a0 0x5626b0a3b590 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70eb297afa6d506 server_cookie=13d0063aad521262 in_seq=0 out_seq=0 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.943+0000 7f56b8c00640 10 client.15567 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56b8c00640 1 -- 172.21.15.60:0/2169302543 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626b0995f30 con 0x5626b0a391a0 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56b8c00640 10 client.15567 handle_command_reply: tid=0 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 client.15567 shutdown 2024-02-26T10:46:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 -- 172.21.15.60:0/2169302543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 msgr2=0x5626b0a34b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a34b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000c00 tx=0x7f56c400b560 comp rx=0 tx=0).stop 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 -- 172.21.15.60:0/2169302543 shutdown_connections 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5626b0a391a0 0x5626b0a3b590 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a28de0 0x5626b0a34b90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a23c70 0x5626b0a2db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/2169302543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56b8c00640 0 client.15567 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 -- 172.21.15.60:0/2169302543 >> 172.21.15.60:0/2169302543 conn(0x5626b0a1e7f0 msgr2=0x5626b0a1da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 -- 172.21.15.60:0/2169302543 shutdown_connections 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 1 -- 172.21.15.60:0/2169302543 wait complete. 2024-02-26T10:46:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.944+0000 7f56d4a1c740 20 client.15567 trim_cache size 0 max 16384 2024-02-26T10:46:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.961+0000 7f56d4a1c740 1 Processor -- start 2024-02-26T10:46:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56d4a1c740 1 -- start start 2024-02-26T10:46:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23c70 0x5626b0a34b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 0x5626b0a2db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1ae2d0 con 0x5626b0a23c70 2024-02-26T10:46:23.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1abe70 con 0x5626b0a23280 2024-02-26T10:46:23.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1a9f10 con 0x5626b0a28de0 2024-02-26T10:46:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56c2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23c70 0x5626b0a34b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56c2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23c70 0x5626b0a34b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45746/0 (socket says 172.21.15.60:45746) 2024-02-26T10:46:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56c2000640 1 -- 172.21.15.60:0/4036773094 learned_addr learned my addr 172.21.15.60:0/4036773094 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38632/0 (socket says 172.21.15.60:38632) 2024-02-26T10:46:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.962+0000 7f56cb400640 1 --2- 172.21.15.60:0/4036773094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 0x5626b0a2db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56c2000640 1 -- 172.21.15.60:0/4036773094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 msgr2=0x5626b0a2db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56c2000640 1 --2- 172.21.15.60:0/4036773094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 0x5626b0a2db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56c2000640 1 -- 172.21.15.60:0/4036773094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 msgr2=0x5626b0a2d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56c2000640 1 --2- 172.21.15.60:0/4036773094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56c2000640 1 -- 172.21.15.60:0/4036773094 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b065d170 con 0x5626b0a23c70 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56cb400640 1 --2- 172.21.15.60:0/4036773094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56c2000640 1 --2- 172.21.15.60:0/4036773094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23c70 0x5626b0a34b50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5626b0a2eaf0 tx=0x7f56b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5259f7fd57e9c2d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56bb400640 1 -- 172.21.15.60:0/4036773094 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b400fc80 con 0x5626b0a23c70 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56bb400640 1 -- 172.21.15.60:0/4036773094 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56b4025070 con 0x5626b0a23c70 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.963+0000 7f56bb400640 1 -- 172.21.15.60:0/4036773094 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4004c60 con 0x5626b0a23c70 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.964+0000 7f56d4a1c740 1 -- 172.21.15.60:0/4036773094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23c70 msgr2=0x5626b0a34b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.964+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/4036773094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23c70 0x5626b0a34b50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5626b0a2eaf0 tx=0x7f56b4005f90 comp rx=0 tx=0).stop 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.964+0000 7f56d4a1c740 1 -- 172.21.15.60:0/4036773094 shutdown_connections 2024-02-26T10:46:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.964+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/4036773094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 0x5626b0a2db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.964+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/4036773094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23c70 0x5626b0a34b50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.964+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/4036773094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23280 0x5626b0a2d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.964+0000 7f56d4a1c740 1 -- 172.21.15.60:0/4036773094 >> 172.21.15.60:0/4036773094 conn(0x5626b0a1e7f0 msgr2=0x5626b0a25190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.964+0000 7f56d4a1c740 1 -- 172.21.15.60:0/4036773094 shutdown_connections 2024-02-26T10:46:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.964+0000 7f56d4a1c740 1 -- 172.21.15.60:0/4036773094 wait complete. 2024-02-26T10:46:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.965+0000 7f56d4a1c740 1 -- start start 2024-02-26T10:46:23.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.965+0000 7f56d4a1c740 1 Processor -- start 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.965+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23280 0x5626b0a34770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.965+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23c70 0x5626b0a32dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.965+0000 7f56d4a1c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 0x5626b0a33310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.965+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1aa510 con 0x5626b0a23280 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.965+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1ab810 con 0x5626b0a23c70 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.965+0000 7f56d4a1c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1aab50 con 0x5626b0a28de0 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.965+0000 7f56d4a1c740 10 client.0 _ll_register_callbacks cb 0x7f56c4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23280 0x5626b0a34770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23280 0x5626b0a34770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45758/0 (socket says 172.21.15.60:45758) 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 -- 172.21.15.60:0/639284123 learned_addr learned my addr 172.21.15.60:0/639284123 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56c2000640 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23c70 0x5626b0a32dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 0x5626b0a33310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 -- 172.21.15.60:0/639284123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 msgr2=0x5626b0a33310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 0x5626b0a33310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 -- 172.21.15.60:0/639284123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23c70 msgr2=0x5626b0a32dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23c70 0x5626b0a32dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 -- 172.21.15.60:0/639284123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626b0a33850 con 0x5626b0a23280 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56cb400640 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23280 0x5626b0a34770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f56c4004370 tx=0x7f56c401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d5cacc9496b385 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.966+0000 7f56b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56b8c00640 1 -- 172.21.15.60:0/639284123 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c403d020 con 0x5626b0a23280 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56b8c00640 1 -- 172.21.15.60:0/639284123 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f56c4011870 con 0x5626b0a23280 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56d4a1c740 1 -- 172.21.15.60:0/639284123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56b4025070 con 0x5626b0a23280 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56b8c00640 1 -- 172.21.15.60:0/639284123 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56c4011a70 con 0x5626b0a23280 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56d4a1c740 10 client.15573 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56d4a1c740 1 -- 172.21.15.60:0/639284123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5626b0a98af0 con 0x5626b0a23280 2024-02-26T10:46:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56b8c00640 1 -- 172.21.15.60:0/639284123 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56b4025070 con 0x5626b0a23280 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56b8c00640 1 -- 172.21.15.60:0/639284123 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f56c402a460 con 0x5626b0a23280 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56d4a1c740 10 client.15573 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56d4a1c740 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56d4a1c740 20 client.15573 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5626b0a99360 0x5626b0a9b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56d4a1c740 4 client.15573 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.967+0000 7f56d4a1c740 1 -- 172.21.15.60:0/639284123 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5626b0990280 con 0x5626b0a99360 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.968+0000 7f56c2000640 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5626b0a99360 0x5626b0a9b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.968+0000 7f56c2000640 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5626b0a99360 0x5626b0a9b780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e16c072d70edac2 server_cookie=f2aa5ed1d3c8a9c1 in_seq=0 out_seq=0 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.968+0000 7f56b8c00640 10 client.15573 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.969+0000 7f56b8c00640 1 -- 172.21.15.60:0/639284123 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5626b0990280 con 0x5626b0a99360 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.969+0000 7f56b8c00640 10 client.15573 handle_command_reply: tid=0 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.969+0000 7f56d4a1c740 1 client.15573 shutdown 2024-02-26T10:46:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 -- 172.21.15.60:0/639284123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23280 msgr2=0x5626b0a34770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23280 0x5626b0a34770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f56c4004370 tx=0x7f56c401ac50 comp rx=0 tx=0).stop 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 -- 172.21.15.60:0/639284123 shutdown_connections 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5626b0a99360 0x5626b0a9b780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626b0a23c70 0x5626b0a32dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5626b0a28de0 0x5626b0a33310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 --2- 172.21.15.60:0/639284123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5626b0a23280 0x5626b0a34770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 -- 172.21.15.60:0/639284123 >> 172.21.15.60:0/639284123 conn(0x5626b0a1e7f0 msgr2=0x5626b0a2d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56b8c00640 0 client.15573 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 -- 172.21.15.60:0/639284123 shutdown_connections 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 1 -- 172.21.15.60:0/639284123 wait complete. 2024-02-26T10:46:23.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.970+0000 7f56d4a1c740 20 client.15573 trim_cache size 0 max 16384 2024-02-26T10:46:23.969 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:46:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-02-26T10:46:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "475885c7-39d3-4f39-b51a-76056d87c482", 2024-02-26T10:46:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-02-26T10:46:23.969 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:46:23.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 -- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f56b00a1e90 msgr2=0x7f56b00a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f56b00a1e90 0x7f56b00a4350 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5626b0a246e0 tx=0x7f56bc0f60e0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 -- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 msgr2=0x7f56cc1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 0x7f56cc1996a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000f80 tx=0x7f56bc035d10 comp rx=0 tx=0).stop 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 -- 172.21.15.60:0/1821297534 shutdown_connections 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f56b00a1e90 0x7f56b00a4350 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc104570 0x7f56cc199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56cc104f30 0x7f56cc1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 --2- 172.21.15.60:0/1821297534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc103380 0x7f56cc1996a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 -- 172.21.15.60:0/1821297534 >> 172.21.15.60:0/1821297534 conn(0x7f56cc0fe910 msgr2=0x7f56cc107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 -- 172.21.15.60:0/1821297534 shutdown_connections 2024-02-26T10:46:23.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:23.971+0000 7f56cbe00640 1 -- 172.21.15.60:0/1821297534 wait complete. 2024-02-26T10:46:23.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T10:46:23.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T10:46:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.207+0000 7f64e6800640 1 -- 172.21.15.60:0/632061696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0106e20 msgr2=0x7f64e0109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.207+0000 7f64e6800640 1 --2- 172.21.15.60:0/632061696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0106e20 0x7f64e0109270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009ca0 tx=0x7f64d802c0f0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.207+0000 7f64e6800640 1 -- 172.21.15.60:0/632061696 shutdown_connections 2024-02-26T10:46:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.207+0000 7f64e6800640 1 --2- 172.21.15.60:0/632061696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0106e20 0x7f64e0109270 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.207+0000 7f64e6800640 1 --2- 172.21.15.60:0/632061696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e01044f0 0x7f64e01068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.207+0000 7f64e6800640 1 --2- 172.21.15.60:0/632061696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0101bc0 0x7f64e0103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.207+0000 7f64e6800640 1 -- 172.21.15.60:0/632061696 >> 172.21.15.60:0/632061696 conn(0x7f64e00fb5d0 msgr2=0x7f64e00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.207+0000 7f64e6800640 1 -- 172.21.15.60:0/632061696 shutdown_connections 2024-02-26T10:46:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.207+0000 7f64e6800640 1 -- 172.21.15.60:0/632061696 wait complete. 2024-02-26T10:46:24.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.208+0000 7f64e6800640 1 Processor -- start 2024-02-26T10:46:24.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.208+0000 7f64e6800640 1 -- start start 2024-02-26T10:46:24.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.208+0000 7f64e6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0101bc0 0x7f64e01995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.208+0000 7f64e6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e01044f0 0x7f64e0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.208+0000 7f64e6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0106e20 0x7f64e01a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.208+0000 7f64e6800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00ffe60 con 0x7f64e0101bc0 2024-02-26T10:46:24.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.208+0000 7f64e6800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00ffce0 con 0x7f64e0106e20 2024-02-26T10:46:24.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.208+0000 7f64e6800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00fffe0 con 0x7f64e01044f0 2024-02-26T10:46:24.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0101bc0 0x7f64e01995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0101bc0 0x7f64e01995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45772/0 (socket says 172.21.15.60:45772) 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5400640 1 -- 172.21.15.60:0/414956138 learned_addr learned my addr 172.21.15.60:0/414956138 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5e00640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0106e20 0x7f64e01a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e4a00640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e01044f0 0x7f64e0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5400640 1 -- 172.21.15.60:0/414956138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e01044f0 msgr2=0x7f64e0199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5400640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e01044f0 0x7f64e0199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5400640 1 -- 172.21.15.60:0/414956138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0106e20 msgr2=0x7f64e01a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5400640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0106e20 0x7f64e01a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5400640 1 -- 172.21.15.60:0/414956138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e01a1210 con 0x7f64e0101bc0 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.209+0000 7f64e5e00640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0106e20 0x7f64e01a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.210+0000 7f64e5400640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0101bc0 0x7f64e01995f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f64d400e9f0 tx=0x7f64d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:24.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.210+0000 7f64de000640 1 -- 172.21.15.60:0/414956138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d400cd80 con 0x7f64e0101bc0 2024-02-26T10:46:24.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.210+0000 7f64de000640 1 -- 172.21.15.60:0/414956138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64d4010910 con 0x7f64e0101bc0 2024-02-26T10:46:24.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.210+0000 7f64e6800640 1 -- 172.21.15.60:0/414956138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e01a1500 con 0x7f64e0101bc0 2024-02-26T10:46:24.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.210+0000 7f64de000640 1 -- 172.21.15.60:0/414956138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4018990 con 0x7f64e0101bc0 2024-02-26T10:46:24.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.210+0000 7f64e6800640 1 -- 172.21.15.60:0/414956138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e01a1a50 con 0x7f64e0101bc0 2024-02-26T10:46:24.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.213+0000 7f64eac3c740 1 Processor -- start 2024-02-26T10:46:24.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64eac3c740 1 -- start start 2024-02-26T10:46:24.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64de000640 1 -- 172.21.15.60:0/414956138 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f64d4010ac0 con 0x7f64e0101bc0 2024-02-26T10:46:24.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 0x5587fbcef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 0x5587fbcf1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64eac3c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbc17060 con 0x5587fbcef840 2024-02-26T10:46:24.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64eac3c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbc4ed00 con 0x5587fbce9ce0 2024-02-26T10:46:24.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64eac3c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbbf7a00 con 0x5587fbcea6d0 2024-02-26T10:46:24.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64e5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 0x5587fbcef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64e5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64e4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 0x5587fbcf1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64e5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcea100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38684/0 (socket says 172.21.15.60:38684) 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.214+0000 7f64e5e00640 1 -- 172.21.15.60:0/374828860 learned_addr learned my addr 172.21.15.60:0/374828860 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.215+0000 7f64e5e00640 1 -- 172.21.15.60:0/374828860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 msgr2=0x5587fbcef1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.215+0000 7f64e5e00640 1 --2- 172.21.15.60:0/374828860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 0x5587fbcef1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.215+0000 7f64e5e00640 1 -- 172.21.15.60:0/374828860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 msgr2=0x5587fbcf1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.215+0000 7f64e5e00640 1 --2- 172.21.15.60:0/374828860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 0x5587fbcf1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.215+0000 7f64e5e00640 1 -- 172.21.15.60:0/374828860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587fba2d7d0 con 0x5587fbce9ce0 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.215+0000 7f64e5e00640 1 --2- 172.21.15.60:0/374828860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcea100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64d802c700 tx=0x7f64d80323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=588f691a7c3e5299 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.215+0000 7f64de000640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f64c80a1e10 0x7f64c80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.215+0000 7f64d2600640 1 -- 172.21.15.60:0/374828860 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8043070 con 0x5587fbce9ce0 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.216+0000 7f64d2600640 1 -- 172.21.15.60:0/374828860 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64d802cca0 con 0x5587fbce9ce0 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.216+0000 7f64d2600640 1 -- 172.21.15.60:0/374828860 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8039040 con 0x5587fbce9ce0 2024-02-26T10:46:24.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.216+0000 7f64de000640 1 -- 172.21.15.60:0/414956138 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f64d40ce600 con 0x7f64e0101bc0 2024-02-26T10:46:24.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.216+0000 7f64e5400640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f64c80a1e10 0x7f64c80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.216+0000 7f64eac3c740 1 -- 172.21.15.60:0/374828860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 msgr2=0x5587fbcea100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.216+0000 7f64eac3c740 1 --2- 172.21.15.60:0/374828860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcea100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64d802c700 tx=0x7f64d80323d0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.217+0000 7f64e5400640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f64c80a1e10 0x7f64c80a42d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5587fbceb140 tx=0x7f64d40d4530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.217+0000 7f64eac3c740 1 -- 172.21.15.60:0/374828860 shutdown_connections 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.217+0000 7f64eac3c740 1 --2- 172.21.15.60:0/374828860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 0x5587fbcf1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.217+0000 7f64eac3c740 1 --2- 172.21.15.60:0/374828860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 0x5587fbcef1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.217+0000 7f64eac3c740 1 --2- 172.21.15.60:0/374828860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcea100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.217+0000 7f64eac3c740 1 -- 172.21.15.60:0/374828860 >> 172.21.15.60:0/374828860 conn(0x5587fbce5250 msgr2=0x5587fbce5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.217+0000 7f64eac3c740 1 -- 172.21.15.60:0/374828860 shutdown_connections 2024-02-26T10:46:24.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.217+0000 7f64eac3c740 1 -- 172.21.15.60:0/374828860 wait complete. 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.217+0000 7f64eac3c740 1 -- start start 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64eac3c740 1 Processor -- start 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcf3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 0x5587fbcfb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64eac3c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbbed090 con 0x5587fbcef840 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64eac3c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbc4ada0 con 0x5587fbce9ce0 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64eac3c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fbcf2790 con 0x5587fbcea6d0 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64eac3c740 10 client.0 _ll_register_callbacks cb 0x5587fbc0a830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:24.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64e5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcf3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64e5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcf3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38692/0 (socket says 172.21.15.60:38692) 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64e5e00640 1 -- 172.21.15.60:0/1563044056 learned_addr learned my addr 172.21.15.60:0/1563044056 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64e4a00640 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.218+0000 7f64e5e00640 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 0x5587fbcfb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64e4a00640 1 -- 172.21.15.60:0/1563044056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 msgr2=0x5587fbcf3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64e4a00640 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcf3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64e4a00640 1 -- 172.21.15.60:0/1563044056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 msgr2=0x5587fbcfb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64e4a00640 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 0x5587fbcfb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64e4a00640 1 -- 172.21.15.60:0/1563044056 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587fb9633f0 con 0x5587fbcea6d0 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64e5e00640 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 0x5587fbcfb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64e4a00640 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64e019ab00 tx=0x7f64cc012b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e5877528ad624d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64c6a00640 1 -- 172.21.15.60:0/1563044056 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc004020 con 0x5587fbcea6d0 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64c6a00640 1 -- 172.21.15.60:0/1563044056 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f64cc0041c0 con 0x5587fbcea6d0 2024-02-26T10:46:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.219+0000 7f64eac3c740 1 -- 172.21.15.60:0/1563044056 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64d802cca0 con 0x5587fbcea6d0 2024-02-26T10:46:24.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.220+0000 7f64c6a00640 1 -- 172.21.15.60:0/1563044056 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64cc025050 con 0x5587fbcea6d0 2024-02-26T10:46:24.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.220+0000 7f64eac3c740 10 client.25711 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:24.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.220+0000 7f64eac3c740 1 -- 172.21.15.60:0/1563044056 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5587fb9230f0 con 0x5587fbcea6d0 2024-02-26T10:46:24.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.220+0000 7f64c6a00640 1 -- 172.21.15.60:0/1563044056 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64d802cca0 con 0x5587fbcea6d0 2024-02-26T10:46:24.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.221+0000 7f64c6a00640 1 -- 172.21.15.60:0/1563044056 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64cc003a60 con 0x5587fbcea6d0 2024-02-26T10:46:24.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.221+0000 7f64eac3c740 10 client.25711 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.221+0000 7f64eac3c740 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.221+0000 7f64eac3c740 20 client.25711 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.221+0000 7f64eac3c740 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5587fbcffaf0 0x5587fbd01f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.221+0000 7f64eac3c740 4 client.25711 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.221+0000 7f64eac3c740 1 -- 172.21.15.60:0/1563044056 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587fbc5c240 con 0x5587fbcffaf0 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.221+0000 7f64e5e00640 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5587fbcffaf0 0x5587fbd01f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.222+0000 7f64e5e00640 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5587fbcffaf0 0x5587fbd01f10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b79b805425a11f74 server_cookie=fd8bad0ac3f07c64 in_seq=0 out_seq=0 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.222+0000 7f64c6a00640 10 client.25711 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.222+0000 7f64c6a00640 1 -- 172.21.15.60:0/1563044056 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587fbc5c240 con 0x5587fbcffaf0 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.222+0000 7f64c6a00640 10 client.25711 handle_command_reply: tid=0 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.222+0000 7f64eac3c740 1 client.25711 shutdown 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 -- 172.21.15.60:0/1563044056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 msgr2=0x5587fbcf44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:24.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64e019ab00 tx=0x7f64cc012b40 comp rx=0 tx=0).stop 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 -- 172.21.15.60:0/1563044056 shutdown_connections 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5587fbcffaf0 0x5587fbd01f10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcef840 0x5587fbcfb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64c6a00640 0 client.25711 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 --2- 172.21.15.60:0/1563044056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcf3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 -- 172.21.15.60:0/1563044056 >> 172.21.15.60:0/1563044056 conn(0x5587fbce5250 msgr2=0x5587fbce4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 -- 172.21.15.60:0/1563044056 shutdown_connections 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 1 -- 172.21.15.60:0/1563044056 wait complete. 2024-02-26T10:46:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.223+0000 7f64eac3c740 20 client.25711 trim_cache size 0 max 16384 2024-02-26T10:46:24.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.242+0000 7f64eac3c740 1 Processor -- start 2024-02-26T10:46:24.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.242+0000 7f64eac3c740 1 -- start start 2024-02-26T10:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcf3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 0x5587fbcfb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64eac3c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01ae2d0 con 0x5587fbcea6d0 2024-02-26T10:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64eac3c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01abe70 con 0x5587fbce9ce0 2024-02-26T10:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64eac3c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01a9f10 con 0x5587fbcef840 2024-02-26T10:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64e4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64e4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45796/0 (socket says 172.21.15.60:45796) 2024-02-26T10:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64e5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcf3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64e4a00640 1 -- 172.21.15.60:0/488022146 learned_addr learned my addr 172.21.15.60:0/488022146 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:24.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.243+0000 7f64e5e00640 1 --2- 172.21.15.60:0/488022146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 0x5587fbcfb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.244+0000 7f64e4a00640 1 -- 172.21.15.60:0/488022146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 msgr2=0x5587fbcfb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:24.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.244+0000 7f64e4a00640 1 --2- 172.21.15.60:0/488022146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 0x5587fbcfb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.244+0000 7f64e4a00640 1 -- 172.21.15.60:0/488022146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 msgr2=0x5587fbcf3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.244+0000 7f64e4a00640 1 --2- 172.21.15.60:0/488022146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcf3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.244+0000 7f64e4a00640 1 -- 172.21.15.60:0/488022146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587fbce0cc0 con 0x5587fbcea6d0 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.244+0000 7f64e5e00640 1 --2- 172.21.15.60:0/488022146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 0x5587fbcfb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.244+0000 7f64e4a00640 1 --2- 172.21.15.60:0/488022146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f64cc003ed0 tx=0x7f64cc009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e089f327808263 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.244+0000 7f64d1c00640 1 -- 172.21.15.60:0/488022146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc02e070 con 0x5587fbcea6d0 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64d1c00640 1 -- 172.21.15.60:0/488022146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64cc003b90 con 0x5587fbcea6d0 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64d1c00640 1 -- 172.21.15.60:0/488022146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc022040 con 0x5587fbcea6d0 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64eac3c740 1 -- 172.21.15.60:0/488022146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcea6d0 msgr2=0x5587fbcf44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64eac3c740 1 --2- 172.21.15.60:0/488022146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f64cc003ed0 tx=0x7f64cc009f00 comp rx=0 tx=0).stop 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64eac3c740 1 -- 172.21.15.60:0/488022146 shutdown_connections 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64eac3c740 1 --2- 172.21.15.60:0/488022146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 0x5587fbcfb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64eac3c740 1 --2- 172.21.15.60:0/488022146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbcea6d0 0x5587fbcf44d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64eac3c740 1 --2- 172.21.15.60:0/488022146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbce9ce0 0x5587fbcf3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64eac3c740 1 -- 172.21.15.60:0/488022146 >> 172.21.15.60:0/488022146 conn(0x5587fbce5250 msgr2=0x5587fbd06870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:24.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64eac3c740 1 -- 172.21.15.60:0/488022146 shutdown_connections 2024-02-26T10:46:24.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.245+0000 7f64eac3c740 1 -- 172.21.15.60:0/488022146 wait complete. 2024-02-26T10:46:24.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.246+0000 7f64eac3c740 1 -- start start 2024-02-26T10:46:24.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.246+0000 7f64eac3c740 1 Processor -- start 2024-02-26T10:46:24.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.246+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbce9ce0 0x5587fbcfb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbcea6d0 0x5587fbcf9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64eac3c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 0x5587fbcfa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64eac3c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01aa510 con 0x5587fbce9ce0 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64eac3c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01ab810 con 0x5587fbcea6d0 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64eac3c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01aab50 con 0x5587fbcef840 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64eac3c740 10 client.0 _ll_register_callbacks cb 0x7f64cc009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbce9ce0 0x5587fbcfb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbce9ce0 0x5587fbcfb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45798/0 (socket says 172.21.15.60:45798) 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e5e00640 1 -- 172.21.15.60:0/2766317655 learned_addr learned my addr 172.21.15.60:0/2766317655 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e4a00640 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbcea6d0 0x5587fbcf9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e5e00640 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 0x5587fbcfa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e4a00640 1 -- 172.21.15.60:0/2766317655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 msgr2=0x5587fbcfa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e4a00640 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 0x5587fbcfa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e4a00640 1 -- 172.21.15.60:0/2766317655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbce9ce0 msgr2=0x5587fbcfb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e4a00640 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbce9ce0 0x5587fbcfb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e4a00640 1 -- 172.21.15.60:0/2766317655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587fbcf2900 con 0x5587fbcea6d0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.247+0000 7f64e5e00640 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbce9ce0 0x5587fbcfb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.248+0000 7f64e4a00640 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbcea6d0 0x5587fbcf9aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x5587fbcea610 tx=0x7f64cc003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad900fc8e3d44be server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.248+0000 7f64c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.248+0000 7f64c6a00640 1 -- 172.21.15.60:0/2766317655 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc02e070 con 0x5587fbcea6d0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.248+0000 7f64eac3c740 1 -- 172.21.15.60:0/2766317655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64cc003b90 con 0x5587fbcea6d0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.248+0000 7f64c6a00640 1 -- 172.21.15.60:0/2766317655 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f64cc01c9e0 con 0x5587fbcea6d0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.248+0000 7f64c6a00640 1 -- 172.21.15.60:0/2766317655 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64cc01cbe0 con 0x5587fbcea6d0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.248+0000 7f64eac3c740 10 client.25712 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.248+0000 7f64eac3c740 1 -- 172.21.15.60:0/2766317655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5587fbd56f30 con 0x5587fbcea6d0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.249+0000 7f64c6a00640 1 -- 172.21.15.60:0/2766317655 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64cc003b90 con 0x5587fbcea6d0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.249+0000 7f64c6a00640 1 -- 172.21.15.60:0/2766317655 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64cc00fb60 con 0x5587fbcea6d0 2024-02-26T10:46:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.249+0000 7f64eac3c740 10 client.25712 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:24.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.249+0000 7f64eac3c740 10 client.25712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:24.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.249+0000 7f64eac3c740 20 client.25712 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:24.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.249+0000 7f64eac3c740 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5587fbd57810 0x5587fbd59c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:24.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.249+0000 7f64eac3c740 4 client.25712 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T10:46:24.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.249+0000 7f64eac3c740 1 -- 172.21.15.60:0/2766317655 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5587fbc56590 con 0x5587fbd57810 2024-02-26T10:46:24.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.249+0000 7f64e5e00640 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5587fbd57810 0x5587fbd59c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:24.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.250+0000 7f64e5e00640 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5587fbd57810 0x5587fbd59c30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70db1442baeb862 server_cookie=ea0cded63dc3ae12 in_seq=0 out_seq=0 2024-02-26T10:46:24.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.250+0000 7f64c6a00640 10 client.25712 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.250+0000 7f64c6a00640 1 -- 172.21.15.60:0/2766317655 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5587fbc56590 con 0x5587fbd57810 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.250+0000 7f64c6a00640 10 client.25712 handle_command_reply: tid=0 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.250+0000 7f64eac3c740 1 client.25712 shutdown 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 -- 172.21.15.60:0/2766317655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbcea6d0 msgr2=0x5587fbcf9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbcea6d0 0x5587fbcf9aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x5587fbcea610 tx=0x7f64cc003960 comp rx=0 tx=0).stop 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 -- 172.21.15.60:0/2766317655 shutdown_connections 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5587fbd57810 0x5587fbd59c30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587fbcea6d0 0x5587fbcf9aa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5587fbcef840 0x5587fbcfa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 --2- 172.21.15.60:0/2766317655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5587fbce9ce0 0x5587fbcfb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 -- 172.21.15.60:0/2766317655 >> 172.21.15.60:0/2766317655 conn(0x5587fbce5250 msgr2=0x5587fbce84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64c6a00640 0 client.25712 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 -- 172.21.15.60:0/2766317655 shutdown_connections 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 1 -- 172.21.15.60:0/2766317655 wait complete. 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.251+0000 7f64eac3c740 20 client.25712 trim_cache size 0 max 16384 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stdout: "475885c7-39d3-4f39-b51a-76056d87c482": { 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "475885c7-39d3-4f39-b51a-76056d87c482", 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-02-26T10:46:24.250 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:46:24.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:46:24.251 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:46:24.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 -- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f64c80a1e10 msgr2=0x7f64c80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:24.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f64c80a1e10 0x7f64c80a42d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5587fbceb140 tx=0x7f64d40d4530 comp rx=0 tx=0).stop 2024-02-26T10:46:24.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 -- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0101bc0 msgr2=0x7f64e01995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:24.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0101bc0 0x7f64e01995f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f64d400e9f0 tx=0x7f64d400eeb0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 -- 172.21.15.60:0/414956138 shutdown_connections 2024-02-26T10:46:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f64c80a1e10 0x7f64c80a42d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0106e20 0x7f64e01a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e01044f0 0x7f64e0199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 --2- 172.21.15.60:0/414956138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0101bc0 0x7f64e01995f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 -- 172.21.15.60:0/414956138 >> 172.21.15.60:0/414956138 conn(0x7f64e00fb5d0 msgr2=0x7f64e01048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 -- 172.21.15.60:0/414956138 shutdown_connections 2024-02-26T10:46:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:24.252+0000 7f64e6800640 1 -- 172.21.15.60:0/414956138 wait complete. 2024-02-26T10:46:24.265 INFO:tasks.cephfs.filesystem:scrub status for tag:475885c7-39d3-4f39-b51a-76056d87c482 - {'path': '/', 'tag': '475885c7-39d3-4f39-b51a-76056d87c482', 'options': 'recursive,force'} 2024-02-26T10:46:24.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:24 smithi060 ceph-mon[31539]: pgmap v438: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T10:46:24.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:24 smithi100 ceph-mon[36051]: pgmap v438: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T10:46:24.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:24 smithi113 ceph-mon[36126]: pgmap v438: 129 pgs: 129 active+clean; 660 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T10:46:25.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:25 smithi060 ceph-mon[31539]: scrub queued for path: / 2024-02-26T10:46:25.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:25 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:25.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:25 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:46:25.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:25 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:25.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:25 smithi100 ceph-mon[36051]: scrub queued for path: / 2024-02-26T10:46:25.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:25 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:25.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:25 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:46:25.376 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:25 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:25.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:25 smithi113 ceph-mon[36126]: scrub queued for path: / 2024-02-26T10:46:25.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:25 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:25.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:25 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:46:25.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:25 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:26.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:26 smithi060 ceph-mon[31539]: pgmap v439: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T10:46:26.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:26 smithi100 ceph-mon[36051]: pgmap v439: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T10:46:26.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:26 smithi113 ceph-mon[36126]: pgmap v439: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-26T10:46:27.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:27 smithi100 ceph-mon[36051]: scrub summary: idle 2024-02-26T10:46:27.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:27 smithi113 ceph-mon[36126]: scrub summary: idle 2024-02-26T10:46:27.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:27 smithi060 ceph-mon[31539]: scrub summary: idle 2024-02-26T10:46:28.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:28 smithi060 ceph-mon[31539]: pgmap v440: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:46:28.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:28 smithi100 ceph-mon[36051]: pgmap v440: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:46:28.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:28 smithi113 ceph-mon[36126]: pgmap v440: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T10:46:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.074+0000 7feed6000640 1 -- 172.21.15.60:0/913249241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed01011a0 msgr2=0x7feed0101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.074+0000 7feed6000640 1 --2- 172.21.15.60:0/913249241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed01011a0 0x7feed0101620 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feebc009900 tx=0x7feebc02bc50 comp rx=0 tx=0).stop 2024-02-26T10:46:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.074+0000 7feed6000640 1 -- 172.21.15.60:0/913249241 shutdown_connections 2024-02-26T10:46:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.074+0000 7feed6000640 1 --2- 172.21.15.60:0/913249241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0101b60 0x7feed01083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.074+0000 7feed6000640 1 --2- 172.21.15.60:0/913249241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed01011a0 0x7feed0101620 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.074+0000 7feed6000640 1 --2- 172.21.15.60:0/913249241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed00fffb0 0x7feed01003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.074+0000 7feed6000640 1 -- 172.21.15.60:0/913249241 >> 172.21.15.60:0/913249241 conn(0x7feed00fb540 msgr2=0x7feed00fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.074+0000 7feed6000640 1 -- 172.21.15.60:0/913249241 shutdown_connections 2024-02-26T10:46:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.074+0000 7feed6000640 1 -- 172.21.15.60:0/913249241 wait complete. 2024-02-26T10:46:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.075+0000 7feed6000640 1 Processor -- start 2024-02-26T10:46:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.075+0000 7feed6000640 1 -- start start 2024-02-26T10:46:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed00fffb0 0x7feed010c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed01011a0 0x7feed010ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0101b60 0x7feed010b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed6000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed010ce50 con 0x7feed0101b60 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00697b0 con 0x7feed00fffb0 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed6000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed010cfd0 con 0x7feed01011a0 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0101b60 0x7feed010b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0101b60 0x7feed010b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45804/0 (socket says 172.21.15.60:45804) 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed5600640 1 -- 172.21.15.60:0/3604585673 learned_addr learned my addr 172.21.15.60:0/3604585673 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feecfe00640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed01011a0 0x7feed010ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed4c00640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed00fffb0 0x7feed010c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.076+0000 7feed5600640 1 -- 172.21.15.60:0/3604585673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed01011a0 msgr2=0x7feed010ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feed5600640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed01011a0 0x7feed010ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feed5600640 1 -- 172.21.15.60:0/3604585673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed00fffb0 msgr2=0x7feed010c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feed5600640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed00fffb0 0x7feed010c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feed5600640 1 -- 172.21.15.60:0/3604585673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed010bbd0 con 0x7feed0101b60 2024-02-26T10:46:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feed4c00640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed00fffb0 0x7feed010c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feed5600640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0101b60 0x7feed010b340 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feec8009fd0 tx=0x7feec800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feecd600640 1 -- 172.21.15.60:0/3604585673 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8017070 con 0x7feed0101b60 2024-02-26T10:46:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feecd600640 1 -- 172.21.15.60:0/3604585673 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feec8003e50 con 0x7feed0101b60 2024-02-26T10:46:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feecd600640 1 -- 172.21.15.60:0/3604585673 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80106c0 con 0x7feed0101b60 2024-02-26T10:46:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feed6000640 1 -- 172.21.15.60:0/3604585673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed01a1500 con 0x7feed0101b60 2024-02-26T10:46:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.077+0000 7feed6000640 1 -- 172.21.15.60:0/3604585673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed01a1a20 con 0x7feed0101b60 2024-02-26T10:46:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.080+0000 7feeda5b4740 1 Processor -- start 2024-02-26T10:46:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.082+0000 7feecd600640 1 -- 172.21.15.60:0/3604585673 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7feec8003970 con 0x7feed0101b60 2024-02-26T10:46:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.082+0000 7feeda5b4740 1 -- start start 2024-02-26T10:46:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.082+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb559e0 0x5630ddb55e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.082+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb563d0 0x5630ddb5aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.082+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 0x5630ddb5d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.082+0000 7feeda5b4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ddabb340 con 0x5630ddb563d0 2024-02-26T10:46:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.082+0000 7feeda5b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630dda64520 con 0x5630ddb559e0 2024-02-26T10:46:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.082+0000 7feeda5b4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ddab73e0 con 0x5630ddb5b540 2024-02-26T10:46:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feed4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb563d0 0x5630ddb5aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feed4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb563d0 0x5630ddb5aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45818/0 (socket says 172.21.15.60:45818) 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feed4c00640 1 -- 172.21.15.60:0/67240076 learned_addr learned my addr 172.21.15.60:0/67240076 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feecfe00640 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb559e0 0x5630ddb55e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feecfe00640 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 0x5630ddb5d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feed4c00640 1 -- 172.21.15.60:0/67240076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 msgr2=0x5630ddb5d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feed4c00640 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 0x5630ddb5d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feed4c00640 1 -- 172.21.15.60:0/67240076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb559e0 msgr2=0x5630ddb55e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feed4c00640 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb559e0 0x5630ddb55e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feed4c00640 1 -- 172.21.15.60:0/67240076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630dda78390 con 0x5630ddb563d0 2024-02-26T10:46:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.083+0000 7feecfe00640 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb559e0 0x5630ddb55e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.084+0000 7feed4c00640 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb563d0 0x5630ddb5aed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feec40114f0 tx=0x7feec40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6712482f263a9c8e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.084+0000 7feec1c00640 1 -- 172.21.15.60:0/67240076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4011250 con 0x5630ddb563d0 2024-02-26T10:46:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.084+0000 7feec1c00640 1 -- 172.21.15.60:0/67240076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feec40035f0 con 0x5630ddb563d0 2024-02-26T10:46:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.084+0000 7feec1c00640 1 -- 172.21.15.60:0/67240076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec400f820 con 0x5630ddb563d0 2024-02-26T10:46:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.084+0000 7feecd600640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7feeb80a1e80 0x7feeb80a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.084+0000 7feecd600640 1 -- 172.21.15.60:0/3604585673 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7feec80cdf20 con 0x7feed0101b60 2024-02-26T10:46:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feecfe00640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7feeb80a1e80 0x7feeb80a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feeda5b4740 1 -- 172.21.15.60:0/67240076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb563d0 msgr2=0x5630ddb5aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feeda5b4740 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb563d0 0x5630ddb5aed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feec40114f0 tx=0x7feec40119b0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feeda5b4740 1 -- 172.21.15.60:0/67240076 shutdown_connections 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feeda5b4740 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 0x5630ddb5d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feeda5b4740 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb563d0 0x5630ddb5aed0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feeda5b4740 1 --2- 172.21.15.60:0/67240076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb559e0 0x5630ddb55e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feeda5b4740 1 -- 172.21.15.60:0/67240076 >> 172.21.15.60:0/67240076 conn(0x5630ddb50f50 msgr2=0x5630ddb51350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feeda5b4740 1 -- 172.21.15.60:0/67240076 shutdown_connections 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feeda5b4740 1 -- 172.21.15.60:0/67240076 wait complete. 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.085+0000 7feecfe00640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7feeb80a1e80 0x7feeb80a4340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feebc02c260 tx=0x7feebc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.086+0000 7feeda5b4740 1 -- start start 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.086+0000 7feeda5b4740 1 Processor -- start 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.086+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.086+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 0x5630ddb60190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.086+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 0x5630ddb671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.086+0000 7feeda5b4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ddb5e4f0 con 0x5630ddb559e0 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.086+0000 7feeda5b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630dda77980 con 0x5630ddb563d0 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.086+0000 7feeda5b4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ddb5e670 con 0x5630ddb5b540 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.086+0000 7feeda5b4740 10 client.0 _ll_register_callbacks cb 0x5630ddab3310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feed4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feed4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45832/0 (socket says 172.21.15.60:45832) 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feed4c00640 1 -- 172.21.15.60:0/2493804 learned_addr learned my addr 172.21.15.60:0/2493804 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feecfe00640 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 0x5630ddb60190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feed5600640 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 0x5630ddb671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feed4c00640 1 -- 172.21.15.60:0/2493804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 msgr2=0x5630ddb671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feed4c00640 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 0x5630ddb671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feed4c00640 1 -- 172.21.15.60:0/2493804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 msgr2=0x5630ddb60190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feed4c00640 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 0x5630ddb60190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.087+0000 7feed4c00640 1 -- 172.21.15.60:0/2493804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630dd7d0210 con 0x5630ddb559e0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.088+0000 7feed4c00640 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5fc50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feec4011eb0 tx=0x7feec4011480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497d127c3e9e1dd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.088+0000 7feeb6000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.088+0000 7feeb6000640 1 -- 172.21.15.60:0/2493804 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4028070 con 0x5630ddb559e0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.088+0000 7feeb6000640 1 -- 172.21.15.60:0/2493804 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7feec40041f0 con 0x5630ddb559e0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.088+0000 7feeda5b4740 1 -- 172.21.15.60:0/2493804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feec40035f0 con 0x5630ddb559e0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.088+0000 7feeb6000640 1 -- 172.21.15.60:0/2493804 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec4034050 con 0x5630ddb559e0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.088+0000 7feeda5b4740 10 client.15603 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.088+0000 7feeda5b4740 1 -- 172.21.15.60:0/2493804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5630dd89c620 con 0x5630ddb559e0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.089+0000 7feeb6000640 1 -- 172.21.15.60:0/2493804 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feec40035f0 con 0x5630ddb559e0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.089+0000 7feeb6000640 1 -- 172.21.15.60:0/2493804 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feec4004920 con 0x5630ddb559e0 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.089+0000 7feeda5b4740 10 client.15603 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.089+0000 7feeda5b4740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.089+0000 7feeda5b4740 20 client.15603 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.089+0000 7feeda5b4740 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5630ddb6b7f0 0x5630ddb6dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.089+0000 7feeda5b4740 4 client.15603 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.089+0000 7feeda5b4740 1 -- 172.21.15.60:0/2493804 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630ddac8880 con 0x5630ddb6b7f0 2024-02-26T10:46:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.089+0000 7feecfe00640 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5630ddb6b7f0 0x5630ddb6dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.090+0000 7feecfe00640 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5630ddb6b7f0 0x5630ddb6dbe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e527b08d8e8aab server_cookie=9a6663a58698e842 in_seq=0 out_seq=0 2024-02-26T10:46:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.090+0000 7feeb6000640 10 client.15603 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeb6000640 1 -- 172.21.15.60:0/2493804 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5630ddac8880 con 0x5630ddb6b7f0 2024-02-26T10:46:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeb6000640 10 client.15603 handle_command_reply: tid=0 2024-02-26T10:46:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 client.15603 shutdown 2024-02-26T10:46:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 -- 172.21.15.60:0/2493804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 msgr2=0x5630ddb5fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5fc50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feec4011eb0 tx=0x7feec4011480 comp rx=0 tx=0).stop 2024-02-26T10:46:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 -- 172.21.15.60:0/2493804 shutdown_connections 2024-02-26T10:46:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5630ddb6b7f0 0x5630ddb6dbe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 0x5630ddb60190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb5b540 0x5630ddb671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 --2- 172.21.15.60:0/2493804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5fc50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeb6000640 0 client.15603 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 -- 172.21.15.60:0/2493804 >> 172.21.15.60:0/2493804 conn(0x5630ddb50f50 msgr2=0x5630ddb501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 -- 172.21.15.60:0/2493804 shutdown_connections 2024-02-26T10:46:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 1 -- 172.21.15.60:0/2493804 wait complete. 2024-02-26T10:46:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.091+0000 7feeda5b4740 20 client.15603 trim_cache size 0 max 16384 2024-02-26T10:46:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.106+0000 7feeda5b4740 1 Processor -- start 2024-02-26T10:46:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.106+0000 7feeda5b4740 1 -- start start 2024-02-26T10:46:30.101 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.106+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.106+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb563d0 0x5630ddb67190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.106+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb5b540 0x5630ddb57610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.106+0000 7feeda5b4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01ae2b0 con 0x5630ddb559e0 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.106+0000 7feeda5b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01abe50 con 0x5630ddb5b540 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.106+0000 7feeda5b4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01a9ef0 con 0x5630ddb563d0 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feed4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feed4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45844/0 (socket says 172.21.15.60:45844) 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feed4c00640 1 -- 172.21.15.60:0/1711296764 learned_addr learned my addr 172.21.15.60:0/1711296764 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feecfe00640 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb563d0 0x5630ddb67190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feed5600640 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb5b540 0x5630ddb57610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feecfe00640 1 -- 172.21.15.60:0/1711296764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb5b540 msgr2=0x5630ddb57610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feecfe00640 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb5b540 0x5630ddb57610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feecfe00640 1 -- 172.21.15.60:0/1711296764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 msgr2=0x5630ddb5ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feecfe00640 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feecfe00640 1 -- 172.21.15.60:0/1711296764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ddb51cc0 con 0x5630ddb563d0 2024-02-26T10:46:30.103 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feed4c00640 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.107+0000 7feecfe00640 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb563d0 0x5630ddb67190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feebc036040 tx=0x7feebc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=529fd50fac3be054 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feec1200640 1 -- 172.21.15.60:0/1711296764 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc004a40 con 0x5630ddb563d0 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feec1200640 1 -- 172.21.15.60:0/1711296764 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feebc004be0 con 0x5630ddb563d0 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feec1200640 1 -- 172.21.15.60:0/1711296764 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0039e0 con 0x5630ddb563d0 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feeda5b4740 1 -- 172.21.15.60:0/1711296764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb563d0 msgr2=0x5630ddb67190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feeda5b4740 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb563d0 0x5630ddb67190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feebc036040 tx=0x7feebc009260 comp rx=0 tx=0).stop 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feeda5b4740 1 -- 172.21.15.60:0/1711296764 shutdown_connections 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feeda5b4740 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb5b540 0x5630ddb57610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feeda5b4740 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb563d0 0x5630ddb67190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feeda5b4740 1 --2- 172.21.15.60:0/1711296764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb559e0 0x5630ddb5ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.108+0000 7feeda5b4740 1 -- 172.21.15.60:0/1711296764 >> 172.21.15.60:0/1711296764 conn(0x5630ddb50f50 msgr2=0x5630ddb72560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.109+0000 7feeda5b4740 1 -- 172.21.15.60:0/1711296764 shutdown_connections 2024-02-26T10:46:30.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.109+0000 7feeda5b4740 1 -- 172.21.15.60:0/1711296764 wait complete. 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.109+0000 7feeda5b4740 1 -- start start 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.109+0000 7feeda5b4740 1 Processor -- start 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb559e0 0x5630ddb64f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 0x5630ddb654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feeda5b4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb5b540 0x5630ddb66490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feeda5b4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01aa4f0 con 0x5630ddb5b540 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feeda5b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01ab7f0 con 0x5630ddb563d0 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feeda5b4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed01aab30 con 0x5630ddb559e0 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feeda5b4740 10 client.0 _ll_register_callbacks cb 0x7feec4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feed5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb5b540 0x5630ddb66490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feed5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb5b540 0x5630ddb66490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45858/0 (socket says 172.21.15.60:45858) 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feed5600640 1 -- 172.21.15.60:0/326549591 learned_addr learned my addr 172.21.15.60:0/326549591 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feed4c00640 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb559e0 0x5630ddb64f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feecfe00640 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 0x5630ddb654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feed5600640 1 -- 172.21.15.60:0/326549591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb559e0 msgr2=0x5630ddb64f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feed5600640 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb559e0 0x5630ddb64f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feed5600640 1 -- 172.21.15.60:0/326549591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 msgr2=0x5630ddb654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feed5600640 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 0x5630ddb654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.110+0000 7feed5600640 1 -- 172.21.15.60:0/326549591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630ddb66a80 con 0x5630ddb5b540 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.111+0000 7feed5600640 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb5b540 0x5630ddb66490 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feec8005c40 tx=0x7feec8096b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb633ea013bf5b5d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.111+0000 7feeb6000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.111+0000 7feeb6000640 1 -- 172.21.15.60:0/326549591 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80b3060 con 0x5630ddb5b540 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.111+0000 7feeb6000640 1 -- 172.21.15.60:0/326549591 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7feec80041b0 con 0x5630ddb5b540 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.111+0000 7feeda5b4740 1 -- 172.21.15.60:0/326549591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feebc004be0 con 0x5630ddb5b540 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.111+0000 7feeb6000640 1 -- 172.21.15.60:0/326549591 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec8020040 con 0x5630ddb5b540 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.111+0000 7feeda5b4740 10 client.15609 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.111+0000 7feeda5b4740 1 -- 172.21.15.60:0/326549591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5630ddbc2470 con 0x5630ddb5b540 2024-02-26T10:46:30.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.111+0000 7feeb6000640 1 -- 172.21.15.60:0/326549591 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feebc004be0 con 0x5630ddb5b540 2024-02-26T10:46:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.112+0000 7feeb6000640 1 -- 172.21.15.60:0/326549591 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feec80ba0b0 con 0x5630ddb5b540 2024-02-26T10:46:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.112+0000 7feeda5b4740 10 client.15609 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.112+0000 7feeda5b4740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.112+0000 7feeda5b4740 20 client.15609 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:30.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.112+0000 7feeda5b4740 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5630ddbc2da0 0x5630ddbc51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:30.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.112+0000 7feeda5b4740 4 client.15609 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:46:30.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.112+0000 7feeda5b4740 1 -- 172.21.15.60:0/326549591 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5630ddac2bd0 con 0x5630ddbc2da0 2024-02-26T10:46:30.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.112+0000 7feed4c00640 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5630ddbc2da0 0x5630ddbc51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:30.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.113+0000 7feed4c00640 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5630ddbc2da0 0x5630ddbc51c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a69ff6660773be server_cookie=3e395fe0d8a69d2a in_seq=0 out_seq=0 2024-02-26T10:46:30.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.113+0000 7feeb6000640 10 client.15609 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeb6000640 1 -- 172.21.15.60:0/326549591 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5630ddac2bd0 con 0x5630ddbc2da0 2024-02-26T10:46:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeb6000640 10 client.15609 handle_command_reply: tid=0 2024-02-26T10:46:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 client.15609 shutdown 2024-02-26T10:46:30.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 -- 172.21.15.60:0/326549591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb5b540 msgr2=0x5630ddb66490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb5b540 0x5630ddb66490 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feec8005c40 tx=0x7feec8096b90 comp rx=0 tx=0).stop 2024-02-26T10:46:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 -- 172.21.15.60:0/326549591 shutdown_connections 2024-02-26T10:46:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5630ddbc2da0 0x5630ddbc51c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ddb563d0 0x5630ddb654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5630ddb5b540 0x5630ddb66490 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 --2- 172.21.15.60:0/326549591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5630ddb559e0 0x5630ddb64f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 -- 172.21.15.60:0/326549591 >> 172.21.15.60:0/326549591 conn(0x5630ddb50f50 msgr2=0x5630ddb57610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeb6000640 0 client.15609 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.120+0000 7feeda5b4740 1 -- 172.21.15.60:0/326549591 shutdown_connections 2024-02-26T10:46:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feeda5b4740 1 -- 172.21.15.60:0/326549591 wait complete. 2024-02-26T10:46:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feeda5b4740 20 client.15609 trim_cache size 0 max 16384 2024-02-26T10:46:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:46:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:46:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:46:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:46:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feed6000640 1 -- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7feeb80a1e80 msgr2=0x7feeb80a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feed6000640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7feeb80a1e80 0x7feeb80a4340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feebc02c260 tx=0x7feebc005fd0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feed6000640 1 -- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0101b60 msgr2=0x7feed010b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feed6000640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0101b60 0x7feed010b340 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feec8009fd0 tx=0x7feec800b260 comp rx=0 tx=0).stop 2024-02-26T10:46:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feed6000640 1 -- 172.21.15.60:0/3604585673 shutdown_connections 2024-02-26T10:46:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feed6000640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7feeb80a1e80 0x7feeb80a4340 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feed6000640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0101b60 0x7feed010b340 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feed6000640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed01011a0 0x7feed010ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.121+0000 7feed6000640 1 --2- 172.21.15.60:0/3604585673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed00fffb0 0x7feed010c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.122+0000 7feed6000640 1 -- 172.21.15.60:0/3604585673 >> 172.21.15.60:0/3604585673 conn(0x7feed00fb540 msgr2=0x7feed01044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.122+0000 7feed6000640 1 -- 172.21.15.60:0/3604585673 shutdown_connections 2024-02-26T10:46:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:30.122+0000 7feed6000640 1 -- 172.21.15.60:0/3604585673 wait complete. 2024-02-26T10:46:30.375 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:30 smithi100 ceph-mon[36051]: pgmap v441: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-02-26T10:46:30.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:30 smithi113 ceph-mon[36126]: pgmap v441: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-02-26T10:46:30.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:30 smithi060 ceph-mon[31539]: pgmap v441: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-02-26T10:46:32.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:32 smithi113 ceph-mon[36126]: pgmap v442: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-26T10:46:32.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:32 smithi060 ceph-mon[31539]: pgmap v442: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-26T10:46:32.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:32 smithi100 ceph-mon[36051]: pgmap v442: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-26T10:46:34.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:34 smithi113 ceph-mon[36126]: pgmap v443: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-02-26T10:46:34.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:34 smithi060 ceph-mon[31539]: pgmap v443: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-02-26T10:46:34.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:34 smithi100 ceph-mon[36051]: pgmap v443: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-02-26T10:46:36.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:36 smithi113 ceph-mon[36126]: pgmap v444: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-26T10:46:36.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:36 smithi060 ceph-mon[31539]: pgmap v444: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-26T10:46:36.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:36 smithi100 ceph-mon[36051]: pgmap v444: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-02-26T10:46:38.427 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:38 smithi060 ceph-mon[31539]: pgmap v445: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 852 B/s wr, 7 op/s 2024-02-26T10:46:38.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:38 smithi113 ceph-mon[36126]: pgmap v445: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 852 B/s wr, 7 op/s 2024-02-26T10:46:38.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:38 smithi100 ceph-mon[36051]: pgmap v445: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 852 B/s wr, 7 op/s 2024-02-26T10:46:40.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:40 smithi113 ceph-mon[36126]: pgmap v446: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-02-26T10:46:40.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:40 smithi060 ceph-mon[31539]: pgmap v446: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-02-26T10:46:40.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:40 smithi100 ceph-mon[36051]: pgmap v446: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-02-26T10:46:42.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:42 smithi113 ceph-mon[36126]: pgmap v447: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-02-26T10:46:42.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:42 smithi060 ceph-mon[31539]: pgmap v447: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-02-26T10:46:42.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:42 smithi100 ceph-mon[36051]: pgmap v447: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-02-26T10:46:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.277+0000 7fa39b000640 1 -- 172.21.15.60:0/1266769543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941022d0 msgr2=0x7fa394102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.277+0000 7fa39b000640 1 --2- 172.21.15.60:0/1266769543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941022d0 0x7fa394102750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa380009900 tx=0x7fa38002bc30 comp rx=0 tx=0).stop 2024-02-26T10:46:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.278+0000 7fa39b000640 1 -- 172.21.15.60:0/1266769543 shutdown_connections 2024-02-26T10:46:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.278+0000 7fa39b000640 1 --2- 172.21.15.60:0/1266769543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941022d0 0x7fa394102750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.278+0000 7fa39b000640 1 --2- 172.21.15.60:0/1266769543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa394102c90 0x7fa394109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.278+0000 7fa39b000640 1 --2- 172.21.15.60:0/1266769543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3941010e0 0x7fa394101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.278+0000 7fa39b000640 1 -- 172.21.15.60:0/1266769543 >> 172.21.15.60:0/1266769543 conn(0x7fa3940fc670 msgr2=0x7fa3940fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.278+0000 7fa39b000640 1 -- 172.21.15.60:0/1266769543 shutdown_connections 2024-02-26T10:46:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.278+0000 7fa39b000640 1 -- 172.21.15.60:0/1266769543 wait complete. 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.279+0000 7fa39b000640 1 Processor -- start 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.279+0000 7fa39b000640 1 -- start start 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.279+0000 7fa39b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941010e0 0x7fa39419da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.279+0000 7fa39b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3941022d0 0x7fa39419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.279+0000 7fa39b000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa394102c90 0x7fa3941a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.279+0000 7fa39b000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39410cdc0 con 0x7fa394102c90 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.279+0000 7fa39b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39410cc40 con 0x7fa3941010e0 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.279+0000 7fa39b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39410cf40 con 0x7fa3941022d0 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa399c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941010e0 0x7fa39419da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa399c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941010e0 0x7fa39419da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58416/0 (socket says 172.21.15.60:58416) 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa399c00640 1 -- 172.21.15.60:0/4157554165 learned_addr learned my addr 172.21.15.60:0/4157554165 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa399200640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3941022d0 0x7fa39419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa39a600640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa394102c90 0x7fa3941a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa399c00640 1 -- 172.21.15.60:0/4157554165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3941022d0 msgr2=0x7fa39419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa399c00640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3941022d0 0x7fa39419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa399c00640 1 -- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa394102c90 msgr2=0x7fa3941a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa399c00640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa394102c90 0x7fa3941a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa399c00640 1 -- 172.21.15.60:0/4157554165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3941a55e0 con 0x7fa3941010e0 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.280+0000 7fa39a600640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa394102c90 0x7fa3941a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:46:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.281+0000 7fa399c00640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941010e0 0x7fa39419da10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa38800e9c0 tx=0x7fa38800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.281+0000 7fa392000640 1 -- 172.21.15.60:0/4157554165 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38800cd80 con 0x7fa3941010e0 2024-02-26T10:46:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.281+0000 7fa392000640 1 -- 172.21.15.60:0/4157554165 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3880035f0 con 0x7fa3941010e0 2024-02-26T10:46:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.281+0000 7fa392000640 1 -- 172.21.15.60:0/4157554165 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388010690 con 0x7fa3941010e0 2024-02-26T10:46:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.281+0000 7fa39b000640 1 -- 172.21.15.60:0/4157554165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3941a58d0 con 0x7fa3941010e0 2024-02-26T10:46:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.281+0000 7fa39b000640 1 -- 172.21.15.60:0/4157554165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3941a5da0 con 0x7fa3941010e0 2024-02-26T10:46:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.286+0000 7fa39f445740 1 Processor -- start 2024-02-26T10:46:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.286+0000 7fa39f445740 1 -- start start 2024-02-26T10:46:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.286+0000 7fa39f445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.286+0000 7fa39f445740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2d5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.286+0000 7fa39f445740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 0x56155c2d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.286+0000 7fa39f445740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155c236340 con 0x56155c2d6540 2024-02-26T10:46:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.286+0000 7fa39f445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155c1df520 con 0x56155c2d09e0 2024-02-26T10:46:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.286+0000 7fa39f445740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155c2323e0 con 0x56155c2d13d0 2024-02-26T10:46:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.287+0000 7fa392000640 1 -- 172.21.15.60:0/4157554165 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa3880037a0 con 0x7fa3941010e0 2024-02-26T10:46:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.287+0000 7fa399c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2d5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.287+0000 7fa39a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.287+0000 7fa399c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2d5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:37464/0 (socket says 172.21.15.60:37464) 2024-02-26T10:46:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.287+0000 7fa399c00640 1 -- 172.21.15.60:0/2507916253 learned_addr learned my addr 172.21.15.60:0/2507916253 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.287+0000 7fa399200640 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 0x56155c2d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.287+0000 7fa39a600640 1 -- 172.21.15.60:0/2507916253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 msgr2=0x56155c2d5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.287+0000 7fa39a600640 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2d5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.288+0000 7fa39a600640 1 -- 172.21.15.60:0/2507916253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 msgr2=0x56155c2d89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.288+0000 7fa39a600640 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 0x56155c2d89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.288+0000 7fa39a600640 1 -- 172.21.15.60:0/2507916253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56155c1f3390 con 0x56155c2d09e0 2024-02-26T10:46:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.288+0000 7fa399c00640 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2d5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.288+0000 7fa399200640 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 0x56155c2d89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.288+0000 7fa39a600640 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d0e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009ec0 tx=0x7fa38c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f86c500f3c5cd76 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.289+0000 7fa392000640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa37c0a1e00 0x7fa37c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.289+0000 7fa386a00640 1 -- 172.21.15.60:0/2507916253 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c003b70 con 0x56155c2d09e0 2024-02-26T10:46:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.289+0000 7fa386a00640 1 -- 172.21.15.60:0/2507916253 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa38c003d10 con 0x56155c2d09e0 2024-02-26T10:46:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.289+0000 7fa386a00640 1 -- 172.21.15.60:0/2507916253 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c011a30 con 0x56155c2d09e0 2024-02-26T10:46:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.289+0000 7fa392000640 1 -- 172.21.15.60:0/4157554165 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa3880ce770 con 0x7fa3941010e0 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.289+0000 7fa399c00640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa37c0a1e00 0x7fa37c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa39f445740 1 -- 172.21.15.60:0/2507916253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 msgr2=0x56155c2d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa39f445740 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d0e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009ec0 tx=0x7fa38c00f730 comp rx=0 tx=0).stop 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa39f445740 1 -- 172.21.15.60:0/2507916253 shutdown_connections 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa39f445740 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 0x56155c2d89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa39f445740 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2d5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa39f445740 1 --2- 172.21.15.60:0/2507916253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d0e00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa39f445740 1 -- 172.21.15.60:0/2507916253 >> 172.21.15.60:0/2507916253 conn(0x56155c2cbf50 msgr2=0x56155c2cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa399c00640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa37c0a1e00 0x7fa37c0a42c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa3880cea10 tx=0x7fa3880d4530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa39f445740 1 -- 172.21.15.60:0/2507916253 shutdown_connections 2024-02-26T10:46:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.290+0000 7fa39f445740 1 -- 172.21.15.60:0/2507916253 wait complete. 2024-02-26T10:46:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.291+0000 7fa39f445740 1 -- start start 2024-02-26T10:46:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.291+0000 7fa39f445740 1 Processor -- start 2024-02-26T10:46:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39f445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39f445740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 0x56155c2db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39f445740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d6540 0x56155c2e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39f445740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155c2d94f0 con 0x56155c2d13d0 2024-02-26T10:46:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39f445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155c1f2980 con 0x56155c2d09e0 2024-02-26T10:46:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39f445740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155c2d9670 con 0x56155c2d6540 2024-02-26T10:46:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39f445740 10 client.0 _ll_register_callbacks cb 0x56155c22e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58436/0 (socket says 172.21.15.60:58436) 2024-02-26T10:46:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39a600640 1 -- 172.21.15.60:0/1782781334 learned_addr learned my addr 172.21.15.60:0/1782781334 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa39a600640 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d6540 0x56155c2e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.292+0000 7fa399200640 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 0x56155c2db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.293+0000 7fa39a600640 1 -- 172.21.15.60:0/1782781334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d6540 msgr2=0x56155c2e2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.293+0000 7fa39a600640 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d6540 0x56155c2e2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.293+0000 7fa39a600640 1 -- 172.21.15.60:0/1782781334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 msgr2=0x56155c2db220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.293+0000 7fa39a600640 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 0x56155c2db220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.293+0000 7fa39a600640 1 -- 172.21.15.60:0/1782781334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56155bf4b210 con 0x56155c2d09e0 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.293+0000 7fa39a600640 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dace0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa38c012e90 tx=0x7fa38c004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=484fc05c367dff5d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.293+0000 7fa37b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.293+0000 7fa37b400640 1 -- 172.21.15.60:0/1782781334 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c024040 con 0x56155c2d09e0 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.293+0000 7fa37b400640 1 -- 172.21.15.60:0/1782781334 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa38c0119b0 con 0x56155c2d09e0 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.294+0000 7fa39f445740 1 -- 172.21.15.60:0/1782781334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa38c003d10 con 0x56155c2d09e0 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.294+0000 7fa37b400640 1 -- 172.21.15.60:0/1782781334 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa38c003b30 con 0x56155c2d09e0 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.294+0000 7fa39f445740 10 client.25742 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.294+0000 7fa39f445740 1 -- 172.21.15.60:0/1782781334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56155c017620 con 0x56155c2d09e0 2024-02-26T10:46:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.294+0000 7fa37b400640 1 -- 172.21.15.60:0/1782781334 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa38c003d10 con 0x56155c2d09e0 2024-02-26T10:46:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.295+0000 7fa37b400640 1 -- 172.21.15.60:0/1782781334 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa38c035420 con 0x56155c2d09e0 2024-02-26T10:46:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.295+0000 7fa39f445740 10 client.25742 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.295+0000 7fa39f445740 10 client.25742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.296+0000 7fa39f445740 20 client.25742 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.296+0000 7fa39f445740 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56155c2e6860 0x56155c2e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.296+0000 7fa39f445740 4 client.25742 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.296+0000 7fa39f445740 1 -- 172.21.15.60:0/1782781334 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56155c243880 con 0x56155c2e6860 2024-02-26T10:46:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.296+0000 7fa399200640 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56155c2e6860 0x56155c2e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.297+0000 7fa399200640 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56155c2e6860 0x56155c2e8c50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffabc5aacdd17cd2 server_cookie=84a07084eff6e908 in_seq=0 out_seq=0 2024-02-26T10:46:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.297+0000 7fa37b400640 10 client.25742 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.297+0000 7fa37b400640 1 -- 172.21.15.60:0/1782781334 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56155c243880 con 0x56155c2e6860 2024-02-26T10:46:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.297+0000 7fa37b400640 10 client.25742 handle_command_reply: tid=0 2024-02-26T10:46:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.297+0000 7fa39f445740 1 client.25742 shutdown 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 -- 172.21.15.60:0/1782781334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 msgr2=0x56155c2dace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dace0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa38c012e90 tx=0x7fa38c004970 comp rx=0 tx=0).stop 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 -- 172.21.15.60:0/1782781334 shutdown_connections 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56155c2e6860 0x56155c2e8c50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d6540 0x56155c2e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 0x56155c2db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa37b400640 0 client.25742 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 --2- 172.21.15.60:0/1782781334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dace0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 -- 172.21.15.60:0/1782781334 >> 172.21.15.60:0/1782781334 conn(0x56155c2cbf50 msgr2=0x56155c2cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 -- 172.21.15.60:0/1782781334 shutdown_connections 2024-02-26T10:46:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 1 -- 172.21.15.60:0/1782781334 wait complete. 2024-02-26T10:46:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.298+0000 7fa39f445740 20 client.25742 trim_cache size 0 max 16384 2024-02-26T10:46:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.325+0000 7fa39f445740 1 Processor -- start 2024-02-26T10:46:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39f445740 1 -- start start 2024-02-26T10:46:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39f445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39f445740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39f445740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 0x56155c2d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39f445740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941b2670 con 0x56155c2d6540 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39f445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941b0210 con 0x56155c2d09e0 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39f445740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941ae360 con 0x56155c2d13d0 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58450/0 (socket says 172.21.15.60:58450) 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39a600640 1 -- 172.21.15.60:0/4179607298 learned_addr learned my addr 172.21.15.60:0/4179607298 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa399200640 1 --2- 172.21.15.60:0/4179607298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.326+0000 7fa39a600640 1 --2- 172.21.15.60:0/4179607298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 0x56155c2d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.327+0000 7fa39a600640 1 -- 172.21.15.60:0/4179607298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 msgr2=0x56155c2e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.327+0000 7fa39a600640 1 --2- 172.21.15.60:0/4179607298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2e2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.327+0000 7fa39a600640 1 -- 172.21.15.60:0/4179607298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 msgr2=0x56155c2d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.327+0000 7fa39a600640 1 --2- 172.21.15.60:0/4179607298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 0x56155c2d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.327+0000 7fa39a600640 1 -- 172.21.15.60:0/4179607298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56155c2e8fc0 con 0x56155c2d09e0 2024-02-26T10:46:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.327+0000 7fa39a600640 1 --2- 172.21.15.60:0/4179607298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d2670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa38c005d70 tx=0x7fa38c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8991eea373fbb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.327+0000 7fa386000640 1 -- 172.21.15.60:0/4179607298 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c015070 con 0x56155c2d09e0 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.327+0000 7fa386000640 1 -- 172.21.15.60:0/4179607298 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa38c03c040 con 0x56155c2d09e0 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.327+0000 7fa386000640 1 -- 172.21.15.60:0/4179607298 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c03b040 con 0x56155c2d09e0 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.328+0000 7fa39f445740 1 -- 172.21.15.60:0/4179607298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 msgr2=0x56155c2d2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.328+0000 7fa39f445740 1 --2- 172.21.15.60:0/4179607298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d2670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa38c005d70 tx=0x7fa38c009500 comp rx=0 tx=0).stop 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.329+0000 7fa39f445740 1 -- 172.21.15.60:0/4179607298 shutdown_connections 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.329+0000 7fa39f445740 1 --2- 172.21.15.60:0/4179607298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d6540 0x56155c2d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.329+0000 7fa39f445740 1 --2- 172.21.15.60:0/4179607298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d13d0 0x56155c2e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.329+0000 7fa39f445740 1 --2- 172.21.15.60:0/4179607298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2d2670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.329+0000 7fa39f445740 1 -- 172.21.15.60:0/4179607298 >> 172.21.15.60:0/4179607298 conn(0x56155c2cbf50 msgr2=0x56155c2ed500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.329+0000 7fa39f445740 1 -- 172.21.15.60:0/4179607298 shutdown_connections 2024-02-26T10:46:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.329+0000 7fa39f445740 1 -- 172.21.15.60:0/4179607298 wait complete. 2024-02-26T10:46:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.329+0000 7fa39f445740 1 -- start start 2024-02-26T10:46:43.325 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.330+0000 7fa39f445740 1 Processor -- start 2024-02-26T10:46:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39f445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39f445740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 0x56155c2e0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39f445740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d6540 0x56155c2e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58464/0 (socket says 172.21.15.60:58464) 2024-02-26T10:46:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39a600640 1 -- 172.21.15.60:0/19670268 learned_addr learned my addr 172.21.15.60:0/19670268 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39f445740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941ae8d0 con 0x56155c2d13d0 2024-02-26T10:46:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941afbb0 con 0x56155c2d09e0 2024-02-26T10:46:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941aeef0 con 0x56155c2d6540 2024-02-26T10:46:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa39f445740 10 client.0 _ll_register_callbacks cb 0x7fa38c009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.331+0000 7fa399200640 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 0x56155c2e0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.332+0000 7fa39a600640 1 -- 172.21.15.60:0/19670268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d6540 msgr2=0x56155c2e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.332+0000 7fa39a600640 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d6540 0x56155c2e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.332+0000 7fa39a600640 1 -- 172.21.15.60:0/19670268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 msgr2=0x56155c2e0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.332+0000 7fa39a600640 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 0x56155c2e0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.332+0000 7fa39a600640 1 -- 172.21.15.60:0/19670268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56155c2e1830 con 0x56155c2d09e0 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.332+0000 7fa399200640 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 0x56155c2e0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.332+0000 7fa39a600640 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dfd40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009360 tx=0x7fa38c035650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1e104d5f088600b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.332+0000 7fa37b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.333+0000 7fa37b400640 1 -- 172.21.15.60:0/19670268 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c046060 con 0x56155c2d09e0 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.333+0000 7fa37b400640 1 -- 172.21.15.60:0/19670268 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa38c004b80 con 0x56155c2d09e0 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.333+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa38c03c040 con 0x56155c2d09e0 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.333+0000 7fa37b400640 1 -- 172.21.15.60:0/19670268 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa38c015070 con 0x56155c2d09e0 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.333+0000 7fa39f445740 10 client.25748 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.333+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56155c33cf70 con 0x56155c2d09e0 2024-02-26T10:46:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.334+0000 7fa37b400640 1 -- 172.21.15.60:0/19670268 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa38c03c040 con 0x56155c2d09e0 2024-02-26T10:46:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.334+0000 7fa37b400640 1 -- 172.21.15.60:0/19670268 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa38c011440 con 0x56155c2d09e0 2024-02-26T10:46:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.334+0000 7fa39f445740 10 client.25748 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.334+0000 7fa39f445740 10 client.25748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.334+0000 7fa39f445740 20 client.25748 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.334+0000 7fa39f445740 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56155c33d810 0x56155c33fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.334+0000 7fa39f445740 4 client.25748 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:46:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.334+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56155c23dbd0 con 0x56155c33d810 2024-02-26T10:46:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.334+0000 7fa399200640 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56155c33d810 0x56155c33fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.335+0000 7fa399200640 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56155c33d810 0x56155c33fc30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1276cc0520bddd09 server_cookie=bd653e8f0e4a5372 in_seq=0 out_seq=0 2024-02-26T10:46:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.335+0000 7fa37b400640 10 client.25748 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.338+0000 7fa37b400640 1 -- 172.21.15.60:0/19670268 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56155c23dbd0 con 0x56155c33d810 2024-02-26T10:46:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.338+0000 7fa37b400640 10 client.25748 handle_command_reply: tid=0 2024-02-26T10:46:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.338+0000 7fa39f445740 1 client.25748 shutdown 2024-02-26T10:46:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 msgr2=0x56155c2dfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dfd40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009360 tx=0x7fa38c035650 comp rx=0 tx=0).stop 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 shutdown_connections 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56155c33d810 0x56155c33fc30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155c2d6540 0x56155c2e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56155c2d13d0 0x56155c2e0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 --2- 172.21.15.60:0/19670268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56155c2d09e0 0x56155c2dfd40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 >> 172.21.15.60:0/19670268 conn(0x56155c2cbf50 msgr2=0x56155c2d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa37b400640 0 client.25748 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 shutdown_connections 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 1 -- 172.21.15.60:0/19670268 wait complete. 2024-02-26T10:46:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.339+0000 7fa39f445740 20 client.25748 trim_cache size 0 max 16384 2024-02-26T10:46:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:46:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:46:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:46:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:46:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 -- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa37c0a1e00 msgr2=0x7fa37c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa37c0a1e00 0x7fa37c0a42c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa3880cea10 tx=0x7fa3880d4530 comp rx=0 tx=0).stop 2024-02-26T10:46:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 -- 172.21.15.60:0/4157554165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941010e0 msgr2=0x7fa39419da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941010e0 0x7fa39419da10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa38800e9c0 tx=0x7fa38800ee80 comp rx=0 tx=0).stop 2024-02-26T10:46:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 -- 172.21.15.60:0/4157554165 shutdown_connections 2024-02-26T10:46:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa37c0a1e00 0x7fa37c0a42c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa394102c90 0x7fa3941a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3941022d0 0x7fa39419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 --2- 172.21.15.60:0/4157554165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3941010e0 0x7fa39419da10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.340+0000 7fa39b000640 1 -- 172.21.15.60:0/4157554165 >> 172.21.15.60:0/4157554165 conn(0x7fa3940fc670 msgr2=0x7fa3941034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.343+0000 7fa39b000640 1 -- 172.21.15.60:0/4157554165 shutdown_connections 2024-02-26T10:46:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:43.343+0000 7fa39b000640 1 -- 172.21.15.60:0/4157554165 wait complete. 2024-02-26T10:46:43.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:43 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:46:43.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:43 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:46:43.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:43 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:46:44.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:44 smithi113 ceph-mon[36126]: pgmap v448: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-26T10:46:44.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:44 smithi060 ceph-mon[31539]: pgmap v448: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-26T10:46:44.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:44 smithi100 ceph-mon[36051]: pgmap v448: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-26T10:46:45.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:46:45.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:46:45.582 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:45 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:46:45.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:46:45.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:46:45.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:45 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:46:45.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:46:45.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:46:45.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:45 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:46:46.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:46 smithi113 ceph-mon[36126]: pgmap v449: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-02-26T10:46:46.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:46 smithi060 ceph-mon[31539]: pgmap v449: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-02-26T10:46:46.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:46 smithi100 ceph-mon[36051]: pgmap v449: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-02-26T10:46:48.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:48 smithi113 ceph-mon[36126]: pgmap v450: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-26T10:46:48.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:48 smithi060 ceph-mon[31539]: pgmap v450: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-26T10:46:48.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:48 smithi100 ceph-mon[36051]: pgmap v450: 129 pgs: 129 active+clean; 671 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-26T10:46:50.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:50 smithi113 ceph-mon[36126]: pgmap v451: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T10:46:50.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:50 smithi060 ceph-mon[31539]: pgmap v451: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T10:46:50.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:50 smithi100 ceph-mon[36051]: pgmap v451: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T10:46:52.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:52 smithi113 ceph-mon[36126]: pgmap v452: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T10:46:52.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:52 smithi060 ceph-mon[31539]: pgmap v452: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T10:46:52.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:52 smithi100 ceph-mon[36051]: pgmap v452: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T10:46:54.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T10:46:54.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T10:46:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.469+0000 7f6349a00640 1 -- 172.21.15.60:0/3387149111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6344101160 msgr2=0x7f6344101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.469+0000 7f6349a00640 1 --2- 172.21.15.60:0/3387149111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6344101160 0x7f6344101580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6334009900 tx=0x7f633402bc40 comp rx=0 tx=0).stop 2024-02-26T10:46:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.469+0000 7f6349a00640 1 -- 172.21.15.60:0/3387149111 shutdown_connections 2024-02-26T10:46:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.469+0000 7f6349a00640 1 --2- 172.21.15.60:0/3387149111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6344102d10 0x7f63441095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.469+0000 7f6349a00640 1 --2- 172.21.15.60:0/3387149111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6344102350 0x7f63441027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.469+0000 7f6349a00640 1 --2- 172.21.15.60:0/3387149111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6344101160 0x7f6344101580 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.469+0000 7f6349a00640 1 -- 172.21.15.60:0/3387149111 >> 172.21.15.60:0/3387149111 conn(0x7f63440fc6d0 msgr2=0x7f63440feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.469+0000 7f6349a00640 1 -- 172.21.15.60:0/3387149111 shutdown_connections 2024-02-26T10:46:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.469+0000 7f6349a00640 1 -- 172.21.15.60:0/3387149111 wait complete. 2024-02-26T10:46:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.470+0000 7f6349a00640 1 Processor -- start 2024-02-26T10:46:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.470+0000 7f6349a00640 1 -- start start 2024-02-26T10:46:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6349a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6344101160 0x7f6344199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6349a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6344102350 0x7f6344199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6349a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6344102d10 0x7f63441a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6349a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634410ce10 con 0x7f6344101160 2024-02-26T10:46:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6349a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634410cc90 con 0x7f6344102350 2024-02-26T10:46:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6349a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634410cf90 con 0x7f6344102d10 2024-02-26T10:46:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6343e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6344101160 0x7f6344199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6343e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6344101160 0x7f6344199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38862/0 (socket says 172.21.15.60:38862) 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6343e00640 1 -- 172.21.15.60:0/60380413 learned_addr learned my addr 172.21.15.60:0/60380413 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6343400640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6344102350 0x7f6344199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.471+0000 7f6349000640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6344102d10 0x7f63441a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6343400640 1 -- 172.21.15.60:0/60380413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6344102d10 msgr2=0x7f63441a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6343400640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6344102d10 0x7f63441a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6343400640 1 -- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6344101160 msgr2=0x7f6344199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6343400640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6344101160 0x7f6344199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6343400640 1 -- 172.21.15.60:0/60380413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63441a1290 con 0x7f6344102350 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6343e00640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6344101160 0x7f6344199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6343400640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6344102350 0x7f6344199bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f633000e9f0 tx=0x7f633000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6340c00640 1 -- 172.21.15.60:0/60380413 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633000ccc0 con 0x7f6344102350 2024-02-26T10:46:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6349a00640 1 -- 172.21.15.60:0/60380413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63441a1580 con 0x7f6344102350 2024-02-26T10:46:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6340c00640 1 -- 172.21.15.60:0/60380413 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6330010910 con 0x7f6344102350 2024-02-26T10:46:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6340c00640 1 -- 172.21.15.60:0/60380413 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330018960 con 0x7f6344102350 2024-02-26T10:46:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.472+0000 7f6349a00640 1 -- 172.21.15.60:0/60380413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63441a1b20 con 0x7f6344102350 2024-02-26T10:46:54.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.477+0000 7f634de0d740 1 Processor -- start 2024-02-26T10:46:54.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.477+0000 7f634de0d740 1 -- start start 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.477+0000 7f6340c00640 1 -- 172.21.15.60:0/60380413 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f6330010ac0 con 0x7f6344102350 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.477+0000 7f634de0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 0x55b4922dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.477+0000 7f634de0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 0x55b4922e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.477+0000 7f634de0d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.477+0000 7f634de0d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b492201060 con 0x55b4922e2050 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.477+0000 7f634de0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b492238d00 con 0x55b4922dcee0 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.477+0000 7f634de0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4921e1a00 con 0x55b4922dc4f0 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6343400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6343400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38864/0 (socket says 172.21.15.60:38864) 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6343400640 1 -- 172.21.15.60:0/1507811039 learned_addr learned my addr 172.21.15.60:0/1507811039 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6343e00640 1 --2- 172.21.15.60:0/1507811039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 0x55b4922e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6349000640 1 --2- 172.21.15.60:0/1507811039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 0x55b4922dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6343400640 1 -- 172.21.15.60:0/1507811039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 msgr2=0x55b4922dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:54.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6343400640 1 --2- 172.21.15.60:0/1507811039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 0x55b4922dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6343400640 1 -- 172.21.15.60:0/1507811039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 msgr2=0x55b4922e19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:54.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6343400640 1 --2- 172.21.15.60:0/1507811039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 0x55b4922e19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.478+0000 7f6343400640 1 -- 172.21.15.60:0/1507811039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4920177d0 con 0x55b4922e2050 2024-02-26T10:46:54.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.479+0000 7f6343400640 1 --2- 172.21.15.60:0/1507811039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e44e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f63300d4bf0 tx=0x7f63300d6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3df03431d2baaf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:54.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.479+0000 7f6339600640 1 -- 172.21.15.60:0/1507811039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300dd070 con 0x55b4922e2050 2024-02-26T10:46:54.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.479+0000 7f6339600640 1 -- 172.21.15.60:0/1507811039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63300cec60 con 0x55b4922e2050 2024-02-26T10:46:54.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.479+0000 7f6339600640 1 -- 172.21.15.60:0/1507811039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300d9730 con 0x55b4922e2050 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.479+0000 7f6340c00640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f63280a1e90 0x7f63280a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.480+0000 7f6340c00640 1 -- 172.21.15.60:0/60380413 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f63300ce670 con 0x7f6344102350 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.480+0000 7f6349000640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f63280a1e90 0x7f63280a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.480+0000 7f634de0d740 1 -- 172.21.15.60:0/1507811039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 msgr2=0x55b4922e44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.480+0000 7f634de0d740 1 --2- 172.21.15.60:0/1507811039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e44e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f63300d4bf0 tx=0x7f63300d6670 comp rx=0 tx=0).stop 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.480+0000 7f634de0d740 1 -- 172.21.15.60:0/1507811039 shutdown_connections 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.480+0000 7f634de0d740 1 --2- 172.21.15.60:0/1507811039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 0x55b4922e19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.480+0000 7f634de0d740 1 --2- 172.21.15.60:0/1507811039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e44e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.481+0000 7f634de0d740 1 --2- 172.21.15.60:0/1507811039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 0x55b4922dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.481+0000 7f634de0d740 1 -- 172.21.15.60:0/1507811039 >> 172.21.15.60:0/1507811039 conn(0x55b4922cf250 msgr2=0x55b4922cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.481+0000 7f6349000640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f63280a1e90 0x7f63280a4350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55b4922dd740 tx=0x7f633c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.481+0000 7f634de0d740 1 -- 172.21.15.60:0/1507811039 shutdown_connections 2024-02-26T10:46:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.481+0000 7f634de0d740 1 -- 172.21.15.60:0/1507811039 wait complete. 2024-02-26T10:46:54.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.481+0000 7f634de0d740 1 -- start start 2024-02-26T10:46:54.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.481+0000 7f634de0d740 1 Processor -- start 2024-02-26T10:46:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f634de0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dc4f0 0x55b4922e67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f634de0d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dcee0 0x55b4922e6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f634de0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922e2050 0x55b4922edd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f634de0d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4921d7090 con 0x55b4922dcee0 2024-02-26T10:46:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f634de0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b492234da0 con 0x55b4922dc4f0 2024-02-26T10:46:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f634de0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4922e5000 con 0x55b4922e2050 2024-02-26T10:46:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f634de0d740 10 client.0 _ll_register_callbacks cb 0x55b4921f4830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f6349000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922e2050 0x55b4922edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f6349000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922e2050 0x55b4922edd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:45980/0 (socket says 172.21.15.60:45980) 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f6349000640 1 -- 172.21.15.60:0/2272535672 learned_addr learned my addr 172.21.15.60:0/2272535672 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.482+0000 7f6343e00640 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dc4f0 0x55b4922e67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f6349000640 1 -- 172.21.15.60:0/2272535672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dc4f0 msgr2=0x55b4922e67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f6349000640 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dc4f0 0x55b4922e67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f6349000640 1 -- 172.21.15.60:0/2272535672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dcee0 msgr2=0x55b4922e6ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f6349000640 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dcee0 0x55b4922e6ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f6349000640 1 -- 172.21.15.60:0/2272535672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b491f4d3f0 con 0x55b4922e2050 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f6343e00640 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dc4f0 0x55b4922e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f6349000640 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922e2050 0x55b4922edd20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f633c009e80 tx=0x7f633c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caebb6a2c69b531b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f632e000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f632e000640 1 -- 172.21.15.60:0/2272535672 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c02d060 con 0x55b4922e2050 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f634de0d740 1 -- 172.21.15.60:0/2272535672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63300cec60 con 0x55b4922e2050 2024-02-26T10:46:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.483+0000 7f632e000640 1 -- 172.21.15.60:0/2272535672 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f633c018030 con 0x55b4922e2050 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f632e000640 1 -- 172.21.15.60:0/2272535672 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f633c034050 con 0x55b4922e2050 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f634de0d740 10 client.25762 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f634de0d740 1 -- 172.21.15.60:0/2272535672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b491f0d0f0 con 0x55b4922e2050 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f632e000640 1 -- 172.21.15.60:0/2272535672 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63300cec60 con 0x55b4922e2050 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f632e000640 1 -- 172.21.15.60:0/2272535672 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f633c028ad0 con 0x55b4922e2050 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f634de0d740 10 client.25762 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f634de0d740 10 client.25762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f634de0d740 20 client.25762 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f634de0d740 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b4922f2270 0x55b4922f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f634de0d740 4 client.25762 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.484+0000 7f634de0d740 1 -- 172.21.15.60:0/2272535672 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b492246240 con 0x55b4922f2270 2024-02-26T10:46:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.485+0000 7f6343e00640 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b4922f2270 0x55b4922f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.485+0000 7f6343e00640 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b4922f2270 0x55b4922f46b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5ea6f9ccd08ed2 server_cookie=ab695a7275bf799d in_seq=0 out_seq=0 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.485+0000 7f632e000640 10 client.25762 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.486+0000 7f632e000640 1 -- 172.21.15.60:0/2272535672 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b492246240 con 0x55b4922f2270 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.486+0000 7f632e000640 10 client.25762 handle_command_reply: tid=0 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.486+0000 7f634de0d740 1 client.25762 shutdown 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.487+0000 7f634de0d740 1 -- 172.21.15.60:0/2272535672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922e2050 msgr2=0x55b4922edd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.487+0000 7f634de0d740 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922e2050 0x55b4922edd20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f633c009e80 tx=0x7f633c004a40 comp rx=0 tx=0).stop 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.487+0000 7f634de0d740 1 -- 172.21.15.60:0/2272535672 shutdown_connections 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.487+0000 7f634de0d740 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b4922f2270 0x55b4922f46b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.487+0000 7f632e000640 0 client.25762 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.487+0000 7f634de0d740 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922e2050 0x55b4922edd20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.487+0000 7f634de0d740 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dcee0 0x55b4922e6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.487+0000 7f634de0d740 1 --2- 172.21.15.60:0/2272535672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dc4f0 0x55b4922e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.487+0000 7f634de0d740 1 -- 172.21.15.60:0/2272535672 >> 172.21.15.60:0/2272535672 conn(0x55b4922cf250 msgr2=0x55b4922ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:54.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.488+0000 7f634de0d740 1 -- 172.21.15.60:0/2272535672 shutdown_connections 2024-02-26T10:46:54.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.488+0000 7f634de0d740 1 -- 172.21.15.60:0/2272535672 wait complete. 2024-02-26T10:46:54.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.488+0000 7f634de0d740 20 client.25762 trim_cache size 0 max 16384 2024-02-26T10:46:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.504+0000 7f634de0d740 1 Processor -- start 2024-02-26T10:46:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.505+0000 7f634de0d740 1 -- start start 2024-02-26T10:46:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.505+0000 7f634de0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 0x55b4922f9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.505+0000 7f634de0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 0x55b4922edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.505+0000 7f634de0d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.505+0000 7f634de0d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441a9f90 con 0x55b4922e2050 2024-02-26T10:46:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.505+0000 7f634de0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441a7b30 con 0x55b4922dcee0 2024-02-26T10:46:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.505+0000 7f634de0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441a5bf0 con 0x55b4922dc4f0 2024-02-26T10:46:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 0x55b4922f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 0x55b4922f9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:45986/0 (socket says 172.21.15.60:45986) 2024-02-26T10:46:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343e00640 1 -- 172.21.15.60:0/2855311023 learned_addr learned my addr 172.21.15.60:0/2855311023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343400640 1 --2- 172.21.15.60:0/2855311023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 0x55b4922edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6349000640 1 --2- 172.21.15.60:0/2855311023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343400640 1 -- 172.21.15.60:0/2855311023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 msgr2=0x55b4922f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343400640 1 --2- 172.21.15.60:0/2855311023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 0x55b4922f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343400640 1 -- 172.21.15.60:0/2855311023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 msgr2=0x55b4922e67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343400640 1 --2- 172.21.15.60:0/2855311023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343400640 1 -- 172.21.15.60:0/2855311023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4922e15d0 con 0x55b4922dcee0 2024-02-26T10:46:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.506+0000 7f6343400640 1 --2- 172.21.15.60:0/2855311023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 0x55b4922edce0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f633009abe0 tx=0x7f633000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda6a8be1a89124b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f6338c00640 1 -- 172.21.15.60:0/2855311023 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300dd030 con 0x55b4922dcee0 2024-02-26T10:46:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f6338c00640 1 -- 172.21.15.60:0/2855311023 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f633009aee0 con 0x55b4922dcee0 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f6338c00640 1 -- 172.21.15.60:0/2855311023 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633009ef80 con 0x55b4922dcee0 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f634de0d740 1 -- 172.21.15.60:0/2855311023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 msgr2=0x55b4922edce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f634de0d740 1 --2- 172.21.15.60:0/2855311023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 0x55b4922edce0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f633009abe0 tx=0x7f633000cf60 comp rx=0 tx=0).stop 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f634de0d740 1 -- 172.21.15.60:0/2855311023 shutdown_connections 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f634de0d740 1 --2- 172.21.15.60:0/2855311023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922dcee0 0x55b4922edce0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f634de0d740 1 --2- 172.21.15.60:0/2855311023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922e2050 0x55b4922e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f634de0d740 1 --2- 172.21.15.60:0/2855311023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dc4f0 0x55b4922f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.507+0000 7f634de0d740 1 -- 172.21.15.60:0/2855311023 >> 172.21.15.60:0/2855311023 conn(0x55b4922cf250 msgr2=0x55b4922e5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.508+0000 7f634de0d740 1 -- 172.21.15.60:0/2855311023 shutdown_connections 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.508+0000 7f634de0d740 1 -- 172.21.15.60:0/2855311023 wait complete. 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.508+0000 7f634de0d740 1 -- start start 2024-02-26T10:46:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.508+0000 7f634de0d740 1 Processor -- start 2024-02-26T10:46:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f634de0d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dc4f0 0x55b4922ebf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f634de0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dcee0 0x55b4922ec480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f634de0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922e2050 0x55b4922ed2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f634de0d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441a61f0 con 0x55b4922dc4f0 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f634de0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441a74d0 con 0x55b4922e2050 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f634de0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441a6810 con 0x55b4922dcee0 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f634de0d740 10 client.0 _ll_register_callbacks cb 0x7f633c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f6343e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dc4f0 0x55b4922ebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f6343400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dcee0 0x55b4922ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f6343e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dc4f0 0x55b4922ebf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38890/0 (socket says 172.21.15.60:38890) 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f6343e00640 1 -- 172.21.15.60:0/2381329416 learned_addr learned my addr 172.21.15.60:0/2381329416 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.509+0000 7f6349000640 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922e2050 0x55b4922ed2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f6343e00640 1 -- 172.21.15.60:0/2381329416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dcee0 msgr2=0x55b4922ec480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f6343e00640 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dcee0 0x55b4922ec480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f6343e00640 1 -- 172.21.15.60:0/2381329416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922e2050 msgr2=0x55b4922ed2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f6343e00640 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922e2050 0x55b4922ed2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f6343e00640 1 -- 172.21.15.60:0/2381329416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4922ed990 con 0x55b4922dc4f0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f6343e00640 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dc4f0 0x55b4922ebf40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f633402c250 tx=0x7f6334000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c16ecedf9def752 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f632e000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f632e000640 1 -- 172.21.15.60:0/2381329416 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340496a0 con 0x55b4922dc4f0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f632e000640 1 -- 172.21.15.60:0/2381329416 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f6334049840 con 0x55b4922dc4f0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.510+0000 7f634de0d740 1 -- 172.21.15.60:0/2381329416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f633009aee0 con 0x55b4922dc4f0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f632e000640 1 -- 172.21.15.60:0/2381329416 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f633405b050 con 0x55b4922dc4f0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f634de0d740 10 client.15639 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f634de0d740 1 -- 172.21.15.60:0/2381329416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b492349220 con 0x55b4922dc4f0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f632e000640 1 -- 172.21.15.60:0/2381329416 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f633009aee0 con 0x55b4922dc4f0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f632e000640 1 -- 172.21.15.60:0/2381329416 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6334047050 con 0x55b4922dc4f0 2024-02-26T10:46:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f634de0d740 10 client.15639 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f634de0d740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f634de0d740 20 client.15639 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f634de0d740 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b492349bb0 0x55b49234bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f634de0d740 4 client.15639 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T10:46:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.511+0000 7f634de0d740 1 -- 172.21.15.60:0/2381329416 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b492240590 con 0x55b492349bb0 2024-02-26T10:46:54.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.512+0000 7f6343400640 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b492349bb0 0x55b49234bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.512+0000 7f6343400640 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b492349bb0 0x55b49234bfd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ff15d34be94f05 server_cookie=b14fc85e949b09e2 in_seq=0 out_seq=0 2024-02-26T10:46:54.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.512+0000 7f632e000640 10 client.15639 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:54.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f632e000640 1 -- 172.21.15.60:0/2381329416 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b492240590 con 0x55b492349bb0 2024-02-26T10:46:54.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f632e000640 10 client.15639 handle_command_reply: tid=0 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 client.15639 shutdown 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 -- 172.21.15.60:0/2381329416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dc4f0 msgr2=0x55b4922ebf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dc4f0 0x55b4922ebf40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f633402c250 tx=0x7f6334000bc0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 -- 172.21.15.60:0/2381329416 shutdown_connections 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b492349bb0 0x55b49234bfd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4922e2050 0x55b4922ed2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4922dcee0 0x55b4922ec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f632e000640 0 client.15639 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 --2- 172.21.15.60:0/2381329416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4922dc4f0 0x55b4922ebf40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 -- 172.21.15.60:0/2381329416 >> 172.21.15.60:0/2381329416 conn(0x55b4922cf250 msgr2=0x55b4922f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 -- 172.21.15.60:0/2381329416 shutdown_connections 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.513+0000 7f634de0d740 1 -- 172.21.15.60:0/2381329416 wait complete. 2024-02-26T10:46:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f634de0d740 20 client.15639 trim_cache size 0 max 16384 2024-02-26T10:46:54.511 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:46:54.511 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-02-26T10:46:54.511 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-02-26T10:46:54.511 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:46:54.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 -- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f63280a1e90 msgr2=0x7f63280a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f63280a1e90 0x7f63280a4350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55b4922dd740 tx=0x7f633c009500 comp rx=0 tx=0).stop 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 -- 172.21.15.60:0/60380413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6344102350 msgr2=0x7f6344199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6344102350 0x7f6344199bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f633000e9f0 tx=0x7f633000eeb0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 -- 172.21.15.60:0/60380413 shutdown_connections 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f63280a1e90 0x7f63280a4350 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6344102350 0x7f6344199bb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6344102d10 0x7f63441a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 --2- 172.21.15.60:0/60380413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6344101160 0x7f6344199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.514+0000 7f6349a00640 1 -- 172.21.15.60:0/60380413 >> 172.21.15.60:0/60380413 conn(0x7f63440fc6d0 msgr2=0x7f6344073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.515+0000 7f6349a00640 1 -- 172.21.15.60:0/60380413 shutdown_connections 2024-02-26T10:46:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.515+0000 7f6349a00640 1 -- 172.21.15.60:0/60380413 wait complete. 2024-02-26T10:46:54.527 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T10:46:54.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:46:54.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:54 smithi060 ceph-mon[31539]: pgmap v453: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T10:46:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.708+0000 7fa2f6e00640 1 -- 172.21.15.60:0/2723630313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 msgr2=0x7fa2f01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.708+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/2723630313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 0x7fa2f01014e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009900 tx=0x7fa2e402bc30 comp rx=0 tx=0).stop 2024-02-26T10:46:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.708+0000 7fa2f6e00640 1 -- 172.21.15.60:0/2723630313 shutdown_connections 2024-02-26T10:46:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.708+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/2723630313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f01022b0 0x7fa2f0102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.708+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/2723630313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2f0102c70 0x7fa2f0109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.708+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/2723630313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 0x7fa2f01014e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.708+0000 7fa2f6e00640 1 -- 172.21.15.60:0/2723630313 >> 172.21.15.60:0/2723630313 conn(0x7fa2f00fc630 msgr2=0x7fa2f00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.708+0000 7fa2f6e00640 1 -- 172.21.15.60:0/2723630313 shutdown_connections 2024-02-26T10:46:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.708+0000 7fa2f6e00640 1 -- 172.21.15.60:0/2723630313 wait complete. 2024-02-26T10:46:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.709+0000 7fa2f6e00640 1 Processor -- start 2024-02-26T10:46:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.709+0000 7fa2f6e00640 1 -- start start 2024-02-26T10:46:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f6e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 0x7fa2f01973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f6e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f01022b0 0x7fa2f0197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f6e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2f0102c70 0x7fa2f019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f6e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f010cd70 con 0x7fa2f0102c70 2024-02-26T10:46:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f6e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f010cbf0 con 0x7fa2f01022b0 2024-02-26T10:46:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f6e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f010cef0 con 0x7fa2f01010c0 2024-02-26T10:46:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 0x7fa2f01973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f6400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2f0102c70 0x7fa2f019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 0x7fa2f01973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46000/0 (socket says 172.21.15.60:46000) 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f5a00640 1 -- 172.21.15.60:0/561307979 learned_addr learned my addr 172.21.15.60:0/561307979 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.710+0000 7fa2f5000640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f01022b0 0x7fa2f0197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2f5a00640 1 -- 172.21.15.60:0/561307979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f01022b0 msgr2=0x7fa2f0197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2f5a00640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f01022b0 0x7fa2f0197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2f5a00640 1 -- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2f0102c70 msgr2=0x7fa2f019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2f5a00640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2f0102c70 0x7fa2f019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2f5a00640 1 -- 172.21.15.60:0/561307979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f019f010 con 0x7fa2f01010c0 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2f6400640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2f0102c70 0x7fa2f019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2f5000640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f01022b0 0x7fa2f0197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2f5a00640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 0x7fa2f01973f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c00 tx=0x7fa2e402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2ee000640 1 -- 172.21.15.60:0/561307979 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4042c60 con 0x7fa2f01010c0 2024-02-26T10:46:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.711+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f0073490 con 0x7fa2f01010c0 2024-02-26T10:46:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.712+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f0073a30 con 0x7fa2f01010c0 2024-02-26T10:46:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.712+0000 7fa2ee000640 1 -- 172.21.15.60:0/561307979 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2e4042e00 con 0x7fa2f01010c0 2024-02-26T10:46:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.712+0000 7fa2ee000640 1 -- 172.21.15.60:0/561307979 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40038e0 con 0x7fa2f01010c0 2024-02-26T10:46:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.713+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d00052c0 con 0x7fa2f01010c0 2024-02-26T10:46:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.716+0000 7fa2ee000640 1 -- 172.21.15.60:0/561307979 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa2e4004040 con 0x7fa2f01010c0 2024-02-26T10:46:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.717+0000 7fa2ee000640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa2d80a1e90 0x7fa2d80a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:54.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.717+0000 7fa2ee000640 1 -- 172.21.15.60:0/561307979 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa2e40f8220 con 0x7fa2f01010c0 2024-02-26T10:46:54.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.718+0000 7fa2f5000640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa2d80a1e90 0x7fa2d80a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:54.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.718+0000 7fa2f5000640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa2d80a1e90 0x7fa2d80a4350 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0198900 tx=0x7fa2dc012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.722+0000 7fa2ee000640 1 -- 172.21.15.60:0/561307979 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2e40c51f0 con 0x7fa2f01010c0 2024-02-26T10:46:54.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:54 smithi113 ceph-mon[36126]: pgmap v453: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T10:46:54.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:54 smithi100 ceph-mon[36051]: pgmap v453: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T10:46:54.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.954+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2d0005880 con 0x7fa2f01010c0 2024-02-26T10:46:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.955+0000 7fa2ee000640 1 -- 172.21.15.60:0/561307979 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11359 (secure 0 0 0) 0x7fa2e40c4af0 con 0x7fa2f01010c0 2024-02-26T10:46:54.950 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:46:54.950 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:45:39.355500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:46:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2024-02-26T10:46:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa2d80a1e90 msgr2=0x7fa2d80a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa2d80a1e90 0x7fa2d80a4350 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0198900 tx=0x7fa2dc012040 comp rx=0 tx=0).stop 2024-02-26T10:46:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 msgr2=0x7fa2f01973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 0x7fa2f01973f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c00 tx=0x7fa2e402c650 comp rx=0 tx=0).stop 2024-02-26T10:46:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 shutdown_connections 2024-02-26T10:46:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa2d80a1e90 0x7fa2d80a4350 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2f01022b0 0x7fa2f0197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2f0102c70 0x7fa2f019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 --2- 172.21.15.60:0/561307979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f01010c0 0x7fa2f01973f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 >> 172.21.15.60:0/561307979 conn(0x7fa2f00fc630 msgr2=0x7fa2f01056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 shutdown_connections 2024-02-26T10:46:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:54.958+0000 7fa2f6e00640 1 -- 172.21.15.60:0/561307979 wait complete. 2024-02-26T10:46:54.969 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:46:54.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T10:46:55.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.155+0000 7fb8caa00640 1 -- 172.21.15.60:0/735477191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8c4073c60 msgr2=0x7fb8c40740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.155+0000 7fb8caa00640 1 --2- 172.21.15.60:0/735477191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8c4073c60 0x7fb8c40740c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009900 tx=0x7fb8b002bcd0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.155+0000 7fb8caa00640 1 -- 172.21.15.60:0/735477191 shutdown_connections 2024-02-26T10:46:55.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.155+0000 7fb8caa00640 1 --2- 172.21.15.60:0/735477191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8c4073c60 0x7fb8c40740c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.155+0000 7fb8caa00640 1 --2- 172.21.15.60:0/735477191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c4108ff0 0x7fb8c410b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.155+0000 7fb8caa00640 1 --2- 172.21.15.60:0/735477191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c4075240 0x7fb8c4073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.155+0000 7fb8caa00640 1 -- 172.21.15.60:0/735477191 >> 172.21.15.60:0/735477191 conn(0x7fb8c40fc6d0 msgr2=0x7fb8c40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.155+0000 7fb8caa00640 1 -- 172.21.15.60:0/735477191 shutdown_connections 2024-02-26T10:46:55.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.155+0000 7fb8caa00640 1 -- 172.21.15.60:0/735477191 wait complete. 2024-02-26T10:46:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.157+0000 7fb8caa00640 1 Processor -- start 2024-02-26T10:46:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.157+0000 7fb8caa00640 1 -- start start 2024-02-26T10:46:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.157+0000 7fb8caa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c4073c60 0x7fb8c4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.157+0000 7fb8caa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c4075240 0x7fb8c4199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.157+0000 7fb8caa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8c4108ff0 0x7fb8c41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.157+0000 7fb8caa00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c410f040 con 0x7fb8c4073c60 2024-02-26T10:46:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.157+0000 7fb8caa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c410eec0 con 0x7fb8c4108ff0 2024-02-26T10:46:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.157+0000 7fb8caa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c410f1c0 con 0x7fb8c4075240 2024-02-26T10:46:55.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c4073c60 0x7fb8c4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c4073c60 0x7fb8c4199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38920/0 (socket says 172.21.15.60:38920) 2024-02-26T10:46:55.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c9600640 1 -- 172.21.15.60:0/3360588882 learned_addr learned my addr 172.21.15.60:0/3360588882 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c4075240 0x7fb8c4199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c9600640 1 -- 172.21.15.60:0/3360588882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c4075240 msgr2=0x7fb8c4199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c9600640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c4075240 0x7fb8c4199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c9600640 1 -- 172.21.15.60:0/3360588882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8c4108ff0 msgr2=0x7fb8c41a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8ca000640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8c4108ff0 0x7fb8c41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c9600640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8c4108ff0 0x7fb8c41a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c9600640 1 -- 172.21.15.60:0/3360588882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c41a1250 con 0x7fb8c4073c60 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c4075240 0x7fb8c4199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.158+0000 7fb8ca000640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8c4108ff0 0x7fb8c41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.159+0000 7fb8c9600640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c4073c60 0x7fb8c4199630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e9f0 tx=0x7fb8b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.159+0000 7fb8c2000640 1 -- 172.21.15.60:0/3360588882 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800cd80 con 0x7fb8c4073c60 2024-02-26T10:46:55.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.159+0000 7fb8caa00640 1 -- 172.21.15.60:0/3360588882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c41a1540 con 0x7fb8c4073c60 2024-02-26T10:46:55.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.159+0000 7fb8c2000640 1 -- 172.21.15.60:0/3360588882 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b8010910 con 0x7fb8c4073c60 2024-02-26T10:46:55.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.159+0000 7fb8c2000640 1 -- 172.21.15.60:0/3360588882 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80188f0 con 0x7fb8c4073c60 2024-02-26T10:46:55.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.160+0000 7fb8caa00640 1 -- 172.21.15.60:0/3360588882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c41a1ab0 con 0x7fb8c4073c60 2024-02-26T10:46:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.164+0000 7fb8c2000640 1 -- 172.21.15.60:0/3360588882 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fb8b8010ac0 con 0x7fb8c4073c60 2024-02-26T10:46:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.164+0000 7fb8cee82740 1 Processor -- start 2024-02-26T10:46:55.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.165+0000 7fb8cee82740 1 -- start start 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.165+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 0x555737cffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.165+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d04b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.165+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d07640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.165+0000 7fb8cee82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555737c5bd30 con 0x555737cff650 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.165+0000 7fb8cee82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555737c04cc0 con 0x555737d051b0 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.165+0000 7fb8cee82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555737c57dd0 con 0x555737d00040 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.166+0000 7fb8c9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d04b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.166+0000 7fb8c9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d04b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46024/0 (socket says 172.21.15.60:46024) 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.166+0000 7fb8c9600640 1 -- 172.21.15.60:0/3678151171 learned_addr learned my addr 172.21.15.60:0/3678151171 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.166+0000 7fb8ca000640 1 --2- 172.21.15.60:0/3678151171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 0x555737cffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.166+0000 7fb8c9600640 1 -- 172.21.15.60:0/3678151171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 msgr2=0x555737d07640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.166+0000 7fb8c9600640 1 --2- 172.21.15.60:0/3678151171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d07640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.166+0000 7fb8c9600640 1 -- 172.21.15.60:0/3678151171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 msgr2=0x555737cffa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.166+0000 7fb8c9600640 1 --2- 172.21.15.60:0/3678151171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 0x555737cffa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.166+0000 7fb8c9600640 1 -- 172.21.15.60:0/3678151171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557379703f0 con 0x555737d00040 2024-02-26T10:46:55.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.167+0000 7fb8c2000640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8ac0a1e90 0x7fb8ac0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.167+0000 7fb8c9600640 1 --2- 172.21.15.60:0/3678151171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d04b40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80d4bd0 tx=0x7fb8b80d6650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdeacbc98c2a1664 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.167+0000 7fb8b6600640 1 -- 172.21.15.60:0/3678151171 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8097460 con 0x555737d00040 2024-02-26T10:46:55.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.167+0000 7fb8c2000640 1 -- 172.21.15.60:0/3360588882 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb8b80cea00 con 0x7fb8c4073c60 2024-02-26T10:46:55.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.167+0000 7fb8b6600640 1 -- 172.21.15.60:0/3678151171 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b801ae30 con 0x555737d00040 2024-02-26T10:46:55.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.167+0000 7fb8b6600640 1 -- 172.21.15.60:0/3678151171 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80a7910 con 0x555737d00040 2024-02-26T10:46:55.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.167+0000 7fb8ca000640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8ac0a1e90 0x7fb8ac0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.168+0000 7fb8cee82740 1 -- 172.21.15.60:0/3678151171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 msgr2=0x555737d04b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.168+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3678151171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d04b40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80d4bd0 tx=0x7fb8b80d6650 comp rx=0 tx=0).stop 2024-02-26T10:46:55.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.168+0000 7fb8ca000640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8ac0a1e90 0x7fb8ac0a4350 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x555737d008a0 tx=0x7fb8bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.169+0000 7fb8cee82740 1 -- 172.21.15.60:0/3678151171 shutdown_connections 2024-02-26T10:46:55.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.169+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3678151171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d07640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.169+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3678151171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d04b40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.169+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3678151171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 0x555737cffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.169+0000 7fb8cee82740 1 -- 172.21.15.60:0/3678151171 >> 172.21.15.60:0/3678151171 conn(0x555737cf23b0 msgr2=0x555737cf27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.169+0000 7fb8cee82740 1 -- 172.21.15.60:0/3678151171 shutdown_connections 2024-02-26T10:46:55.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.169+0000 7fb8cee82740 1 -- 172.21.15.60:0/3678151171 wait complete. 2024-02-26T10:46:55.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.169+0000 7fb8cee82740 1 -- start start 2024-02-26T10:46:55.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.169+0000 7fb8cee82740 1 Processor -- start 2024-02-26T10:46:55.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 0x555737d09b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d0a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8cee82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555737d08100 con 0x555737cff650 2024-02-26T10:46:55.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8cee82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555737bfa1f0 con 0x555737d051b0 2024-02-26T10:46:55.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8cee82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555737d08280 con 0x555737d00040 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8cee82740 10 client.0 _ll_register_callbacks cb 0x55573796fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8c9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 0x555737d09b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8c8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d0a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8c8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d0a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46030/0 (socket says 172.21.15.60:46030) 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.170+0000 7fb8c8c00640 1 -- 172.21.15.60:0/813204372 learned_addr learned my addr 172.21.15.60:0/813204372 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.171+0000 7fb8ca000640 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.171+0000 7fb8c8c00640 1 -- 172.21.15.60:0/813204372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 msgr2=0x555737d110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.171+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.171+0000 7fb8c8c00640 1 -- 172.21.15.60:0/813204372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 msgr2=0x555737d09b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.171+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 0x555737d09b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.171+0000 7fb8c8c00640 1 -- 172.21.15.60:0/813204372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555737a3a7d0 con 0x555737d00040 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.171+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d0a060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009900 tx=0x7fb8b0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9c36294adff701 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.171+0000 7fb8aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:46:55.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.172+0000 7fb8aaa00640 1 -- 172.21.15.60:0/813204372 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0044020 con 0x555737d00040 2024-02-26T10:46:55.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.172+0000 7fb8cee82740 1 -- 172.21.15.60:0/813204372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8b801ae30 con 0x555737d00040 2024-02-26T10:46:55.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.172+0000 7fb8aaa00640 1 -- 172.21.15.60:0/813204372 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb8b0005240 con 0x555737d00040 2024-02-26T10:46:55.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.172+0000 7fb8aaa00640 1 -- 172.21.15.60:0/813204372 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8b004f050 con 0x555737d00040 2024-02-26T10:46:55.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.172+0000 7fb8cee82740 10 client.25786 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:55.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.172+0000 7fb8cee82740 1 -- 172.21.15.60:0/813204372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555737d144d0 con 0x555737d00040 2024-02-26T10:46:55.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.172+0000 7fb8aaa00640 1 -- 172.21.15.60:0/813204372 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b801ae30 con 0x555737d00040 2024-02-26T10:46:55.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.172+0000 7fb8aaa00640 1 -- 172.21.15.60:0/813204372 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb8b00035f0 con 0x555737d00040 2024-02-26T10:46:55.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.173+0000 7fb8cee82740 10 client.25786 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:55.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.173+0000 7fb8cee82740 10 client.25786 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-26T10:46:55.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.173+0000 7fb8cee82740 20 client.25786 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:55.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.173+0000 7fb8cee82740 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x555737d15b60 0x555737d17fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.173+0000 7fb8cee82740 4 client.25786 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:55.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.173+0000 7fb8cee82740 1 -- 172.21.15.60:0/813204372 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555737c69270 con 0x555737d15b60 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.173+0000 7fb8c9600640 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 0x555737d09b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.174+0000 7fb8c9600640 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x555737d15b60 0x555737d17fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.174+0000 7fb8aaa00640 10 client.25786 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.174+0000 7fb8c9600640 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x555737d15b60 0x555737d17fe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9662d1d00a24075 server_cookie=2335c016aed5f32 in_seq=0 out_seq=0 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8aaa00640 1 -- 172.21.15.60:0/813204372 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555737c69270 con 0x555737d15b60 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8aaa00640 10 client.25786 handle_command_reply: tid=0 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 client.25786 shutdown 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 -- 172.21.15.60:0/813204372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 msgr2=0x555737d0a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d0a060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009900 tx=0x7fb8b0004270 comp rx=0 tx=0).stop 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 -- 172.21.15.60:0/813204372 shutdown_connections 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x555737d15b60 0x555737d17fe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d00040 0x555737d0a060 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 --2- 172.21.15.60:0/813204372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737cff650 0x555737d09b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 -- 172.21.15.60:0/813204372 >> 172.21.15.60:0/813204372 conn(0x555737cf23b0 msgr2=0x555737cf16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8aaa00640 0 client.25786 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:46:55.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 -- 172.21.15.60:0/813204372 shutdown_connections 2024-02-26T10:46:55.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.175+0000 7fb8cee82740 1 -- 172.21.15.60:0/813204372 wait complete. 2024-02-26T10:46:55.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.176+0000 7fb8cee82740 20 client.25786 trim_cache size 0 max 16384 2024-02-26T10:46:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.189+0000 7fb8cee82740 1 Processor -- start 2024-02-26T10:46:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.189+0000 7fb8cee82740 1 -- start start 2024-02-26T10:46:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.189+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737cff650 0x555737cf0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.189+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d11060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.189+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d051b0 0x555737cf1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.189+0000 7fb8cee82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a9e60 con 0x555737d00040 2024-02-26T10:46:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.189+0000 7fb8cee82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a7a00 con 0x555737cff650 2024-02-26T10:46:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.189+0000 7fb8cee82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a5ac0 con 0x555737d051b0 2024-02-26T10:46:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.189+0000 7fb8c9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737cff650 0x555737cf0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8c9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737cff650 0x555737cf0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40816/0 (socket says 172.21.15.60:40816) 2024-02-26T10:46:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8c9600640 1 -- 172.21.15.60:0/4159665408 learned_addr learned my addr 172.21.15.60:0/4159665408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d11060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8ca000640 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d051b0 0x555737cf1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8c9600640 1 -- 172.21.15.60:0/4159665408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d051b0 msgr2=0x555737cf1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8c9600640 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d051b0 0x555737cf1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8c9600640 1 -- 172.21.15.60:0/4159665408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 msgr2=0x555737d11060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8c9600640 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d11060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8c9600640 1 -- 172.21.15.60:0/4159665408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555737d008d0 con 0x555737cff650 2024-02-26T10:46:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8ca000640 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d051b0 0x555737cf1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.190+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d11060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.191+0000 7fb8c9600640 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737cff650 0x555737cf0e40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8097b40 tx=0x7fb8b8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f1a00918bd0aad3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.191+0000 7fb8b5c00640 1 -- 172.21.15.60:0/4159665408 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80a7070 con 0x555737cff650 2024-02-26T10:46:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.191+0000 7fb8b5c00640 1 -- 172.21.15.60:0/4159665408 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b809b8c0 con 0x555737cff650 2024-02-26T10:46:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.191+0000 7fb8b5c00640 1 -- 172.21.15.60:0/4159665408 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80a7b10 con 0x555737cff650 2024-02-26T10:46:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.192+0000 7fb8cee82740 1 -- 172.21.15.60:0/4159665408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737cff650 msgr2=0x555737cf0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.192+0000 7fb8cee82740 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737cff650 0x555737cf0e40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8097b40 tx=0x7fb8b8009310 comp rx=0 tx=0).stop 2024-02-26T10:46:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.192+0000 7fb8cee82740 1 -- 172.21.15.60:0/4159665408 shutdown_connections 2024-02-26T10:46:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.192+0000 7fb8cee82740 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737d051b0 0x555737cf1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.192+0000 7fb8cee82740 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d11060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.192+0000 7fb8cee82740 1 --2- 172.21.15.60:0/4159665408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737cff650 0x555737cf0e40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.192+0000 7fb8cee82740 1 -- 172.21.15.60:0/4159665408 >> 172.21.15.60:0/4159665408 conn(0x555737cf23b0 msgr2=0x555737d015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.192+0000 7fb8cee82740 1 -- 172.21.15.60:0/4159665408 shutdown_connections 2024-02-26T10:46:55.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.192+0000 7fb8cee82740 1 -- 172.21.15.60:0/4159665408 wait complete. 2024-02-26T10:46:55.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.193+0000 7fb8cee82740 1 -- start start 2024-02-26T10:46:55.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.193+0000 7fb8cee82740 1 Processor -- start 2024-02-26T10:46:55.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.193+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737cff650 0x555737d0f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.193+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d0f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.193+0000 7fb8cee82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d10560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8cee82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a60c0 con 0x555737d00040 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8cee82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a73a0 con 0x555737d051b0 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8cee82740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a66e0 con 0x555737cff650 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8cee82740 10 client.0 _ll_register_callbacks cb 0x7fb8b80d43e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8ca000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d10560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8ca000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d10560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40832/0 (socket says 172.21.15.60:40832) 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8c9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737cff650 0x555737d0f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8ca000640 1 -- 172.21.15.60:0/3320304118 learned_addr learned my addr 172.21.15.60:0/3320304118 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8c9600640 1 -- 172.21.15.60:0/3320304118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 msgr2=0x555737d10560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8c9600640 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d10560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d0f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8c9600640 1 -- 172.21.15.60:0/3320304118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 msgr2=0x555737d0f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8c9600640 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d0f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.194+0000 7fb8c9600640 1 -- 172.21.15.60:0/3320304118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555737d10c50 con 0x555737cff650 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.195+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d0f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.195+0000 7fb8c9600640 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737cff650 0x555737d0f1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x555737cff410 tx=0x7fb8b8023280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f132a610a6ef76 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.195+0000 7fb8aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.195+0000 7fb8aaa00640 1 -- 172.21.15.60:0/3320304118 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80a7050 con 0x555737cff650 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.195+0000 7fb8aaa00640 1 -- 172.21.15.60:0/3320304118 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fb8b80d46b0 con 0x555737cff650 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.195+0000 7fb8cee82740 1 -- 172.21.15.60:0/3320304118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8b809b8c0 con 0x555737cff650 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.195+0000 7fb8aaa00640 1 -- 172.21.15.60:0/3320304118 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8b80a7d10 con 0x555737cff650 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.195+0000 7fb8cee82740 10 client.25792 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.195+0000 7fb8cee82740 1 -- 172.21.15.60:0/3320304118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555737d6ca30 con 0x555737cff650 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.196+0000 7fb8aaa00640 1 -- 172.21.15.60:0/3320304118 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b809b8c0 con 0x555737cff650 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.196+0000 7fb8aaa00640 1 -- 172.21.15.60:0/3320304118 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb8b80a9030 con 0x555737cff650 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.196+0000 7fb8cee82740 10 client.25792 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.196+0000 7fb8cee82740 10 client.25792 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-26T10:46:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.196+0000 7fb8cee82740 20 client.25792 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.196+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x555737d6ce80 0x555737d6f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.196+0000 7fb8cee82740 4 client.25792 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:46:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.196+0000 7fb8cee82740 1 -- 172.21.15.60:0/3320304118 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555737c635c0 con 0x555737d6ce80 2024-02-26T10:46:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.196+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x555737d6ce80 0x555737d6f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8c8c00640 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x555737d6ce80 0x555737d6f2a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21bb27ff0591b29c server_cookie=d84f6eedacbd492b in_seq=0 out_seq=0 2024-02-26T10:46:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8aaa00640 10 client.25792 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:46:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8aaa00640 1 -- 172.21.15.60:0/3320304118 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555737c635c0 con 0x555737d6ce80 2024-02-26T10:46:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8aaa00640 10 client.25792 handle_command_reply: tid=0 2024-02-26T10:46:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8cee82740 1 client.25792 shutdown 2024-02-26T10:46:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8cee82740 1 -- 172.21.15.60:0/3320304118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737cff650 msgr2=0x555737d0f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737cff650 0x555737d0f1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x555737cff410 tx=0x7fb8b8023280 comp rx=0 tx=0).stop 2024-02-26T10:46:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8cee82740 1 -- 172.21.15.60:0/3320304118 shutdown_connections 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x555737d6ce80 0x555737d6f2a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555737d051b0 0x555737d10560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555737d00040 0x555737d0f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8cee82740 1 --2- 172.21.15.60:0/3320304118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555737cff650 0x555737d0f1a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8aaa00640 0 client.25792 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.197+0000 7fb8cee82740 1 -- 172.21.15.60:0/3320304118 >> 172.21.15.60:0/3320304118 conn(0x555737cf23b0 msgr2=0x555737cf5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.198+0000 7fb8cee82740 1 -- 172.21.15.60:0/3320304118 shutdown_connections 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.198+0000 7fb8cee82740 1 -- 172.21.15.60:0/3320304118 wait complete. 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.198+0000 7fb8cee82740 20 client.25792 trim_cache size 0 max 16384 2024-02-26T10:46:55.195 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:46:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.198+0000 7fb8caa00640 1 -- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8ac0a1e90 msgr2=0x7fb8ac0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.198+0000 7fb8caa00640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8ac0a1e90 0x7fb8ac0a4350 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x555737d008a0 tx=0x7fb8bc009500 comp rx=0 tx=0).stop 2024-02-26T10:46:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.198+0000 7fb8caa00640 1 -- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c4073c60 msgr2=0x7fb8c4199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.198+0000 7fb8caa00640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c4073c60 0x7fb8c4199630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e9f0 tx=0x7fb8b800eeb0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.199+0000 7fb8caa00640 1 -- 172.21.15.60:0/3360588882 shutdown_connections 2024-02-26T10:46:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.199+0000 7fb8caa00640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fb8ac0a1e90 0x7fb8ac0a4350 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.199+0000 7fb8caa00640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8c4108ff0 0x7fb8c41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.199+0000 7fb8caa00640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c4075240 0x7fb8c4199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.199+0000 7fb8caa00640 1 --2- 172.21.15.60:0/3360588882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c4073c60 0x7fb8c4199630 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.199+0000 7fb8caa00640 1 -- 172.21.15.60:0/3360588882 >> 172.21.15.60:0/3360588882 conn(0x7fb8c40fc6d0 msgr2=0x7fb8c4104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.199+0000 7fb8caa00640 1 -- 172.21.15.60:0/3360588882 shutdown_connections 2024-02-26T10:46:55.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.199+0000 7fb8caa00640 1 -- 172.21.15.60:0/3360588882 wait complete. 2024-02-26T10:46:55.209 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:46:55.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T10:46:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.400+0000 7f6e5ae00640 1 -- 172.21.15.60:0/321449834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e54102350 msgr2=0x7f6e541027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.400+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/321449834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e54102350 0x7f6e541027d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009900 tx=0x7f6e4002bc30 comp rx=0 tx=0).stop 2024-02-26T10:46:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.400+0000 7f6e5ae00640 1 -- 172.21.15.60:0/321449834 shutdown_connections 2024-02-26T10:46:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.400+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/321449834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e54102d10 0x7f6e541095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.400+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/321449834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e54102350 0x7f6e541027d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.400+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/321449834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e54101160 0x7f6e54101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.400+0000 7f6e5ae00640 1 -- 172.21.15.60:0/321449834 >> 172.21.15.60:0/321449834 conn(0x7f6e540fc6d0 msgr2=0x7f6e540feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.400+0000 7f6e5ae00640 1 -- 172.21.15.60:0/321449834 shutdown_connections 2024-02-26T10:46:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.400+0000 7f6e5ae00640 1 -- 172.21.15.60:0/321449834 wait complete. 2024-02-26T10:46:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.402+0000 7f6e5ae00640 1 Processor -- start 2024-02-26T10:46:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.402+0000 7f6e5ae00640 1 -- start start 2024-02-26T10:46:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.402+0000 7f6e5ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e54101160 0x7f6e54199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e5ae00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e54102350 0x7f6e54199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e5ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e54102d10 0x7f6e541a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e5ae00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5410cdf0 con 0x7f6e54102350 2024-02-26T10:46:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e5ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5410cc70 con 0x7f6e54102d10 2024-02-26T10:46:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e5ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5410cf70 con 0x7f6e54101160 2024-02-26T10:46:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e59a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e54101160 0x7f6e54199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e59a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e54101160 0x7f6e54199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46072/0 (socket says 172.21.15.60:46072) 2024-02-26T10:46:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e59a00640 1 -- 172.21.15.60:0/3411625768 learned_addr learned my addr 172.21.15.60:0/3411625768 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e5a400640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e54102d10 0x7f6e541a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.403+0000 7f6e59000640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e54102350 0x7f6e54199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.404+0000 7f6e5a400640 1 -- 172.21.15.60:0/3411625768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e54101160 msgr2=0x7f6e54199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.404+0000 7f6e5a400640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e54101160 0x7f6e54199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.404+0000 7f6e5a400640 1 -- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e54102350 msgr2=0x7f6e54199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.404+0000 7f6e5a400640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e54102350 0x7f6e54199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.404+0000 7f6e5a400640 1 -- 172.21.15.60:0/3411625768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e541a1250 con 0x7f6e54102d10 2024-02-26T10:46:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.404+0000 7f6e5a400640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e54102d10 0x7f6e541a0be0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009fd0 tx=0x7f6e4c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.404+0000 7f6e52000640 1 -- 172.21.15.60:0/3411625768 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00ce20 con 0x7f6e54102d10 2024-02-26T10:46:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.404+0000 7f6e52000640 1 -- 172.21.15.60:0/3411625768 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c005020 con 0x7f6e54102d10 2024-02-26T10:46:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.404+0000 7f6e52000640 1 -- 172.21.15.60:0/3411625768 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c010720 con 0x7f6e54102d10 2024-02-26T10:46:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.405+0000 7f6e5ae00640 1 -- 172.21.15.60:0/3411625768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e541a1540 con 0x7f6e54102d10 2024-02-26T10:46:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.405+0000 7f6e5ae00640 1 -- 172.21.15.60:0/3411625768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e541a1a10 con 0x7f6e54102d10 2024-02-26T10:46:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.409+0000 7f6e5f403740 1 Processor -- start 2024-02-26T10:46:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.410+0000 7f6e52000640 1 -- 172.21.15.60:0/3411625768 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f6e4c005540 con 0x7f6e54102d10 2024-02-26T10:46:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.410+0000 7f6e5f403740 1 -- start start 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.410+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff7faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.410+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff7ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.410+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff802640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.410+0000 7f6e5f403740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff756d30 con 0x561fff7fa650 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.410+0000 7f6e5f403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff6ffcc0 con 0x561fff7fb040 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.410+0000 7f6e5f403740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff752dd0 con 0x561fff8001b0 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.411+0000 7f6e59a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff7ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.411+0000 7f6e59000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff7faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.411+0000 7f6e59a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff7ffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40862/0 (socket says 172.21.15.60:40862) 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.411+0000 7f6e59a00640 1 -- 172.21.15.60:0/1791883618 learned_addr learned my addr 172.21.15.60:0/1791883618 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.411+0000 7f6e59000640 1 --2- 172.21.15.60:0/1791883618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff802640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.412+0000 7f6e59a00640 1 -- 172.21.15.60:0/1791883618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 msgr2=0x561fff802640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.412+0000 7f6e59a00640 1 --2- 172.21.15.60:0/1791883618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff802640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.412+0000 7f6e59a00640 1 -- 172.21.15.60:0/1791883618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 msgr2=0x561fff7faa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.412+0000 7f6e59a00640 1 --2- 172.21.15.60:0/1791883618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff7faa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.412+0000 7f6e59a00640 1 -- 172.21.15.60:0/1791883618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fff46b3f0 con 0x561fff7fb040 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.412+0000 7f6e59a00640 1 --2- 172.21.15.60:0/1791883618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff7ffb40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800be50 tx=0x7f6e480107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c27dc9fa9a16d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.412+0000 7f6e46a00640 1 -- 172.21.15.60:0/1791883618 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48011640 con 0x561fff7fb040 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.412+0000 7f6e46a00640 1 -- 172.21.15.60:0/1791883618 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e48011cc0 con 0x561fff7fb040 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.413+0000 7f6e46a00640 1 -- 172.21.15.60:0/1791883618 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48019c60 con 0x561fff7fb040 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.413+0000 7f6e52000640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6e3c0a1e10 0x7f6e3c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.413+0000 7f6e52000640 1 -- 172.21.15.60:0/3411625768 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f6e4c0ced40 con 0x7f6e54102d10 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e59000640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6e3c0a1e10 0x7f6e3c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e5f403740 1 -- 172.21.15.60:0/1791883618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 msgr2=0x561fff7ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e5f403740 1 --2- 172.21.15.60:0/1791883618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff7ffb40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800be50 tx=0x7f6e480107a0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e5f403740 1 -- 172.21.15.60:0/1791883618 shutdown_connections 2024-02-26T10:46:55.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e5f403740 1 --2- 172.21.15.60:0/1791883618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff7ffb40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e5f403740 1 --2- 172.21.15.60:0/1791883618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff802640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e5f403740 1 --2- 172.21.15.60:0/1791883618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff7faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e5f403740 1 -- 172.21.15.60:0/1791883618 >> 172.21.15.60:0/1791883618 conn(0x561fff7ed3b0 msgr2=0x561fff7ed7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e5f403740 1 -- 172.21.15.60:0/1791883618 shutdown_connections 2024-02-26T10:46:55.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.414+0000 7f6e5f403740 1 -- 172.21.15.60:0/1791883618 wait complete. 2024-02-26T10:46:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.415+0000 7f6e5f403740 1 -- start start 2024-02-26T10:46:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.415+0000 7f6e5f403740 1 Processor -- start 2024-02-26T10:46:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.415+0000 7f6e59000640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6e3c0a1e10 0x7f6e3c0a42d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6e4002c7b0 tx=0x7f6e40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.415+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff804ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.415+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff804ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.415+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.415+0000 7f6e5f403740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff803160 con 0x561fff7fb040 2024-02-26T10:46:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e5f403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff6f51f0 con 0x561fff8001b0 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e5f403740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff8032e0 con 0x561fff7fa650 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e59000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff804ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e59000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff804ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38990/0 (socket says 172.21.15.60:38990) 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e59000640 1 -- 172.21.15.60:0/2288236455 learned_addr learned my addr 172.21.15.60:0/2288236455 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e5a400640 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e5f403740 10 client.0 _ll_register_callbacks cb 0x561fff46abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e59a00640 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff804ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e5a400640 1 -- 172.21.15.60:0/2288236455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 msgr2=0x561fff804ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.416+0000 7f6e5a400640 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff804ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e5a400640 1 -- 172.21.15.60:0/2288236455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 msgr2=0x561fff804ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e5a400640 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff804ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e5a400640 1 -- 172.21.15.60:0/2288236455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fff5357d0 con 0x561fff8001b0 2024-02-26T10:46:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e59a00640 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff804ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e59000640 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff804ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e5f403740 1 -- 172.21.15.60:0/2288236455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e48011cc0 con 0x561fff8001b0 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e5a400640 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80c030 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0cc2e0 tx=0x7f6e4c09bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=647bcd2565c4eb02 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e3b400640 10 client.25796 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e3b400640 1 -- 172.21.15.60:0/2288236455 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c09f650 con 0x561fff8001b0 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e3b400640 1 -- 172.21.15.60:0/2288236455 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f6e4c0051d0 con 0x561fff8001b0 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e3b400640 1 -- 172.21.15.60:0/2288236455 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e4c0a3ca0 con 0x561fff8001b0 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e5f403740 10 client.25796 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.417+0000 7f6e5f403740 1 -- 172.21.15.60:0/2288236455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x561fff80f350 con 0x561fff8001b0 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.418+0000 7f6e3b400640 1 -- 172.21.15.60:0/2288236455 <== mon.2 v2:172.21.15.100:3300/0 4 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6e4c020070 con 0x561fff8001b0 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.418+0000 7f6e5f403740 10 client.25796 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.418+0000 7f6e5f403740 10 client.25796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.418+0000 7f6e5f403740 20 client.25796 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.418+0000 7f6e5f403740 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x561fff810920 0x561fff812d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.418+0000 7f6e5f403740 4 client.25796 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:55.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.418+0000 7f6e5f403740 1 -- 172.21.15.60:0/2288236455 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fff764270 con 0x561fff810920 2024-02-26T10:46:55.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.419+0000 7f6e59a00640 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x561fff810920 0x561fff812d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.419+0000 7f6e59a00640 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x561fff810920 0x561fff812d30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4cc8bcd1eaa76010 server_cookie=2627faf0fcf8871a in_seq=0 out_seq=0 2024-02-26T10:46:55.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.419+0000 7f6e3b400640 10 client.25796 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:46:55.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.420+0000 7f6e3b400640 1 -- 172.21.15.60:0/2288236455 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561fff764270 con 0x561fff810920 2024-02-26T10:46:55.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.420+0000 7f6e3b400640 10 client.25796 handle_command_reply: tid=0 2024-02-26T10:46:55.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.420+0000 7f6e5f403740 1 client.25796 shutdown 2024-02-26T10:46:55.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.420+0000 7f6e5f403740 1 -- 172.21.15.60:0/2288236455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 msgr2=0x561fff80c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.420+0000 7f6e5f403740 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80c030 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0cc2e0 tx=0x7f6e4c09bc00 comp rx=0 tx=0).stop 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e5f403740 1 -- 172.21.15.60:0/2288236455 shutdown_connections 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e5f403740 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80c030 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e5f403740 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x561fff810920 0x561fff812d30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e5f403740 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff804ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e5f403740 1 --2- 172.21.15.60:0/2288236455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff804ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e5f403740 1 -- 172.21.15.60:0/2288236455 >> 172.21.15.60:0/2288236455 conn(0x561fff7ed3b0 msgr2=0x561fff7ec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e3b400640 0 client.25796 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e5f403740 1 -- 172.21.15.60:0/2288236455 shutdown_connections 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e5f403740 1 -- 172.21.15.60:0/2288236455 wait complete. 2024-02-26T10:46:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.421+0000 7f6e5f403740 20 client.25796 trim_cache size 0 max 16384 2024-02-26T10:46:55.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.434+0000 7f6e5f403740 1 Processor -- start 2024-02-26T10:46:55.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.434+0000 7f6e5f403740 1 -- start start 2024-02-26T10:46:55.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.434+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff817690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.434+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff80bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.434+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff7fc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.434+0000 7f6e5f403740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541a9de0 con 0x561fff7fa650 2024-02-26T10:46:55.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.434+0000 7f6e5f403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541a7980 con 0x561fff7fb040 2024-02-26T10:46:55.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.434+0000 7f6e5f403740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541a5a60 con 0x561fff8001b0 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.435+0000 7f6e59a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff817690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.435+0000 7f6e5a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff7fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.435+0000 7f6e59a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff817690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38998/0 (socket says 172.21.15.60:38998) 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.435+0000 7f6e5a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff7fc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46116/0 (socket says 172.21.15.60:46116) 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.435+0000 7f6e59a00640 1 -- 172.21.15.60:0/3279287218 learned_addr learned my addr 172.21.15.60:0/3279287218 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.435+0000 7f6e59000640 1 --2- 172.21.15.60:0/3279287218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff80bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.435+0000 7f6e5a400640 1 -- 172.21.15.60:0/3279287218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 msgr2=0x561fff80bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.435+0000 7f6e5a400640 1 --2- 172.21.15.60:0/3279287218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff80bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.435+0000 7f6e5a400640 1 -- 172.21.15.60:0/3279287218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 msgr2=0x561fff817690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.436+0000 7f6e5a400640 1 --2- 172.21.15.60:0/3279287218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff817690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.436+0000 7f6e5a400640 1 -- 172.21.15.60:0/3279287218 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fff7ff780 con 0x561fff8001b0 2024-02-26T10:46:55.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.436+0000 7f6e59a00640 1 --2- 172.21.15.60:0/3279287218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff817690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:55.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.436+0000 7f6e5a400640 1 --2- 172.21.15.60:0/3279287218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff7fc500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c097ed0 tx=0x7f6e4c09d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da986bd62f416038 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.436+0000 7f6e46000640 1 -- 172.21.15.60:0/3279287218 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c081060 con 0x561fff8001b0 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.436+0000 7f6e46000640 1 -- 172.21.15.60:0/3279287218 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e48011cc0 con 0x561fff8001b0 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.437+0000 7f6e5f403740 1 -- 172.21.15.60:0/3279287218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 msgr2=0x561fff7fc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.437+0000 7f6e5f403740 1 --2- 172.21.15.60:0/3279287218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff7fc500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c097ed0 tx=0x7f6e4c09d000 comp rx=0 tx=0).stop 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.437+0000 7f6e5f403740 1 -- 172.21.15.60:0/3279287218 shutdown_connections 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.437+0000 7f6e5f403740 1 --2- 172.21.15.60:0/3279287218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff7fb040 0x561fff80bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.437+0000 7f6e5f403740 1 --2- 172.21.15.60:0/3279287218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff8001b0 0x561fff7fc500 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.437+0000 7f6e5f403740 1 --2- 172.21.15.60:0/3279287218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fa650 0x561fff817690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.437+0000 7f6e5f403740 1 -- 172.21.15.60:0/3279287218 >> 172.21.15.60:0/3279287218 conn(0x561fff7ed3b0 msgr2=0x561fff7ed790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.437+0000 7f6e5f403740 1 -- 172.21.15.60:0/3279287218 shutdown_connections 2024-02-26T10:46:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.437+0000 7f6e5f403740 1 -- 172.21.15.60:0/3279287218 wait complete. 2024-02-26T10:46:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.438+0000 7f6e5f403740 1 -- start start 2024-02-26T10:46:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.438+0000 7f6e5f403740 1 Processor -- start 2024-02-26T10:46:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff809fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff80a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e5f403740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e59a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff809fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e59a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff809fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46130/0 (socket says 172.21.15.60:46130) 2024-02-26T10:46:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e5f403740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541a6060 con 0x561fff7fb040 2024-02-26T10:46:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e5f403740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541a7320 con 0x561fff8001b0 2024-02-26T10:46:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e5f403740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541a6660 con 0x561fff7fa650 2024-02-26T10:46:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e5f403740 10 client.0 _ll_register_callbacks cb 0x7f6e4800b880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.439+0000 7f6e59a00640 1 -- 172.21.15.60:0/4089931739 learned_addr learned my addr 172.21.15.60:0/4089931739 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.440+0000 7f6e59000640 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff80a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.440+0000 7f6e5a400640 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.440+0000 7f6e59a00640 1 -- 172.21.15.60:0/4089931739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 msgr2=0x561fff80b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.440+0000 7f6e59a00640 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.440+0000 7f6e59a00640 1 -- 172.21.15.60:0/4089931739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 msgr2=0x561fff80a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.440+0000 7f6e59a00640 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff80a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.440+0000 7f6e59a00640 1 -- 172.21.15.60:0/4089931739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fff80bb00 con 0x561fff7fa650 2024-02-26T10:46:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.440+0000 7f6e5a400640 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:46:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.440+0000 7f6e59000640 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff80a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.441+0000 7f6e59a00640 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff809fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x561fff7fbab0 tx=0x7f6e48000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c9917c81b6ddf0a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.441+0000 7f6e3b400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:46:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.441+0000 7f6e3b400640 1 -- 172.21.15.60:0/4089931739 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4801f040 con 0x561fff7fa650 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.441+0000 7f6e3b400640 1 -- 172.21.15.60:0/4089931739 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f6e48005040 con 0x561fff7fa650 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.441+0000 7f6e5f403740 1 -- 172.21.15.60:0/4089931739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e48011cc0 con 0x561fff7fa650 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.443+0000 7f6e3b400640 1 -- 172.21.15.60:0/4089931739 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e48038050 con 0x561fff7fa650 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.443+0000 7f6e5f403740 10 client.25804 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.443+0000 7f6e5f403740 1 -- 172.21.15.60:0/4089931739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561fff8676e0 con 0x561fff7fa650 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.443+0000 7f6e3b400640 1 -- 172.21.15.60:0/4089931739 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e4802d020 con 0x561fff7fa650 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.443+0000 7f6e3b400640 1 -- 172.21.15.60:0/4089931739 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6e480196e0 con 0x561fff7fa650 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.443+0000 7f6e5f403740 10 client.25804 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.443+0000 7f6e5f403740 10 client.25804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.443+0000 7f6e5f403740 20 client.25804 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.444+0000 7f6e5f403740 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x561fff867d50 0x561fff86a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.444+0000 7f6e5f403740 4 client.25804 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:46:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.444+0000 7f6e5f403740 1 -- 172.21.15.60:0/4089931739 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f6e48011cc0 con 0x561fff867d50 2024-02-26T10:46:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.444+0000 7f6e59000640 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x561fff867d50 0x561fff86a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.445+0000 7f6e3b400640 10 client.25804 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:46:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.445+0000 7f6e59000640 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x561fff867d50 0x561fff86a140 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9f8d5bc975009f0 server_cookie=b4510c0198950315 in_seq=0 out_seq=0 2024-02-26T10:46:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.445+0000 7f6e3b400640 1 -- 172.21.15.60:0/4089931739 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f6e48011cc0 con 0x561fff867d50 2024-02-26T10:46:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.445+0000 7f6e3b400640 10 client.25804 handle_command_reply: tid=0 2024-02-26T10:46:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.445+0000 7f6e5f403740 1 client.25804 shutdown 2024-02-26T10:46:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.445+0000 7f6e5f403740 1 -- 172.21.15.60:0/4089931739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 msgr2=0x561fff809fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.445+0000 7f6e5f403740 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff809fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x561fff7fbab0 tx=0x7f6e48000bc0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e5f403740 1 -- 172.21.15.60:0/4089931739 shutdown_connections 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e5f403740 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561fff8001b0 0x561fff80b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e5f403740 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x561fff867d50 0x561fff86a140 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e5f403740 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561fff7fb040 0x561fff80a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e5f403740 1 --2- 172.21.15.60:0/4089931739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561fff7fa650 0x561fff809fc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e5f403740 1 -- 172.21.15.60:0/4089931739 >> 172.21.15.60:0/4089931739 conn(0x561fff7ed3b0 msgr2=0x561fff817690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e3b400640 0 client.25804 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e5f403740 1 -- 172.21.15.60:0/4089931739 shutdown_connections 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e5f403740 1 -- 172.21.15.60:0/4089931739 wait complete. 2024-02-26T10:46:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.446+0000 7f6e5f403740 20 client.25804 trim_cache size 0 max 16384 2024-02-26T10:46:55.442 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:46:55.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 -- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6e3c0a1e10 msgr2=0x7f6e3c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6e3c0a1e10 0x7f6e3c0a42d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6e4002c7b0 tx=0x7f6e40005fd0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 -- 172.21.15.60:0/3411625768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e54102d10 msgr2=0x7f6e541a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e54102d10 0x7f6e541a0be0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009fd0 tx=0x7f6e4c00ef10 comp rx=0 tx=0).stop 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 -- 172.21.15.60:0/3411625768 shutdown_connections 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f6e3c0a1e10 0x7f6e3c0a42d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e54102d10 0x7f6e541a0be0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e54102350 0x7f6e54199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 --2- 172.21.15.60:0/3411625768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e54101160 0x7f6e54199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.447+0000 7f6e5ae00640 1 -- 172.21.15.60:0/3411625768 >> 172.21.15.60:0/3411625768 conn(0x7f6e540fc6d0 msgr2=0x7f6e54073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.448+0000 7f6e5ae00640 1 -- 172.21.15.60:0/3411625768 shutdown_connections 2024-02-26T10:46:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.448+0000 7f6e5ae00640 1 -- 172.21.15.60:0/3411625768 wait complete. 2024-02-26T10:46:55.466 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:46:55.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T10:46:55.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:55 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/561307979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:46:55.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.653+0000 7f71f2400640 1 -- 172.21.15.60:0/810426171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec102d10 msgr2=0x7f71ec1095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.653+0000 7f71f2400640 1 --2- 172.21.15.60:0/810426171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec102d10 0x7f71ec1095a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009be0 tx=0x7f71e402bef0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.653+0000 7f71f2400640 1 -- 172.21.15.60:0/810426171 shutdown_connections 2024-02-26T10:46:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.653+0000 7f71f2400640 1 --2- 172.21.15.60:0/810426171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec102d10 0x7f71ec1095a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.653+0000 7f71f2400640 1 --2- 172.21.15.60:0/810426171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71ec102350 0x7f71ec1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.653+0000 7f71f2400640 1 --2- 172.21.15.60:0/810426171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71ec101160 0x7f71ec101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.653+0000 7f71f2400640 1 -- 172.21.15.60:0/810426171 >> 172.21.15.60:0/810426171 conn(0x7f71ec0fc6d0 msgr2=0x7f71ec0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.653+0000 7f71f2400640 1 -- 172.21.15.60:0/810426171 shutdown_connections 2024-02-26T10:46:55.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.653+0000 7f71f2400640 1 -- 172.21.15.60:0/810426171 wait complete. 2024-02-26T10:46:55.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.654+0000 7f71f2400640 1 Processor -- start 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.654+0000 7f71f2400640 1 -- start start 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec101160 0x7f71ec1951f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71ec102350 0x7f71ec195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f2400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71ec102d10 0x7f71ec19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f2400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec10ac50 con 0x7f71ec102d10 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f2400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec10aad0 con 0x7f71ec101160 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f2400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec10add0 con 0x7f71ec102350 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec101160 0x7f71ec1951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec101160 0x7f71ec1951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40910/0 (socket says 172.21.15.60:40910) 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f1000640 1 -- 172.21.15.60:0/640243854 learned_addr learned my addr 172.21.15.60:0/640243854 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71ebe00640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71ec102350 0x7f71ec195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f1a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71ec102d10 0x7f71ec19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f1000640 1 -- 172.21.15.60:0/640243854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71ec102350 msgr2=0x7f71ec195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.655+0000 7f71f1000640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71ec102350 0x7f71ec195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.656+0000 7f71f1000640 1 -- 172.21.15.60:0/640243854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71ec102d10 msgr2=0x7f71ec19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.656+0000 7f71f1000640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71ec102d10 0x7f71ec19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.656+0000 7f71f1000640 1 -- 172.21.15.60:0/640243854 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec19ce10 con 0x7f71ec101160 2024-02-26T10:46:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.656+0000 7f71f1000640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec101160 0x7f71ec1951f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f71e00140c0 tx=0x7f71e0014580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.656+0000 7f71e9600640 1 -- 172.21.15.60:0/640243854 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0021070 con 0x7f71ec101160 2024-02-26T10:46:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.656+0000 7f71f2400640 1 -- 172.21.15.60:0/640243854 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec19d100 con 0x7f71ec101160 2024-02-26T10:46:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.656+0000 7f71f2400640 1 -- 172.21.15.60:0/640243854 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec19d650 con 0x7f71ec101160 2024-02-26T10:46:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.656+0000 7f71e9600640 1 -- 172.21.15.60:0/640243854 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e0004230 con 0x7f71ec101160 2024-02-26T10:46:55.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.656+0000 7f71e9600640 1 -- 172.21.15.60:0/640243854 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00035f0 con 0x7f71ec101160 2024-02-26T10:46:55.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.660+0000 7f71f69c0740 1 Processor -- start 2024-02-26T10:46:55.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.660+0000 7f71f69c0740 1 -- start start 2024-02-26T10:46:55.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.660+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 0x556eb562a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.660+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562a830 0x556eb562f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.660+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb562f9a0 0x556eb5631e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.660+0000 7f71f69c0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb558ed30 con 0x556eb562f9a0 2024-02-26T10:46:55.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.660+0000 7f71f69c0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb5537cc0 con 0x556eb562a830 2024-02-26T10:46:55.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.660+0000 7f71f69c0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb558add0 con 0x556eb5629e40 2024-02-26T10:46:55.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.661+0000 7f71ebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb562f9a0 0x556eb5631e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.661+0000 7f71ebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb562f9a0 0x556eb5631e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39034/0 (socket says 172.21.15.60:39034) 2024-02-26T10:46:55.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.661+0000 7f71ebe00640 1 -- 172.21.15.60:0/1319842973 learned_addr learned my addr 172.21.15.60:0/1319842973 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.661+0000 7f71f1a00640 1 --2- 172.21.15.60:0/1319842973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 0x556eb562a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.661+0000 7f71ebe00640 1 -- 172.21.15.60:0/1319842973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 msgr2=0x556eb562a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.661+0000 7f71ebe00640 1 --2- 172.21.15.60:0/1319842973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 0x556eb562a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.661+0000 7f71ebe00640 1 -- 172.21.15.60:0/1319842973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562a830 msgr2=0x556eb562f330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:46:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.661+0000 7f71ebe00640 1 --2- 172.21.15.60:0/1319842973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562a830 0x556eb562f330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.661+0000 7f71ebe00640 1 -- 172.21.15.60:0/1319842973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eb52a33f0 con 0x556eb562f9a0 2024-02-26T10:46:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.662+0000 7f71e9600640 1 -- 172.21.15.60:0/640243854 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f71e0024070 con 0x7f71ec101160 2024-02-26T10:46:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.662+0000 7f71ebe00640 1 --2- 172.21.15.60:0/1319842973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb562f9a0 0x556eb5631e30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f71d800dde0 tx=0x7f71d800d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9017b60bc941e98 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.662+0000 7f71de000640 1 -- 172.21.15.60:0/1319842973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d801c020 con 0x556eb562f9a0 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.662+0000 7f71de000640 1 -- 172.21.15.60:0/1319842973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71d80035f0 con 0x556eb562f9a0 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.662+0000 7f71de000640 1 -- 172.21.15.60:0/1319842973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8018660 con 0x556eb562f9a0 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.663+0000 7f71f69c0740 1 -- 172.21.15.60:0/1319842973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb562f9a0 msgr2=0x556eb5631e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.663+0000 7f71f69c0740 1 --2- 172.21.15.60:0/1319842973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb562f9a0 0x556eb5631e30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f71d800dde0 tx=0x7f71d800d670 comp rx=0 tx=0).stop 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.663+0000 7f71f69c0740 1 -- 172.21.15.60:0/1319842973 shutdown_connections 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.663+0000 7f71f69c0740 1 --2- 172.21.15.60:0/1319842973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562a830 0x556eb562f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.663+0000 7f71f69c0740 1 --2- 172.21.15.60:0/1319842973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb562f9a0 0x556eb5631e30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.663+0000 7f71f69c0740 1 --2- 172.21.15.60:0/1319842973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 0x556eb562a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.663+0000 7f71f69c0740 1 -- 172.21.15.60:0/1319842973 >> 172.21.15.60:0/1319842973 conn(0x556eb56253b0 msgr2=0x556eb56257b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.664+0000 7f71e9600640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71d40a1e90 0x7f71d40a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.665+0000 7f71f69c0740 1 -- 172.21.15.60:0/1319842973 shutdown_connections 2024-02-26T10:46:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.665+0000 7f71e9600640 1 -- 172.21.15.60:0/640243854 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f71e00d5fc0 con 0x7f71ec101160 2024-02-26T10:46:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.665+0000 7f71f69c0740 1 -- 172.21.15.60:0/1319842973 wait complete. 2024-02-26T10:46:55.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.665+0000 7f71ebe00640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71d40a1e90 0x7f71d40a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.665+0000 7f71f69c0740 1 -- start start 2024-02-26T10:46:55.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.666+0000 7f71f69c0740 1 Processor -- start 2024-02-26T10:46:55.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.666+0000 7f71ebe00640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71d40a1e90 0x7f71d40a4350 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d800d330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb5629e40 0x556eb56342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5634820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562f9a0 0x556eb563b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f69c0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb5632950 con 0x556eb5629e40 2024-02-26T10:46:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f69c0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb552d1f0 con 0x556eb562f9a0 2024-02-26T10:46:55.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f69c0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eb5632ad0 con 0x556eb562a830 2024-02-26T10:46:55.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f69c0740 10 client.0 _ll_register_callbacks cb 0x556eb52a2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:55.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb5629e40 0x556eb56342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb5629e40 0x556eb56342e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39046/0 (socket says 172.21.15.60:39046) 2024-02-26T10:46:55.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f1000640 1 -- 172.21.15.60:0/653827851 learned_addr learned my addr 172.21.15.60:0/653827851 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71ebe00640 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5634820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f1a00640 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562f9a0 0x556eb563b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.667+0000 7f71f1000640 1 -- 172.21.15.60:0/653827851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 msgr2=0x556eb5634820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71f1000640 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5634820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71f1000640 1 -- 172.21.15.60:0/653827851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562f9a0 msgr2=0x556eb563b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71f1000640 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562f9a0 0x556eb563b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71f1000640 1 -- 172.21.15.60:0/653827851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556eb536d7d0 con 0x556eb5629e40 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71f1a00640 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562f9a0 0x556eb563b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71f1000640 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb5629e40 0x556eb56342e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f71e00a27c0 tx=0x7f71e00a28f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4052ed7d8781275b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71d2a00640 1 -- 172.21.15.60:0/653827851 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00aaf20 con 0x556eb5629e40 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71d2a00640 1 -- 172.21.15.60:0/653827851 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f71e008d0d0 con 0x556eb5629e40 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.668+0000 7f71f69c0740 1 -- 172.21.15.60:0/653827851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71d80035f0 con 0x556eb5629e40 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.669+0000 7f71d2a00640 1 -- 172.21.15.60:0/653827851 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e00d37b0 con 0x556eb5629e40 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.669+0000 7f71f69c0740 10 client.15687 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.669+0000 7f71f69c0740 1 -- 172.21.15.60:0/653827851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556eb563ec40 con 0x556eb5629e40 2024-02-26T10:46:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.669+0000 7f71d2a00640 1 -- 172.21.15.60:0/653827851 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71d80035f0 con 0x556eb5629e40 2024-02-26T10:46:55.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.670+0000 7f71d2a00640 1 -- 172.21.15.60:0/653827851 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71e00aab90 con 0x556eb5629e40 2024-02-26T10:46:55.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.670+0000 7f71f69c0740 10 client.15687 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:55.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.670+0000 7f71f69c0740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.670+0000 7f71f69c0740 20 client.15687 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.670+0000 7f71f69c0740 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556eb56402f0 0x556eb5642770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.670+0000 7f71f69c0740 4 client.15687 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.670+0000 7f71f69c0740 1 -- 172.21.15.60:0/653827851 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556eb559c270 con 0x556eb56402f0 2024-02-26T10:46:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.671+0000 7f71ebe00640 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556eb56402f0 0x556eb5642770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.671+0000 7f71ebe00640 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556eb56402f0 0x556eb5642770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f39664c7df0f45cf server_cookie=1ba6cf48da4aa41e in_seq=0 out_seq=0 2024-02-26T10:46:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.672+0000 7f71d2a00640 10 client.15687 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.672+0000 7f71d2a00640 1 -- 172.21.15.60:0/653827851 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556eb559c270 con 0x556eb56402f0 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.672+0000 7f71d2a00640 10 client.15687 handle_command_reply: tid=0 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.672+0000 7f71f69c0740 1 client.15687 shutdown 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 -- 172.21.15.60:0/653827851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb5629e40 msgr2=0x556eb56342e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb5629e40 0x556eb56342e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f71e00a27c0 tx=0x7f71e00a28f0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 -- 172.21.15.60:0/653827851 shutdown_connections 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556eb56402f0 0x556eb5642770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562f9a0 0x556eb563b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5634820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 --2- 172.21.15.60:0/653827851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb5629e40 0x556eb56342e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 -- 172.21.15.60:0/653827851 >> 172.21.15.60:0/653827851 conn(0x556eb56253b0 msgr2=0x556eb56246d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71d2a00640 0 client.15687 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 -- 172.21.15.60:0/653827851 shutdown_connections 2024-02-26T10:46:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 1 -- 172.21.15.60:0/653827851 wait complete. 2024-02-26T10:46:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.673+0000 7f71f69c0740 20 client.15687 trim_cache size 0 max 16384 2024-02-26T10:46:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.698+0000 7f71f69c0740 1 Processor -- start 2024-02-26T10:46:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.698+0000 7f71f69c0740 1 -- start start 2024-02-26T10:46:55.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb5623e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb5629e40 0x556eb563b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5624380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f69c0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc011fa0 con 0x556eb56253b0 2024-02-26T10:46:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f69c0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc00fb40 con 0x556eb5629e40 2024-02-26T10:46:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f69c0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc00dbe0 con 0x556eb562a830 2024-02-26T10:46:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5624380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5624380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46188/0 (socket says 172.21.15.60:46188) 2024-02-26T10:46:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f1a00640 1 -- 172.21.15.60:0/556919523 learned_addr learned my addr 172.21.15.60:0/556919523 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71ebe00640 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb5629e40 0x556eb563b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.699+0000 7f71f1000640 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb5623e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.700+0000 7f71f1a00640 1 -- 172.21.15.60:0/556919523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb5629e40 msgr2=0x556eb563b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.700+0000 7f71f1a00640 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb5629e40 0x556eb563b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.700+0000 7f71f1a00640 1 -- 172.21.15.60:0/556919523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 msgr2=0x556eb5623e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.700+0000 7f71f1a00640 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb5623e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.700+0000 7f71f1a00640 1 -- 172.21.15.60:0/556919523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eb56350a0 con 0x556eb562a830 2024-02-26T10:46:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.700+0000 7f71f1000640 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb5623e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.700+0000 7f71f1a00640 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5624380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f71e402c500 tx=0x7f71e4033b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c41c68f4b9eecdd3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.700+0000 7f71dd600640 1 -- 172.21.15.60:0/556919523 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4033d70 con 0x556eb562a830 2024-02-26T10:46:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.700+0000 7f71dd600640 1 -- 172.21.15.60:0/556919523 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e4003970 con 0x556eb562a830 2024-02-26T10:46:55.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.701+0000 7f71dd600640 1 -- 172.21.15.60:0/556919523 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e402cdc0 con 0x556eb562a830 2024-02-26T10:46:55.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.701+0000 7f71f69c0740 1 -- 172.21.15.60:0/556919523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 msgr2=0x556eb5624380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.701+0000 7f71f69c0740 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5624380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f71e402c500 tx=0x7f71e4033b40 comp rx=0 tx=0).stop 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.701+0000 7f71f69c0740 1 -- 172.21.15.60:0/556919523 shutdown_connections 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.702+0000 7f71f69c0740 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb5629e40 0x556eb563b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.702+0000 7f71f69c0740 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb562a830 0x556eb5624380 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.702+0000 7f71f69c0740 1 --2- 172.21.15.60:0/556919523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb5623e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.702+0000 7f71f69c0740 1 -- 172.21.15.60:0/556919523 >> 172.21.15.60:0/556919523 conn(0x556eb562b610 msgr2=0x556eb5625790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.702+0000 7f71f69c0740 1 -- 172.21.15.60:0/556919523 shutdown_connections 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.702+0000 7f71f69c0740 1 -- 172.21.15.60:0/556919523 wait complete. 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.702+0000 7f71f69c0740 1 -- start start 2024-02-26T10:46:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.703+0000 7f71f69c0740 1 Processor -- start 2024-02-26T10:46:55.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.703+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb563b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.703+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 0x556eb5639d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.703+0000 7f71f69c0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562a830 0x556eb563a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.703+0000 7f71f69c0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc00e1e0 con 0x556eb56253b0 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.703+0000 7f71f69c0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc00f4e0 con 0x556eb562a830 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.703+0000 7f71f69c0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc00e820 con 0x556eb5629e40 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.703+0000 7f71f69c0740 10 client.0 _ll_register_callbacks cb 0x7f71d800da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71f1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb563b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71f1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb563b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39064/0 (socket says 172.21.15.60:39064) 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71f1000640 1 -- 172.21.15.60:0/2221882595 learned_addr learned my addr 172.21.15.60:0/2221882595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71ebe00640 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 0x556eb5639d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71f1a00640 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562a830 0x556eb563a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71f1000640 1 -- 172.21.15.60:0/2221882595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 msgr2=0x556eb5639d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71f1000640 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 0x556eb5639d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71f1000640 1 -- 172.21.15.60:0/2221882595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562a830 msgr2=0x556eb563a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71f1000640 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562a830 0x556eb563a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.704+0000 7f71f1000640 1 -- 172.21.15.60:0/2221882595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556eb563aaa0 con 0x556eb56253b0 2024-02-26T10:46:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.705+0000 7f71f1000640 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb563b700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f71e00058c0 tx=0x7f71e00a2200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01ab45d9207c3f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.705+0000 7f71d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.705+0000 7f71d2a00640 1 -- 172.21.15.60:0/2221882595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00d3950 con 0x556eb56253b0 2024-02-26T10:46:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.705+0000 7f71f69c0740 1 -- 172.21.15.60:0/2221882595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71e4003970 con 0x556eb56253b0 2024-02-26T10:46:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.705+0000 7f71d2a00640 1 -- 172.21.15.60:0/2221882595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f71e00d3b70 con 0x556eb56253b0 2024-02-26T10:46:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.705+0000 7f71d2a00640 1 -- 172.21.15.60:0/2221882595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e00aaf20 con 0x556eb56253b0 2024-02-26T10:46:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.705+0000 7f71f69c0740 10 client.15693 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.705+0000 7f71f69c0740 1 -- 172.21.15.60:0/2221882595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556eb56978e0 con 0x556eb56253b0 2024-02-26T10:46:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.706+0000 7f71d2a00640 1 -- 172.21.15.60:0/2221882595 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e4003970 con 0x556eb56253b0 2024-02-26T10:46:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.706+0000 7f71d2a00640 1 -- 172.21.15.60:0/2221882595 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71e00a1da0 con 0x556eb56253b0 2024-02-26T10:46:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.707+0000 7f71f69c0740 10 client.15693 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.707+0000 7f71f69c0740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.707+0000 7f71f69c0740 20 client.15693 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.707+0000 7f71f69c0740 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556eb562f9a0 0x556eb569a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.707+0000 7f71f69c0740 4 client.15693 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:46:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.707+0000 7f71f69c0740 1 -- 172.21.15.60:0/2221882595 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556eb55965c0 con 0x556eb562f9a0 2024-02-26T10:46:55.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.707+0000 7f71ebe00640 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556eb562f9a0 0x556eb569a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:55.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.708+0000 7f71ebe00640 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556eb562f9a0 0x556eb569a2d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6671744dad2ef9 server_cookie=8aafa96d7a7c5bf4 in_seq=0 out_seq=0 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.708+0000 7f71d2a00640 10 client.15693 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.708+0000 7f71d2a00640 1 -- 172.21.15.60:0/2221882595 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556eb55965c0 con 0x556eb562f9a0 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.708+0000 7f71d2a00640 10 client.15693 handle_command_reply: tid=0 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.708+0000 7f71f69c0740 1 client.15693 shutdown 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 -- 172.21.15.60:0/2221882595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 msgr2=0x556eb563b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb563b700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f71e00058c0 tx=0x7f71e00a2200 comp rx=0 tx=0).stop 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 -- 172.21.15.60:0/2221882595 shutdown_connections 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x556eb562f9a0 0x556eb569a2d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556eb562a830 0x556eb563a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556eb5629e40 0x556eb5639d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 --2- 172.21.15.60:0/2221882595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556eb56253b0 0x556eb563b700 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 -- 172.21.15.60:0/2221882595 >> 172.21.15.60:0/2221882595 conn(0x556eb562b610 msgr2=0x556eb5628610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71d2a00640 0 client.15693 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:55.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 -- 172.21.15.60:0/2221882595 shutdown_connections 2024-02-26T10:46:55.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 1 -- 172.21.15.60:0/2221882595 wait complete. 2024-02-26T10:46:55.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.709+0000 7f71f69c0740 20 client.15693 trim_cache size 0 max 16384 2024-02-26T10:46:55.705 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:46:55.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.710+0000 7f71f2400640 1 -- 172.21.15.60:0/640243854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71d40a1e90 msgr2=0x7f71d40a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.710+0000 7f71f2400640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71d40a1e90 0x7f71d40a4350 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d800d330 comp rx=0 tx=0).stop 2024-02-26T10:46:55.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.710+0000 7f71f2400640 1 -- 172.21.15.60:0/640243854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec101160 msgr2=0x7f71ec1951f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:55.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.710+0000 7f71f2400640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec101160 0x7f71ec1951f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f71e00140c0 tx=0x7f71e0014580 comp rx=0 tx=0).stop 2024-02-26T10:46:55.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.712+0000 7f71f2400640 1 -- 172.21.15.60:0/640243854 shutdown_connections 2024-02-26T10:46:55.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.712+0000 7f71f2400640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f71d40a1e90 0x7f71d40a4350 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.712+0000 7f71f2400640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71ec102d10 0x7f71ec19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.712+0000 7f71f2400640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71ec102350 0x7f71ec195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.712+0000 7f71f2400640 1 --2- 172.21.15.60:0/640243854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71ec101160 0x7f71ec1951f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:55.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.712+0000 7f71f2400640 1 -- 172.21.15.60:0/640243854 >> 172.21.15.60:0/640243854 conn(0x7f71ec0fc6d0 msgr2=0x7f71ec1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:55.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.712+0000 7f71f2400640 1 -- 172.21.15.60:0/640243854 shutdown_connections 2024-02-26T10:46:55.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:55.712+0000 7f71f2400640 1 -- 172.21.15.60:0/640243854 wait complete. 2024-02-26T10:46:55.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:55 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/561307979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:46:55.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:55 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/561307979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:46:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.529+0000 7fd826e00640 1 -- 172.21.15.60:0/332722579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd820076e60 msgr2=0x7fd8200772c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.529+0000 7fd826e00640 1 --2- 172.21.15.60:0/332722579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd820076e60 0x7fd8200772c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009860 tx=0x7fd80c02bb70 comp rx=0 tx=0).stop 2024-02-26T10:46:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.529+0000 7fd826e00640 1 -- 172.21.15.60:0/332722579 shutdown_connections 2024-02-26T10:46:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.529+0000 7fd826e00640 1 --2- 172.21.15.60:0/332722579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd820076e60 0x7fd8200772c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.529+0000 7fd826e00640 1 --2- 172.21.15.60:0/332722579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd820077800 0x7fd8200780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.529+0000 7fd826e00640 1 --2- 172.21.15.60:0/332722579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd820075c20 0x7fd820076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.529+0000 7fd826e00640 1 -- 172.21.15.60:0/332722579 >> 172.21.15.60:0/332722579 conn(0x7fd8200fea20 msgr2=0x7fd820100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.530+0000 7fd826e00640 1 -- 172.21.15.60:0/332722579 shutdown_connections 2024-02-26T10:46:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.530+0000 7fd826e00640 1 -- 172.21.15.60:0/332722579 wait complete. 2024-02-26T10:46:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.531+0000 7fd826e00640 1 Processor -- start 2024-02-26T10:46:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.531+0000 7fd826e00640 1 -- start start 2024-02-26T10:46:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd826e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd820075c20 0x7fd82019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd826e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd820076e60 0x7fd82019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd826e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd820077800 0x7fd8201a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd826e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820113540 con 0x7fd820075c20 2024-02-26T10:46:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd826e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201133c0 con 0x7fd820076e60 2024-02-26T10:46:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd826e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201136c0 con 0x7fd820077800 2024-02-26T10:46:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd820075c20 0x7fd82019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd81f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd820076e60 0x7fd82019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd820075c20 0x7fd82019db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39082/0 (socket says 172.21.15.60:39082) 2024-02-26T10:46:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd81fe00640 1 -- 172.21.15.60:0/4189320075 learned_addr learned my addr 172.21.15.60:0/4189320075 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.532+0000 7fd824e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd820077800 0x7fd8201a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.533+0000 7fd81fe00640 1 -- 172.21.15.60:0/4189320075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd820077800 msgr2=0x7fd8201a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.533+0000 7fd81fe00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd820077800 0x7fd8201a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.533+0000 7fd81fe00640 1 -- 172.21.15.60:0/4189320075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd820076e60 msgr2=0x7fd82019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.533+0000 7fd81fe00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd820076e60 0x7fd82019e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.533+0000 7fd81fe00640 1 -- 172.21.15.60:0/4189320075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8201a56f0 con 0x7fd820075c20 2024-02-26T10:46:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.533+0000 7fd81f400640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd820076e60 0x7fd82019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.533+0000 7fd824e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd820077800 0x7fd8201a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.533+0000 7fd81fe00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd820075c20 0x7fd82019db20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd81000d950 tx=0x7fd81000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.534+0000 7fd81cc00640 1 -- 172.21.15.60:0/4189320075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810017030 con 0x7fd820075c20 2024-02-26T10:46:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.534+0000 7fd81cc00640 1 -- 172.21.15.60:0/4189320075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd810004020 con 0x7fd820075c20 2024-02-26T10:46:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.534+0000 7fd826e00640 1 -- 172.21.15.60:0/4189320075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8201a59e0 con 0x7fd820075c20 2024-02-26T10:46:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.534+0000 7fd81cc00640 1 -- 172.21.15.60:0/4189320075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100047a0 con 0x7fd820075c20 2024-02-26T10:46:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.535+0000 7fd826e00640 1 -- 172.21.15.60:0/4189320075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8201a5f50 con 0x7fd820075c20 2024-02-26T10:46:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.538+0000 7fd829d70740 1 Processor -- start 2024-02-26T10:46:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.538+0000 7fd829d70740 1 -- start start 2024-02-26T10:46:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.538+0000 7fd81cc00640 1 -- 172.21.15.60:0/4189320075 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fd8100041d0 con 0x7fd820075c20 2024-02-26T10:46:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.538+0000 7fd829d70740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d886820 0x55984d886c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd829d70740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d887210 0x55984d88bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd829d70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d88c380 0x55984d88e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd829d70740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d7ec340 con 0x55984d886820 2024-02-26T10:46:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd829d70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d795520 con 0x55984d88c380 2024-02-26T10:46:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd829d70740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d7e83e0 con 0x55984d887210 2024-02-26T10:46:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd824e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d886820 0x55984d886c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd81f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d88c380 0x55984d88e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd824e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d886820 0x55984d886c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39088/0 (socket says 172.21.15.60:39088) 2024-02-26T10:46:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd824e00640 1 -- 172.21.15.60:0/1887777187 learned_addr learned my addr 172.21.15.60:0/1887777187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd81fe00640 1 --2- 172.21.15.60:0/1887777187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d887210 0x55984d88bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd81f400640 1 -- 172.21.15.60:0/1887777187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d887210 msgr2=0x55984d88bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.539+0000 7fd81f400640 1 --2- 172.21.15.60:0/1887777187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d887210 0x55984d88bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.540+0000 7fd81f400640 1 -- 172.21.15.60:0/1887777187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d886820 msgr2=0x55984d886c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.540+0000 7fd81f400640 1 --2- 172.21.15.60:0/1887777187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d886820 0x55984d886c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.540+0000 7fd81f400640 1 -- 172.21.15.60:0/1887777187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55984d7a9390 con 0x55984d88c380 2024-02-26T10:46:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.540+0000 7fd81f400640 1 --2- 172.21.15.60:0/1887777187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d88c380 0x55984d88e810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd80c02c0b0 tx=0x7fd80c034ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfd4e73285dd11c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.540+0000 7fd81cc00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd8040a1f40 0x7fd8040a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.540+0000 7fd815600640 1 -- 172.21.15.60:0/1887777187 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c044020 con 0x55984d88c380 2024-02-26T10:46:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.540+0000 7fd815600640 1 -- 172.21.15.60:0/1887777187 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd80c034640 con 0x55984d88c380 2024-02-26T10:46:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.540+0000 7fd815600640 1 -- 172.21.15.60:0/1887777187 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c039040 con 0x55984d88c380 2024-02-26T10:46:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.541+0000 7fd81cc00640 1 -- 172.21.15.60:0/4189320075 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fd8100cd720 con 0x7fd820075c20 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.541+0000 7fd824e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd8040a1f40 0x7fd8040a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.541+0000 7fd829d70740 1 -- 172.21.15.60:0/1887777187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d88c380 msgr2=0x55984d88e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.541+0000 7fd829d70740 1 --2- 172.21.15.60:0/1887777187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d88c380 0x55984d88e810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd80c02c0b0 tx=0x7fd80c034ab0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.541+0000 7fd824e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd8040a1f40 0x7fd8040a4400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55984d887a70 tx=0x7fd818011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.542+0000 7fd829d70740 1 -- 172.21.15.60:0/1887777187 shutdown_connections 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.542+0000 7fd829d70740 1 --2- 172.21.15.60:0/1887777187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d88c380 0x55984d88e810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.542+0000 7fd829d70740 1 --2- 172.21.15.60:0/1887777187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d887210 0x55984d88bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.542+0000 7fd829d70740 1 --2- 172.21.15.60:0/1887777187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d886820 0x55984d886c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.542+0000 7fd829d70740 1 -- 172.21.15.60:0/1887777187 >> 172.21.15.60:0/1887777187 conn(0x55984d881d90 msgr2=0x55984d882190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.542+0000 7fd829d70740 1 -- 172.21.15.60:0/1887777187 shutdown_connections 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.542+0000 7fd829d70740 1 -- 172.21.15.60:0/1887777187 wait complete. 2024-02-26T10:46:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.542+0000 7fd829d70740 1 -- start start 2024-02-26T10:46:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.542+0000 7fd829d70740 1 Processor -- start 2024-02-26T10:46:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.543+0000 7fd829d70740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d890ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.543+0000 7fd829d70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d891010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.543+0000 7fd829d70740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.543+0000 7fd829d70740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d88f2d0 con 0x55984d88c380 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.543+0000 7fd829d70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d7a8980 con 0x55984d887210 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.543+0000 7fd829d70740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55984d88f450 con 0x55984d886820 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.543+0000 7fd829d70740 10 client.0 _ll_register_callbacks cb 0x55984d7e4310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd824e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd824e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39104/0 (socket says 172.21.15.60:39104) 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd824e00640 1 -- 172.21.15.60:0/2896423993 learned_addr learned my addr 172.21.15.60:0/2896423993 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd81fe00640 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d890ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd81f400640 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d891010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd824e00640 1 -- 172.21.15.60:0/2896423993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 msgr2=0x55984d890ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd824e00640 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d890ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd824e00640 1 -- 172.21.15.60:0/2896423993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 msgr2=0x55984d891010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd824e00640 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d891010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.544+0000 7fd824e00640 1 -- 172.21.15.60:0/2896423993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55984d501210 con 0x55984d88c380 2024-02-26T10:46:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.545+0000 7fd824e00640 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd81800bed0 tx=0x7fd818004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2d4ede3477c818 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.545+0000 7fd80a000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.545+0000 7fd80a000640 1 -- 172.21.15.60:0/2896423993 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818021040 con 0x55984d88c380 2024-02-26T10:46:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.545+0000 7fd80a000640 1 -- 172.21.15.60:0/2896423993 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fd818025420 con 0x55984d88c380 2024-02-26T10:46:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.545+0000 7fd829d70740 1 -- 172.21.15.60:0/2896423993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd80c034640 con 0x55984d88c380 2024-02-26T10:46:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd80a000640 1 -- 172.21.15.60:0/2896423993 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd818033050 con 0x55984d88c380 2024-02-26T10:46:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd829d70740 10 client.15705 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd829d70740 1 -- 172.21.15.60:0/2896423993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55984d5cd620 con 0x55984d88c380 2024-02-26T10:46:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd80a000640 1 -- 172.21.15.60:0/2896423993 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd80c034640 con 0x55984d88c380 2024-02-26T10:46:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd80a000640 1 -- 172.21.15.60:0/2896423993 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd81800b040 con 0x55984d88c380 2024-02-26T10:46:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd829d70740 10 client.15705 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd829d70740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd829d70740 20 client.15705 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd829d70740 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55984d89c930 0x55984d89ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd829d70740 4 client.15705 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.546+0000 7fd829d70740 1 -- 172.21.15.60:0/2896423993 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55984d7f9880 con 0x55984d89c930 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.547+0000 7fd81fe00640 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55984d89c930 0x55984d89ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.547+0000 7fd81fe00640 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55984d89c930 0x55984d89ed20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe2b09b77ea780f server_cookie=bd504e4451467cae in_seq=0 out_seq=0 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.547+0000 7fd80a000640 10 client.15705 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd80a000640 1 -- 172.21.15.60:0/2896423993 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55984d7f9880 con 0x55984d89c930 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd80a000640 10 client.15705 handle_command_reply: tid=0 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd829d70740 1 client.15705 shutdown 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd829d70740 1 -- 172.21.15.60:0/2896423993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 msgr2=0x55984d897fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd829d70740 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd81800bed0 tx=0x7fd818004ab0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd829d70740 1 -- 172.21.15.60:0/2896423993 shutdown_connections 2024-02-26T10:46:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd829d70740 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55984d89c930 0x55984d89ed20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd829d70740 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d891010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd829d70740 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fe0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd829d70740 1 --2- 172.21.15.60:0/2896423993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d890ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.548+0000 7fd829d70740 1 -- 172.21.15.60:0/2896423993 >> 172.21.15.60:0/2896423993 conn(0x55984d881d90 msgr2=0x55984d880fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.549+0000 7fd80a000640 0 client.15705 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.549+0000 7fd829d70740 1 -- 172.21.15.60:0/2896423993 shutdown_connections 2024-02-26T10:46:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.549+0000 7fd829d70740 1 -- 172.21.15.60:0/2896423993 wait complete. 2024-02-26T10:46:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.549+0000 7fd829d70740 20 client.15705 trim_cache size 0 max 16384 2024-02-26T10:46:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.563+0000 7fd829d70740 1 Processor -- start 2024-02-26T10:46:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.563+0000 7fd829d70740 1 -- start start 2024-02-26T10:46:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.563+0000 7fd829d70740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d8884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.563+0000 7fd829d70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d888a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.563+0000 7fd829d70740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.563+0000 7fd829d70740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201b2820 con 0x55984d88c380 2024-02-26T10:46:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.563+0000 7fd829d70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201b03c0 con 0x55984d887210 2024-02-26T10:46:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.563+0000 7fd829d70740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201ae460 con 0x55984d886820 2024-02-26T10:46:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd824e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd81f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d888a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd824e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39118/0 (socket says 172.21.15.60:39118) 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd824e00640 1 -- 172.21.15.60:0/3278418221 learned_addr learned my addr 172.21.15.60:0/3278418221 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd81f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d888a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40988/0 (socket says 172.21.15.60:40988) 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd81fe00640 1 --2- 172.21.15.60:0/3278418221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d8884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd81f400640 1 -- 172.21.15.60:0/3278418221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 msgr2=0x55984d8884f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd81f400640 1 --2- 172.21.15.60:0/3278418221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d8884f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd81f400640 1 -- 172.21.15.60:0/3278418221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 msgr2=0x55984d897fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd81f400640 1 --2- 172.21.15.60:0/3278418221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd81f400640 1 -- 172.21.15.60:0/3278418221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55984d8875f0 con 0x55984d887210 2024-02-26T10:46:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.564+0000 7fd81f400640 1 --2- 172.21.15.60:0/3278418221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d888a30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd80c000c00 tx=0x7fd80c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dab6dd24cc0d66f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.565+0000 7fd814c00640 1 -- 172.21.15.60:0/3278418221 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c04f070 con 0x55984d887210 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.565+0000 7fd814c00640 1 -- 172.21.15.60:0/3278418221 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd80c034640 con 0x55984d887210 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.565+0000 7fd814c00640 1 -- 172.21.15.60:0/3278418221 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c044040 con 0x55984d887210 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 -- 172.21.15.60:0/3278418221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 msgr2=0x55984d888a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 --2- 172.21.15.60:0/3278418221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d888a30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd80c000c00 tx=0x7fd80c005d30 comp rx=0 tx=0).stop 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 -- 172.21.15.60:0/3278418221 shutdown_connections 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 --2- 172.21.15.60:0/3278418221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d888a30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 --2- 172.21.15.60:0/3278418221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d897fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 --2- 172.21.15.60:0/3278418221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d8884f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 -- 172.21.15.60:0/3278418221 >> 172.21.15.60:0/3278418221 conn(0x55984d881d90 msgr2=0x55984d8a36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 -- 172.21.15.60:0/3278418221 shutdown_connections 2024-02-26T10:46:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 -- 172.21.15.60:0/3278418221 wait complete. 2024-02-26T10:46:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 -- start start 2024-02-26T10:46:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.566+0000 7fd829d70740 1 Processor -- start 2024-02-26T10:46:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd829d70740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d897870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd829d70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d897db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd829d70740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d895e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd829d70740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201aea60 con 0x55984d88c380 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd829d70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201afd60 con 0x55984d887210 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd829d70740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201af0a0 con 0x55984d886820 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd829d70740 10 client.0 _ll_register_callbacks cb 0x7fd818009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd81f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d897db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d897870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd81f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d897db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40994/0 (socket says 172.21.15.60:40994) 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d897870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46258/0 (socket says 172.21.15.60:46258) 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd81f400640 1 -- 172.21.15.60:0/1936937233 learned_addr learned my addr 172.21.15.60:0/1936937233 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.567+0000 7fd824e00640 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d895e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.568+0000 7fd81f400640 1 -- 172.21.15.60:0/1936937233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 msgr2=0x55984d897870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.568+0000 7fd81f400640 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d897870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.568+0000 7fd81f400640 1 -- 172.21.15.60:0/1936937233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 msgr2=0x55984d895e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.568+0000 7fd81f400640 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d895e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.568+0000 7fd81f400640 1 -- 172.21.15.60:0/1936937233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55984d8965c0 con 0x55984d887210 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.568+0000 7fd81f400640 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d897db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0349e0 tx=0x7fd80c0347f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62eb92df247cf8b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.568+0000 7fd80a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.568+0000 7fd80a000640 1 -- 172.21.15.60:0/1936937233 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c044040 con 0x55984d887210 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.569+0000 7fd829d70740 1 -- 172.21.15.60:0/1936937233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd80c034640 con 0x55984d887210 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.569+0000 7fd80a000640 1 -- 172.21.15.60:0/1936937233 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fd80c004240 con 0x55984d887210 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.569+0000 7fd80a000640 1 -- 172.21.15.60:0/1936937233 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd80c062050 con 0x55984d887210 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.569+0000 7fd829d70740 10 client.25832 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.569+0000 7fd829d70740 1 -- 172.21.15.60:0/1936937233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55984d8f3860 con 0x55984d887210 2024-02-26T10:46:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.569+0000 7fd80a000640 1 -- 172.21.15.60:0/1936937233 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd80c034640 con 0x55984d887210 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.570+0000 7fd80a000640 1 -- 172.21.15.60:0/1936937233 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd80c032640 con 0x55984d887210 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.570+0000 7fd829d70740 10 client.25832 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.570+0000 7fd829d70740 10 client.25832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.570+0000 7fd829d70740 20 client.25832 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.570+0000 7fd829d70740 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55984d8f3c50 0x55984d8f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.570+0000 7fd829d70740 4 client.25832 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.570+0000 7fd829d70740 1 -- 172.21.15.60:0/1936937233 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55984d7f3bd0 con 0x55984d8f3c50 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.570+0000 7fd81fe00640 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55984d8f3c50 0x55984d8f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.571+0000 7fd81fe00640 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55984d8f3c50 0x55984d8f6040 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83673ea2d0e794cd server_cookie=ed7a460837321c0a in_seq=0 out_seq=0 2024-02-26T10:46:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.571+0000 7fd80a000640 10 client.25832 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.576+0000 7fd80a000640 1 -- 172.21.15.60:0/1936937233 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55984d7f3bd0 con 0x55984d8f3c50 2024-02-26T10:46:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.576+0000 7fd80a000640 10 client.25832 handle_command_reply: tid=0 2024-02-26T10:46:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.576+0000 7fd829d70740 1 client.25832 shutdown 2024-02-26T10:46:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.576+0000 7fd829d70740 1 -- 172.21.15.60:0/1936937233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 msgr2=0x55984d897db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.576+0000 7fd829d70740 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d897db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0349e0 tx=0x7fd80c0347f0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.576+0000 7fd829d70740 1 -- 172.21.15.60:0/1936937233 shutdown_connections 2024-02-26T10:46:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.576+0000 7fd829d70740 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55984d8f3c50 0x55984d8f6040 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd829d70740 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55984d887210 0x55984d897db0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd829d70740 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55984d88c380 0x55984d895e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd829d70740 1 --2- 172.21.15.60:0/1936937233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55984d886820 0x55984d897870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd829d70740 1 -- 172.21.15.60:0/1936937233 >> 172.21.15.60:0/1936937233 conn(0x55984d881d90 msgr2=0x55984d8884f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd80a000640 0 client.25832 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd829d70740 1 -- 172.21.15.60:0/1936937233 shutdown_connections 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd829d70740 1 -- 172.21.15.60:0/1936937233 wait complete. 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd829d70740 20 client.25832 trim_cache size 0 max 16384 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:46:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:46:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:46:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:46:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd826e00640 1 -- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd8040a1f40 msgr2=0x7fd8040a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.577+0000 7fd826e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd8040a1f40 0x7fd8040a4400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55984d887a70 tx=0x7fd818011040 comp rx=0 tx=0).stop 2024-02-26T10:46:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 -- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd820075c20 msgr2=0x7fd82019db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd820075c20 0x7fd82019db20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd81000d950 tx=0x7fd81000de10 comp rx=0 tx=0).stop 2024-02-26T10:46:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 -- 172.21.15.60:0/4189320075 shutdown_connections 2024-02-26T10:46:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd8040a1f40 0x7fd8040a4400 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd820076e60 0x7fd82019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd820077800 0x7fd8201a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 --2- 172.21.15.60:0/4189320075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd820075c20 0x7fd82019db20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 -- 172.21.15.60:0/4189320075 >> 172.21.15.60:0/4189320075 conn(0x7fd8200fea20 msgr2=0x7fd820100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 -- 172.21.15.60:0/4189320075 shutdown_connections 2024-02-26T10:46:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:46:56.578+0000 7fd826e00640 1 -- 172.21.15.60:0/4189320075 wait complete. 2024-02-26T10:46:56.732 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T10:46:56.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:46:56.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T10:46:56.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:56 smithi060 ceph-mon[31539]: pgmap v454: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T10:46:56.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:56 smithi113 ceph-mon[36126]: pgmap v454: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T10:46:56.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:56 smithi100 ceph-mon[36051]: pgmap v454: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T10:46:56.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.941+0000 7fd4cfe00640 1 -- 172.21.15.60:0/63387455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0075240 msgr2=0x7fd4d0073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.941+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/63387455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0075240 0x7fd4d0073690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009900 tx=0x7fd4c002bce0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.941+0000 7fd4cfe00640 1 -- 172.21.15.60:0/63387455 shutdown_connections 2024-02-26T10:46:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.941+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/63387455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d0108ff0 0x7fd4d010b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.941+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/63387455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d0073c60 0x7fd4d00740c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.941+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/63387455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0075240 0x7fd4d0073690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.941+0000 7fd4cfe00640 1 -- 172.21.15.60:0/63387455 >> 172.21.15.60:0/63387455 conn(0x7fd4d00fc6d0 msgr2=0x7fd4d00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.941+0000 7fd4cfe00640 1 -- 172.21.15.60:0/63387455 shutdown_connections 2024-02-26T10:46:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.941+0000 7fd4cfe00640 1 -- 172.21.15.60:0/63387455 wait complete. 2024-02-26T10:46:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.943+0000 7fd4cfe00640 1 Processor -- start 2024-02-26T10:46:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.943+0000 7fd4cfe00640 1 -- start start 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.943+0000 7fd4cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0073c60 0x7fd4d0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.943+0000 7fd4cfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d0075240 0x7fd4d0199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.943+0000 7fd4cfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d0108ff0 0x7fd4d01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.943+0000 7fd4cfe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d010f040 con 0x7fd4d0108ff0 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.943+0000 7fd4cfe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d010eec0 con 0x7fd4d0073c60 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.943+0000 7fd4cfe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d010f1c0 con 0x7fd4d0075240 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0073c60 0x7fd4d0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0073c60 0x7fd4d0199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:41008/0 (socket says 172.21.15.60:41008) 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4cea00640 1 -- 172.21.15.60:0/2297563691 learned_addr learned my addr 172.21.15.60:0/2297563691 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4cf400640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d0108ff0 0x7fd4d01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4ce000640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d0075240 0x7fd4d0199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4cea00640 1 -- 172.21.15.60:0/2297563691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d0075240 msgr2=0x7fd4d0199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4cea00640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d0075240 0x7fd4d0199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4cea00640 1 -- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d0108ff0 msgr2=0x7fd4d01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4cea00640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d0108ff0 0x7fd4d01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4cea00640 1 -- 172.21.15.60:0/2297563691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d01a1260 con 0x7fd4d0073c60 2024-02-26T10:46:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.944+0000 7fd4ce000640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d0075240 0x7fd4d0199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:46:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.945+0000 7fd4cea00640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0073c60 0x7fd4d0199640 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002c2f0 tx=0x7fd4c002c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.945+0000 7fd4c7400640 1 -- 172.21.15.60:0/2297563691 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0043020 con 0x7fd4d0073c60 2024-02-26T10:46:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.945+0000 7fd4cfe00640 1 -- 172.21.15.60:0/2297563691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d01a14f0 con 0x7fd4d0073c60 2024-02-26T10:46:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.945+0000 7fd4c7400640 1 -- 172.21.15.60:0/2297563691 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4c00054e0 con 0x7fd4d0073c60 2024-02-26T10:46:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.945+0000 7fd4c7400640 1 -- 172.21.15.60:0/2297563691 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0003c50 con 0x7fd4d0073c60 2024-02-26T10:46:56.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.945+0000 7fd4cfe00640 1 -- 172.21.15.60:0/2297563691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d01a1a90 con 0x7fd4d0073c60 2024-02-26T10:46:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.949+0000 7fd4d8801740 1 Processor -- start 2024-02-26T10:46:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.949+0000 7fd4d8801740 1 -- start start 2024-02-26T10:46:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.949+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea9676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.949+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967c70 0x5634ea96c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.949+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea96cde0 0x5634ea96f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.949+0000 7fd4d8801740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ea87adf0 con 0x5634ea967c70 2024-02-26T10:46:56.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.949+0000 7fd4d8801740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ea8cc9f0 con 0x5634ea967280 2024-02-26T10:46:56.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.949+0000 7fd4d8801740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ea875ad0 con 0x5634ea96cde0 2024-02-26T10:46:56.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4ce000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea96cde0 0x5634ea96f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4cf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea9676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4ce000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea96cde0 0x5634ea96f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46292/0 (socket says 172.21.15.60:46292) 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4ce000640 1 -- 172.21.15.60:0/2934398777 learned_addr learned my addr 172.21.15.60:0/2934398777 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4ce000640 1 -- 172.21.15.60:0/2934398777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 msgr2=0x5634ea9676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4ce000640 1 --2- 172.21.15.60:0/2934398777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea9676a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4ce000640 1 -- 172.21.15.60:0/2934398777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967c70 msgr2=0x5634ea96c770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4ce000640 1 --2- 172.21.15.60:0/2934398777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967c70 0x5634ea96c770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4ce000640 1 -- 172.21.15.60:0/2934398777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634ea88ade0 con 0x5634ea96cde0 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4c7400640 1 -- 172.21.15.60:0/2297563691 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fd4c0046020 con 0x7fd4d0073c60 2024-02-26T10:46:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.950+0000 7fd4ce000640 1 --2- 172.21.15.60:0/2934398777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea96cde0 0x5634ea96f270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0114f0 tx=0x7fd4bc0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50fce434d21a576 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.951+0000 7fd4bbe00640 1 -- 172.21.15.60:0/2934398777 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc01e070 con 0x5634ea96cde0 2024-02-26T10:46:56.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.951+0000 7fd4bbe00640 1 -- 172.21.15.60:0/2934398777 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc0035f0 con 0x5634ea96cde0 2024-02-26T10:46:56.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.951+0000 7fd4bbe00640 1 -- 172.21.15.60:0/2934398777 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00f820 con 0x5634ea96cde0 2024-02-26T10:46:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.952+0000 7fd4d8801740 1 -- 172.21.15.60:0/2934398777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea96cde0 msgr2=0x5634ea96f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.952+0000 7fd4d8801740 1 --2- 172.21.15.60:0/2934398777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea96cde0 0x5634ea96f270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0114f0 tx=0x7fd4bc0119b0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.952+0000 7fd4d8801740 1 -- 172.21.15.60:0/2934398777 shutdown_connections 2024-02-26T10:46:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.952+0000 7fd4c7400640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4b40a1e90 0x7fd4b40a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.952+0000 7fd4d8801740 1 --2- 172.21.15.60:0/2934398777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea96cde0 0x5634ea96f270 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.952+0000 7fd4c7400640 1 -- 172.21.15.60:0/2297563691 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fd4c00f0290 con 0x7fd4d0073c60 2024-02-26T10:46:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.952+0000 7fd4d8801740 1 --2- 172.21.15.60:0/2934398777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967c70 0x5634ea96c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.953+0000 7fd4d8801740 1 --2- 172.21.15.60:0/2934398777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea9676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.953+0000 7fd4d8801740 1 -- 172.21.15.60:0/2934398777 >> 172.21.15.60:0/2934398777 conn(0x5634ea9627f0 msgr2=0x5634ea962bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.953+0000 7fd4d8801740 1 -- 172.21.15.60:0/2934398777 shutdown_connections 2024-02-26T10:46:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.953+0000 7fd4cf400640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4b40a1e90 0x7fd4b40a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.953+0000 7fd4d8801740 1 -- 172.21.15.60:0/2934398777 wait complete. 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.954+0000 7fd4d8801740 1 -- start start 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.954+0000 7fd4cf400640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4b40a1e90 0x7fd4b40a4350 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5634ea9684d0 tx=0x7fd4c8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.954+0000 7fd4d8801740 1 Processor -- start 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.954+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 0x5634ea9716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.954+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea971c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 0x5634ea978c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4d8801740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ea888f10 con 0x5634ea967280 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4d8801740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ea8c8a90 con 0x5634ea96cde0 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4d8801740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634ea96fd90 con 0x5634ea967c70 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4d8801740 10 client.0 _ll_register_callbacks cb 0x5634ea8c7a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4ce000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea971c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4ce000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea971c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46300/0 (socket says 172.21.15.60:46300) 2024-02-26T10:46:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4ce000640 1 -- 172.21.15.60:0/1925067580 learned_addr learned my addr 172.21.15.60:0/1925067580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4cea00640 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 0x5634ea9716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.955+0000 7fd4cf400640 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 0x5634ea978c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.956+0000 7fd4ce000640 1 -- 172.21.15.60:0/1925067580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 msgr2=0x5634ea978c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.956+0000 7fd4ce000640 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 0x5634ea978c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.956+0000 7fd4ce000640 1 -- 172.21.15.60:0/1925067580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 msgr2=0x5634ea9716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.956+0000 7fd4ce000640 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 0x5634ea9716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.956+0000 7fd4ce000640 1 -- 172.21.15.60:0/1925067580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634ea6a97e0 con 0x5634ea967c70 2024-02-26T10:46:56.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.956+0000 7fd4cea00640 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 0x5634ea9716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:56.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.957+0000 7fd4ce000640 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea971c10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc011eb0 tx=0x7fd4bc004210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fdd9f039e706c17 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.957+0000 7fd4b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:46:56.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.957+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1925067580 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc027020 con 0x5634ea967c70 2024-02-26T10:46:56.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.957+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1925067580 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fd4bc005250 con 0x5634ea967c70 2024-02-26T10:46:56.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.957+0000 7fd4d8801740 1 -- 172.21.15.60:0/1925067580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4bc0035f0 con 0x5634ea967c70 2024-02-26T10:46:56.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.957+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1925067580 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4bc032050 con 0x5634ea967c70 2024-02-26T10:46:56.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.957+0000 7fd4d8801740 10 client.25852 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.957+0000 7fd4d8801740 1 -- 172.21.15.60:0/1925067580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5634ea709f70 con 0x5634ea967c70 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.958+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1925067580 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc01f040 con 0x5634ea967c70 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.958+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1925067580 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd4bc00f5c0 con 0x5634ea967c70 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.958+0000 7fd4d8801740 10 client.25852 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.958+0000 7fd4d8801740 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.958+0000 7fd4d8801740 20 client.25852 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.958+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5634ea97d1f0 0x5634ea97f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.959+0000 7fd4d8801740 4 client.25852 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.959+0000 7fd4d8801740 1 -- 172.21.15.60:0/1925067580 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4bc0035f0 con 0x5634ea97d1f0 2024-02-26T10:46:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.959+0000 7fd4cea00640 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5634ea97d1f0 0x5634ea97f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.959+0000 7fd4b8c00640 10 client.25852 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.959+0000 7fd4cea00640 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5634ea97d1f0 0x5634ea97f5e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146aaee37e554edb server_cookie=acd6d7bbe6824ee in_seq=0 out_seq=0 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.960+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1925067580 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fd4bc0035f0 con 0x5634ea97d1f0 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.960+0000 7fd4b8c00640 10 client.25852 handle_command_reply: tid=0 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.960+0000 7fd4d8801740 1 client.25852 shutdown 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 -- 172.21.15.60:0/1925067580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 msgr2=0x5634ea971c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea971c10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc011eb0 tx=0x7fd4bc004210 comp rx=0 tx=0).stop 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 -- 172.21.15.60:0/1925067580 shutdown_connections 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5634ea97d1f0 0x5634ea97f5e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 0x5634ea978c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea971c10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4b8c00640 0 client.25852 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1925067580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 0x5634ea9716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 -- 172.21.15.60:0/1925067580 >> 172.21.15.60:0/1925067580 conn(0x5634ea9627f0 msgr2=0x5634ea961a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 -- 172.21.15.60:0/1925067580 shutdown_connections 2024-02-26T10:46:56.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 1 -- 172.21.15.60:0/1925067580 wait complete. 2024-02-26T10:46:56.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.961+0000 7fd4d8801740 20 client.25852 trim_cache size 0 max 16384 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.993+0000 7fd4d8801740 1 Processor -- start 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.993+0000 7fd4d8801740 1 -- start start 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 0x5634ea970160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea9706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 0x5634ea978c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4d8801740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01ae290 con 0x5634ea967280 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4d8801740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01abe30 con 0x5634ea96cde0 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4d8801740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01a9ed0 con 0x5634ea967c70 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 0x5634ea970160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4ce000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea9706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4cf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 0x5634ea978c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4ce000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea9706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46306/0 (socket says 172.21.15.60:46306) 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4cf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 0x5634ea978c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:41022/0 (socket says 172.21.15.60:41022) 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.994+0000 7fd4ce000640 1 -- 172.21.15.60:0/711912392 learned_addr learned my addr 172.21.15.60:0/711912392 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.995+0000 7fd4ce000640 1 -- 172.21.15.60:0/711912392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 msgr2=0x5634ea978c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.995+0000 7fd4ce000640 1 --2- 172.21.15.60:0/711912392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 0x5634ea978c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.995+0000 7fd4ce000640 1 -- 172.21.15.60:0/711912392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 msgr2=0x5634ea970160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.995+0000 7fd4ce000640 1 --2- 172.21.15.60:0/711912392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 0x5634ea970160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.995+0000 7fd4ce000640 1 -- 172.21.15.60:0/711912392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634ea97f950 con 0x5634ea967c70 2024-02-26T10:46:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.995+0000 7fd4ce000640 1 --2- 172.21.15.60:0/711912392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea9706a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0096c0 tx=0x7fd4bc0096f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55b0f5b552e44576 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.995+0000 7fd4bb400640 1 -- 172.21.15.60:0/711912392 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc016070 con 0x5634ea967c70 2024-02-26T10:46:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.995+0000 7fd4bb400640 1 -- 172.21.15.60:0/711912392 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc0035f0 con 0x5634ea967c70 2024-02-26T10:46:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.995+0000 7fd4bb400640 1 -- 172.21.15.60:0/711912392 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc027070 con 0x5634ea967c70 2024-02-26T10:46:56.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.996+0000 7fd4d8801740 1 -- 172.21.15.60:0/711912392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 msgr2=0x5634ea9706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.996+0000 7fd4d8801740 1 --2- 172.21.15.60:0/711912392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea9706a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0096c0 tx=0x7fd4bc0096f0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.996+0000 7fd4d8801740 1 -- 172.21.15.60:0/711912392 shutdown_connections 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.996+0000 7fd4d8801740 1 --2- 172.21.15.60:0/711912392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea96cde0 0x5634ea978c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.996+0000 7fd4d8801740 1 --2- 172.21.15.60:0/711912392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea9706a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.996+0000 7fd4d8801740 1 --2- 172.21.15.60:0/711912392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea967280 0x5634ea970160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.996+0000 7fd4d8801740 1 -- 172.21.15.60:0/711912392 >> 172.21.15.60:0/711912392 conn(0x5634ea9627f0 msgr2=0x5634ea9691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.996+0000 7fd4d8801740 1 -- 172.21.15.60:0/711912392 shutdown_connections 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.996+0000 7fd4d8801740 1 -- 172.21.15.60:0/711912392 wait complete. 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.997+0000 7fd4d8801740 1 -- start start 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.997+0000 7fd4d8801740 1 Processor -- start 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea978c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea977290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4d8801740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea96cde0 0x5634ea977810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4d8801740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01aa4d0 con 0x5634ea96cde0 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4d8801740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01ab7d0 con 0x5634ea967280 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4d8801740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01aab10 con 0x5634ea967c70 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4d8801740 10 client.0 _ll_register_callbacks cb 0x7fd4bc009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea978c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea978c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:41030/0 (socket says 172.21.15.60:41030) 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4cea00640 1 -- 172.21.15.60:0/1946882812 learned_addr learned my addr 172.21.15.60:0/1946882812 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4cea00640 1 -- 172.21.15.60:0/1946882812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 msgr2=0x5634ea977290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.998+0000 7fd4ce000640 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea977290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4cea00640 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea977290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4cea00640 1 -- 172.21.15.60:0/1946882812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea96cde0 msgr2=0x5634ea977810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4cf400640 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea96cde0 0x5634ea977810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4cea00640 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea96cde0 0x5634ea977810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4cea00640 1 -- 172.21.15.60:0/1946882812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634ea5a1170 con 0x5634ea967280 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4cf400640 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea96cde0 0x5634ea977810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4cea00640 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea978c30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002bd20 tx=0x7fd4c00ef040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea9ba8f8f87a05c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1946882812 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00c0d10 con 0x5634ea967280 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1946882812 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fd4c00baec0 con 0x5634ea967280 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:56.999+0000 7fd4d8801740 1 -- 172.21.15.60:0/1946882812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4bc0035f0 con 0x5634ea967280 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1946882812 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4c00a4050 con 0x5634ea967280 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4d8801740 10 client.25847 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4d8801740 1 -- 172.21.15.60:0/1946882812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5634ea9d4660 con 0x5634ea967280 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1946882812 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc0035f0 con 0x5634ea967280 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1946882812 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd4c00c4ed0 con 0x5634ea967280 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4d8801740 10 client.25847 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4d8801740 10 client.25847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4d8801740 20 client.25847 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5634ea9d4ef0 0x5634ea9d7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4d8801740 4 client.25847 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T10:46:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.000+0000 7fd4d8801740 1 -- 172.21.15.60:0/1946882812 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5634ea8d9f30 con 0x5634ea9d4ef0 2024-02-26T10:46:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.001+0000 7fd4ce000640 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5634ea9d4ef0 0x5634ea9d7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.001+0000 7fd4ce000640 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5634ea9d4ef0 0x5634ea9d7330 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff03c18423209585 server_cookie=c012913c05b304da in_seq=0 out_seq=0 2024-02-26T10:46:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.001+0000 7fd4b8c00640 10 client.25847 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.002+0000 7fd4b8c00640 1 -- 172.21.15.60:0/1946882812 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5634ea8d9f30 con 0x5634ea9d4ef0 2024-02-26T10:46:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.002+0000 7fd4b8c00640 10 client.25847 handle_command_reply: tid=0 2024-02-26T10:46:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.002+0000 7fd4d8801740 1 client.25847 shutdown 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 -- 172.21.15.60:0/1946882812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 msgr2=0x5634ea978c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea978c30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002bd20 tx=0x7fd4c00ef040 comp rx=0 tx=0).stop 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 -- 172.21.15.60:0/1946882812 shutdown_connections 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5634ea9d4ef0 0x5634ea9d7330 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5634ea96cde0 0x5634ea977810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634ea967c70 0x5634ea977290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 --2- 172.21.15.60:0/1946882812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634ea967280 0x5634ea978c30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 -- 172.21.15.60:0/1946882812 >> 172.21.15.60:0/1946882812 conn(0x5634ea9627f0 msgr2=0x5634ea9691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4b8c00640 0 client.25847 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 -- 172.21.15.60:0/1946882812 shutdown_connections 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.003+0000 7fd4d8801740 1 -- 172.21.15.60:0/1946882812 wait complete. 2024-02-26T10:46:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.004+0000 7fd4d8801740 20 client.25847 trim_cache size 0 max 16384 2024-02-26T10:46:57.001 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:46:57.001 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-02-26T10:46:57.001 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "9b9d82ce-fec1-45bd-89df-7cfc1a5b410a", 2024-02-26T10:46:57.001 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-02-26T10:46:57.001 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:46:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 -- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4b40a1e90 msgr2=0x7fd4b40a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4b40a1e90 0x7fd4b40a4350 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5634ea9684d0 tx=0x7fd4c8009500 comp rx=0 tx=0).stop 2024-02-26T10:46:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 -- 172.21.15.60:0/2297563691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0073c60 msgr2=0x7fd4d0199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0073c60 0x7fd4d0199640 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002c2f0 tx=0x7fd4c002c680 comp rx=0 tx=0).stop 2024-02-26T10:46:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 -- 172.21.15.60:0/2297563691 shutdown_connections 2024-02-26T10:46:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd4b40a1e90 0x7fd4b40a4350 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d0108ff0 0x7fd4d01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d0075240 0x7fd4d0199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 --2- 172.21.15.60:0/2297563691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4d0073c60 0x7fd4d0199640 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 -- 172.21.15.60:0/2297563691 >> 172.21.15.60:0/2297563691 conn(0x7fd4d00fc6d0 msgr2=0x7fd4d0104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:57.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 -- 172.21.15.60:0/2297563691 shutdown_connections 2024-02-26T10:46:57.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.005+0000 7fd4cfe00640 1 -- 172.21.15.60:0/2297563691 wait complete. 2024-02-26T10:46:57.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T10:46:57.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T10:46:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.252+0000 7f2c7e600640 1 -- 172.21.15.60:0/3621638209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78101160 msgr2=0x7f2c78101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.252+0000 7f2c7e600640 1 --2- 172.21.15.60:0/3621638209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78101160 0x7f2c78101580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009900 tx=0x7f2c6c02bc50 comp rx=0 tx=0).stop 2024-02-26T10:46:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.253+0000 7f2c7e600640 1 -- 172.21.15.60:0/3621638209 shutdown_connections 2024-02-26T10:46:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.253+0000 7f2c7e600640 1 --2- 172.21.15.60:0/3621638209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c78102d10 0x7f2c781095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.253+0000 7f2c7e600640 1 --2- 172.21.15.60:0/3621638209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c78102350 0x7f2c781027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.253+0000 7f2c7e600640 1 --2- 172.21.15.60:0/3621638209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78101160 0x7f2c78101580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.253+0000 7f2c7e600640 1 -- 172.21.15.60:0/3621638209 >> 172.21.15.60:0/3621638209 conn(0x7f2c780fc6d0 msgr2=0x7f2c780feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.253+0000 7f2c7e600640 1 -- 172.21.15.60:0/3621638209 shutdown_connections 2024-02-26T10:46:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.253+0000 7f2c7e600640 1 -- 172.21.15.60:0/3621638209 wait complete. 2024-02-26T10:46:57.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.254+0000 7f2c7e600640 1 Processor -- start 2024-02-26T10:46:57.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.254+0000 7f2c7e600640 1 -- start start 2024-02-26T10:46:57.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.254+0000 7f2c7e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c78101160 0x7f2c78199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.254+0000 7f2c7e600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78102350 0x7f2c78199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.254+0000 7f2c7e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c78102d10 0x7f2c781a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.254+0000 7f2c7e600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7810ce40 con 0x7f2c78102350 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.254+0000 7f2c7e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7810ccc0 con 0x7f2c78102d10 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.254+0000 7f2c7e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7810cfc0 con 0x7f2c78101160 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c78101160 0x7f2c78199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c78101160 0x7f2c78199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46330/0 (socket says 172.21.15.60:46330) 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7d200640 1 -- 172.21.15.60:0/444128772 learned_addr learned my addr 172.21.15.60:0/444128772 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c77e00640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78102350 0x7f2c78199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7dc00640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c78102d10 0x7f2c781a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7d200640 1 -- 172.21.15.60:0/444128772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c78102d10 msgr2=0x7f2c781a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7d200640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c78102d10 0x7f2c781a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7d200640 1 -- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78102350 msgr2=0x7f2c78199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7d200640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78102350 0x7f2c78199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7d200640 1 -- 172.21.15.60:0/444128772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c781a1270 con 0x7f2c78101160 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c77e00640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78102350 0x7f2c78199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.255+0000 7f2c7dc00640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c78102d10 0x7f2c781a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.256+0000 7f2c7d200640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c78101160 0x7f2c78199650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009850 tx=0x7f2c6c004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.256+0000 7f2c75600640 1 -- 172.21.15.60:0/444128772 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c041070 con 0x7f2c78101160 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.256+0000 7f2c75600640 1 -- 172.21.15.60:0/444128772 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c6c0054e0 con 0x7f2c78101160 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.256+0000 7f2c75600640 1 -- 172.21.15.60:0/444128772 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c003df0 con 0x7f2c78101160 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.256+0000 7f2c7e600640 1 -- 172.21.15.60:0/444128772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c781a1500 con 0x7f2c78101160 2024-02-26T10:46:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.256+0000 7f2c7e600640 1 -- 172.21.15.60:0/444128772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c781a1a00 con 0x7f2c78101160 2024-02-26T10:46:57.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.259+0000 7f2c829f8740 1 Processor -- start 2024-02-26T10:46:57.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.259+0000 7f2c829f8740 1 -- start start 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf5ce0 0x56215ccf6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 0x56215ccfb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 0x56215ccfdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c829f8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215cc23060 con 0x56215ccfb840 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c829f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215cc5ad00 con 0x56215ccf5ce0 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c829f8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215cc03a00 con 0x56215ccf66d0 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c7dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf5ce0 0x56215ccf6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c7dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf5ce0 0x56215ccf6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:41062/0 (socket says 172.21.15.60:41062) 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c7dc00640 1 -- 172.21.15.60:0/2261421764 learned_addr learned my addr 172.21.15.60:0/2261421764 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.260+0000 7f2c77e00640 1 --2- 172.21.15.60:0/2261421764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 0x56215ccfdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.261+0000 7f2c7dc00640 1 -- 172.21.15.60:0/2261421764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 msgr2=0x56215ccfb1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:46:57.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.261+0000 7f2c7dc00640 1 --2- 172.21.15.60:0/2261421764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 0x56215ccfb1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.261+0000 7f2c7dc00640 1 -- 172.21.15.60:0/2261421764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 msgr2=0x56215ccfdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:57.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.261+0000 7f2c7dc00640 1 --2- 172.21.15.60:0/2261421764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 0x56215ccfdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.261+0000 7f2c7dc00640 1 -- 172.21.15.60:0/2261421764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56215ca397d0 con 0x56215ccf5ce0 2024-02-26T10:46:57.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.261+0000 7f2c75600640 1 -- 172.21.15.60:0/444128772 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f2c6c044070 con 0x7f2c78101160 2024-02-26T10:46:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.261+0000 7f2c7dc00640 1 --2- 172.21.15.60:0/2261421764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf5ce0 0x56215ccf6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009fd0 tx=0x7f2c700108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b63d06e5e99a9e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.261+0000 7f2c6a000640 1 -- 172.21.15.60:0/2261421764 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70011760 con 0x56215ccf5ce0 2024-02-26T10:46:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.262+0000 7f2c6a000640 1 -- 172.21.15.60:0/2261421764 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c70011de0 con 0x56215ccf5ce0 2024-02-26T10:46:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.262+0000 7f2c6a000640 1 -- 172.21.15.60:0/2261421764 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000e3c0 con 0x56215ccf5ce0 2024-02-26T10:46:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.262+0000 7f2c829f8740 1 -- 172.21.15.60:0/2261421764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf5ce0 msgr2=0x56215ccf6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.262+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2261421764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf5ce0 0x56215ccf6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009fd0 tx=0x7f2c700108e0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.262+0000 7f2c829f8740 1 -- 172.21.15.60:0/2261421764 shutdown_connections 2024-02-26T10:46:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.262+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2261421764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 0x56215ccfdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.263+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2261421764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 0x56215ccfb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.263+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2261421764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf5ce0 0x56215ccf6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.263+0000 7f2c829f8740 1 -- 172.21.15.60:0/2261421764 >> 172.21.15.60:0/2261421764 conn(0x56215ccf1250 msgr2=0x56215ccf1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:57.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.264+0000 7f2c75600640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c600a1f50 0x7f2c600a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.264+0000 7f2c829f8740 1 -- 172.21.15.60:0/2261421764 shutdown_connections 2024-02-26T10:46:57.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.264+0000 7f2c829f8740 1 -- 172.21.15.60:0/2261421764 wait complete. 2024-02-26T10:46:57.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.264+0000 7f2c75600640 1 -- 172.21.15.60:0/444128772 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2c6c0f00e0 con 0x7f2c78101160 2024-02-26T10:46:57.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.264+0000 7f2c77e00640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c600a1f50 0x7f2c600a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.264+0000 7f2c829f8740 1 -- start start 2024-02-26T10:46:57.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.265+0000 7f2c829f8740 1 Processor -- start 2024-02-26T10:46:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.265+0000 7f2c77e00640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c600a1f50 0x7f2c600a4410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x56215ccf21c0 tx=0x7f2c64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.265+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf5ce0 0x56215ccffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.265+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf66d0 0x56215cd003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.265+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccfb840 0x56215cd07410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.265+0000 7f2c829f8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215cbf9090 con 0x56215ccf5ce0 2024-02-26T10:46:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.265+0000 7f2c829f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215cc56da0 con 0x56215ccf66d0 2024-02-26T10:46:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.265+0000 7f2c829f8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56215ccfe7f0 con 0x56215ccfb840 2024-02-26T10:46:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.265+0000 7f2c829f8740 10 client.0 _ll_register_callbacks cb 0x56215cc16830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c7d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf5ce0 0x56215ccffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c7d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf5ce0 0x56215ccffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39198/0 (socket says 172.21.15.60:39198) 2024-02-26T10:46:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c7d200640 1 -- 172.21.15.60:0/2254916184 learned_addr learned my addr 172.21.15.60:0/2254916184 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c77e00640 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf66d0 0x56215cd003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c7dc00640 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccfb840 0x56215cd07410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c7d200640 1 -- 172.21.15.60:0/2254916184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccfb840 msgr2=0x56215cd07410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c7d200640 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccfb840 0x56215cd07410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c7d200640 1 -- 172.21.15.60:0/2254916184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf66d0 msgr2=0x56215cd003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c7d200640 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf66d0 0x56215cd003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.266+0000 7f2c7d200640 1 -- 172.21.15.60:0/2254916184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56215c96f3f0 con 0x56215ccf5ce0 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.267+0000 7f2c7d200640 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf5ce0 0x56215ccffe90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c02c260 tx=0x7f2c6c004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c95e89e7960c57a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.267+0000 7f2c5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.267+0000 7f2c5ea00640 1 -- 172.21.15.60:0/2254916184 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c04b040 con 0x56215ccf5ce0 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.267+0000 7f2c5ea00640 1 -- 172.21.15.60:0/2254916184 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2c6c0bcc10 con 0x56215ccf5ce0 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.267+0000 7f2c829f8740 1 -- 172.21.15.60:0/2254916184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c70011de0 con 0x56215ccf5ce0 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c5ea00640 1 -- 172.21.15.60:0/2254916184 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c6c0bce90 con 0x56215ccf5ce0 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c829f8740 10 client.15735 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c829f8740 1 -- 172.21.15.60:0/2254916184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56215c92f0f0 con 0x56215ccf5ce0 2024-02-26T10:46:57.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c5ea00640 1 -- 172.21.15.60:0/2254916184 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c70011de0 con 0x56215ccf5ce0 2024-02-26T10:46:57.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c5ea00640 1 -- 172.21.15.60:0/2254916184 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c6c0bdce0 con 0x56215ccf5ce0 2024-02-26T10:46:57.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c829f8740 10 client.15735 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:57.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c829f8740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:57.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c829f8740 20 client.15735 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:57.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56215cd0b7f0 0x56215cd0dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c829f8740 4 client.15735 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:46:57.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.268+0000 7f2c829f8740 1 -- 172.21.15.60:0/2254916184 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56215cc68240 con 0x56215cd0b7f0 2024-02-26T10:46:57.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.269+0000 7f2c77e00640 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56215cd0b7f0 0x56215cd0dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.270+0000 7f2c77e00640 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56215cd0b7f0 0x56215cd0dbe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdc7f6e32f279eb7 server_cookie=8fed56f08e1a9232 in_seq=0 out_seq=0 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.270+0000 7f2c5ea00640 10 client.15735 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.270+0000 7f2c5ea00640 1 -- 172.21.15.60:0/2254916184 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56215cc68240 con 0x56215cd0b7f0 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.270+0000 7f2c5ea00640 10 client.15735 handle_command_reply: tid=0 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.270+0000 7f2c829f8740 1 client.15735 shutdown 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 -- 172.21.15.60:0/2254916184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf5ce0 msgr2=0x56215ccffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf5ce0 0x56215ccffe90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c02c260 tx=0x7f2c6c004510 comp rx=0 tx=0).stop 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 -- 172.21.15.60:0/2254916184 shutdown_connections 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56215cd0b7f0 0x56215cd0dbe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccf66d0 0x56215cd003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccfb840 0x56215cd07410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2254916184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf5ce0 0x56215ccffe90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 -- 172.21.15.60:0/2254916184 >> 172.21.15.60:0/2254916184 conn(0x56215ccf1250 msgr2=0x56215ccf0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c5ea00640 0 client.15735 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 -- 172.21.15.60:0/2254916184 shutdown_connections 2024-02-26T10:46:57.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 1 -- 172.21.15.60:0/2254916184 wait complete. 2024-02-26T10:46:57.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.271+0000 7f2c829f8740 20 client.15735 trim_cache size 0 max 16384 2024-02-26T10:46:57.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.288+0000 7f2c829f8740 1 Processor -- start 2024-02-26T10:46:57.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.288+0000 7f2c829f8740 1 -- start start 2024-02-26T10:46:57.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.288+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf66d0 0x56215ccf60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.288+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccfb840 0x56215cd073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.288+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf8160 0x56215ccf44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.288+0000 7f2c829f8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c781a9ea0 con 0x56215ccf66d0 2024-02-26T10:46:57.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.288+0000 7f2c829f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c781a7a40 con 0x56215ccfb840 2024-02-26T10:46:57.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.288+0000 7f2c829f8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c781a5b00 con 0x56215ccf8160 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccfb840 0x56215cd073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c7d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf66d0 0x56215ccf60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccfb840 0x56215cd073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:41080/0 (socket says 172.21.15.60:41080) 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c77e00640 1 -- 172.21.15.60:0/2578017401 learned_addr learned my addr 172.21.15.60:0/2578017401 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c7dc00640 1 --2- 172.21.15.60:0/2578017401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf8160 0x56215ccf44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c77e00640 1 -- 172.21.15.60:0/2578017401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf8160 msgr2=0x56215ccf44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c77e00640 1 --2- 172.21.15.60:0/2578017401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf8160 0x56215ccf44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c77e00640 1 -- 172.21.15.60:0/2578017401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf66d0 msgr2=0x56215ccf60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c77e00640 1 --2- 172.21.15.60:0/2578017401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf66d0 0x56215ccf60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c77e00640 1 -- 172.21.15.60:0/2578017401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56215cd12df0 con 0x56215ccfb840 2024-02-26T10:46:57.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.289+0000 7f2c77e00640 1 --2- 172.21.15.60:0/2578017401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccfb840 0x56215cd073a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x56215cd0e510 tx=0x7f2c64022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4caa19e534947e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:57.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.290+0000 7f2c69600640 1 -- 172.21.15.60:0/2578017401 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64027d90 con 0x56215ccfb840 2024-02-26T10:46:57.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.290+0000 7f2c69600640 1 -- 172.21.15.60:0/2578017401 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c64032890 con 0x56215ccfb840 2024-02-26T10:46:57.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.290+0000 7f2c69600640 1 -- 172.21.15.60:0/2578017401 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640047a0 con 0x56215ccfb840 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.290+0000 7f2c829f8740 1 -- 172.21.15.60:0/2578017401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccfb840 msgr2=0x56215cd073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.290+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2578017401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccfb840 0x56215cd073a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x56215cd0e510 tx=0x7f2c64022000 comp rx=0 tx=0).stop 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.291+0000 7f2c829f8740 1 -- 172.21.15.60:0/2578017401 shutdown_connections 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.291+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2578017401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215ccfb840 0x56215cd073a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.291+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2578017401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf8160 0x56215ccf44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.291+0000 7f2c829f8740 1 --2- 172.21.15.60:0/2578017401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccf66d0 0x56215ccf60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.291+0000 7f2c829f8740 1 -- 172.21.15.60:0/2578017401 >> 172.21.15.60:0/2578017401 conn(0x56215ccf1250 msgr2=0x56215ccf7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.291+0000 7f2c829f8740 1 -- 172.21.15.60:0/2578017401 shutdown_connections 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.291+0000 7f2c829f8740 1 -- 172.21.15.60:0/2578017401 wait complete. 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.291+0000 7f2c829f8740 1 -- start start 2024-02-26T10:46:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.291+0000 7f2c829f8740 1 Processor -- start 2024-02-26T10:46:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 0x56215cd04090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 0x56215cd045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c829f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215cd075a0 0x56215cd72ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c829f8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c781a6100 con 0x56215ccfb840 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c829f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c781a73e0 con 0x56215cd075a0 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c829f8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c781a6720 con 0x56215ccf66d0 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c829f8740 10 client.0 _ll_register_callbacks cb 0x7f2c6c0045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c7d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 0x56215cd04090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c7d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 0x56215cd04090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46358/0 (socket says 172.21.15.60:46358) 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.292+0000 7f2c7d200640 1 -- 172.21.15.60:0/3377835753 learned_addr learned my addr 172.21.15.60:0/3377835753 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.293+0000 7f2c77e00640 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 0x56215cd045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.293+0000 7f2c7d200640 1 -- 172.21.15.60:0/3377835753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215cd075a0 msgr2=0x56215cd72ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.293+0000 7f2c7d200640 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215cd075a0 0x56215cd72ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.293+0000 7f2c7d200640 1 -- 172.21.15.60:0/3377835753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 msgr2=0x56215cd045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.293+0000 7f2c7d200640 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 0x56215cd045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.293+0000 7f2c7d200640 1 -- 172.21.15.60:0/3377835753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56215cd73250 con 0x56215ccf66d0 2024-02-26T10:46:57.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.293+0000 7f2c7d200640 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 0x56215cd04090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0b9330 tx=0x7f2c6c0bce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6915d7190d63ac43 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.293+0000 7f2c5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.293+0000 7f2c5ea00640 1 -- 172.21.15.60:0/3377835753 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0b8c50 con 0x56215ccf66d0 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c829f8740 1 -- 172.21.15.60:0/3377835753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c64032890 con 0x56215ccf66d0 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c5ea00640 1 -- 172.21.15.60:0/3377835753 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2c6c0c4ed0 con 0x56215ccf66d0 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c5ea00640 1 -- 172.21.15.60:0/3377835753 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c6c0bc310 con 0x56215ccf66d0 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c829f8740 10 client.25870 fetch_fsmap learned FSMap version 24 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c829f8740 1 -- 172.21.15.60:0/3377835753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56215cd745d0 con 0x56215ccf66d0 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c5ea00640 1 -- 172.21.15.60:0/3377835753 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c64032890 con 0x56215ccf66d0 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c5ea00640 1 -- 172.21.15.60:0/3377835753 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c6c0bdce0 con 0x56215ccf66d0 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c829f8740 10 client.25870 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c829f8740 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c829f8740 20 client.25870 populate_metadata read hostname 'smithi060' 2024-02-26T10:46:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c829f8740 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56215cd74ce0 0x56215cd77100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:46:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c829f8740 4 client.25870 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T10:46:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.294+0000 7f2c829f8740 1 -- 172.21.15.60:0/3377835753 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56215cc62590 con 0x56215cd74ce0 2024-02-26T10:46:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.295+0000 7f2c77e00640 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56215cd74ce0 0x56215cd77100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:46:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.295+0000 7f2c77e00640 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56215cd74ce0 0x56215cd77100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bae4bc23128d8f5 server_cookie=bc72db853f8ee91 in_seq=0 out_seq=0 2024-02-26T10:46:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.295+0000 7f2c5ea00640 10 client.25870 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c5ea00640 1 -- 172.21.15.60:0/3377835753 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56215cc62590 con 0x56215cd74ce0 2024-02-26T10:46:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c5ea00640 10 client.25870 handle_command_reply: tid=0 2024-02-26T10:46:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 client.25870 shutdown 2024-02-26T10:46:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 -- 172.21.15.60:0/3377835753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 msgr2=0x56215cd04090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 0x56215cd04090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0b9330 tx=0x7f2c6c0bce20 comp rx=0 tx=0).stop 2024-02-26T10:46:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 -- 172.21.15.60:0/3377835753 shutdown_connections 2024-02-26T10:46:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x56215cd74ce0 0x56215cd77100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56215cd075a0 0x56215cd72ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56215ccfb840 0x56215cd045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 --2- 172.21.15.60:0/3377835753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56215ccf66d0 0x56215cd04090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c5ea00640 0 client.25870 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:46:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 -- 172.21.15.60:0/3377835753 >> 172.21.15.60:0/3377835753 conn(0x56215ccf1250 msgr2=0x56215ccfac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 -- 172.21.15.60:0/3377835753 shutdown_connections 2024-02-26T10:46:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 1 -- 172.21.15.60:0/3377835753 wait complete. 2024-02-26T10:46:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.296+0000 7f2c829f8740 20 client.25870 trim_cache size 0 max 16384 2024-02-26T10:46:57.295 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:46:57.296 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T10:46:57.296 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-02-26T10:46:57.296 INFO:teuthology.orchestra.run.smithi060.stdout: "9b9d82ce-fec1-45bd-89df-7cfc1a5b410a": { 2024-02-26T10:46:57.296 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-02-26T10:46:57.296 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "9b9d82ce-fec1-45bd-89df-7cfc1a5b410a", 2024-02-26T10:46:57.297 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-02-26T10:46:57.297 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:46:57.297 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:46:57.297 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 -- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c600a1f50 msgr2=0x7f2c600a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c600a1f50 0x7f2c600a4410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x56215ccf21c0 tx=0x7f2c64005f90 comp rx=0 tx=0).stop 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 -- 172.21.15.60:0/444128772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c78101160 msgr2=0x7f2c78199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c78101160 0x7f2c78199650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009850 tx=0x7f2c6c004e30 comp rx=0 tx=0).stop 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 -- 172.21.15.60:0/444128772 shutdown_connections 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c600a1f50 0x7f2c600a4410 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c78102d10 0x7f2c781a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c78102350 0x7f2c78199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 --2- 172.21.15.60:0/444128772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c78101160 0x7f2c78199650 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 -- 172.21.15.60:0/444128772 >> 172.21.15.60:0/444128772 conn(0x7f2c780fc6d0 msgr2=0x7f2c78073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.297+0000 7f2c7e600640 1 -- 172.21.15.60:0/444128772 shutdown_connections 2024-02-26T10:46:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:46:57.298+0000 7f2c7e600640 1 -- 172.21.15.60:0/444128772 wait complete. 2024-02-26T10:46:57.316 INFO:tasks.cephfs.filesystem:scrub status for tag:9b9d82ce-fec1-45bd-89df-7cfc1a5b410a - {'path': '/', 'tag': '9b9d82ce-fec1-45bd-89df-7cfc1a5b410a', 'options': 'recursive,force'} 2024-02-26T10:46:57.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:57 smithi113 ceph-mon[36126]: scrub queued for path: / 2024-02-26T10:46:57.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:57 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:57.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:57 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:46:57.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:57 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:57.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:57 smithi060 ceph-mon[31539]: scrub queued for path: / 2024-02-26T10:46:57.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:57 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:57.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:57 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:46:57.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:57 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:57.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:57 smithi100 ceph-mon[36051]: scrub queued for path: / 2024-02-26T10:46:57.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:57 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:57.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:57 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:46:57.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:57 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:46:58.775 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:46:58 smithi100 ceph-mon[36051]: pgmap v455: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T10:46:58.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:46:58 smithi113 ceph-mon[36126]: pgmap v455: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T10:46:58.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:46:58 smithi060 ceph-mon[31539]: pgmap v455: 129 pgs: 129 active+clean; 672 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T10:47:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:00 smithi113 ceph-mon[36126]: pgmap v456: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 8 op/s 2024-02-26T10:47:00.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:00 smithi060 ceph-mon[31539]: pgmap v456: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 8 op/s 2024-02-26T10:47:00.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:00 smithi100 ceph-mon[36051]: pgmap v456: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 8 op/s 2024-02-26T10:47:01.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:01 smithi113 ceph-mon[36126]: scrub summary: idle 2024-02-26T10:47:01.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:01 smithi060 ceph-mon[31539]: scrub summary: idle 2024-02-26T10:47:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:01 smithi100 ceph-mon[36051]: scrub summary: idle 2024-02-26T10:47:02.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:02 smithi113 ceph-mon[36126]: pgmap v457: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:02.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:02 smithi060 ceph-mon[31539]: pgmap v457: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:02.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:02 smithi100 ceph-mon[36051]: pgmap v457: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:04.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:04 smithi113 ceph-mon[36126]: pgmap v458: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:04.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:04 smithi060 ceph-mon[31539]: pgmap v458: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:04.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:04 smithi100 ceph-mon[36051]: pgmap v458: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:06.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:06 smithi113 ceph-mon[36126]: pgmap v459: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-02-26T10:47:06.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:06 smithi060 ceph-mon[31539]: pgmap v459: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-02-26T10:47:06.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:06 smithi100 ceph-mon[36051]: pgmap v459: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-02-26T10:47:08.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:08 smithi113 ceph-mon[36126]: pgmap v460: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:08.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:08 smithi113 ceph-mon[36126]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-02-26T10:47:08.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:08 smithi060 ceph-mon[31539]: pgmap v460: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:08.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:08 smithi060 ceph-mon[31539]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-02-26T10:47:08.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:08 smithi100 ceph-mon[36051]: pgmap v460: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:08.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:08 smithi100 ceph-mon[36051]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-02-26T10:47:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.717+0000 7fa831a00640 1 -- 172.21.15.60:0/2541801660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1046b0 msgr2=0x7fa82c104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.717+0000 7fa831a00640 1 --2- 172.21.15.60:0/2541801660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1046b0 0x7fa82c104b30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa818009900 tx=0x7fa81802bc30 comp rx=0 tx=0).stop 2024-02-26T10:47:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.717+0000 7fa831a00640 1 -- 172.21.15.60:0/2541801660 shutdown_connections 2024-02-26T10:47:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.717+0000 7fa831a00640 1 --2- 172.21.15.60:0/2541801660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c105070 0x7fa82c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.717+0000 7fa831a00640 1 --2- 172.21.15.60:0/2541801660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1046b0 0x7fa82c104b30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.717+0000 7fa831a00640 1 --2- 172.21.15.60:0/2541801660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c1034c0 0x7fa82c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.717+0000 7fa831a00640 1 -- 172.21.15.60:0/2541801660 >> 172.21.15.60:0/2541801660 conn(0x7fa82c0fea50 msgr2=0x7fa82c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.717+0000 7fa831a00640 1 -- 172.21.15.60:0/2541801660 shutdown_connections 2024-02-26T10:47:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.717+0000 7fa831a00640 1 -- 172.21.15.60:0/2541801660 wait complete. 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.718+0000 7fa831a00640 1 Processor -- start 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.718+0000 7fa831a00640 1 -- start start 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa831a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c1034c0 0x7fa82c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa831a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1046b0 0x7fa82c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa831a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c105070 0x7fa82c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa831a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c10f1a0 con 0x7fa82c1046b0 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa831a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c10f020 con 0x7fa82c105070 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa831a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c10f320 con 0x7fa82c1034c0 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa82a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1046b0 0x7fa82c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c1034c0 0x7fa82c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa82a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1046b0 0x7fa82c199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47286/0 (socket says 172.21.15.60:47286) 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa82a000640 1 -- 172.21.15.60:0/4232546492 learned_addr learned my addr 172.21.15.60:0/4232546492 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.719+0000 7fa82b400640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c105070 0x7fa82c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa82aa00640 1 -- 172.21.15.60:0/4232546492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c105070 msgr2=0x7fa82c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa82aa00640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c105070 0x7fa82c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa82aa00640 1 -- 172.21.15.60:0/4232546492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1046b0 msgr2=0x7fa82c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa82aa00640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1046b0 0x7fa82c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa82aa00640 1 -- 172.21.15.60:0/4232546492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c1a1350 con 0x7fa82c1034c0 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa82aa00640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c1034c0 0x7fa82c199730 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00e9c0 tx=0x7fa81c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa823400640 1 -- 172.21.15.60:0/4232546492 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00ccc0 con 0x7fa82c1034c0 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa831a00640 1 -- 172.21.15.60:0/4232546492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c1a1640 con 0x7fa82c1034c0 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa823400640 1 -- 172.21.15.60:0/4232546492 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa81c0035f0 con 0x7fa82c1034c0 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa823400640 1 -- 172.21.15.60:0/4232546492 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00ce60 con 0x7fa82c1034c0 2024-02-26T10:47:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.720+0000 7fa831a00640 1 -- 172.21.15.60:0/4232546492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c1a1b10 con 0x7fa82c1034c0 2024-02-26T10:47:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.723+0000 7fa823400640 1 -- 172.21.15.60:0/4232546492 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa81c0037a0 con 0x7fa82c1034c0 2024-02-26T10:47:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.724+0000 7fa8349cb740 1 Processor -- start 2024-02-26T10:47:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.724+0000 7fa8349cb740 1 -- start start 2024-02-26T10:47:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.724+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c6031cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.724+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c60321d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.724+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c60322380 0x559c60324810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.724+0000 7fa8349cb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60282340 con 0x559c6031c820 2024-02-26T10:47:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.724+0000 7fa8349cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6022b520 con 0x559c6031d210 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.724+0000 7fa8349cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6027e3e0 con 0x559c60322380 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c6031cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c60321d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c6031cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47294/0 (socket says 172.21.15.60:47294) 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82b400640 1 -- 172.21.15.60:0/1912050070 learned_addr learned my addr 172.21.15.60:0/1912050070 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82a000640 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c60322380 0x559c60324810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82b400640 1 -- 172.21.15.60:0/1912050070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c60322380 msgr2=0x559c60324810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82b400640 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c60322380 0x559c60324810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82b400640 1 -- 172.21.15.60:0/1912050070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 msgr2=0x559c60321d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82b400640 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c60321d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa823400640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8100a1e20 0x7fa8100a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82b400640 1 -- 172.21.15.60:0/1912050070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c6023f390 con 0x559c6031c820 2024-02-26T10:47:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82aa00640 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c60321d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa82a000640 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c60322380 0x559c60324810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:47:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.725+0000 7fa823400640 1 -- 172.21.15.60:0/4232546492 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa81c0ce4b0 con 0x7fa82c1034c0 2024-02-26T10:47:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.726+0000 7fa82b400640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8100a1e20 0x7fa8100a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.726+0000 7fa82b400640 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c6031cc40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa824011440 tx=0x7fa824011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17eaf37fa4853d30 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.726+0000 7fa817e00640 1 -- 172.21.15.60:0/1912050070 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824004250 con 0x559c6031c820 2024-02-26T10:47:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.726+0000 7fa817e00640 1 -- 172.21.15.60:0/1912050070 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa824003cd0 con 0x559c6031c820 2024-02-26T10:47:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.726+0000 7fa817e00640 1 -- 172.21.15.60:0/1912050070 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82401bc80 con 0x559c6031c820 2024-02-26T10:47:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa8349cb740 1 -- 172.21.15.60:0/1912050070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 msgr2=0x559c6031cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c6031cc40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa824011440 tx=0x7fa824011900 comp rx=0 tx=0).stop 2024-02-26T10:47:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa82b400640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8100a1e20 0x7fa8100a42e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa824020000 tx=0x7fa824021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa8349cb740 1 -- 172.21.15.60:0/1912050070 shutdown_connections 2024-02-26T10:47:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c60321d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c60322380 0x559c60324810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1912050070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c6031cc40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa8349cb740 1 -- 172.21.15.60:0/1912050070 >> 172.21.15.60:0/1912050070 conn(0x559c60317d90 msgr2=0x559c60318190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa8349cb740 1 -- 172.21.15.60:0/1912050070 shutdown_connections 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.727+0000 7fa8349cb740 1 -- 172.21.15.60:0/1912050070 wait complete. 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.728+0000 7fa8349cb740 1 -- start start 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.728+0000 7fa8349cb740 1 Processor -- start 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.728+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c60326b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031d210 0x559c60327090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c60322380 0x559c6032e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa8349cb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60325330 con 0x559c6031c820 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa8349cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6023e980 con 0x559c60322380 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa8349cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c603254b0 con 0x559c6031d210 2024-02-26T10:47:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa8349cb740 10 client.0 _ll_register_callbacks cb 0x559c6027a310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c60326b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c60326b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47300/0 (socket says 172.21.15.60:47300) 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82aa00640 1 -- 172.21.15.60:0/1469255046 learned_addr learned my addr 172.21.15.60:0/1469255046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82a000640 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031d210 0x559c60327090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82b400640 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c60322380 0x559c6032e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82aa00640 1 -- 172.21.15.60:0/1469255046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031d210 msgr2=0x559c60327090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82aa00640 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031d210 0x559c60327090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82aa00640 1 -- 172.21.15.60:0/1469255046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c60322380 msgr2=0x559c6032e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82aa00640 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c60322380 0x559c6032e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.729+0000 7fa82aa00640 1 -- 172.21.15.60:0/1469255046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c5ff97210 con 0x559c6031c820 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.730+0000 7fa82aa00640 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c60326b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa81c097230 tx=0x7fa81c09aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9c3673f507e830 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.730+0000 7fa814c00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.730+0000 7fa814c00640 1 -- 172.21.15.60:0/1469255046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c09b4d0 con 0x559c6031c820 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.730+0000 7fa814c00640 1 -- 172.21.15.60:0/1469255046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa81c003b50 con 0x559c6031c820 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.730+0000 7fa8349cb740 1 -- 172.21.15.60:0/1469255046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa824003cd0 con 0x559c6031c820 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.730+0000 7fa814c00640 1 -- 172.21.15.60:0/1469255046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81c09add0 con 0x559c6031c820 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.730+0000 7fa8349cb740 10 client.15753 fetch_fsmap learned FSMap version 24 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.730+0000 7fa8349cb740 1 -- 172.21.15.60:0/1469255046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559c60063620 con 0x559c6031c820 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.731+0000 7fa814c00640 1 -- 172.21.15.60:0/1469255046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa824003cd0 con 0x559c6031c820 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.731+0000 7fa814c00640 1 -- 172.21.15.60:0/1469255046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa81c0a30e0 con 0x559c6031c820 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.731+0000 7fa8349cb740 10 client.15753 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.731+0000 7fa8349cb740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.731+0000 7fa8349cb740 20 client.15753 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.731+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x559c603328c0 0x559c60334cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.731+0000 7fa8349cb740 4 client.15753 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.731+0000 7fa8349cb740 1 -- 172.21.15.60:0/1469255046 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c6028f880 con 0x559c603328c0 2024-02-26T10:47:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.731+0000 7fa82a000640 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x559c603328c0 0x559c60334cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.732+0000 7fa82a000640 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x559c603328c0 0x559c60334cd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a191acc4400a3c server_cookie=be29470e1750982 in_seq=0 out_seq=0 2024-02-26T10:47:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.732+0000 7fa814c00640 10 client.15753 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa814c00640 1 -- 172.21.15.60:0/1469255046 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c6028f880 con 0x559c603328c0 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa814c00640 10 client.15753 handle_command_reply: tid=0 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 client.15753 shutdown 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 -- 172.21.15.60:0/1469255046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 msgr2=0x559c60326b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c60326b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa81c097230 tx=0x7fa81c09aba0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 -- 172.21.15.60:0/1469255046 shutdown_connections 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x559c603328c0 0x559c60334cd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c60322380 0x559c6032e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031d210 0x559c60327090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 --2- 172.21.15.60:0/1469255046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c6031c820 0x559c60326b50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa814c00640 0 client.15753 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 -- 172.21.15.60:0/1469255046 >> 172.21.15.60:0/1469255046 conn(0x559c60317d90 msgr2=0x559c60316fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 -- 172.21.15.60:0/1469255046 shutdown_connections 2024-02-26T10:47:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 1 -- 172.21.15.60:0/1469255046 wait complete. 2024-02-26T10:47:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.733+0000 7fa8349cb740 20 client.15753 trim_cache size 0 max 16384 2024-02-26T10:47:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.748+0000 7fa8349cb740 1 Processor -- start 2024-02-26T10:47:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.748+0000 7fa8349cb740 1 -- start start 2024-02-26T10:47:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.748+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c60324770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.748+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c6032e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.748+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c60318170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.748+0000 7fa8349cb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1ae4a0 con 0x559c60322380 2024-02-26T10:47:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.748+0000 7fa8349cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1ac040 con 0x559c6031d210 2024-02-26T10:47:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.748+0000 7fa8349cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1aa0e0 con 0x559c6031c820 2024-02-26T10:47:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c60318170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c60324770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c60318170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47312/0 (socket says 172.21.15.60:47312) 2024-02-26T10:47:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82b400640 1 -- 172.21.15.60:0/231605729 learned_addr learned my addr 172.21.15.60:0/231605729 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82a000640 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c6032e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82aa00640 1 -- 172.21.15.60:0/231605729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 msgr2=0x559c6032e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82aa00640 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c6032e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82aa00640 1 -- 172.21.15.60:0/231605729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 msgr2=0x559c60318170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82aa00640 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c60318170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82aa00640 1 -- 172.21.15.60:0/231605729 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c603176e0 con 0x559c6031c820 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82a000640 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c6032e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.749+0000 7fa82b400640 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c60318170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.750+0000 7fa82aa00640 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c60324770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa81c099080 tx=0x7fa81c097100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0890358a454035a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.750+0000 7fa817400640 1 -- 172.21.15.60:0/231605729 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c09a390 con 0x559c6031c820 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.750+0000 7fa817400640 1 -- 172.21.15.60:0/231605729 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa81c09b4d0 con 0x559c6031c820 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.750+0000 7fa817400640 1 -- 172.21.15.60:0/231605729 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c09a910 con 0x559c6031c820 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 -- 172.21.15.60:0/231605729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 msgr2=0x559c60324770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c60324770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa81c099080 tx=0x7fa81c097100 comp rx=0 tx=0).stop 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 -- 172.21.15.60:0/231605729 shutdown_connections 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c6032e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c60318170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 --2- 172.21.15.60:0/231605729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c60324770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 -- 172.21.15.60:0/231605729 >> 172.21.15.60:0/231605729 conn(0x559c60317d90 msgr2=0x559c60321b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 -- 172.21.15.60:0/231605729 shutdown_connections 2024-02-26T10:47:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 -- 172.21.15.60:0/231605729 wait complete. 2024-02-26T10:47:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 -- start start 2024-02-26T10:47:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.751+0000 7fa8349cb740 1 Processor -- start 2024-02-26T10:47:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c6032be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c6032c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa8349cb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c6032d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa8349cb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1aa6e0 con 0x559c60322380 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa8349cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1ab9e0 con 0x559c6031d210 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa8349cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1aad20 con 0x559c6031c820 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa8349cb740 10 client.0 _ll_register_callbacks cb 0x7fa824009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c6032be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c6032be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:44010/0 (socket says 172.21.15.60:44010) 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa82aa00640 1 -- 172.21.15.60:0/735786003 learned_addr learned my addr 172.21.15.60:0/735786003 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa82b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c6032d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.752+0000 7fa82a000640 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c6032c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa82aa00640 1 -- 172.21.15.60:0/735786003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 msgr2=0x559c6032c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa82aa00640 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c6032c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa82aa00640 1 -- 172.21.15.60:0/735786003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 msgr2=0x559c6032d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa82aa00640 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c6032d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa82aa00640 1 -- 172.21.15.60:0/735786003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c6032d980 con 0x559c6031c820 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa82aa00640 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c6032be40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c6a0 tx=0x7fa81c09aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0b9d222ac422e88 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa814c00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa814c00640 1 -- 172.21.15.60:0/735786003 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c09a390 con 0x559c6031c820 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa814c00640 1 -- 172.21.15.60:0/735786003 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa81c066220 con 0x559c6031c820 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa8349cb740 1 -- 172.21.15.60:0/735786003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa81c09b4d0 con 0x559c6031c820 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa814c00640 1 -- 172.21.15.60:0/735786003 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81c003d60 con 0x559c6031c820 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa8349cb740 10 client.25885 fetch_fsmap learned FSMap version 24 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.753+0000 7fa8349cb740 1 -- 172.21.15.60:0/735786003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559c6032d980 con 0x559c6031c820 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.754+0000 7fa814c00640 1 -- 172.21.15.60:0/735786003 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa81c09b4d0 con 0x559c6031c820 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.754+0000 7fa814c00640 1 -- 172.21.15.60:0/735786003 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa81c083fc0 con 0x559c6031c820 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.754+0000 7fa8349cb740 10 client.25885 fetch_fsmap finished waiting for FSMap version 24 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.754+0000 7fa8349cb740 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.754+0000 7fa8349cb740 20 client.25885 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.754+0000 7fa8349cb740 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x559c60389a10 0x559c6038be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.754+0000 7fa8349cb740 4 client.25885 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:47:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.754+0000 7fa8349cb740 1 -- 172.21.15.60:0/735786003 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c60289bd0 con 0x559c60389a10 2024-02-26T10:47:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.754+0000 7fa82a000640 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x559c60389a10 0x559c6038be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.755+0000 7fa814c00640 10 client.25885 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.755+0000 7fa82a000640 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x559c60389a10 0x559c6038be30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8611cd345d294218 server_cookie=173b00c56873a312 in_seq=0 out_seq=0 2024-02-26T10:47:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa814c00640 1 -- 172.21.15.60:0/735786003 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c60289bd0 con 0x559c60389a10 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa814c00640 10 client.25885 handle_command_reply: tid=0 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 client.25885 shutdown 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 -- 172.21.15.60:0/735786003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 msgr2=0x559c6032be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c6032be40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c6a0 tx=0x7fa81c09aba0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 -- 172.21.15.60:0/735786003 shutdown_connections 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x559c60389a10 0x559c6038be30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c6031d210 0x559c6032c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559c60322380 0x559c6032d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 --2- 172.21.15.60:0/735786003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c6031c820 0x559c6032be40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa814c00640 0 client.25885 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 -- 172.21.15.60:0/735786003 >> 172.21.15.60:0/735786003 conn(0x559c60317d90 msgr2=0x559c60321750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 -- 172.21.15.60:0/735786003 shutdown_connections 2024-02-26T10:47:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 1 -- 172.21.15.60:0/735786003 wait complete. 2024-02-26T10:47:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.761+0000 7fa8349cb740 20 client.25885 trim_cache size 0 max 16384 2024-02-26T10:47:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:47:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:47:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:47:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:47:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 -- 172.21.15.60:0/4232546492 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8100a1e20 msgr2=0x7fa8100a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8100a1e20 0x7fa8100a42e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa824020000 tx=0x7fa824021000 comp rx=0 tx=0).stop 2024-02-26T10:47:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 -- 172.21.15.60:0/4232546492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c1034c0 msgr2=0x7fa82c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c1034c0 0x7fa82c199730 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00e9c0 tx=0x7fa81c00edc0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 -- 172.21.15.60:0/4232546492 shutdown_connections 2024-02-26T10:47:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa8100a1e20 0x7fa8100a42e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa82c105070 0x7fa82c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa82c1046b0 0x7fa82c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 --2- 172.21.15.60:0/4232546492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c1034c0 0x7fa82c199730 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 -- 172.21.15.60:0/4232546492 >> 172.21.15.60:0/4232546492 conn(0x7fa82c0fea50 msgr2=0x7fa82c1078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 -- 172.21.15.60:0/4232546492 shutdown_connections 2024-02-26T10:47:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:09.762+0000 7fa831a00640 1 -- 172.21.15.60:0/4232546492 wait complete. 2024-02-26T10:47:10.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:10 smithi113 ceph-mon[36126]: pgmap v461: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-02-26T10:47:10.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:10 smithi060 ceph-mon[31539]: pgmap v461: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-02-26T10:47:10.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:10 smithi100 ceph-mon[36051]: pgmap v461: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-02-26T10:47:11.495 INFO:tasks.workunit.client.0.smithi060.stderr:Updating files: 59% (7339/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 99% (12181/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr:Note: switching to '6085b91fefeb74bf4310344a87d81d3217ef7893'. 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr:state without impacting any branches by switching back to a branch. 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr: git switch -c 2024-02-26T10:47:11.901 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T10:47:11.902 INFO:tasks.workunit.client.0.smithi060.stderr:Or undo this operation with: 2024-02-26T10:47:11.902 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T10:47:11.902 INFO:tasks.workunit.client.0.smithi060.stderr: git switch - 2024-02-26T10:47:11.902 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T10:47:11.902 INFO:tasks.workunit.client.0.smithi060.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-26T10:47:11.902 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T10:47:11.902 INFO:tasks.workunit.client.0.smithi060.stderr:HEAD is now at 6085b91fefe qa/cephfs: add `MON_DOWN` to ignorelist 2024-02-26T10:47:11.905 DEBUG:teuthology.orchestra.run.smithi060:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-26T10:47:11.961 INFO:tasks.workunit.client.0.smithi060.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-26T10:47:11.971 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-26T10:47:11.972 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-26T10:47:12.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:12 smithi113 ceph-mon[36126]: pgmap v462: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T10:47:12.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:12 smithi060 ceph-mon[31539]: pgmap v462: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T10:47:12.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:12 smithi100 ceph-mon[36051]: pgmap v462: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-26T10:47:12.914 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-26T10:47:13.032 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-26T10:47:13.124 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-26T10:47:13.132 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-26T10:47:13.132 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-26T10:47:13.219 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-26T10:47:13.230 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:47:13.230 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-26T10:47:13.282 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-26T10:47:13.283 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-26T10:47:13.283 DEBUG:teuthology.orchestra.run.smithi060:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6085b91fefeb74bf4310344a87d81d3217ef7893 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-26T10:47:13.355 INFO:tasks.workunit.client.0.smithi060.stderr:+ mkdir -p fsstress 2024-02-26T10:47:13.359 INFO:tasks.workunit.client.0.smithi060.stderr:+ pushd fsstress 2024-02-26T10:47:13.360 INFO:tasks.workunit.client.0.smithi060.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-26T10:47:13.360 INFO:tasks.workunit.client.0.smithi060.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-26T10:47:14.050 INFO:tasks.workunit.client.0.smithi060.stderr:+ tar xzf ltp-full.tgz 2024-02-26T10:47:14.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:14 smithi113 ceph-mon[36126]: pgmap v463: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:14.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:14 smithi060 ceph-mon[31539]: pgmap v463: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:14.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:14 smithi100 ceph-mon[36051]: pgmap v463: 129 pgs: 129 active+clean; 673 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T10:47:16.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:16 smithi113 ceph-mon[36126]: pgmap v464: 129 pgs: 129 active+clean; 674 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-02-26T10:47:16.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:16 smithi060 ceph-mon[31539]: pgmap v464: 129 pgs: 129 active+clean; 674 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-02-26T10:47:16.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:16 smithi100 ceph-mon[36051]: pgmap v464: 129 pgs: 129 active+clean; 674 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-02-26T10:47:17.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:17 smithi113 ceph-mon[36126]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:17.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:17 smithi060 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:17.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:17 smithi100 ceph-mon[36051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:18.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:18 smithi113 ceph-mon[36126]: pgmap v465: 129 pgs: 129 active+clean; 674 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-26T10:47:18.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:18 smithi060 ceph-mon[31539]: pgmap v465: 129 pgs: 129 active+clean; 674 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-26T10:47:18.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:18 smithi100 ceph-mon[36051]: pgmap v465: 129 pgs: 129 active+clean; 674 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-26T10:47:20.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:20 smithi113 ceph-mon[36126]: pgmap v466: 129 pgs: 129 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 376 KiB/s wr, 16 op/s 2024-02-26T10:47:20.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:20 smithi060 ceph-mon[31539]: pgmap v466: 129 pgs: 129 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 376 KiB/s wr, 16 op/s 2024-02-26T10:47:20.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:20 smithi100 ceph-mon[36051]: pgmap v466: 129 pgs: 129 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 376 KiB/s wr, 16 op/s 2024-02-26T10:47:22.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:22 smithi113 ceph-mon[36126]: pgmap v467: 129 pgs: 129 active+clean; 6.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 609 KiB/s wr, 66 op/s 2024-02-26T10:47:22.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:22 smithi060 ceph-mon[31539]: pgmap v467: 129 pgs: 129 active+clean; 6.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 609 KiB/s wr, 66 op/s 2024-02-26T10:47:22.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:22 smithi100 ceph-mon[36051]: pgmap v467: 129 pgs: 129 active+clean; 6.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 609 KiB/s wr, 66 op/s 2024-02-26T10:47:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.908+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1914437101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0109150 msgr2=0x7f0ec010b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.908+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1914437101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0109150 0x7f0ec010b5f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009ca0 tx=0x7f0eb802c190 comp rx=0 tx=0).stop 2024-02-26T10:47:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.908+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1914437101 shutdown_connections 2024-02-26T10:47:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.908+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1914437101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0109150 0x7f0ec010b5f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.908+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1914437101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0073c60 0x7f0ec00740c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.908+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1914437101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec0075240 0x7f0ec0073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.908+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1914437101 >> 172.21.15.60:0/1914437101 conn(0x7f0ec00fc830 msgr2=0x7f0ec00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.908+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1914437101 shutdown_connections 2024-02-26T10:47:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.908+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1914437101 wait complete. 2024-02-26T10:47:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.909+0000 7f0ec6a00640 1 Processor -- start 2024-02-26T10:47:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.909+0000 7f0ec6a00640 1 -- start start 2024-02-26T10:47:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ec6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0073c60 0x7f0ec0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ec6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec0075240 0x7f0ec0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ec6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0109150 0x7f0ec01a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ec6a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec010f150 con 0x7f0ec0075240 2024-02-26T10:47:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ec6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec010efd0 con 0x7f0ec0073c60 2024-02-26T10:47:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ec6a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec010f2d0 con 0x7f0ec0109150 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0073c60 0x7f0ec0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0073c60 0x7f0ec0199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51140/0 (socket says 172.21.15.60:51140) 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ebfe00640 1 -- 172.21.15.60:0/1265360040 learned_addr learned my addr 172.21.15.60:0/1265360040 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ebf400640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec0075240 0x7f0ec0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0109150 0x7f0ec01a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ebfe00640 1 -- 172.21.15.60:0/1265360040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0109150 msgr2=0x7f0ec01a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0109150 0x7f0ec01a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ebfe00640 1 -- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec0075240 msgr2=0x7f0ec0199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec0075240 0x7f0ec0199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.910+0000 7f0ebfe00640 1 -- 172.21.15.60:0/1265360040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec01a12b0 con 0x7f0ec0073c60 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.911+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0073c60 0x7f0ec0199690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000e9f0 tx=0x7f0eb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.911+0000 7f0ebcc00640 1 -- 172.21.15.60:0/1265360040 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000cd80 con 0x7f0ec0073c60 2024-02-26T10:47:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.911+0000 7f0ebcc00640 1 -- 172.21.15.60:0/1265360040 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0010910 con 0x7f0ec0073c60 2024-02-26T10:47:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.911+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1265360040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec01a15a0 con 0x7f0ec0073c60 2024-02-26T10:47:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.911+0000 7f0ebcc00640 1 -- 172.21.15.60:0/1265360040 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0018990 con 0x7f0ec0073c60 2024-02-26T10:47:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.911+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1265360040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec01a1af0 con 0x7f0ec0073c60 2024-02-26T10:47:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.915+0000 7f0ebcc00640 1 -- 172.21.15.60:0/1265360040 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0eb0010ac0 con 0x7f0ec0073c60 2024-02-26T10:47:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.915+0000 7f0ec9972740 1 Processor -- start 2024-02-26T10:47:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.915+0000 7f0ec9972740 1 -- start start 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.915+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b574354450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.915+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 0x55b574359520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.915+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574359b90 0x55b57435c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.915+0000 7f0ec9972740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5742b1340 con 0x55b574354a20 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.915+0000 7f0ec9972740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57425a520 con 0x55b574359b90 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.915+0000 7f0ec9972740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5742ad3e0 con 0x55b574354030 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574359b90 0x55b57435c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574359b90 0x55b57435c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51144/0 (socket says 172.21.15.60:51144) 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebf400640 1 -- 172.21.15.60:0/22637350 learned_addr learned my addr 172.21.15.60:0/22637350 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/22637350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b574354450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/22637350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 0x55b574359520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebf400640 1 -- 172.21.15.60:0/22637350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 msgr2=0x55b574354450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebf400640 1 --2- 172.21.15.60:0/22637350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b574354450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebf400640 1 -- 172.21.15.60:0/22637350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 msgr2=0x55b574359520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebf400640 1 --2- 172.21.15.60:0/22637350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 0x55b574359520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebf400640 1 -- 172.21.15.60:0/22637350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57426e390 con 0x55b574359b90 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebcc00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ea40a1e20 0x7f0ea40a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebf400640 1 --2- 172.21.15.60:0/22637350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574359b90 0x55b57435c020 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009ca0 tx=0x7f0eac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=404a13507d4fe18 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.916+0000 7f0ebcc00640 1 -- 172.21.15.60:0/1265360040 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f0eb00ce490 con 0x7f0ec0073c60 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.917+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ea40a1e20 0x7f0ea40a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.917+0000 7f0eb5200640 1 -- 172.21.15.60:0/22637350 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac01e070 con 0x55b574359b90 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.917+0000 7f0eb5200640 1 -- 172.21.15.60:0/22637350 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eac019b10 con 0x55b574359b90 2024-02-26T10:47:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.917+0000 7f0eb5200640 1 -- 172.21.15.60:0/22637350 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00ea80 con 0x55b574359b90 2024-02-26T10:47:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.917+0000 7f0ec9972740 1 -- 172.21.15.60:0/22637350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574359b90 msgr2=0x55b57435c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.917+0000 7f0ec9972740 1 --2- 172.21.15.60:0/22637350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574359b90 0x55b57435c020 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009ca0 tx=0x7f0eac011810 comp rx=0 tx=0).stop 2024-02-26T10:47:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.917+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ea40a1e20 0x7f0ea40a42e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0eb802c6d0 tx=0x7f0eb8032670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.918+0000 7f0ec9972740 1 -- 172.21.15.60:0/22637350 shutdown_connections 2024-02-26T10:47:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.918+0000 7f0ec9972740 1 --2- 172.21.15.60:0/22637350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574359b90 0x55b57435c020 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.918+0000 7f0ec9972740 1 --2- 172.21.15.60:0/22637350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 0x55b574359520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.918+0000 7f0ec9972740 1 --2- 172.21.15.60:0/22637350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b574354450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.918+0000 7f0ec9972740 1 -- 172.21.15.60:0/22637350 >> 172.21.15.60:0/22637350 conn(0x55b574346d90 msgr2=0x55b574347190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.918+0000 7f0ec9972740 1 -- 172.21.15.60:0/22637350 shutdown_connections 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.918+0000 7f0ec9972740 1 -- 172.21.15.60:0/22637350 wait complete. 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.918+0000 7f0ec9972740 1 -- start start 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.918+0000 7f0ec9972740 1 Processor -- start 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b57435e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574354a20 0x55b57435e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 0x55b574365890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ec9972740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57435cb40 con 0x55b574359b90 2024-02-26T10:47:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ec9972740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57426d980 con 0x55b574354a20 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ec9972740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57435ccc0 con 0x55b574354030 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ec9972740 10 client.0 _ll_register_callbacks cb 0x55b5742a9310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b57435e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ebf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574354a20 0x55b57435e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ec4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 0x55b574365890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ebf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574354a20 0x55b57435e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51156/0 (socket says 172.21.15.60:51156) 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ebf400640 1 -- 172.21.15.60:0/3354191705 learned_addr learned my addr 172.21.15.60:0/3354191705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b57435e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:38348/0 (socket says 172.21.15.60:38348) 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.919+0000 7f0ebfe00640 1 -- 172.21.15.60:0/3354191705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574354a20 msgr2=0x55b57435e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574354a20 0x55b57435e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ebfe00640 1 -- 172.21.15.60:0/3354191705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 msgr2=0x55b574365890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 0x55b574365890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ebfe00640 1 -- 172.21.15.60:0/3354191705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b573fc6210 con 0x55b574354030 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ebf400640 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574354a20 0x55b57435e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b57435e310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00cbc30 tx=0x7f0eb0097250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60dd20eea7c73c0b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ea9600640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ea9600640 1 -- 172.21.15.60:0/3354191705 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb009aec0 con 0x55b574354030 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ea9600640 1 -- 172.21.15.60:0/3354191705 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f0eb00ce1e0 con 0x55b574354030 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ec9972740 1 -- 172.21.15.60:0/3354191705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0eac019b10 con 0x55b574354030 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ea9600640 1 -- 172.21.15.60:0/3354191705 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eb009ef60 con 0x55b574354030 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ec9972740 10 client.25897 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.920+0000 7f0ec9972740 1 -- 172.21.15.60:0/3354191705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b574092620 con 0x55b574354030 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.921+0000 7f0ea9600640 1 -- 172.21.15.60:0/3354191705 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eac019b10 con 0x55b574354030 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.921+0000 7f0ea9600640 1 -- 172.21.15.60:0/3354191705 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0eb009a4a0 con 0x55b574354030 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.921+0000 7f0ec9972740 10 client.25897 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.921+0000 7f0ec9972740 10 client.25897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.921+0000 7f0ec9972740 20 client.25897 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.921+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b57436a220 0x55b57436c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.921+0000 7f0ec9972740 4 client.25897 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.921+0000 7f0ec9972740 1 -- 172.21.15.60:0/3354191705 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5742be880 con 0x55b57436a220 2024-02-26T10:47:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.921+0000 7f0ebf400640 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b57436a220 0x55b57436c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.922+0000 7f0ea9600640 10 client.25897 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.922+0000 7f0ebf400640 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b57436a220 0x55b57436c690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f707bb1ddd004033 server_cookie=34a04622ba38e866 in_seq=0 out_seq=0 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.923+0000 7f0ea9600640 1 -- 172.21.15.60:0/3354191705 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5742be880 con 0x55b57436a220 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.923+0000 7f0ea9600640 10 client.25897 handle_command_reply: tid=0 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.923+0000 7f0ec9972740 1 client.25897 shutdown 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.923+0000 7f0ec9972740 1 -- 172.21.15.60:0/3354191705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 msgr2=0x55b57435e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.923+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b57435e310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00cbc30 tx=0x7f0eb0097250 comp rx=0 tx=0).stop 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ec9972740 1 -- 172.21.15.60:0/3354191705 shutdown_connections 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b57436a220 0x55b57436c690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b574354a20 0x55b57435e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 0x55b574365890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3354191705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354030 0x55b57435e310 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00cbc30 tx=0x7f0eb0097250 comp rx=0 tx=0).stop 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ec9972740 1 -- 172.21.15.60:0/3354191705 >> 172.21.15.60:0/3354191705 conn(0x55b574346d90 msgr2=0x55b574345fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ea9600640 0 client.25897 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ec9972740 1 -- 172.21.15.60:0/3354191705 shutdown_connections 2024-02-26T10:47:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ec9972740 1 -- 172.21.15.60:0/3354191705 wait complete. 2024-02-26T10:47:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.924+0000 7f0ec9972740 20 client.25897 trim_cache size 0 max 16384 2024-02-26T10:47:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.940+0000 7f0ec9972740 1 Processor -- start 2024-02-26T10:47:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.940+0000 7f0ec9972740 1 -- start start 2024-02-26T10:47:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.940+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 0x55b574371030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574359b90 0x55b574365850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b574355d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec9972740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01a9fb0 con 0x55b574354a20 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec9972740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01a7b50 con 0x55b57435e970 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec9972740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01a5bf0 con 0x55b574359b90 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b574355d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 0x55b574371030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b574355d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51158/0 (socket says 172.21.15.60:51158) 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec4a00640 1 -- 172.21.15.60:0/3588865199 learned_addr learned my addr 172.21.15.60:0/3588865199 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ebf400640 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574359b90 0x55b574365850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec4a00640 1 -- 172.21.15.60:0/3588865199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574359b90 msgr2=0x55b574365850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574359b90 0x55b574365850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec4a00640 1 -- 172.21.15.60:0/3588865199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 msgr2=0x55b574371030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 0x55b574371030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ec4a00640 1 -- 172.21.15.60:0/3588865199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b574354e00 con 0x55b57435e970 2024-02-26T10:47:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ebf400640 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574359b90 0x55b574365850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:47:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.941+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 0x55b574371030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.942+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b574355d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009ca0 tx=0x7f0eb8041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394e0f5a826c7dbd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.942+0000 7f0eabe00640 1 -- 172.21.15.60:0/3588865199 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8052070 con 0x55b57435e970 2024-02-26T10:47:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.942+0000 7f0eabe00640 1 -- 172.21.15.60:0/3588865199 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eb804e940 con 0x55b57435e970 2024-02-26T10:47:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.942+0000 7f0eabe00640 1 -- 172.21.15.60:0/3588865199 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8039040 con 0x55b57435e970 2024-02-26T10:47:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.943+0000 7f0ec9972740 1 -- 172.21.15.60:0/3588865199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 msgr2=0x55b574355d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.943+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b574355d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009ca0 tx=0x7f0eb8041000 comp rx=0 tx=0).stop 2024-02-26T10:47:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.943+0000 7f0ec9972740 1 -- 172.21.15.60:0/3588865199 shutdown_connections 2024-02-26T10:47:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.943+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b574355d00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.943+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574359b90 0x55b574365850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.943+0000 7f0ec9972740 1 --2- 172.21.15.60:0/3588865199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574354a20 0x55b574371030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.943+0000 7f0ec9972740 1 -- 172.21.15.60:0/3588865199 >> 172.21.15.60:0/3588865199 conn(0x55b574346d90 msgr2=0x55b574347170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.943+0000 7f0ec9972740 1 -- 172.21.15.60:0/3588865199 shutdown_connections 2024-02-26T10:47:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.943+0000 7f0ec9972740 1 -- 172.21.15.60:0/3588865199 wait complete. 2024-02-26T10:47:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.944+0000 7f0ec9972740 1 -- start start 2024-02-26T10:47:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.944+0000 7f0ec9972740 1 Processor -- start 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354a20 0x55b574364460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 0x55b5743649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec9972740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b5743c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec9972740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01a61f0 con 0x55b574359b90 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec9972740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01a74f0 con 0x55b57435e970 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec9972740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01a6830 con 0x55b574354a20 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec9972740 10 client.0 _ll_register_callbacks cb 0x7f0eac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b5743c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b5743c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51166/0 (socket says 172.21.15.60:51166) 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec4a00640 1 -- 172.21.15.60:0/4257420159 learned_addr learned my addr 172.21.15.60:0/4257420159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ebf400640 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 0x55b5743649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354a20 0x55b574364460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec4a00640 1 -- 172.21.15.60:0/4257420159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354a20 msgr2=0x55b574364460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354a20 0x55b574364460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec4a00640 1 -- 172.21.15.60:0/4257420159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 msgr2=0x55b5743649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 0x55b5743649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.945+0000 7f0ec4a00640 1 -- 172.21.15.60:0/4257420159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5743c0ce0 con 0x55b57435e970 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.946+0000 7f0ec4a00640 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b5743c07a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009bf0 tx=0x7f0eb802c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=babdf0b5ad17835a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.946+0000 7f0ea9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.946+0000 7f0ea9600640 1 -- 172.21.15.60:0/4257420159 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb805b020 con 0x55b57435e970 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.946+0000 7f0ec9972740 1 -- 172.21.15.60:0/4257420159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0eb804e940 con 0x55b57435e970 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.946+0000 7f0ea9600640 1 -- 172.21.15.60:0/4257420159 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f0eb804eaf0 con 0x55b57435e970 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.946+0000 7f0ea9600640 1 -- 172.21.15.60:0/4257420159 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eb8066050 con 0x55b57435e970 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.946+0000 7f0ec9972740 10 client.25898 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.946+0000 7f0ec9972740 1 -- 172.21.15.60:0/4257420159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b5743c1e00 con 0x55b57435e970 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.947+0000 7f0ea9600640 1 -- 172.21.15.60:0/4257420159 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eb804e940 con 0x55b57435e970 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.947+0000 7f0ea9600640 1 -- 172.21.15.60:0/4257420159 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0eb8034c00 con 0x55b57435e970 2024-02-26T10:47:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.947+0000 7f0ec9972740 10 client.25898 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.947+0000 7f0ec9972740 10 client.25898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.947+0000 7f0ec9972740 20 client.25898 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.947+0000 7f0ec9972740 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b5743c27f0 0x55b5743c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.947+0000 7f0ec9972740 4 client.25898 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:47:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.947+0000 7f0ec9972740 1 -- 172.21.15.60:0/4257420159 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b5742b8bd0 con 0x55b5743c27f0 2024-02-26T10:47:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.947+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b5743c27f0 0x55b5743c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.948+0000 7f0ebfe00640 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b5743c27f0 0x55b5743c4c10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7b77c267fae052 server_cookie=167b5d53a06b0d59 in_seq=0 out_seq=0 2024-02-26T10:47:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:22.948+0000 7f0ea9600640 10 client.25898 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ea9600640 1 -- 172.21.15.60:0/4257420159 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b5742b8bd0 con 0x55b5743c27f0 2024-02-26T10:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ea9600640 10 client.25898 handle_command_reply: tid=0 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 client.25898 shutdown 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 -- 172.21.15.60:0/4257420159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 msgr2=0x55b5743c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b5743c07a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009bf0 tx=0x7f0eb802c910 comp rx=0 tx=0).stop 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 -- 172.21.15.60:0/4257420159 shutdown_connections 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55b5743c27f0 0x55b5743c4c10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b57435e970 0x55b5743c07a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b574359b90 0x55b5743649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 --2- 172.21.15.60:0/4257420159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b574354a20 0x55b574364460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ea9600640 0 client.25898 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 -- 172.21.15.60:0/4257420159 >> 172.21.15.60:0/4257420159 conn(0x55b574346d90 msgr2=0x55b574358f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 -- 172.21.15.60:0/4257420159 shutdown_connections 2024-02-26T10:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 1 -- 172.21.15.60:0/4257420159 wait complete. 2024-02-26T10:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.221+0000 7f0ec9972740 20 client.25898 trim_cache size 0 max 16384 2024-02-26T10:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ea40a1e20 msgr2=0x7f0ea40a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ea40a1e20 0x7f0ea40a42e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0eb802c6d0 tx=0x7f0eb8032670 comp rx=0 tx=0).stop 2024-02-26T10:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1265360040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0073c60 msgr2=0x7f0ec0199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0073c60 0x7f0ec0199690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000e9f0 tx=0x7f0eb000eeb0 comp rx=0 tx=0).stop 2024-02-26T10:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1265360040 shutdown_connections 2024-02-26T10:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f0ea40a1e20 0x7f0ea40a42e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0109150 0x7f0ec01a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ec0075240 0x7f0ec0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 --2- 172.21.15.60:0/1265360040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ec0073c60 0x7f0ec0199690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1265360040 >> 172.21.15.60:0/1265360040 conn(0x7f0ec00fc830 msgr2=0x7f0ec01050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1265360040 shutdown_connections 2024-02-26T10:47:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:24.223+0000 7f0ec6a00640 1 -- 172.21.15.60:0/1265360040 wait complete. 2024-02-26T10:47:24.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:24 smithi113 ceph-mon[36126]: pgmap v468: 129 pgs: 129 active+clean; 13 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-02-26T10:47:24.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:24 smithi060 ceph-mon[31539]: pgmap v468: 129 pgs: 129 active+clean; 13 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-02-26T10:47:24.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:24 smithi100 ceph-mon[36051]: pgmap v468: 129 pgs: 129 active+clean; 13 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-02-26T10:47:26.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:26 smithi113 ceph-mon[36126]: pgmap v469: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 561 op/s 2024-02-26T10:47:26.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:26 smithi060 ceph-mon[31539]: pgmap v469: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 561 op/s 2024-02-26T10:47:26.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:26 smithi100 ceph-mon[36051]: pgmap v469: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 561 op/s 2024-02-26T10:47:27.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T10:47:27.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T10:47:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.567+0000 7fe697a00640 1 -- 172.21.15.60:0/2213349823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe690101d10 msgr2=0x7fe6901085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.567+0000 7fe697a00640 1 --2- 172.21.15.60:0/2213349823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe690101d10 0x7fe6901085a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe688009ca0 tx=0x7fe68802c0f0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.567+0000 7fe697a00640 1 -- 172.21.15.60:0/2213349823 shutdown_connections 2024-02-26T10:47:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.567+0000 7fe697a00640 1 --2- 172.21.15.60:0/2213349823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe690101d10 0x7fe6901085a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.567+0000 7fe697a00640 1 --2- 172.21.15.60:0/2213349823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe690101350 0x7fe6901017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.567+0000 7fe697a00640 1 --2- 172.21.15.60:0/2213349823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe690100160 0x7fe690100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.567+0000 7fe697a00640 1 -- 172.21.15.60:0/2213349823 >> 172.21.15.60:0/2213349823 conn(0x7fe6900fb6f0 msgr2=0x7fe6900fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.567+0000 7fe697a00640 1 -- 172.21.15.60:0/2213349823 shutdown_connections 2024-02-26T10:47:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.567+0000 7fe697a00640 1 -- 172.21.15.60:0/2213349823 wait complete. 2024-02-26T10:47:27.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.568+0000 7fe697a00640 1 Processor -- start 2024-02-26T10:47:27.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe697a00640 1 -- start start 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe697a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe690100160 0x7fe690197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe697a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe690101350 0x7fe690197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe697a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe690101d10 0x7fe69019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe697a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69010bdf0 con 0x7fe690101d10 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe697a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69010bc70 con 0x7fe690101350 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe697a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69010bf70 con 0x7fe690100160 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe690101350 0x7fe690197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe695000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe690100160 0x7fe690197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe690101350 0x7fe690197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53380/0 (socket says 172.21.15.60:53380) 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe68fe00640 1 -- 172.21.15.60:0/3970691931 learned_addr learned my addr 172.21.15.60:0/3970691931 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.569+0000 7fe695a00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe690101d10 0x7fe69019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe68fe00640 1 -- 172.21.15.60:0/3970691931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe690100160 msgr2=0x7fe690197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe68fe00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe690100160 0x7fe690197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe68fe00640 1 -- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe690101d10 msgr2=0x7fe69019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe68fe00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe690101d10 0x7fe69019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe68fe00640 1 -- 172.21.15.60:0/3970691931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69019f160 con 0x7fe690101350 2024-02-26T10:47:27.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe695000640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe690100160 0x7fe690197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:27.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe68fe00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe690101350 0x7fe690197a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00e9f0 tx=0x7fe67c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:27.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe68d600640 1 -- 172.21.15.60:0/3970691931 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00cd80 con 0x7fe690101350 2024-02-26T10:47:27.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe68d600640 1 -- 172.21.15.60:0/3970691931 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe67c005500 con 0x7fe690101350 2024-02-26T10:47:27.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe697a00640 1 -- 172.21.15.60:0/3970691931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69019f450 con 0x7fe690101350 2024-02-26T10:47:27.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe68d600640 1 -- 172.21.15.60:0/3970691931 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c010690 con 0x7fe690101350 2024-02-26T10:47:27.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.570+0000 7fe697a00640 1 -- 172.21.15.60:0/3970691931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69019f950 con 0x7fe690101350 2024-02-26T10:47:27.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.573+0000 7fe68d600640 1 -- 172.21.15.60:0/3970691931 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe67c0046c0 con 0x7fe690101350 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe69a9b5740 1 Processor -- start 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe69a9b5740 1 -- start start 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc90350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc95420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cc97f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8cbb9060 con 0x55ae8cc95a90 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8cbf0d00 con 0x55ae8cc90920 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8cb99a00 con 0x55ae8cc8ff30 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe695000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc95420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe695a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc90350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe695000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc95420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53394/0 (socket says 172.21.15.60:53394) 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.574+0000 7fe695a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc90350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:45944/0 (socket says 172.21.15.60:45944) 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe695000640 1 -- 172.21.15.60:0/1119036504 learned_addr learned my addr 172.21.15.60:0/1119036504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe68fe00640 1 --2- 172.21.15.60:0/1119036504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cc97f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe695000640 1 -- 172.21.15.60:0/1119036504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 msgr2=0x55ae8cc90350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe695000640 1 --2- 172.21.15.60:0/1119036504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc90350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe695000640 1 -- 172.21.15.60:0/1119036504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 msgr2=0x55ae8cc97f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe695000640 1 --2- 172.21.15.60:0/1119036504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cc97f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe695000640 1 -- 172.21.15.60:0/1119036504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8c9cf7d0 con 0x55ae8cc90920 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe68d600640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6780a1d80 0x7fe6780a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe695000640 1 --2- 172.21.15.60:0/1119036504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc95420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6840114c0 tx=0x7fe684011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7ff1a7bda4e37d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe68d600640 1 -- 172.21.15.60:0/3970691931 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe67c0ce5b0 con 0x7fe690101350 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe695a00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6780a1d80 0x7fe6780a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.575+0000 7fe682000640 1 -- 172.21.15.60:0/1119036504 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684004020 con 0x55ae8cc90920 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe682000640 1 -- 172.21.15.60:0/1119036504 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe684005500 con 0x55ae8cc90920 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe682000640 1 -- 172.21.15.60:0/1119036504 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684004e10 con 0x55ae8cc90920 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1119036504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 msgr2=0x55ae8cc95420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1119036504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc95420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6840114c0 tx=0x7fe684011980 comp rx=0 tx=0).stop 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe695a00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6780a1d80 0x7fe6780a4240 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe68802c630 tx=0x7fe688032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1119036504 shutdown_connections 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1119036504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc95420 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1119036504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cc97f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1119036504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc90350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1119036504 >> 172.21.15.60:0/1119036504 conn(0x55ae8cc87090 msgr2=0x55ae8cc87490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.576+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1119036504 shutdown_connections 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.577+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1119036504 wait complete. 2024-02-26T10:47:27.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.577+0000 7fe69a9b5740 1 -- start start 2024-02-26T10:47:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.577+0000 7fe69a9b5740 1 Processor -- start 2024-02-26T10:47:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc9a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc9a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cca1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8cb8f090 con 0x55ae8cc95a90 2024-02-26T10:47:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8cbecda0 con 0x55ae8cc90920 2024-02-26T10:47:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8cc98a40 con 0x55ae8cc8ff30 2024-02-26T10:47:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe69a9b5740 10 client.0 _ll_register_callbacks cb 0x55ae8cbac830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc9a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc9a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53408/0 (socket says 172.21.15.60:53408) 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe68fe00640 1 -- 172.21.15.60:0/3381103652 learned_addr learned my addr 172.21.15.60:0/3381103652 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe695000640 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc9a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe695a00640 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cca1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe68fe00640 1 -- 172.21.15.60:0/3381103652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 msgr2=0x55ae8cc9a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe68fe00640 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc9a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe68fe00640 1 -- 172.21.15.60:0/3381103652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 msgr2=0x55ae8cca1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe68fe00640 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cca1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.578+0000 7fe68fe00640 1 -- 172.21.15.60:0/3381103652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae8c9053f0 con 0x55ae8cc90920 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe68fe00640 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc9a810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe67c097330 tx=0x7fe67c09aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38be69a4d71ab633 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe676a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe676a00640 1 -- 172.21.15.60:0/3381103652 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c09b5d0 con 0x55ae8cc90920 2024-02-26T10:47:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe676a00640 1 -- 172.21.15.60:0/3381103652 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe67c0a31e0 con 0x55ae8cc90920 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe69a9b5740 1 -- 172.21.15.60:0/3381103652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe684005500 con 0x55ae8cc90920 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe676a00640 1 -- 172.21.15.60:0/3381103652 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe67c09aed0 con 0x55ae8cc90920 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe69a9b5740 10 client.25910 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe69a9b5740 1 -- 172.21.15.60:0/3381103652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae8c8c50f0 con 0x55ae8cc90920 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe676a00640 1 -- 172.21.15.60:0/3381103652 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe684005500 con 0x55ae8cc90920 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.579+0000 7fe676a00640 1 -- 172.21.15.60:0/3381103652 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe67c0b91f0 con 0x55ae8cc90920 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.580+0000 7fe69a9b5740 10 client.25910 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.580+0000 7fe69a9b5740 10 client.25910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.580+0000 7fe69a9b5740 20 client.25910 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.580+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ae8cca5de0 0x55ae8cca81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.580+0000 7fe69a9b5740 4 client.25910 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.580+0000 7fe69a9b5740 1 -- 172.21.15.60:0/3381103652 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae8cbfe240 con 0x55ae8cca5de0 2024-02-26T10:47:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.580+0000 7fe695000640 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ae8cca5de0 0x55ae8cca81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.580+0000 7fe695000640 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ae8cca5de0 0x55ae8cca81d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14aba4fbcf0afbb3 server_cookie=b94f54e8a96491a0 in_seq=0 out_seq=0 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.580+0000 7fe676a00640 10 client.25910 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe676a00640 1 -- 172.21.15.60:0/3381103652 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae8cbfe240 con 0x55ae8cca5de0 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe676a00640 10 client.25910 handle_command_reply: tid=0 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 client.25910 shutdown 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 -- 172.21.15.60:0/3381103652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 msgr2=0x55ae8cc9a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc9a810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe67c097330 tx=0x7fe67c09aa10 comp rx=0 tx=0).stop 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 -- 172.21.15.60:0/3381103652 shutdown_connections 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ae8cca5de0 0x55ae8cca81d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cc9a810 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe676a00640 0 client.25910 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:27.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cca1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/3381103652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc9a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 -- 172.21.15.60:0/3381103652 >> 172.21.15.60:0/3381103652 conn(0x55ae8cc87090 msgr2=0x55ae8cc86330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:27.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 -- 172.21.15.60:0/3381103652 shutdown_connections 2024-02-26T10:47:27.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 1 -- 172.21.15.60:0/3381103652 wait complete. 2024-02-26T10:47:27.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.582+0000 7fe69a9b5740 20 client.25910 trim_cache size 0 max 16384 2024-02-26T10:47:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.599+0000 7fe69a9b5740 1 Processor -- start 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe69a9b5740 1 -- start start 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc98de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc90920 0x55ae8cca1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc95a90 0x55ae8cc99320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901aa0b0 con 0x55ae8cc8ff30 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901a7c50 con 0x55ae8cc95a90 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901a5cf0 con 0x55ae8cc90920 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe695a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc95a90 0x55ae8cc99320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe695000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc98de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe695a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc95a90 0x55ae8cc99320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53412/0 (socket says 172.21.15.60:53412) 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe695a00640 1 -- 172.21.15.60:0/2838634537 learned_addr learned my addr 172.21.15.60:0/2838634537 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.600+0000 7fe68fe00640 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc90920 0x55ae8cca1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe695a00640 1 -- 172.21.15.60:0/2838634537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc90920 msgr2=0x55ae8cca1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe695a00640 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc90920 0x55ae8cca1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe695a00640 1 -- 172.21.15.60:0/2838634537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc8ff30 msgr2=0x55ae8cc98de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe695a00640 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc98de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe695a00640 1 -- 172.21.15.60:0/2838634537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8ccad3c0 con 0x55ae8cc95a90 2024-02-26T10:47:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe68fe00640 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc90920 0x55ae8cca1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe695000640 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc98de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe695a00640 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc95a90 0x55ae8cc99320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe688009ca0 tx=0x7fe688041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d283612214c8f6a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe681600640 1 -- 172.21.15.60:0/2838634537 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688039040 con 0x55ae8cc95a90 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe681600640 1 -- 172.21.15.60:0/2838634537 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe688003e50 con 0x55ae8cc95a90 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.601+0000 7fe681600640 1 -- 172.21.15.60:0/2838634537 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68804e6c0 con 0x55ae8cc95a90 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.602+0000 7fe69a9b5740 1 -- 172.21.15.60:0/2838634537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc95a90 msgr2=0x55ae8cc99320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.602+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc95a90 0x55ae8cc99320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe688009ca0 tx=0x7fe688041000 comp rx=0 tx=0).stop 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.602+0000 7fe69a9b5740 1 -- 172.21.15.60:0/2838634537 shutdown_connections 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.602+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc95a90 0x55ae8cc99320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.602+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc90920 0x55ae8cca1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.602+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/2838634537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc8ff30 0x55ae8cc98de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.602+0000 7fe69a9b5740 1 -- 172.21.15.60:0/2838634537 >> 172.21.15.60:0/2838634537 conn(0x55ae8cc87090 msgr2=0x55ae8cc9a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.602+0000 7fe69a9b5740 1 -- 172.21.15.60:0/2838634537 shutdown_connections 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.602+0000 7fe69a9b5740 1 -- 172.21.15.60:0/2838634537 wait complete. 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe69a9b5740 1 -- start start 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe69a9b5740 1 Processor -- start 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cca10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cca1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe69a9b5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cc9f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901a62f0 con 0x55ae8cc95a90 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901a75f0 con 0x55ae8cc90920 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe69a9b5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901a6930 con 0x55ae8cc8ff30 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe69a9b5740 10 client.0 _ll_register_callbacks cb 0x7fe67c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cca1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cca1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53416/0 (socket says 172.21.15.60:53416) 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.603+0000 7fe68fe00640 1 -- 172.21.15.60:0/1410935421 learned_addr learned my addr 172.21.15.60:0/1410935421 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe695a00640 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cc9f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe695000640 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cca10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe68fe00640 1 -- 172.21.15.60:0/1410935421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 msgr2=0x55ae8cca10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe68fe00640 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cca10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe68fe00640 1 -- 172.21.15.60:0/1410935421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 msgr2=0x55ae8cc9f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe68fe00640 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cc9f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe68fe00640 1 -- 172.21.15.60:0/1410935421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae8cc9fde0 con 0x55ae8cc90920 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe695a00640 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cc9f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe68fe00640 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cca1600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00c6a0 tx=0x7fe67c09a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c258ddd7a738bea2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe676a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe676a00640 1 -- 172.21.15.60:0/1410935421 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c09a830 con 0x55ae8cc90920 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1410935421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe688003e50 con 0x55ae8cc90920 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.604+0000 7fe676a00640 1 -- 172.21.15.60:0/1410935421 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe67c096e20 con 0x55ae8cc90920 2024-02-26T10:47:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe676a00640 1 -- 172.21.15.60:0/1410935421 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe67c097170 con 0x55ae8cc90920 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe69a9b5740 10 client.25916 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1410935421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae8ccfcd20 con 0x55ae8cc90920 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe676a00640 1 -- 172.21.15.60:0/1410935421 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe688003e50 con 0x55ae8cc90920 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe676a00640 1 -- 172.21.15.60:0/1410935421 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe67c085be0 con 0x55ae8cc90920 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe69a9b5740 10 client.25916 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe69a9b5740 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe69a9b5740 20 client.25916 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ae8ccfd650 0x55ae8ccffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe69a9b5740 4 client.25916 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1410935421 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ae8cbf8590 con 0x55ae8ccfd650 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.605+0000 7fe695000640 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ae8ccfd650 0x55ae8ccffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.606+0000 7fe695000640 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ae8ccfd650 0x55ae8ccffa70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28eea1eaf1d6bc37 server_cookie=42bea23dd821a956 in_seq=0 out_seq=0 2024-02-26T10:47:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.606+0000 7fe676a00640 10 client.25916 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe676a00640 1 -- 172.21.15.60:0/1410935421 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ae8cbf8590 con 0x55ae8ccfd650 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe676a00640 10 client.25916 handle_command_reply: tid=0 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 client.25916 shutdown 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1410935421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 msgr2=0x55ae8cca1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cca1600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00c6a0 tx=0x7fe67c09a6d0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1410935421 shutdown_connections 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ae8ccfd650 0x55ae8ccffa70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae8cc90920 0x55ae8cca1600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ae8cc95a90 0x55ae8cc9f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe676a00640 0 client.25916 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 --2- 172.21.15.60:0/1410935421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ae8cc8ff30 0x55ae8cca10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1410935421 >> 172.21.15.60:0/1410935421 conn(0x55ae8cc87090 msgr2=0x55ae8cc9a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1410935421 shutdown_connections 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 1 -- 172.21.15.60:0/1410935421 wait complete. 2024-02-26T10:47:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.607+0000 7fe69a9b5740 20 client.25916 trim_cache size 0 max 16384 2024-02-26T10:47:27.606 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:47:27.606 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-02-26T10:47:27.606 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-02-26T10:47:27.606 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 -- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6780a1d80 msgr2=0x7fe6780a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6780a1d80 0x7fe6780a4240 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe68802c630 tx=0x7fe688032590 comp rx=0 tx=0).stop 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 -- 172.21.15.60:0/3970691931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe690101350 msgr2=0x7fe690197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe690101350 0x7fe690197a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00e9f0 tx=0x7fe67c00edf0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 -- 172.21.15.60:0/3970691931 shutdown_connections 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6780a1d80 0x7fe6780a4240 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe690101350 0x7fe690197a80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe690101d10 0x7fe69019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 --2- 172.21.15.60:0/3970691931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe690100160 0x7fe690197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 -- 172.21.15.60:0/3970691931 >> 172.21.15.60:0/3970691931 conn(0x7fe6900fb6f0 msgr2=0x7fe6900683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 -- 172.21.15.60:0/3970691931 shutdown_connections 2024-02-26T10:47:27.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.608+0000 7fe697a00640 1 -- 172.21.15.60:0/3970691931 wait complete. 2024-02-26T10:47:27.618 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T10:47:27.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:47:27.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.841+0000 7f4d30a00640 1 -- 172.21.15.60:0/1630471064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c1034a0 msgr2=0x7f4d2c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.841+0000 7f4d30a00640 1 --2- 172.21.15.60:0/1630471064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c1034a0 0x7f4d2c1038c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009900 tx=0x7f4d1c02bc40 comp rx=0 tx=0).stop 2024-02-26T10:47:27.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.841+0000 7f4d30a00640 1 -- 172.21.15.60:0/1630471064 shutdown_connections 2024-02-26T10:47:27.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.841+0000 7f4d30a00640 1 --2- 172.21.15.60:0/1630471064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d2c105050 0x7f4d2c10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.841+0000 7f4d30a00640 1 --2- 172.21.15.60:0/1630471064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d2c104690 0x7f4d2c104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.841+0000 7f4d30a00640 1 --2- 172.21.15.60:0/1630471064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c1034a0 0x7f4d2c1038c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.841+0000 7f4d30a00640 1 -- 172.21.15.60:0/1630471064 >> 172.21.15.60:0/1630471064 conn(0x7f4d2c0fea50 msgr2=0x7f4d2c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.841+0000 7f4d30a00640 1 -- 172.21.15.60:0/1630471064 shutdown_connections 2024-02-26T10:47:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.841+0000 7f4d30a00640 1 -- 172.21.15.60:0/1630471064 wait complete. 2024-02-26T10:47:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.842+0000 7f4d30a00640 1 Processor -- start 2024-02-26T10:47:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d30a00640 1 -- start start 2024-02-26T10:47:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d30a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d2c1034a0 0x7f4d2c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d30a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c104690 0x7f4d2c199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d30a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d2c105050 0x7f4d2c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d30a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c10f100 con 0x7f4d2c1034a0 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d30a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c10ef80 con 0x7f4d2c104690 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d30a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c10f280 con 0x7f4d2c105050 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d29e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d2c1034a0 0x7f4d2c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d29400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c104690 0x7f4d2c199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d29e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d2c1034a0 0x7f4d2c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56498/0 (socket says 172.21.15.60:56498) 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d29e00640 1 -- 172.21.15.60:0/2335056522 learned_addr learned my addr 172.21.15.60:0/2335056522 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.843+0000 7f4d2a800640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d2c105050 0x7f4d2c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.844+0000 7f4d29e00640 1 -- 172.21.15.60:0/2335056522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d2c105050 msgr2=0x7f4d2c1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.844+0000 7f4d29e00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d2c105050 0x7f4d2c1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.844+0000 7f4d29e00640 1 -- 172.21.15.60:0/2335056522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c104690 msgr2=0x7f4d2c199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:27.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.844+0000 7f4d29e00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c104690 0x7f4d2c199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:27.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.844+0000 7f4d29e00640 1 -- 172.21.15.60:0/2335056522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c1a13c0 con 0x7f4d2c1034a0 2024-02-26T10:47:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.844+0000 7f4d29400640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c104690 0x7f4d2c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:47:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.844+0000 7f4d29e00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d2c1034a0 0x7f4d2c1997a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005ab0 tx=0x7f4d1c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.845+0000 7f4d22a00640 1 -- 172.21.15.60:0/2335056522 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c041070 con 0x7f4d2c1034a0 2024-02-26T10:47:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.845+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c1a1650 con 0x7f4d2c1034a0 2024-02-26T10:47:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.845+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c1a1b90 con 0x7f4d2c1034a0 2024-02-26T10:47:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.845+0000 7f4d22a00640 1 -- 172.21.15.60:0/2335056522 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d1c0054e0 con 0x7f4d2c1034a0 2024-02-26T10:47:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.845+0000 7f4d22a00640 1 -- 172.21.15.60:0/2335056522 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c034440 con 0x7f4d2c1034a0 2024-02-26T10:47:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.846+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0c0052c0 con 0x7f4d2c1034a0 2024-02-26T10:47:27.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.849+0000 7f4d22a00640 1 -- 172.21.15.60:0/2335056522 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4d1c044070 con 0x7f4d2c1034a0 2024-02-26T10:47:27.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.851+0000 7f4d22a00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4d100a1e00 0x7f4d100a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:27.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.851+0000 7f4d22a00640 1 -- 172.21.15.60:0/2335056522 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f4d1c0efd40 con 0x7f4d2c1034a0 2024-02-26T10:47:27.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.851+0000 7f4d29400640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4d100a1e00 0x7f4d100a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:27.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.852+0000 7f4d29400640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4d100a1e00 0x7f4d100a42c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005d00 tx=0x7f4d18005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:27.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:27.857+0000 7f4d22a00640 1 -- 172.21.15.60:0/2335056522 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d1c0bcd60 con 0x7f4d2c1034a0 2024-02-26T10:47:28.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.117+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d0c0055b0 con 0x7f4d2c1034a0 2024-02-26T10:47:28.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.119+0000 7f4d22a00640 1 -- 172.21.15.60:0/2335056522 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f4d1c0bc660 con 0x7f4d2c1034a0 2024-02-26T10:47:28.113 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:47:28.113 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:47:16.131092+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:47:28.113 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 25 2024-02-26T10:47:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4d100a1e00 msgr2=0x7f4d100a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4d100a1e00 0x7f4d100a42c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005d00 tx=0x7f4d18005c70 comp rx=0 tx=0).stop 2024-02-26T10:47:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d2c1034a0 msgr2=0x7f4d2c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d2c1034a0 0x7f4d2c1997a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005ab0 tx=0x7f4d1c005dc0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 shutdown_connections 2024-02-26T10:47:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f4d100a1e00 0x7f4d100a42c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c104690 0x7f4d2c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d2c105050 0x7f4d2c1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 --2- 172.21.15.60:0/2335056522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d2c1034a0 0x7f4d2c1997a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 >> 172.21.15.60:0/2335056522 conn(0x7f4d2c0fea50 msgr2=0x7f4d2c107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 shutdown_connections 2024-02-26T10:47:28.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.122+0000 7f4d30a00640 1 -- 172.21.15.60:0/2335056522 wait complete. 2024-02-26T10:47:28.134 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:47:28.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T10:47:28.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.366+0000 7fe6dca00640 1 -- 172.21.15.60:0/23897791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8075c20 msgr2=0x7fe6d8076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.366+0000 7fe6dca00640 1 --2- 172.21.15.60:0/23897791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8075c20 0x7fe6d8076040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c802bc50 comp rx=0 tx=0).stop 2024-02-26T10:47:28.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.366+0000 7fe6dca00640 1 -- 172.21.15.60:0/23897791 shutdown_connections 2024-02-26T10:47:28.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.366+0000 7fe6dca00640 1 --2- 172.21.15.60:0/23897791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8076e60 0x7fe6d80772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.366+0000 7fe6dca00640 1 --2- 172.21.15.60:0/23897791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6d8077800 0x7fe6d80780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.366+0000 7fe6dca00640 1 --2- 172.21.15.60:0/23897791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8075c20 0x7fe6d8076040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.366+0000 7fe6dca00640 1 -- 172.21.15.60:0/23897791 >> 172.21.15.60:0/23897791 conn(0x7fe6d80fe910 msgr2=0x7fe6d8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.367+0000 7fe6dca00640 1 -- 172.21.15.60:0/23897791 shutdown_connections 2024-02-26T10:47:28.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.367+0000 7fe6dca00640 1 -- 172.21.15.60:0/23897791 wait complete. 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.367+0000 7fe6dca00640 1 Processor -- start 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6dca00640 1 -- start start 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6dca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8075c20 0x7fe6d819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6dca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6d8076e60 0x7fe6d819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6dca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8077800 0x7fe6d81a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6dca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81133f0 con 0x7fe6d8077800 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6dca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d8113270 con 0x7fe6d8075c20 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6dca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d8113570 con 0x7fe6d8076e60 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6d7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8077800 0x7fe6d81a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6d7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8077800 0x7fe6d81a5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56520/0 (socket says 172.21.15.60:56520) 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6d7e00640 1 -- 172.21.15.60:0/1100051389 learned_addr learned my addr 172.21.15.60:0/1100051389 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6d6a00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6d8076e60 0x7fe6d819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.368+0000 7fe6d7400640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8075c20 0x7fe6d819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6d7e00640 1 -- 172.21.15.60:0/1100051389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6d8076e60 msgr2=0x7fe6d819e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6d7e00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6d8076e60 0x7fe6d819e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6d7e00640 1 -- 172.21.15.60:0/1100051389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8075c20 msgr2=0x7fe6d819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6d7e00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8075c20 0x7fe6d819db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6d7e00640 1 -- 172.21.15.60:0/1100051389 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d81a56b0 con 0x7fe6d8077800 2024-02-26T10:47:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6d7e00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8077800 0x7fe6d81a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000efd0 tx=0x7fe6d000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6cfe00640 1 -- 172.21.15.60:0/1100051389 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0003b70 con 0x7fe6d8077800 2024-02-26T10:47:28.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6cfe00640 1 -- 172.21.15.60:0/1100051389 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0003d10 con 0x7fe6d8077800 2024-02-26T10:47:28.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6cfe00640 1 -- 172.21.15.60:0/1100051389 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00109f0 con 0x7fe6d8077800 2024-02-26T10:47:28.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.369+0000 7fe6dca00640 1 -- 172.21.15.60:0/1100051389 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d81a59a0 con 0x7fe6d8077800 2024-02-26T10:47:28.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.370+0000 7fe6dca00640 1 -- 172.21.15.60:0/1100051389 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d81a5ea0 con 0x7fe6d8077800 2024-02-26T10:47:28.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.373+0000 7fe6cfe00640 1 -- 172.21.15.60:0/1100051389 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe6d0004020 con 0x7fe6d8077800 2024-02-26T10:47:28.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.373+0000 7fe6e0e45740 1 Processor -- start 2024-02-26T10:47:28.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.373+0000 7fe6e0e45740 1 -- start start 2024-02-26T10:47:28.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 0x558155368260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x55815536d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x55815536fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581552ccd30 con 0x558155367e40 2024-02-26T10:47:28.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558155275cc0 con 0x55815536d9a0 2024-02-26T10:47:28.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581552c8dd0 con 0x558155368830 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x55815536d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6d6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 0x558155368260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x55815536d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:45986/0 (socket says 172.21.15.60:45986) 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6d6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 0x558155368260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56522/0 (socket says 172.21.15.60:56522) 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6d7400640 1 -- 172.21.15.60:0/725135280 learned_addr learned my addr 172.21.15.60:0/725135280 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.374+0000 7fe6d6a00640 1 --2- 172.21.15.60:0/725135280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x55815536fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6d7400640 1 -- 172.21.15.60:0/725135280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 msgr2=0x55815536fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6d7400640 1 --2- 172.21.15.60:0/725135280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x55815536fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6d7400640 1 -- 172.21.15.60:0/725135280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 msgr2=0x558155368260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6d7400640 1 --2- 172.21.15.60:0/725135280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 0x558155368260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6d7400640 1 -- 172.21.15.60:0/725135280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558154fe13f0 con 0x558155368830 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6cfe00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6bc0a1e00 0x7fe6bc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6d7400640 1 --2- 172.21.15.60:0/725135280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x55815536d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6c802c900 tx=0x7fe6c8009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e158268f46e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6cfe00640 1 -- 172.21.15.60:0/1100051389 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe6d00ce570 con 0x7fe6d8077800 2024-02-26T10:47:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6d6a00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6bc0a1e00 0x7fe6bc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6c3e00640 1 -- 172.21.15.60:0/725135280 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8042070 con 0x558155368830 2024-02-26T10:47:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6c3e00640 1 -- 172.21.15.60:0/725135280 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c802cba0 con 0x558155368830 2024-02-26T10:47:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.375+0000 7fe6c3e00640 1 -- 172.21.15.60:0/725135280 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8005130 con 0x558155368830 2024-02-26T10:47:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6e0e45740 1 -- 172.21.15.60:0/725135280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 msgr2=0x55815536d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6d6a00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6bc0a1e00 0x7fe6bc0a42c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x558155369090 tx=0x7fe6c4015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/725135280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x55815536d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6c802c900 tx=0x7fe6c8009560 comp rx=0 tx=0).stop 2024-02-26T10:47:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6e0e45740 1 -- 172.21.15.60:0/725135280 shutdown_connections 2024-02-26T10:47:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/725135280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x55815536fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/725135280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x55815536d330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/725135280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 0x558155368260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6e0e45740 1 -- 172.21.15.60:0/725135280 >> 172.21.15.60:0/725135280 conn(0x5581553633b0 msgr2=0x5581553637b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6e0e45740 1 -- 172.21.15.60:0/725135280 shutdown_connections 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.376+0000 7fe6e0e45740 1 -- 172.21.15.60:0/725135280 wait complete. 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.377+0000 7fe6e0e45740 1 -- start start 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.377+0000 7fe6e0e45740 1 Processor -- start 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.377+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558155367e40 0x5581553722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.377+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155372830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.377+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55815536d9a0 0x558155379870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.377+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581553708f0 con 0x558155368830 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.377+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55815526b1f0 con 0x558155367e40 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.377+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x558155370a70 con 0x55815536d9a0 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6e0e45740 10 client.0 _ll_register_callbacks cb 0x558154fe0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155372830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558155367e40 0x5581553722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55815536d9a0 0x558155379870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155372830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56534/0 (socket says 172.21.15.60:56534) 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d6a00640 1 -- 172.21.15.60:0/2268851791 learned_addr learned my addr 172.21.15.60:0/2268851791 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d6a00640 1 -- 172.21.15.60:0/2268851791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55815536d9a0 msgr2=0x558155379870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d6a00640 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55815536d9a0 0x558155379870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d6a00640 1 -- 172.21.15.60:0/2268851791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558155367e40 msgr2=0x5581553722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d6a00640 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558155367e40 0x5581553722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d6a00640 1 -- 172.21.15.60:0/2268851791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581550ab7d0 con 0x558155368830 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d7e00640 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55815536d9a0 0x558155379870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.378+0000 7fe6d7400640 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558155367e40 0x5581553722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.379+0000 7fe6d6a00640 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155372830 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40045f0 tx=0x7fe6c4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd6922c18d4cda3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.379+0000 7fe6c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.379+0000 7fe6c0c00640 1 -- 172.21.15.60:0/2268851791 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400f800 con 0x558155368830 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.379+0000 7fe6c0c00640 1 -- 172.21.15.60:0/2268851791 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe6c400f9a0 con 0x558155368830 2024-02-26T10:47:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.379+0000 7fe6e0e45740 1 -- 172.21.15.60:0/2268851791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6c802cba0 con 0x558155368830 2024-02-26T10:47:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.379+0000 7fe6c0c00640 1 -- 172.21.15.60:0/2268851791 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6c400fba0 con 0x558155368830 2024-02-26T10:47:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.379+0000 7fe6e0e45740 10 client.15813 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.379+0000 7fe6e0e45740 1 -- 172.21.15.60:0/2268851791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55815537cc50 con 0x558155368830 2024-02-26T10:47:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.380+0000 7fe6c0c00640 1 -- 172.21.15.60:0/2268851791 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c802cba0 con 0x558155368830 2024-02-26T10:47:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.380+0000 7fe6c0c00640 1 -- 172.21.15.60:0/2268851791 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe6c4021420 con 0x558155368830 2024-02-26T10:47:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.380+0000 7fe6e0e45740 10 client.15813 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.380+0000 7fe6e0e45740 10 client.15813 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.380+0000 7fe6e0e45740 20 client.15813 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.380+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55815537e2e0 0x558155380760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.380+0000 7fe6e0e45740 4 client.15813 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.380+0000 7fe6e0e45740 1 -- 172.21.15.60:0/2268851791 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581552da270 con 0x55815537e2e0 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.380+0000 7fe6d7400640 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55815537e2e0 0x558155380760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.381+0000 7fe6d7400640 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55815537e2e0 0x558155380760 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3559a8994d3202e5 server_cookie=d8091ec39d645d19 in_seq=0 out_seq=0 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.381+0000 7fe6c0c00640 10 client.15813 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6c0c00640 1 -- 172.21.15.60:0/2268851791 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581552da270 con 0x55815537e2e0 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6c0c00640 10 client.15813 handle_command_reply: tid=0 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 client.15813 shutdown 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 -- 172.21.15.60:0/2268851791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 msgr2=0x558155372830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155372830 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40045f0 tx=0x7fe6c4005f90 comp rx=0 tx=0).stop 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 -- 172.21.15.60:0/2268851791 shutdown_connections 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55815537e2e0 0x558155380760 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55815536d9a0 0x558155379870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155372830 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/2268851791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558155367e40 0x5581553722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 -- 172.21.15.60:0/2268851791 >> 172.21.15.60:0/2268851791 conn(0x5581553633b0 msgr2=0x5581553626d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6c0c00640 0 client.15813 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:47:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 -- 172.21.15.60:0/2268851791 shutdown_connections 2024-02-26T10:47:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 1 -- 172.21.15.60:0/2268851791 wait complete. 2024-02-26T10:47:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.382+0000 7fe6e0e45740 20 client.15813 trim_cache size 0 max 16384 2024-02-26T10:47:28.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.396+0000 7fe6e0e45740 1 Processor -- start 2024-02-26T10:47:28.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.396+0000 7fe6e0e45740 1 -- start start 2024-02-26T10:47:28.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.396+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155367e40 0x558155369cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.396+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155379830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.396+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x55815536a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.396+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81b2600 con 0x558155368830 2024-02-26T10:47:28.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.396+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81b01a0 con 0x55815536d9a0 2024-02-26T10:47:28.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.396+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81ae240 con 0x558155367e40 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155367e40 0x558155369cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155367e40 0x558155369cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46010/0 (socket says 172.21.15.60:46010) 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x55815536a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7400640 1 -- 172.21.15.60:0/86162509 learned_addr learned my addr 172.21.15.60:0/86162509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d6a00640 1 --2- 172.21.15.60:0/86162509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155379830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7400640 1 -- 172.21.15.60:0/86162509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 msgr2=0x55815536a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7400640 1 --2- 172.21.15.60:0/86162509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x55815536a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7400640 1 -- 172.21.15.60:0/86162509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 msgr2=0x558155379830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7400640 1 --2- 172.21.15.60:0/86162509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155379830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7400640 1 -- 172.21.15.60:0/86162509 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55815536cfc0 con 0x558155367e40 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.397+0000 7fe6d7400640 1 --2- 172.21.15.60:0/86162509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155367e40 0x558155369cf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c8005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acecc1f4a3b2ecbe server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.398+0000 7fe6c3400640 1 -- 172.21.15.60:0/86162509 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8042040 con 0x558155367e40 2024-02-26T10:47:28.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.398+0000 7fe6c3400640 1 -- 172.21.15.60:0/86162509 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8004ed0 con 0x558155367e40 2024-02-26T10:47:28.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.398+0000 7fe6c3400640 1 -- 172.21.15.60:0/86162509 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80522e0 con 0x558155367e40 2024-02-26T10:47:28.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.398+0000 7fe6e0e45740 1 -- 172.21.15.60:0/86162509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155367e40 msgr2=0x558155369cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.398+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/86162509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155367e40 0x558155369cf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c8005e20 comp rx=0 tx=0).stop 2024-02-26T10:47:28.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.398+0000 7fe6e0e45740 1 -- 172.21.15.60:0/86162509 shutdown_connections 2024-02-26T10:47:28.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.398+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/86162509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x55815536a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.398+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/86162509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155368830 0x558155379830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.398+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/86162509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155367e40 0x558155369cf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.399+0000 7fe6e0e45740 1 -- 172.21.15.60:0/86162509 >> 172.21.15.60:0/86162509 conn(0x5581553633b0 msgr2=0x558155363790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.399+0000 7fe6e0e45740 1 -- 172.21.15.60:0/86162509 shutdown_connections 2024-02-26T10:47:28.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.399+0000 7fe6e0e45740 1 -- 172.21.15.60:0/86162509 wait complete. 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.399+0000 7fe6e0e45740 1 -- start start 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.399+0000 7fe6e0e45740 1 Processor -- start 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 0x558155379910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x558155377f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6e0e45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x558155378550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81ae840 con 0x558155367e40 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81afb40 con 0x55815536d9a0 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6e0e45740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81aee80 con 0x558155368830 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6e0e45740 10 client.0 _ll_register_callbacks cb 0x7fe6d00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 0x558155379910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x558155377f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x558155378550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x558155377f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46014/0 (socket says 172.21.15.60:46014) 2024-02-26T10:47:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d6a00640 1 -- 172.21.15.60:0/3475642440 learned_addr learned my addr 172.21.15.60:0/3475642440 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d7e00640 1 -- 172.21.15.60:0/3475642440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 msgr2=0x558155377f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d7e00640 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x558155377f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d7e00640 1 -- 172.21.15.60:0/3475642440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 msgr2=0x558155379910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d7e00640 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 0x558155379910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.400+0000 7fe6d7e00640 1 -- 172.21.15.60:0/3475642440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581553d4830 con 0x55815536d9a0 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6d7e00640 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x558155378550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000cba0 tx=0x7fe6d00971f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8e72b0dbddd66e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6c0c00640 1 -- 172.21.15.60:0/3475642440 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d009efe0 con 0x55815536d9a0 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6e0e45740 1 -- 172.21.15.60:0/3475642440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6c8004ed0 con 0x55815536d9a0 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6c0c00640 1 -- 172.21.15.60:0/3475642440 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe6d0004620 con 0x55815536d9a0 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6c0c00640 1 -- 172.21.15.60:0/3475642440 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d0081050 con 0x55815536d9a0 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6e0e45740 10 client.25940 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6e0e45740 1 -- 172.21.15.60:0/3475642440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5581553d5aa0 con 0x55815536d9a0 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6c0c00640 1 -- 172.21.15.60:0/3475642440 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8004ed0 con 0x55815536d9a0 2024-02-26T10:47:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6c0c00640 1 -- 172.21.15.60:0/3475642440 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe6d0084080 con 0x55815536d9a0 2024-02-26T10:47:28.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6e0e45740 10 client.25940 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:28.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6e0e45740 10 client.25940 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-26T10:47:28.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.401+0000 7fe6e0e45740 20 client.25940 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:28.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.402+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x5581553d63d0 0x5581553d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.402+0000 7fe6e0e45740 4 client.25940 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:47:28.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.402+0000 7fe6e0e45740 1 -- 172.21.15.60:0/3475642440 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581552d45c0 con 0x5581553d63d0 2024-02-26T10:47:28.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.402+0000 7fe6d7400640 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x5581553d63d0 0x5581553d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.402+0000 7fe6d7400640 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x5581553d63d0 0x5581553d87f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e91eb3c018e145a server_cookie=54fa25ce11a8b54f in_seq=0 out_seq=0 2024-02-26T10:47:28.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.402+0000 7fe6c0c00640 10 client.25940 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:47:28.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.402+0000 7fe6c0c00640 1 -- 172.21.15.60:0/3475642440 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581552d45c0 con 0x5581553d63d0 2024-02-26T10:47:28.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.402+0000 7fe6c0c00640 10 client.25940 handle_command_reply: tid=0 2024-02-26T10:47:28.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 client.25940 shutdown 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 -- 172.21.15.60:0/3475642440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 msgr2=0x558155378550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x558155378550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000cba0 tx=0x7fe6d00971f0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 -- 172.21.15.60:0/3475642440 shutdown_connections 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x5581553d63d0 0x5581553d87f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55815536d9a0 0x558155378550 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558155368830 0x558155377f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 --2- 172.21.15.60:0/3475642440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558155367e40 0x558155379910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6c0c00640 0 client.25940 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 -- 172.21.15.60:0/3475642440 >> 172.21.15.60:0/3475642440 conn(0x5581553633b0 msgr2=0x5581553722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 -- 172.21.15.60:0/3475642440 shutdown_connections 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 1 -- 172.21.15.60:0/3475642440 wait complete. 2024-02-26T10:47:28.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.403+0000 7fe6e0e45740 20 client.25940 trim_cache size 0 max 16384 2024-02-26T10:47:28.402 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:47:28.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 -- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6bc0a1e00 msgr2=0x7fe6bc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6bc0a1e00 0x7fe6bc0a42c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x558155369090 tx=0x7fe6c4015040 comp rx=0 tx=0).stop 2024-02-26T10:47:28.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 -- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8077800 msgr2=0x7fe6d81a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8077800 0x7fe6d81a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000efd0 tx=0x7fe6d000c6a0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 -- 172.21.15.60:0/1100051389 shutdown_connections 2024-02-26T10:47:28.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe6bc0a1e00 0x7fe6bc0a42c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d8077800 0x7fe6d81a5040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6d8076e60 0x7fe6d819e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 --2- 172.21.15.60:0/1100051389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8075c20 0x7fe6d819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 -- 172.21.15.60:0/1100051389 >> 172.21.15.60:0/1100051389 conn(0x7fe6d80fe910 msgr2=0x7fe6d8100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 -- 172.21.15.60:0/1100051389 shutdown_connections 2024-02-26T10:47:28.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.404+0000 7fe6dca00640 1 -- 172.21.15.60:0/1100051389 wait complete. 2024-02-26T10:47:28.415 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:47:28.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T10:47:28.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.635+0000 7fa514c00640 1 -- 172.21.15.60:0/2704736569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa510109000 msgr2=0x7fa51010b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.635+0000 7fa514c00640 1 --2- 172.21.15.60:0/2704736569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa510109000 0x7fa51010b490 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa508009ca0 tx=0x7fa50802c100 comp rx=0 tx=0).stop 2024-02-26T10:47:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.635+0000 7fa514c00640 1 -- 172.21.15.60:0/2704736569 shutdown_connections 2024-02-26T10:47:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.635+0000 7fa514c00640 1 --2- 172.21.15.60:0/2704736569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa510109000 0x7fa51010b490 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.635+0000 7fa514c00640 1 --2- 172.21.15.60:0/2704736569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa510073cc0 0x7fa510074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.635+0000 7fa514c00640 1 --2- 172.21.15.60:0/2704736569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5100752a0 0x7fa5100736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.635+0000 7fa514c00640 1 -- 172.21.15.60:0/2704736569 >> 172.21.15.60:0/2704736569 conn(0x7fa5100fc6d0 msgr2=0x7fa5100feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.635+0000 7fa514c00640 1 -- 172.21.15.60:0/2704736569 shutdown_connections 2024-02-26T10:47:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.635+0000 7fa514c00640 1 -- 172.21.15.60:0/2704736569 wait complete. 2024-02-26T10:47:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.636+0000 7fa514c00640 1 Processor -- start 2024-02-26T10:47:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa514c00640 1 -- start start 2024-02-26T10:47:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa514c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa510073cc0 0x7fa5101995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa514c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5100752a0 0x7fa510199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa514c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa510109000 0x7fa5101a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa514c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51010ef80 con 0x7fa510109000 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa514c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51010ee00 con 0x7fa510073cc0 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa514c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51010f100 con 0x7fa5100752a0 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa50f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa510073cc0 0x7fa5101995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa50f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa510073cc0 0x7fa5101995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53520/0 (socket says 172.21.15.60:53520) 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa50f400640 1 -- 172.21.15.60:0/419827266 learned_addr learned my addr 172.21.15.60:0/419827266 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa507e00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5100752a0 0x7fa510199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.637+0000 7fa50fe00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa510109000 0x7fa5101a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa50f400640 1 -- 172.21.15.60:0/419827266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5100752a0 msgr2=0x7fa510199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa50f400640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5100752a0 0x7fa510199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa50f400640 1 -- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa510109000 msgr2=0x7fa5101a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa50f400640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa510109000 0x7fa5101a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa50f400640 1 -- 172.21.15.60:0/419827266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5101a11d0 con 0x7fa510073cc0 2024-02-26T10:47:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa50f400640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa510073cc0 0x7fa5101995b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa50000d950 tx=0x7fa50000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa50cc00640 1 -- 172.21.15.60:0/419827266 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500017030 con 0x7fa510073cc0 2024-02-26T10:47:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa50cc00640 1 -- 172.21.15.60:0/419827266 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa500004020 con 0x7fa510073cc0 2024-02-26T10:47:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa514c00640 1 -- 172.21.15.60:0/419827266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5101a14c0 con 0x7fa510073cc0 2024-02-26T10:47:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa50cc00640 1 -- 172.21.15.60:0/419827266 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000047a0 con 0x7fa510073cc0 2024-02-26T10:47:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.638+0000 7fa514c00640 1 -- 172.21.15.60:0/419827266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5101a1a10 con 0x7fa510073cc0 2024-02-26T10:47:28.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.641+0000 7fa519026740 1 Processor -- start 2024-02-26T10:47:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.642+0000 7fa50cc00640 1 -- 172.21.15.60:0/419827266 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa5000041d0 con 0x7fa510073cc0 2024-02-26T10:47:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.642+0000 7fa519026740 1 -- start start 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.642+0000 7fa519026740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa34e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.642+0000 7fa519026740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa353330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.642+0000 7fa519026740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa355e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.642+0000 7fa519026740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fa2b2d30 con 0x5597fa34e830 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.642+0000 7fa519026740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fa25bcc0 con 0x5597fa3539a0 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.642+0000 7fa519026740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fa2aedd0 con 0x5597fa34de40 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.642+0000 7fa50f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa353330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa50f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa353330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56576/0 (socket says 172.21.15.60:56576) 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa50f400640 1 -- 172.21.15.60:0/2534258261 learned_addr learned my addr 172.21.15.60:0/2534258261 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa50fe00640 1 --2- 172.21.15.60:0/2534258261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa34e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa507e00640 1 --2- 172.21.15.60:0/2534258261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa355e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa50f400640 1 -- 172.21.15.60:0/2534258261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 msgr2=0x5597fa34e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa50f400640 1 --2- 172.21.15.60:0/2534258261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa34e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa50f400640 1 -- 172.21.15.60:0/2534258261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 msgr2=0x5597fa355e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa50f400640 1 --2- 172.21.15.60:0/2534258261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa355e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa50f400640 1 -- 172.21.15.60:0/2534258261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597f9fc73f0 con 0x5597fa34e830 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.643+0000 7fa50f400640 1 --2- 172.21.15.60:0/2534258261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa353330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa5000d1ae0 tx=0x7fa5000d61c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b66b63cdb1e90 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.644+0000 7fa50cc00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa4f40a1d80 0x7fa4f40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.644+0000 7fa4ffe00640 1 -- 172.21.15.60:0/2534258261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500096700 con 0x5597fa34e830 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.644+0000 7fa4ffe00640 1 -- 172.21.15.60:0/2534258261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5000bb8b0 con 0x5597fa34e830 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.644+0000 7fa4ffe00640 1 -- 172.21.15.60:0/2534258261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500099be0 con 0x5597fa34e830 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.644+0000 7fa50cc00640 1 -- 172.21.15.60:0/419827266 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa5000cdb40 con 0x7fa510073cc0 2024-02-26T10:47:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.644+0000 7fa50fe00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa4f40a1d80 0x7fa4f40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 -- 172.21.15.60:0/2534258261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 msgr2=0x5597fa353330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 --2- 172.21.15.60:0/2534258261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa353330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa5000d1ae0 tx=0x7fa5000d61c0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 -- 172.21.15.60:0/2534258261 shutdown_connections 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 --2- 172.21.15.60:0/2534258261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa355e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 --2- 172.21.15.60:0/2534258261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa353330 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 --2- 172.21.15.60:0/2534258261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa34e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 -- 172.21.15.60:0/2534258261 >> 172.21.15.60:0/2534258261 conn(0x5597fa3493b0 msgr2=0x5597fa3497b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa50fe00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa4f40a1d80 0x7fa4f40a4240 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa508005e30 tx=0x7fa508032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 -- 172.21.15.60:0/2534258261 shutdown_connections 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 -- 172.21.15.60:0/2534258261 wait complete. 2024-02-26T10:47:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.645+0000 7fa519026740 1 -- start start 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa519026740 1 Processor -- start 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa519026740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34de40 0x5597fa358320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa519026740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34e830 0x5597fa358860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa519026740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa519026740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fa356950 con 0x5597fa34de40 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa519026740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fa2511f0 con 0x5597fa3539a0 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa519026740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597fa356ad0 con 0x5597fa34e830 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa519026740 10 client.0 _ll_register_callbacks cb 0x5597f9fc6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa50f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34de40 0x5597fa358320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.646+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53538/0 (socket says 172.21.15.60:53538) 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa50fe00640 1 -- 172.21.15.60:0/1061393314 learned_addr learned my addr 172.21.15.60:0/1061393314 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa507e00640 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34e830 0x5597fa358860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa50f400640 1 -- 172.21.15.60:0/1061393314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34e830 msgr2=0x5597fa358860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa50f400640 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34e830 0x5597fa358860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa50f400640 1 -- 172.21.15.60:0/1061393314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 msgr2=0x5597fa35f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa50f400640 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa50f400640 1 -- 172.21.15.60:0/1061393314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597fa0917d0 con 0x5597fa34de40 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa50f400640 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34de40 0x5597fa358320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa5000d1ae0 tx=0x7fa5000cddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed9afb86fdbfbf8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa4fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.647+0000 7fa4fcc00640 1 -- 172.21.15.60:0/1061393314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500099dc0 con 0x5597fa34de40 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa4fcc00640 1 -- 172.21.15.60:0/1061393314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa500098500 con 0x5597fa34de40 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa519026740 1 -- 172.21.15.60:0/1061393314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5000bb8b0 con 0x5597fa34de40 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa4fcc00640 1 -- 172.21.15.60:0/1061393314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa500099fa0 con 0x5597fa34de40 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa519026740 10 client.15831 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa519026740 1 -- 172.21.15.60:0/1061393314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5597fa362c10 con 0x5597fa34de40 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa4fcc00640 1 -- 172.21.15.60:0/1061393314 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5000bb8b0 con 0x5597fa34de40 2024-02-26T10:47:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa4fcc00640 1 -- 172.21.15.60:0/1061393314 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa500096ad0 con 0x5597fa34de40 2024-02-26T10:47:28.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa519026740 10 client.15831 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:28.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa519026740 10 client.15831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T10:47:28.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa519026740 20 client.15831 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:28.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa519026740 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x5597fa363f60 0x5597fa366350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa519026740 4 client.15831 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:28.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.648+0000 7fa519026740 1 -- 172.21.15.60:0/1061393314 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597fa2c0270 con 0x5597fa363f60 2024-02-26T10:47:28.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.649+0000 7fa507e00640 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x5597fa363f60 0x5597fa366350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.649+0000 7fa507e00640 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x5597fa363f60 0x5597fa366350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4d2f5f3b8d9bb34 server_cookie=ad2a75e8c89ecd85 in_seq=0 out_seq=0 2024-02-26T10:47:28.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.649+0000 7fa4fcc00640 10 client.15831 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:47:28.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.650+0000 7fa4fcc00640 1 -- 172.21.15.60:0/1061393314 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5597fa2c0270 con 0x5597fa363f60 2024-02-26T10:47:28.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.650+0000 7fa4fcc00640 10 client.15831 handle_command_reply: tid=0 2024-02-26T10:47:28.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.650+0000 7fa519026740 1 client.15831 shutdown 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 -- 172.21.15.60:0/1061393314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34de40 msgr2=0x5597fa358320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34de40 0x5597fa358320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa5000d1ae0 tx=0x7fa5000cddc0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 -- 172.21.15.60:0/1061393314 shutdown_connections 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x5597fa363f60 0x5597fa366350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34e830 0x5597fa358860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 --2- 172.21.15.60:0/1061393314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34de40 0x5597fa358320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa4fcc00640 0 client.15831 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 -- 172.21.15.60:0/1061393314 >> 172.21.15.60:0/1061393314 conn(0x5597fa3493b0 msgr2=0x5597fa3486d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 -- 172.21.15.60:0/1061393314 shutdown_connections 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 1 -- 172.21.15.60:0/1061393314 wait complete. 2024-02-26T10:47:28.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.651+0000 7fa519026740 20 client.15831 trim_cache size 0 max 16384 2024-02-26T10:47:28.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.664+0000 7fa519026740 1 Processor -- start 2024-02-26T10:47:28.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.664+0000 7fa519026740 1 -- start start 2024-02-26T10:47:28.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.664+0000 7fa519026740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa357f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.664+0000 7fa519026740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa34e830 0x5597fa35f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.664+0000 7fa519026740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa3539a0 0x5597fa358470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.664+0000 7fa519026740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101ae270 con 0x5597fa3539a0 2024-02-26T10:47:28.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.664+0000 7fa519026740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101abe10 con 0x5597fa34e830 2024-02-26T10:47:28.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.664+0000 7fa519026740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101a9eb0 con 0x5597fa34de40 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa3539a0 0x5597fa358470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa3539a0 0x5597fa358470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56580/0 (socket says 172.21.15.60:56580) 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50fe00640 1 -- 172.21.15.60:0/1051007336 learned_addr learned my addr 172.21.15.60:0/1051007336 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa507e00640 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa34e830 0x5597fa35f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50f400640 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa357f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50fe00640 1 -- 172.21.15.60:0/1051007336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 msgr2=0x5597fa357f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50fe00640 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa357f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50fe00640 1 -- 172.21.15.60:0/1051007336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa34e830 msgr2=0x5597fa35f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50fe00640 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa34e830 0x5597fa35f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50fe00640 1 -- 172.21.15.60:0/1051007336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597fa349e20 con 0x5597fa3539a0 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50f400640 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa357f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.665+0000 7fa50fe00640 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa3539a0 0x5597fa358470 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa5080324c0 tx=0x7fa508041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19aaa9664004c71c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa4ff400640 1 -- 172.21.15.60:0/1051007336 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508039040 con 0x5597fa3539a0 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa4ff400640 1 -- 172.21.15.60:0/1051007336 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa508003b70 con 0x5597fa3539a0 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa4ff400640 1 -- 172.21.15.60:0/1051007336 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50804e7e0 con 0x5597fa3539a0 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa519026740 1 -- 172.21.15.60:0/1051007336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa3539a0 msgr2=0x5597fa358470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa519026740 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa3539a0 0x5597fa358470 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa5080324c0 tx=0x7fa508041000 comp rx=0 tx=0).stop 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa519026740 1 -- 172.21.15.60:0/1051007336 shutdown_connections 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa519026740 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa34e830 0x5597fa35f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa519026740 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa3539a0 0x5597fa358470 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa519026740 1 --2- 172.21.15.60:0/1051007336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa357f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa519026740 1 -- 172.21.15.60:0/1051007336 >> 172.21.15.60:0/1051007336 conn(0x5597fa3493b0 msgr2=0x5597fa34fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa519026740 1 -- 172.21.15.60:0/1051007336 shutdown_connections 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.666+0000 7fa519026740 1 -- 172.21.15.60:0/1051007336 wait complete. 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.667+0000 7fa519026740 1 -- start start 2024-02-26T10:47:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.667+0000 7fa519026740 1 Processor -- start 2024-02-26T10:47:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.667+0000 7fa519026740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa35e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.667+0000 7fa519026740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa35e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.667+0000 7fa519026740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.667+0000 7fa519026740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101aa4b0 con 0x5597fa34e830 2024-02-26T10:47:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.667+0000 7fa519026740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101ab7b0 con 0x5597fa3539a0 2024-02-26T10:47:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.667+0000 7fa519026740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101aaaf0 con 0x5597fa34de40 2024-02-26T10:47:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.667+0000 7fa519026740 10 client.0 _ll_register_callbacks cb 0x7fa5000d1450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa507e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa35e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53546/0 (socket says 172.21.15.60:53546) 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50fe00640 1 -- 172.21.15.60:0/4229677213 learned_addr learned my addr 172.21.15.60:0/4229677213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50f400640 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa35e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50fe00640 1 -- 172.21.15.60:0/4229677213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 msgr2=0x5597fa35e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50fe00640 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa35e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50fe00640 1 -- 172.21.15.60:0/4229677213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 msgr2=0x5597fa35e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50fe00640 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa35e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50fe00640 1 -- 172.21.15.60:0/4229677213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597fa3ba460 con 0x5597fa3539a0 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa507e00640 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa35e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa50fe00640 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35ed40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa508033000 tx=0x7fa508009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a39ec36009e14ec6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.668+0000 7fa4fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:47:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.669+0000 7fa4fcc00640 1 -- 172.21.15.60:0/4229677213 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508039030 con 0x5597fa3539a0 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.669+0000 7fa519026740 1 -- 172.21.15.60:0/4229677213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa508003b70 con 0x5597fa3539a0 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.669+0000 7fa4fcc00640 1 -- 172.21.15.60:0/4229677213 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa508003fe0 con 0x5597fa3539a0 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.669+0000 7fa4fcc00640 1 -- 172.21.15.60:0/4229677213 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa508067050 con 0x5597fa3539a0 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.669+0000 7fa519026740 10 client.25958 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.669+0000 7fa519026740 1 -- 172.21.15.60:0/4229677213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5597fa3bb7d0 con 0x5597fa3539a0 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.669+0000 7fa4fcc00640 1 -- 172.21.15.60:0/4229677213 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa508003b70 con 0x5597fa3539a0 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.669+0000 7fa4fcc00640 1 -- 172.21.15.60:0/4229677213 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa508034440 con 0x5597fa3539a0 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.670+0000 7fa519026740 10 client.25958 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.670+0000 7fa519026740 10 client.25958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.670+0000 7fa519026740 20 client.25958 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.670+0000 7fa519026740 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x5597fa3bc110 0x5597fa3be530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.670+0000 7fa519026740 4 client.25958 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.670+0000 7fa519026740 1 -- 172.21.15.60:0/4229677213 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5597fa2ba5c0 con 0x5597fa3bc110 2024-02-26T10:47:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.670+0000 7fa50f400640 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x5597fa3bc110 0x5597fa3be530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.670+0000 7fa50f400640 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x5597fa3bc110 0x5597fa3be530 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14798155984e79de server_cookie=4f20b07399bf73a6 in_seq=0 out_seq=0 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.670+0000 7fa4fcc00640 10 client.25958 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa4fcc00640 1 -- 172.21.15.60:0/4229677213 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5597fa2ba5c0 con 0x5597fa3bc110 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa4fcc00640 10 client.25958 handle_command_reply: tid=0 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 client.25958 shutdown 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 -- 172.21.15.60:0/4229677213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 msgr2=0x5597fa35ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35ed40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa508033000 tx=0x7fa508009360 comp rx=0 tx=0).stop 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 -- 172.21.15.60:0/4229677213 shutdown_connections 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5597fa3539a0 0x5597fa35ed40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x5597fa3bc110 0x5597fa3be530 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597fa34e830 0x5597fa35e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 --2- 172.21.15.60:0/4229677213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5597fa34de40 0x5597fa35e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa4fcc00640 0 client.25958 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:47:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 -- 172.21.15.60:0/4229677213 >> 172.21.15.60:0/4229677213 conn(0x5597fa3493b0 msgr2=0x5597fa357f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 -- 172.21.15.60:0/4229677213 shutdown_connections 2024-02-26T10:47:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 1 -- 172.21.15.60:0/4229677213 wait complete. 2024-02-26T10:47:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.671+0000 7fa519026740 20 client.25958 trim_cache size 0 max 16384 2024-02-26T10:47:28.670 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 -- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa4f40a1d80 msgr2=0x7fa4f40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa4f40a1d80 0x7fa4f40a4240 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa508005e30 tx=0x7fa508032590 comp rx=0 tx=0).stop 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 -- 172.21.15.60:0/419827266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa510073cc0 msgr2=0x7fa5101995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa510073cc0 0x7fa5101995b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa50000d950 tx=0x7fa50000de10 comp rx=0 tx=0).stop 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 -- 172.21.15.60:0/419827266 shutdown_connections 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa4f40a1d80 0x7fa4f40a4240 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa510109000 0x7fa5101a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5100752a0 0x7fa510199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 --2- 172.21.15.60:0/419827266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa510073cc0 0x7fa5101995b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 -- 172.21.15.60:0/419827266 >> 172.21.15.60:0/419827266 conn(0x7fa5100fc6d0 msgr2=0x7fa510104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 -- 172.21.15.60:0/419827266 shutdown_connections 2024-02-26T10:47:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.672+0000 7fa514c00640 1 -- 172.21.15.60:0/419827266 wait complete. 2024-02-26T10:47:28.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:28 smithi060 ceph-mon[31539]: pgmap v470: 129 pgs: 129 active+clean; 30 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 697 op/s 2024-02-26T10:47:28.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:28 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2335056522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:47:28.684 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:47:28.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T10:47:28.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:28 smithi113 ceph-mon[36126]: pgmap v470: 129 pgs: 129 active+clean; 30 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 697 op/s 2024-02-26T10:47:28.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:28 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2335056522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:47:28.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:28 smithi100 ceph-mon[36051]: pgmap v470: 129 pgs: 129 active+clean; 30 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 697 op/s 2024-02-26T10:47:28.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:28 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2335056522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:47:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.937+0000 7f443b000640 1 -- 172.21.15.60:0/1282690646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4434104570 msgr2=0x7f44341049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.937+0000 7f443b000640 1 --2- 172.21.15.60:0/1282690646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4434104570 0x7f44341049f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4424009900 tx=0x7f442402bc40 comp rx=0 tx=0).stop 2024-02-26T10:47:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.937+0000 7f443b000640 1 -- 172.21.15.60:0/1282690646 shutdown_connections 2024-02-26T10:47:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.937+0000 7f443b000640 1 --2- 172.21.15.60:0/1282690646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4434104570 0x7f44341049f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.937+0000 7f443b000640 1 --2- 172.21.15.60:0/1282690646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4434104f30 0x7f443410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.937+0000 7f443b000640 1 --2- 172.21.15.60:0/1282690646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4434103380 0x7f44341037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.937+0000 7f443b000640 1 -- 172.21.15.60:0/1282690646 >> 172.21.15.60:0/1282690646 conn(0x7f44340fe910 msgr2=0x7f4434100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.937+0000 7f443b000640 1 -- 172.21.15.60:0/1282690646 shutdown_connections 2024-02-26T10:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.937+0000 7f443b000640 1 -- 172.21.15.60:0/1282690646 wait complete. 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.938+0000 7f443b000640 1 Processor -- start 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.938+0000 7f443b000640 1 -- start start 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f443b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4434103380 0x7f4434199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f443b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4434104570 0x7f4434199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f443b000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4434104f30 0x7f44341a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f443b000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443410f060 con 0x7f4434104f30 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f443b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443410eee0 con 0x7f4434103380 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f443b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443410f1e0 con 0x7f4434104570 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f4439c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4434103380 0x7f4434199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f4439c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4434103380 0x7f4434199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53550/0 (socket says 172.21.15.60:53550) 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f4439200640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4434104570 0x7f4434199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f4439c00640 1 -- 172.21.15.60:0/3297948120 learned_addr learned my addr 172.21.15.60:0/3297948120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f443a600640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4434104f30 0x7f44341a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f4439200640 1 -- 172.21.15.60:0/3297948120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4434103380 msgr2=0x7f4434199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f4439200640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4434103380 0x7f4434199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f4439200640 1 -- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4434104f30 msgr2=0x7f44341a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f4439200640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4434104f30 0x7f44341a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f4439200640 1 -- 172.21.15.60:0/3297948120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44341a12a0 con 0x7f4434104570 2024-02-26T10:47:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.939+0000 7f443a600640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4434104f30 0x7f44341a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.940+0000 7f4439200640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4434104570 0x7f4434199bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4424036040 tx=0x7f442402c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.940+0000 7f4432000640 1 -- 172.21.15.60:0/3297948120 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424041070 con 0x7f4434104570 2024-02-26T10:47:28.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.940+0000 7f443b000640 1 -- 172.21.15.60:0/3297948120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44341a1530 con 0x7f4434104570 2024-02-26T10:47:28.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.940+0000 7f4432000640 1 -- 172.21.15.60:0/3297948120 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4424004450 con 0x7f4434104570 2024-02-26T10:47:28.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.940+0000 7f4432000640 1 -- 172.21.15.60:0/3297948120 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424003df0 con 0x7f4434104570 2024-02-26T10:47:28.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.940+0000 7f443b000640 1 -- 172.21.15.60:0/3297948120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44341a19f0 con 0x7f4434104570 2024-02-26T10:47:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443f5a5740 1 Processor -- start 2024-02-26T10:47:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443f5a5740 1 -- start start 2024-02-26T10:47:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f4432000640 1 -- 172.21.15.60:0/3297948120 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4424044070 con 0x7f4434104570 2024-02-26T10:47:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a74e40 0x55be09a75260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 0x55be09a7a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a7a9a0 0x55be09a7ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443f5a5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be099d9d30 con 0x55be09a7a9a0 2024-02-26T10:47:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443f5a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be09982cc0 con 0x55be09a74e40 2024-02-26T10:47:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443f5a5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be099d5dd0 con 0x55be09a75830 2024-02-26T10:47:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f4439200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a7a9a0 0x55be09a7ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a74e40 0x55be09a75260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f4439c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 0x55be09a7a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a74e40 0x55be09a75260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53552/0 (socket says 172.21.15.60:53552) 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f443a600640 1 -- 172.21.15.60:0/48803425 learned_addr learned my addr 172.21.15.60:0/48803425 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.944+0000 7f4439200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a7a9a0 0x55be09a7ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56604/0 (socket says 172.21.15.60:56604) 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.945+0000 7f4439c00640 1 -- 172.21.15.60:0/48803425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a74e40 msgr2=0x55be09a75260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.945+0000 7f4439c00640 1 --2- 172.21.15.60:0/48803425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a74e40 0x55be09a75260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.945+0000 7f4439c00640 1 -- 172.21.15.60:0/48803425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a7a9a0 msgr2=0x55be09a7ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.945+0000 7f4439c00640 1 --2- 172.21.15.60:0/48803425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a7a9a0 0x55be09a7ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.945+0000 7f4439c00640 1 -- 172.21.15.60:0/48803425 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be096ee3f0 con 0x55be09a75830 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.945+0000 7f4439c00640 1 --2- 172.21.15.60:0/48803425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 0x55be09a7a330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f44280058c0 tx=0x7f4428011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ce1b3a1985567e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.945+0000 7f4422a00640 1 -- 172.21.15.60:0/48803425 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428004220 con 0x55be09a75830 2024-02-26T10:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.945+0000 7f4422a00640 1 -- 172.21.15.60:0/48803425 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4428009e20 con 0x55be09a75830 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.945+0000 7f4422a00640 1 -- 172.21.15.60:0/48803425 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428019840 con 0x55be09a75830 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f4432000640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f441c0a1d80 0x7f441c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f4432000640 1 -- 172.21.15.60:0/3297948120 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f44240efbd0 con 0x7f4434104570 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443f5a5740 1 -- 172.21.15.60:0/48803425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 msgr2=0x55be09a7a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443f5a5740 1 --2- 172.21.15.60:0/48803425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 0x55be09a7a330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f44280058c0 tx=0x7f4428011810 comp rx=0 tx=0).stop 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443a600640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f441c0a1d80 0x7f441c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443f5a5740 1 -- 172.21.15.60:0/48803425 shutdown_connections 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443f5a5740 1 --2- 172.21.15.60:0/48803425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a7a9a0 0x55be09a7ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443f5a5740 1 --2- 172.21.15.60:0/48803425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 0x55be09a7a330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443f5a5740 1 --2- 172.21.15.60:0/48803425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a74e40 0x55be09a75260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443f5a5740 1 -- 172.21.15.60:0/48803425 >> 172.21.15.60:0/48803425 conn(0x55be09a703b0 msgr2=0x55be09a707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443f5a5740 1 -- 172.21.15.60:0/48803425 shutdown_connections 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.946+0000 7f443f5a5740 1 -- 172.21.15.60:0/48803425 wait complete. 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.947+0000 7f443a600640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f441c0a1d80 0x7f441c0a4240 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55be09a76090 tx=0x7f442c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.947+0000 7f443f5a5740 1 -- start start 2024-02-26T10:47:28.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.947+0000 7f443f5a5740 1 Processor -- start 2024-02-26T10:47:28.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a7f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 0x55be09a7f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f443f5a5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be09a7d950 con 0x55be09a74e40 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f443f5a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be099781f0 con 0x55be09a75830 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f443f5a5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be09a7dad0 con 0x55be09a7a9a0 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f443f5a5740 10 client.0 _ll_register_callbacks cb 0x55be096edbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f4439c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a7f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f4439c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a7f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56606/0 (socket says 172.21.15.60:56606) 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f4439c00640 1 -- 172.21.15.60:0/199440310 learned_addr learned my addr 172.21.15.60:0/199440310 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f4439200640 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 0x55be09a7f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f443a600640 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f4439c00640 1 -- 172.21.15.60:0/199440310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 msgr2=0x55be09a868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f4439c00640 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a868b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f4439c00640 1 -- 172.21.15.60:0/199440310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 msgr2=0x55be09a7f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f4439c00640 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 0x55be09a7f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f4439c00640 1 -- 172.21.15.60:0/199440310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be097b87d0 con 0x55be09a74e40 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.948+0000 7f443a600640 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.949+0000 7f4439c00640 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a7f330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4428011d10 tx=0x7f4428003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f04d35b0fb76b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.949+0000 7f4417400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.949+0000 7f4417400640 1 -- 172.21.15.60:0/199440310 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428028020 con 0x55be09a74e40 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.949+0000 7f4417400640 1 -- 172.21.15.60:0/199440310 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f4428005240 con 0x55be09a74e40 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.949+0000 7f443f5a5740 1 -- 172.21.15.60:0/199440310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4428009e20 con 0x55be09a74e40 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.949+0000 7f4417400640 1 -- 172.21.15.60:0/199440310 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4428033050 con 0x55be09a74e40 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f443f5a5740 10 client.15849 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f443f5a5740 1 -- 172.21.15.60:0/199440310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be09a89c20 con 0x55be09a74e40 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f4417400640 1 -- 172.21.15.60:0/199440310 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4428009e20 con 0x55be09a74e40 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f4417400640 1 -- 172.21.15.60:0/199440310 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4428004ba0 con 0x55be09a74e40 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f443f5a5740 10 client.15849 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f443f5a5740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f443f5a5740 20 client.15849 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f443f5a5740 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55be09a8b170 0x55be09a8d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f443f5a5740 4 client.15849 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.950+0000 7f443f5a5740 1 -- 172.21.15.60:0/199440310 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be099e7270 con 0x55be09a8b170 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.951+0000 7f4439200640 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55be09a8b170 0x55be09a8d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.951+0000 7f4439200640 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55be09a8b170 0x55be09a8d5c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a05ca9f7c5d938b server_cookie=e66d72299d11fafe in_seq=0 out_seq=0 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.951+0000 7f4417400640 10 client.15849 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f4417400640 1 -- 172.21.15.60:0/199440310 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be099e7270 con 0x55be09a8b170 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f4417400640 10 client.15849 handle_command_reply: tid=0 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 client.15849 shutdown 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 -- 172.21.15.60:0/199440310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 msgr2=0x55be09a7f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a7f330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4428011d10 tx=0x7f4428003820 comp rx=0 tx=0).stop 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 -- 172.21.15.60:0/199440310 shutdown_connections 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55be09a8b170 0x55be09a8d5c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 0x55be09a7f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 --2- 172.21.15.60:0/199440310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a7f330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f4417400640 0 client.15849 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 -- 172.21.15.60:0/199440310 >> 172.21.15.60:0/199440310 conn(0x55be09a703b0 msgr2=0x55be09a6f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 -- 172.21.15.60:0/199440310 shutdown_connections 2024-02-26T10:47:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 1 -- 172.21.15.60:0/199440310 wait complete. 2024-02-26T10:47:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.952+0000 7f443f5a5740 20 client.15849 trim_cache size 0 max 16384 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.965+0000 7f443f5a5740 1 Processor -- start 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.965+0000 7f443f5a5740 1 -- start start 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a6ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 0x55be09a86870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a7a9a0 0x55be09a6f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f443f5a5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341ae210 con 0x55be09a74e40 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f443f5a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341abdb0 con 0x55be09a7a9a0 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f443f5a5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341a9e50 con 0x55be09a75830 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f4439c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a6ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f4439c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a6ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56616/0 (socket says 172.21.15.60:56616) 2024-02-26T10:47:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f4439c00640 1 -- 172.21.15.60:0/1698065160 learned_addr learned my addr 172.21.15.60:0/1698065160 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f4439200640 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 0x55be09a86870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f443a600640 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a7a9a0 0x55be09a6f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f4439c00640 1 -- 172.21.15.60:0/1698065160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 msgr2=0x55be09a86870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f4439c00640 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 0x55be09a86870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f4439c00640 1 -- 172.21.15.60:0/1698065160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a7a9a0 msgr2=0x55be09a6f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f4439c00640 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a7a9a0 0x55be09a6f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f4439c00640 1 -- 172.21.15.60:0/1698065160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be09a7e400 con 0x55be09a74e40 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.966+0000 7f443a600640 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a7a9a0 0x55be09a6f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.967+0000 7f4439c00640 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a6ee40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4428011850 tx=0x7f4428012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0671ea1a2964be server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.967+0000 7f4422000640 1 -- 172.21.15.60:0/1698065160 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428028070 con 0x55be09a74e40 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.967+0000 7f4422000640 1 -- 172.21.15.60:0/1698065160 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f442801e040 con 0x55be09a74e40 2024-02-26T10:47:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.967+0000 7f4422000640 1 -- 172.21.15.60:0/1698065160 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442801d070 con 0x55be09a74e40 2024-02-26T10:47:28.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.967+0000 7f443f5a5740 1 -- 172.21.15.60:0/1698065160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 msgr2=0x55be09a6ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.967+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a6ee40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4428011850 tx=0x7f4428012f70 comp rx=0 tx=0).stop 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.968+0000 7f443f5a5740 1 -- 172.21.15.60:0/1698065160 shutdown_connections 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.968+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a7a9a0 0x55be09a6f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.968+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a75830 0x55be09a86870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.968+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1698065160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a6ee40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.968+0000 7f443f5a5740 1 -- 172.21.15.60:0/1698065160 >> 172.21.15.60:0/1698065160 conn(0x55be09a703b0 msgr2=0x55be09a76d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.968+0000 7f443f5a5740 1 -- 172.21.15.60:0/1698065160 shutdown_connections 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.968+0000 7f443f5a5740 1 -- 172.21.15.60:0/1698065160 wait complete. 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.968+0000 7f443f5a5740 1 -- start start 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.968+0000 7f443f5a5740 1 Processor -- start 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 0x55be09a84b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f443f5a5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a85070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f443f5a5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341aa450 con 0x55be09a74e40 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f443f5a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341ab750 con 0x55be09a75830 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f443f5a5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341aaa90 con 0x55be09a7a9a0 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f443f5a5740 10 client.0 _ll_register_callbacks cb 0x7f4428009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a864d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56626/0 (socket says 172.21.15.60:56626) 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439c00640 1 -- 172.21.15.60:0/1088970632 learned_addr learned my addr 172.21.15.60:0/1088970632 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439200640 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 0x55be09a84b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f443a600640 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a85070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439c00640 1 -- 172.21.15.60:0/1088970632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 msgr2=0x55be09a85070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439c00640 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a85070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439c00640 1 -- 172.21.15.60:0/1088970632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 msgr2=0x55be09a84b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439c00640 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 0x55be09a84b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439c00640 1 -- 172.21.15.60:0/1088970632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be09a855b0 con 0x55be09a74e40 2024-02-26T10:47:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f4439200640 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 0x55be09a84b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.969+0000 7f443a600640 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a85070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.970+0000 7f4439c00640 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a864d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4428005cd0 tx=0x7f4428005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53aeb309961ef9ae server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.970+0000 7f4417400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.970+0000 7f4417400640 1 -- 172.21.15.60:0/1088970632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428045020 con 0x55be09a74e40 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.970+0000 7f443f5a5740 1 -- 172.21.15.60:0/1088970632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f442801e040 con 0x55be09a74e40 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.970+0000 7f4417400640 1 -- 172.21.15.60:0/1088970632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f44280051c0 con 0x55be09a74e40 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.970+0000 7f4417400640 1 -- 172.21.15.60:0/1088970632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f442804d050 con 0x55be09a74e40 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.970+0000 7f443f5a5740 10 client.15855 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.970+0000 7f443f5a5740 1 -- 172.21.15.60:0/1088970632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be09ae2400 con 0x55be09a74e40 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.971+0000 7f4417400640 1 -- 172.21.15.60:0/1088970632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f442801e040 con 0x55be09a74e40 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.971+0000 7f4417400640 1 -- 172.21.15.60:0/1088970632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f44280195d0 con 0x55be09a74e40 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.971+0000 7f443f5a5740 10 client.15855 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.971+0000 7f443f5a5740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.971+0000 7f443f5a5740 20 client.15855 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.971+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55be09ae2d00 0x55be09ae5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.971+0000 7f443f5a5740 4 client.15855 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:47:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.971+0000 7f443f5a5740 1 -- 172.21.15.60:0/1088970632 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be099e15c0 con 0x55be09ae2d00 2024-02-26T10:47:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.971+0000 7f4439200640 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55be09ae2d00 0x55be09ae5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.972+0000 7f4439200640 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55be09ae2d00 0x55be09ae5120 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a74fc2082fbd3b server_cookie=2fbfd4dff762dd07 in_seq=0 out_seq=0 2024-02-26T10:47:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.972+0000 7f4417400640 10 client.15855 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:28.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.972+0000 7f4417400640 1 -- 172.21.15.60:0/1088970632 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be099e15c0 con 0x55be09ae2d00 2024-02-26T10:47:28.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.972+0000 7f4417400640 10 client.15855 handle_command_reply: tid=0 2024-02-26T10:47:28.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.972+0000 7f443f5a5740 1 client.15855 shutdown 2024-02-26T10:47:28.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 -- 172.21.15.60:0/1088970632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 msgr2=0x55be09a864d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a864d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4428005cd0 tx=0x7f4428005d00 comp rx=0 tx=0).stop 2024-02-26T10:47:28.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 -- 172.21.15.60:0/1088970632 shutdown_connections 2024-02-26T10:47:28.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55be09ae2d00 0x55be09ae5120 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be09a75830 0x55be09a84b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be09a7a9a0 0x55be09a85070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 --2- 172.21.15.60:0/1088970632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55be09a74e40 0x55be09a864d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 -- 172.21.15.60:0/1088970632 >> 172.21.15.60:0/1088970632 conn(0x55be09a703b0 msgr2=0x55be09a73610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f4417400640 0 client.15855 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 -- 172.21.15.60:0/1088970632 shutdown_connections 2024-02-26T10:47:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 1 -- 172.21.15.60:0/1088970632 wait complete. 2024-02-26T10:47:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.973+0000 7f443f5a5740 20 client.15855 trim_cache size 0 max 16384 2024-02-26T10:47:28.971 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 -- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f441c0a1d80 msgr2=0x7f441c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f441c0a1d80 0x7f441c0a4240 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55be09a76090 tx=0x7f442c009500 comp rx=0 tx=0).stop 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 -- 172.21.15.60:0/3297948120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4434104570 msgr2=0x7f4434199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4434104570 0x7f4434199bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4424036040 tx=0x7f442402c660 comp rx=0 tx=0).stop 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 -- 172.21.15.60:0/3297948120 shutdown_connections 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f441c0a1d80 0x7f441c0a4240 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4434104f30 0x7f44341a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4434104570 0x7f4434199bc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 --2- 172.21.15.60:0/3297948120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4434103380 0x7f4434199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 -- 172.21.15.60:0/3297948120 >> 172.21.15.60:0/3297948120 conn(0x7f44340fe910 msgr2=0x7f44341078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 -- 172.21.15.60:0/3297948120 shutdown_connections 2024-02-26T10:47:28.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:28.974+0000 7f443b000640 1 -- 172.21.15.60:0/3297948120 wait complete. 2024-02-26T10:47:29.987 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T10:47:29.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:47:29.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T10:47:30.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.245+0000 7f1487000640 1 -- 172.21.15.60:0/3590381920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 msgr2=0x7f1480076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.245+0000 7f1487000640 1 --2- 172.21.15.60:0/3590381920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 0x7f1480076040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1474009900 tx=0x7f147402bc30 comp rx=0 tx=0).stop 2024-02-26T10:47:30.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.245+0000 7f1487000640 1 -- 172.21.15.60:0/3590381920 shutdown_connections 2024-02-26T10:47:30.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.245+0000 7f1487000640 1 --2- 172.21.15.60:0/3590381920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1480077800 0x7f14800780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.245+0000 7f1487000640 1 --2- 172.21.15.60:0/3590381920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1480076e60 0x7f14800772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.245+0000 7f1487000640 1 --2- 172.21.15.60:0/3590381920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 0x7f1480076040 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.245+0000 7f1487000640 1 -- 172.21.15.60:0/3590381920 >> 172.21.15.60:0/3590381920 conn(0x7f14800fe8d0 msgr2=0x7f1480100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.245+0000 7f1487000640 1 -- 172.21.15.60:0/3590381920 shutdown_connections 2024-02-26T10:47:30.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.245+0000 7f1487000640 1 -- 172.21.15.60:0/3590381920 wait complete. 2024-02-26T10:47:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.246+0000 7f1487000640 1 Processor -- start 2024-02-26T10:47:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.246+0000 7f1487000640 1 -- start start 2024-02-26T10:47:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.246+0000 7f1487000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 0x7f148019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.246+0000 7f1487000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1480076e60 0x7f148019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.246+0000 7f1487000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1480077800 0x7f14801a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.246+0000 7f1487000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480113430 con 0x7f1480076e60 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1487000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801132b0 con 0x7f1480075c20 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1487000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801135b0 con 0x7f1480077800 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1485c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 0x7f148019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1485c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 0x7f148019da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53590/0 (socket says 172.21.15.60:53590) 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1485200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1480076e60 0x7f148019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1485c00640 1 -- 172.21.15.60:0/950048330 learned_addr learned my addr 172.21.15.60:0/950048330 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1486600640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1480077800 0x7f14801a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1485c00640 1 -- 172.21.15.60:0/950048330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1480077800 msgr2=0x7f14801a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1485c00640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1480077800 0x7f14801a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1485c00640 1 -- 172.21.15.60:0/950048330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1480076e60 msgr2=0x7f148019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1485c00640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1480076e60 0x7f148019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.247+0000 7f1485c00640 1 -- 172.21.15.60:0/950048330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14801a5650 con 0x7f1480075c20 2024-02-26T10:47:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.248+0000 7f1485c00640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 0x7f148019da30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1474005ab0 tx=0x7f147402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.248+0000 7f147e000640 1 -- 172.21.15.60:0/950048330 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474043020 con 0x7f1480075c20 2024-02-26T10:47:30.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.248+0000 7f147e000640 1 -- 172.21.15.60:0/950048330 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14740054e0 con 0x7f1480075c20 2024-02-26T10:47:30.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.248+0000 7f147e000640 1 -- 172.21.15.60:0/950048330 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474003ce0 con 0x7f1480075c20 2024-02-26T10:47:30.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.248+0000 7f1487000640 1 -- 172.21.15.60:0/950048330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14801a58e0 con 0x7f1480075c20 2024-02-26T10:47:30.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.248+0000 7f1487000640 1 -- 172.21.15.60:0/950048330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14801a5de0 con 0x7f1480075c20 2024-02-26T10:47:30.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.251+0000 7f148b40a740 1 Processor -- start 2024-02-26T10:47:30.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.251+0000 7f148b40a740 1 -- start start 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.251+0000 7f147e000640 1 -- 172.21.15.60:0/950048330 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1474046020 con 0x7f1480075c20 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f148b40a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404280 0x55854e4046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f148b40a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e409770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f148b40a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e409de0 0x55854e40c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f148b40a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854e317df0 con 0x55854e409de0 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f148b40a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854e3699f0 con 0x55854e404c70 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f148b40a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854e312ad0 con 0x55854e404280 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1486600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404280 0x55854e4046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1485c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e409770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1486600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404280 0x55854e4046a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46152/0 (socket says 172.21.15.60:46152) 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1486600640 1 -- 172.21.15.60:0/3651303242 learned_addr learned my addr 172.21.15.60:0/3651303242 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1485200640 1 --2- 172.21.15.60:0/3651303242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e409de0 0x55854e40c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1486600640 1 -- 172.21.15.60:0/3651303242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 msgr2=0x55854e409770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1486600640 1 --2- 172.21.15.60:0/3651303242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e409770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1486600640 1 -- 172.21.15.60:0/3651303242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e409de0 msgr2=0x55854e40c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1486600640 1 --2- 172.21.15.60:0/3651303242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e409de0 0x55854e40c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.252+0000 7f1486600640 1 -- 172.21.15.60:0/3651303242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55854e327de0 con 0x55854e404280 2024-02-26T10:47:30.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.253+0000 7f1486600640 1 --2- 172.21.15.60:0/3651303242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404280 0x55854e4046a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1478009f50 tx=0x7f14780118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d277706df512322c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.253+0000 7f147e000640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f14680a1e00 0x7f14680a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.253+0000 7f1472a00640 1 -- 172.21.15.60:0/3651303242 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478004d20 con 0x55854e404280 2024-02-26T10:47:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.253+0000 7f1472a00640 1 -- 172.21.15.60:0/3651303242 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1478004050 con 0x55854e404280 2024-02-26T10:47:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.253+0000 7f1472a00640 1 -- 172.21.15.60:0/3651303242 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478005240 con 0x55854e404280 2024-02-26T10:47:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.253+0000 7f147e000640 1 -- 172.21.15.60:0/950048330 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f14740f01c0 con 0x7f1480075c20 2024-02-26T10:47:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.253+0000 7f1485c00640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f14680a1e00 0x7f14680a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f148b40a740 1 -- 172.21.15.60:0/3651303242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404280 msgr2=0x55854e4046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f148b40a740 1 --2- 172.21.15.60:0/3651303242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404280 0x55854e4046a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1478009f50 tx=0x7f14780118d0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f1485c00640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f14680a1e00 0x7f14680a42c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f14740edc80 tx=0x7f14740f3ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f148b40a740 1 -- 172.21.15.60:0/3651303242 shutdown_connections 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f148b40a740 1 --2- 172.21.15.60:0/3651303242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e409770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f148b40a740 1 --2- 172.21.15.60:0/3651303242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e409de0 0x55854e40c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f148b40a740 1 --2- 172.21.15.60:0/3651303242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404280 0x55854e4046a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f148b40a740 1 -- 172.21.15.60:0/3651303242 >> 172.21.15.60:0/3651303242 conn(0x55854e3ff7f0 msgr2=0x55854e3ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f148b40a740 1 -- 172.21.15.60:0/3651303242 shutdown_connections 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.254+0000 7f148b40a740 1 -- 172.21.15.60:0/3651303242 wait complete. 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.255+0000 7f148b40a740 1 -- start start 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.255+0000 7f148b40a740 1 Processor -- start 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.255+0000 7f148b40a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e40e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.255+0000 7f148b40a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e40ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.255+0000 7f148b40a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 0x55854e415b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.255+0000 7f148b40a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854e325f10 con 0x55854e404280 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f148b40a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854e365a90 con 0x55854e409de0 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f148b40a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55854e40cd90 con 0x55854e404c70 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f148b40a740 10 client.0 _ll_register_callbacks cb 0x55854e364a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1486600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e40e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1485200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e40ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1486600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e40e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56654/0 (socket says 172.21.15.60:56654) 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1485200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e40ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46166/0 (socket says 172.21.15.60:46166) 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1486600640 1 -- 172.21.15.60:0/3868112664 learned_addr learned my addr 172.21.15.60:0/3868112664 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1486600640 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 0x55854e415b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1485200640 1 -- 172.21.15.60:0/3868112664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 msgr2=0x55854e415b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1485200640 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 0x55854e415b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1485200640 1 -- 172.21.15.60:0/3868112664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 msgr2=0x55854e40e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1485200640 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e40e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1485200640 1 -- 172.21.15.60:0/3868112664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55854e1467e0 con 0x55854e404c70 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.256+0000 7f1486600640 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 0x55854e415b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.257+0000 7f1485200640 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e40ead0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f148019ef40 tx=0x7f146c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79799b3cd798db7b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.257+0000 7f1467400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.257+0000 7f1467400640 1 -- 172.21.15.60:0/3868112664 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c004020 con 0x55854e404c70 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.257+0000 7f1467400640 1 -- 172.21.15.60:0/3868112664 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f146c0041c0 con 0x55854e404c70 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.257+0000 7f148b40a740 1 -- 172.21.15.60:0/3868112664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1478004050 con 0x55854e404c70 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.257+0000 7f1467400640 1 -- 172.21.15.60:0/3868112664 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f146c024050 con 0x55854e404c70 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.257+0000 7f148b40a740 10 client.25993 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.257+0000 7f148b40a740 1 -- 172.21.15.60:0/3868112664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55854e1a6f70 con 0x55854e404c70 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.257+0000 7f1467400640 1 -- 172.21.15.60:0/3868112664 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1478004050 con 0x55854e404c70 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.258+0000 7f1467400640 1 -- 172.21.15.60:0/3868112664 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f146c0038f0 con 0x55854e404c70 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.258+0000 7f148b40a740 10 client.25993 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.258+0000 7f148b40a740 10 client.25993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.258+0000 7f148b40a740 20 client.25993 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.258+0000 7f148b40a740 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55854e41a3f0 0x55854e41c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.258+0000 7f148b40a740 4 client.25993 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.258+0000 7f148b40a740 1 -- 172.21.15.60:0/3868112664 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55854e376f30 con 0x55854e41a3f0 2024-02-26T10:47:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.258+0000 7f1486600640 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55854e41a3f0 0x55854e41c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.259+0000 7f1486600640 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55854e41a3f0 0x55854e41c850 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3576c52d3b7325 server_cookie=fd186636255f4501 in_seq=0 out_seq=0 2024-02-26T10:47:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.259+0000 7f1467400640 10 client.25993 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.259+0000 7f1467400640 1 -- 172.21.15.60:0/3868112664 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55854e376f30 con 0x55854e41a3f0 2024-02-26T10:47:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.259+0000 7f1467400640 10 client.25993 handle_command_reply: tid=0 2024-02-26T10:47:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.259+0000 7f148b40a740 1 client.25993 shutdown 2024-02-26T10:47:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 -- 172.21.15.60:0/3868112664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 msgr2=0x55854e40ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e40ead0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f148019ef40 tx=0x7f146c0118d0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 -- 172.21.15.60:0/3868112664 shutdown_connections 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55854e41a3f0 0x55854e41c850 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 0x55854e415b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e40ead0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 --2- 172.21.15.60:0/3868112664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e40e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f1467400640 0 client.25993 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 -- 172.21.15.60:0/3868112664 >> 172.21.15.60:0/3868112664 conn(0x55854e3ff7f0 msgr2=0x55854e3fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 -- 172.21.15.60:0/3868112664 shutdown_connections 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 1 -- 172.21.15.60:0/3868112664 wait complete. 2024-02-26T10:47:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.260+0000 7f148b40a740 20 client.25993 trim_cache size 0 max 16384 2024-02-26T10:47:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.279+0000 7f148b40a740 1 Processor -- start 2024-02-26T10:47:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.279+0000 7f148b40a740 1 -- start start 2024-02-26T10:47:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f148b40a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e40e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f148b40a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e40ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f148b40a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e409de0 0x55854e415ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f148b40a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801b2640 con 0x55854e404280 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f148b40a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801b01e0 con 0x55854e404c70 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f148b40a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801ae280 con 0x55854e409de0 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1485200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e40ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1485200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e40ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53624/0 (socket says 172.21.15.60:53624) 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1485200640 1 -- 172.21.15.60:0/932713303 learned_addr learned my addr 172.21.15.60:0/932713303 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1486600640 1 --2- 172.21.15.60:0/932713303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e40e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1486600640 1 --2- 172.21.15.60:0/932713303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e409de0 0x55854e415ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1485200640 1 -- 172.21.15.60:0/932713303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e409de0 msgr2=0x55854e415ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1485200640 1 --2- 172.21.15.60:0/932713303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e409de0 0x55854e415ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1485200640 1 -- 172.21.15.60:0/932713303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 msgr2=0x55854e40e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1485200640 1 --2- 172.21.15.60:0/932713303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e40e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.280+0000 7f1485200640 1 -- 172.21.15.60:0/932713303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55854e03e170 con 0x55854e404c70 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.281+0000 7f1485200640 1 --2- 172.21.15.60:0/932713303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e40ead0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f146c001eb0 tx=0x7f146c00bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab9664e18f42872 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.281+0000 7f1472000640 1 -- 172.21.15.60:0/932713303 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c02c020 con 0x55854e404c70 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.281+0000 7f1472000640 1 -- 172.21.15.60:0/932713303 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f146c00b480 con 0x55854e404c70 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.281+0000 7f1472000640 1 -- 172.21.15.60:0/932713303 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c0050a0 con 0x55854e404c70 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.282+0000 7f148b40a740 1 -- 172.21.15.60:0/932713303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 msgr2=0x55854e40ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.282+0000 7f148b40a740 1 --2- 172.21.15.60:0/932713303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e40ead0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f146c001eb0 tx=0x7f146c00bb70 comp rx=0 tx=0).stop 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.282+0000 7f148b40a740 1 -- 172.21.15.60:0/932713303 shutdown_connections 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.282+0000 7f148b40a740 1 --2- 172.21.15.60:0/932713303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e404c70 0x55854e40ead0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.282+0000 7f148b40a740 1 --2- 172.21.15.60:0/932713303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e409de0 0x55854e415ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.282+0000 7f148b40a740 1 --2- 172.21.15.60:0/932713303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e40e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.282+0000 7f148b40a740 1 -- 172.21.15.60:0/932713303 >> 172.21.15.60:0/932713303 conn(0x55854e3ff7f0 msgr2=0x55854e4211f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.282+0000 7f148b40a740 1 -- 172.21.15.60:0/932713303 shutdown_connections 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.282+0000 7f148b40a740 1 -- 172.21.15.60:0/932713303 wait complete. 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.283+0000 7f148b40a740 1 -- start start 2024-02-26T10:47:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.283+0000 7f148b40a740 1 Processor -- start 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f148b40a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e415640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f148b40a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e415b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f148b40a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 0x55854e413d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f148b40a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801ae880 con 0x55854e404280 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f148b40a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801afb80 con 0x55854e409de0 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f148b40a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801aeec0 con 0x55854e404c70 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f148b40a740 10 client.0 _ll_register_callbacks cb 0x7f1478009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1486600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e415640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1485200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e415b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1486600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e415640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56670/0 (socket says 172.21.15.60:56670) 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1486600640 1 -- 172.21.15.60:0/2744802370 learned_addr learned my addr 172.21.15.60:0/2744802370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1486600640 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 0x55854e413d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1485200640 1 -- 172.21.15.60:0/2744802370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 msgr2=0x55854e413d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1485200640 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 0x55854e413d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1485200640 1 -- 172.21.15.60:0/2744802370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 msgr2=0x55854e415640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1485200640 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e415640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.284+0000 7f1485200640 1 -- 172.21.15.60:0/2744802370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55854e4145c0 con 0x55854e404c70 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.285+0000 7f1485200640 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e415b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55854e3f4110 tx=0x7f146c005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c69b9825080569 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.285+0000 7f1467400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.285+0000 7f1467400640 1 -- 172.21.15.60:0/2744802370 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c035020 con 0x55854e404c70 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.285+0000 7f148b40a740 1 -- 172.21.15.60:0/2744802370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f146c00b480 con 0x55854e404c70 2024-02-26T10:47:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.285+0000 7f1467400640 1 -- 172.21.15.60:0/2744802370 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f146c003800 con 0x55854e404c70 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.285+0000 7f1467400640 1 -- 172.21.15.60:0/2744802370 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f146c040050 con 0x55854e404c70 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.285+0000 7f148b40a740 10 client.25999 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.285+0000 7f148b40a740 1 -- 172.21.15.60:0/2744802370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55854e4715c0 con 0x55854e404c70 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.286+0000 7f1467400640 1 -- 172.21.15.60:0/2744802370 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f146c00b480 con 0x55854e404c70 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.286+0000 7f1467400640 1 -- 172.21.15.60:0/2744802370 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f146c031560 con 0x55854e404c70 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.286+0000 7f148b40a740 10 client.25999 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.286+0000 7f148b40a740 10 client.25999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.286+0000 7f148b40a740 20 client.25999 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.286+0000 7f148b40a740 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55854e471ef0 0x55854e474310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.286+0000 7f148b40a740 4 client.25999 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T10:47:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.286+0000 7f148b40a740 1 -- 172.21.15.60:0/2744802370 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55854e371280 con 0x55854e471ef0 2024-02-26T10:47:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.286+0000 7f1486600640 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55854e471ef0 0x55854e474310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.287+0000 7f1486600640 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55854e471ef0 0x55854e474310 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41d1cd1562bfcf1 server_cookie=8280742e749770b3 in_seq=0 out_seq=0 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.287+0000 7f1467400640 10 client.25999 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f1467400640 1 -- 172.21.15.60:0/2744802370 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55854e371280 con 0x55854e471ef0 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f1467400640 10 client.25999 handle_command_reply: tid=0 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 client.25999 shutdown 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 -- 172.21.15.60:0/2744802370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 msgr2=0x55854e415b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e415b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55854e3f4110 tx=0x7f146c005760 comp rx=0 tx=0).stop 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 -- 172.21.15.60:0/2744802370 shutdown_connections 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55854e471ef0 0x55854e474310 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55854e409de0 0x55854e413d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55854e404c70 0x55854e415b80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 --2- 172.21.15.60:0/2744802370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55854e404280 0x55854e415640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 -- 172.21.15.60:0/2744802370 >> 172.21.15.60:0/2744802370 conn(0x55854e3ff7f0 msgr2=0x55854e4060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f1467400640 0 client.25999 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 -- 172.21.15.60:0/2744802370 shutdown_connections 2024-02-26T10:47:30.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 1 -- 172.21.15.60:0/2744802370 wait complete. 2024-02-26T10:47:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.288+0000 7f148b40a740 20 client.25999 trim_cache size 0 max 16384 2024-02-26T10:47:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:47:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-02-26T10:47:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "b5900c1b-1dbb-41f1-8afe-0365bf0d963d", 2024-02-26T10:47:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-02-26T10:47:30.285 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 -- 172.21.15.60:0/950048330 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f14680a1e00 msgr2=0x7f14680a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f14680a1e00 0x7f14680a42c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f14740edc80 tx=0x7f14740f3ca0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 -- 172.21.15.60:0/950048330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 msgr2=0x7f148019da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 0x7f148019da30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1474005ab0 tx=0x7f147402c6c0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 -- 172.21.15.60:0/950048330 shutdown_connections 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f14680a1e00 0x7f14680a42c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1480077800 0x7f14801a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1480076e60 0x7f148019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 --2- 172.21.15.60:0/950048330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1480075c20 0x7f148019da30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 -- 172.21.15.60:0/950048330 >> 172.21.15.60:0/950048330 conn(0x7f14800fe8d0 msgr2=0x7f14800fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.289+0000 7f1487000640 1 -- 172.21.15.60:0/950048330 shutdown_connections 2024-02-26T10:47:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.290+0000 7f1487000640 1 -- 172.21.15.60:0/950048330 wait complete. 2024-02-26T10:47:30.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T10:47:30.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T10:47:30.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.534+0000 7f21bb000640 1 -- 172.21.15.60:0/3637086900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b4101d00 msgr2=0x7f21b41040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.534+0000 7f21bb000640 1 --2- 172.21.15.60:0/3637086900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b4101d00 0x7f21b41040f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009900 tx=0x7f21a402bc50 comp rx=0 tx=0).stop 2024-02-26T10:47:30.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.534+0000 7f21bb000640 1 -- 172.21.15.60:0/3637086900 shutdown_connections 2024-02-26T10:47:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.534+0000 7f21bb000640 1 --2- 172.21.15.60:0/3637086900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21b4106f60 0x7f21b41093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.534+0000 7f21bb000640 1 --2- 172.21.15.60:0/3637086900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b4104630 0x7f21b4106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.534+0000 7f21bb000640 1 --2- 172.21.15.60:0/3637086900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b4101d00 0x7f21b41040f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.534+0000 7f21bb000640 1 -- 172.21.15.60:0/3637086900 >> 172.21.15.60:0/3637086900 conn(0x7f21b40fb710 msgr2=0x7f21b40fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.534+0000 7f21bb000640 1 -- 172.21.15.60:0/3637086900 shutdown_connections 2024-02-26T10:47:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.534+0000 7f21bb000640 1 -- 172.21.15.60:0/3637086900 wait complete. 2024-02-26T10:47:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.535+0000 7f21bb000640 1 Processor -- start 2024-02-26T10:47:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.535+0000 7f21bb000640 1 -- start start 2024-02-26T10:47:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21bb000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21b4101d00 0x7f21b41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21bb000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b4104630 0x7f21b4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21bb000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b4106f60 0x7f21b419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21bb000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b410cf60 con 0x7f21b4104630 2024-02-26T10:47:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21bb000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b410cde0 con 0x7f21b4106f60 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21bb000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b410d0e0 con 0x7f21b4101d00 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21b4101d00 0x7f21b41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21ab400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b4104630 0x7f21b4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21b4101d00 0x7f21b41975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46202/0 (socket says 172.21.15.60:46202) 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21b3e00640 1 -- 172.21.15.60:0/3466416690 learned_addr learned my addr 172.21.15.60:0/3466416690 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.536+0000 7f21b9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b4106f60 0x7f21b419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21b9000640 1 -- 172.21.15.60:0/3466416690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21b4101d00 msgr2=0x7f21b41975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21b9000640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21b4101d00 0x7f21b41975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21b9000640 1 -- 172.21.15.60:0/3466416690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b4104630 msgr2=0x7f21b4197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21b9000640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b4104630 0x7f21b4197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21b9000640 1 -- 172.21.15.60:0/3466416690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b419f1e0 con 0x7f21b4106f60 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21b9000640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b4106f60 0x7f21b419eb70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00ec50 tx=0x7f21ac00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21b1600640 1 -- 172.21.15.60:0/3466416690 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac003b70 con 0x7f21b4106f60 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21b1600640 1 -- 172.21.15.60:0/3466416690 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21ac003d10 con 0x7f21b4106f60 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21b1600640 1 -- 172.21.15.60:0/3466416690 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac010890 con 0x7f21b4106f60 2024-02-26T10:47:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21bb000640 1 -- 172.21.15.60:0/3466416690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b419f4d0 con 0x7f21b4106f60 2024-02-26T10:47:30.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.537+0000 7f21bb000640 1 -- 172.21.15.60:0/3466416690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b419f9d0 con 0x7f21b4106f60 2024-02-26T10:47:30.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.541+0000 7f21b1600640 1 -- 172.21.15.60:0/3466416690 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f21ac003fc0 con 0x7f21b4106f60 2024-02-26T10:47:30.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.541+0000 7f21bdfa9740 1 Processor -- start 2024-02-26T10:47:30.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.541+0000 7f21bdfa9740 1 -- start start 2024-02-26T10:47:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7cf30 0x558622a7d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7d920 0x558622a82420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 0x558622a84f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586229a6060 con 0x558622a7cf30 2024-02-26T10:47:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586229ddd00 con 0x558622a7d920 2024-02-26T10:47:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622986a00 con 0x558622a82a90 2024-02-26T10:47:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7d920 0x558622a82420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21ab400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7cf30 0x558622a7d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7d920 0x558622a82420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53664/0 (socket says 172.21.15.60:53664) 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21b3e00640 1 -- 172.21.15.60:0/3378021990 learned_addr learned my addr 172.21.15.60:0/3378021990 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.542+0000 7f21ab400640 1 --2- 172.21.15.60:0/3378021990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 0x558622a84f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21b3e00640 1 -- 172.21.15.60:0/3378021990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 msgr2=0x558622a84f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21b3e00640 1 --2- 172.21.15.60:0/3378021990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 0x558622a84f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21b3e00640 1 -- 172.21.15.60:0/3378021990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7cf30 msgr2=0x558622a7d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21b3e00640 1 --2- 172.21.15.60:0/3378021990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7cf30 0x558622a7d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21b3e00640 1 -- 172.21.15.60:0/3378021990 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586227bc7d0 con 0x558622a7d920 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21ab400640 1 --2- 172.21.15.60:0/3378021990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 0x558622a84f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21b1600640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f21980a1e00 0x7f21980a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21b3e00640 1 --2- 172.21.15.60:0/3378021990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7d920 0x558622a82420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f21a4036040 tx=0x7f21a4009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11918a640de38094 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21ab400640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f21980a1e00 0x7f21980a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21b1600640 1 -- 172.21.15.60:0/3466416690 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f21ac0ce650 con 0x7f21b4106f60 2024-02-26T10:47:30.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.543+0000 7f21a9600640 1 -- 172.21.15.60:0/3378021990 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4042070 con 0x558622a7d920 2024-02-26T10:47:30.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.544+0000 7f21a9600640 1 -- 172.21.15.60:0/3378021990 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a402cba0 con 0x558622a7d920 2024-02-26T10:47:30.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.544+0000 7f21a9600640 1 -- 172.21.15.60:0/3378021990 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4005130 con 0x558622a7d920 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21ab400640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f21980a1e00 0x7f21980a42c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x558622a7e180 tx=0x7f219c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 -- 172.21.15.60:0/3378021990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7d920 msgr2=0x558622a82420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/3378021990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7d920 0x558622a82420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f21a4036040 tx=0x7f21a4009560 comp rx=0 tx=0).stop 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 -- 172.21.15.60:0/3378021990 shutdown_connections 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/3378021990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7d920 0x558622a82420 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/3378021990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 0x558622a84f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/3378021990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7cf30 0x558622a7d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 -- 172.21.15.60:0/3378021990 >> 172.21.15.60:0/3378021990 conn(0x558622a74090 msgr2=0x558622a74490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 -- 172.21.15.60:0/3378021990 shutdown_connections 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 -- 172.21.15.60:0/3378021990 wait complete. 2024-02-26T10:47:30.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.545+0000 7f21bdfa9740 1 -- start start 2024-02-26T10:47:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.546+0000 7f21bdfa9740 1 Processor -- start 2024-02-26T10:47:30.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.546+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7cf30 0x558622a87230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.546+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 0x558622a87770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.546+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a82a90 0x558622a8e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.546+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862297c090 con 0x558622a7d920 2024-02-26T10:47:30.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.546+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586229d9da0 con 0x558622a82a90 2024-02-26T10:47:30.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.546+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x558622a859e0 con 0x558622a7cf30 2024-02-26T10:47:30.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a82a90 0x558622a8e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a82a90 0x558622a8e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53678/0 (socket says 172.21.15.60:53678) 2024-02-26T10:47:30.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b9000640 1 -- 172.21.15.60:0/1818102730 learned_addr learned my addr 172.21.15.60:0/1818102730 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b3e00640 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7cf30 0x558622a87230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21bdfa9740 10 client.0 _ll_register_callbacks cb 0x558622999830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21ab400640 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 0x558622a87770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b9000640 1 -- 172.21.15.60:0/1818102730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7cf30 msgr2=0x558622a87230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b9000640 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7cf30 0x558622a87230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b9000640 1 -- 172.21.15.60:0/1818102730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 msgr2=0x558622a87770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b9000640 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 0x558622a87770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b9000640 1 -- 172.21.15.60:0/1818102730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586226f23f0 con 0x558622a82a90 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21bdfa9740 1 -- 172.21.15.60:0/1818102730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21a402cba0 con 0x558622a82a90 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21b9000640 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a82a90 0x558622a8e7b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f21ac09af60 tx=0x7f21ac09a930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbc05cc0b5156729 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.547+0000 7f21a2000640 10 client.26006 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.548+0000 7f21a2000640 1 -- 172.21.15.60:0/1818102730 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0b8060 con 0x558622a82a90 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.548+0000 7f21a2000640 1 -- 172.21.15.60:0/1818102730 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f21ac0a32e0 con 0x558622a82a90 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.548+0000 7f21a2000640 1 -- 172.21.15.60:0/1818102730 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21ac0a3520 con 0x558622a82a90 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.548+0000 7f21bdfa9740 10 client.26006 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:30.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.548+0000 7f21bdfa9740 1 -- 172.21.15.60:0/1818102730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x5586226b20f0 con 0x558622a82a90 2024-02-26T10:47:30.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.548+0000 7f21a2000640 1 -- 172.21.15.60:0/1818102730 <== mon.2 v2:172.21.15.100:3300/0 4 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f21ac0b9250 con 0x558622a82a90 2024-02-26T10:47:30.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.549+0000 7f21bdfa9740 10 client.26006 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:30.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.549+0000 7f21bdfa9740 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.549+0000 7f21bdfa9740 20 client.26006 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.549+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x558622a79ca0 0x558622a7c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.549+0000 7f21bdfa9740 4 client.26006 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.549+0000 7f21bdfa9740 1 -- 172.21.15.60:0/1818102730 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586229eb240 con 0x558622a79ca0 2024-02-26T10:47:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.549+0000 7f21b3e00640 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x558622a79ca0 0x558622a7c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.550+0000 7f21b3e00640 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x558622a79ca0 0x558622a7c0c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71890ee5f466762 server_cookie=14ad1a228e6c1ce in_seq=0 out_seq=0 2024-02-26T10:47:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.550+0000 7f21a2000640 10 client.26006 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:30.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21a2000640 1 -- 172.21.15.60:0/1818102730 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586229eb240 con 0x558622a79ca0 2024-02-26T10:47:30.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21a2000640 10 client.26006 handle_command_reply: tid=0 2024-02-26T10:47:30.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 client.26006 shutdown 2024-02-26T10:47:30.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 -- 172.21.15.60:0/1818102730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a82a90 msgr2=0x558622a8e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a82a90 0x558622a8e7b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f21ac09af60 tx=0x7f21ac09a930 comp rx=0 tx=0).stop 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 -- 172.21.15.60:0/1818102730 shutdown_connections 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x558622a79ca0 0x558622a7c0c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a82a90 0x558622a8e7b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 0x558622a87770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/1818102730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7cf30 0x558622a87230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 -- 172.21.15.60:0/1818102730 >> 172.21.15.60:0/1818102730 conn(0x558622a74090 msgr2=0x558622a73330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21a2000640 0 client.26006 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.553+0000 7f21bdfa9740 1 -- 172.21.15.60:0/1818102730 shutdown_connections 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.554+0000 7f21bdfa9740 1 -- 172.21.15.60:0/1818102730 wait complete. 2024-02-26T10:47:30.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.554+0000 7f21bdfa9740 20 client.26006 trim_cache size 0 max 16384 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.574+0000 7f21bdfa9740 1 Processor -- start 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.574+0000 7f21bdfa9740 1 -- start start 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.574+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a87230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.574+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 0x558622a8e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.574+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 0x558622a87770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.574+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41a9f40 con 0x558622a7d920 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.574+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41a7ae0 con 0x558622a7cf30 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.574+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41a5b80 con 0x558622a82a90 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a87230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 0x558622a87770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a87230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53682/0 (socket says 172.21.15.60:53682) 2024-02-26T10:47:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b3e00640 1 -- 172.21.15.60:0/137980849 learned_addr learned my addr 172.21.15.60:0/137980849 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21ab400640 1 --2- 172.21.15.60:0/137980849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 0x558622a8e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b3e00640 1 -- 172.21.15.60:0/137980849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 msgr2=0x558622a87770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b3e00640 1 --2- 172.21.15.60:0/137980849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 0x558622a87770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b3e00640 1 -- 172.21.15.60:0/137980849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 msgr2=0x558622a8e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b3e00640 1 --2- 172.21.15.60:0/137980849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 0x558622a8e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b3e00640 1 -- 172.21.15.60:0/137980849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558622a9e790 con 0x558622a7cf30 2024-02-26T10:47:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.575+0000 7f21b3e00640 1 --2- 172.21.15.60:0/137980849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a87230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f21a402c260 tx=0x7f21a4040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d61d2796cfd6030 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.576+0000 7f21a8c00640 1 -- 172.21.15.60:0/137980849 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4042040 con 0x558622a7cf30 2024-02-26T10:47:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.576+0000 7f21a8c00640 1 -- 172.21.15.60:0/137980849 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a4050430 con 0x558622a7cf30 2024-02-26T10:47:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.576+0000 7f21a8c00640 1 -- 172.21.15.60:0/137980849 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4004ed0 con 0x558622a7cf30 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.576+0000 7f21bdfa9740 1 -- 172.21.15.60:0/137980849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 msgr2=0x558622a87230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.576+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/137980849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a87230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f21a402c260 tx=0x7f21a4040040 comp rx=0 tx=0).stop 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.577+0000 7f21bdfa9740 1 -- 172.21.15.60:0/137980849 shutdown_connections 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.577+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/137980849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a82a90 0x558622a87770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.577+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/137980849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a7d920 0x558622a8e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.577+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/137980849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a87230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.577+0000 7f21bdfa9740 1 -- 172.21.15.60:0/137980849 >> 172.21.15.60:0/137980849 conn(0x558622a74090 msgr2=0x558622a7eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.577+0000 7f21bdfa9740 1 -- 172.21.15.60:0/137980849 shutdown_connections 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.577+0000 7f21bdfa9740 1 -- 172.21.15.60:0/137980849 wait complete. 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.577+0000 7f21bdfa9740 1 -- start start 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.577+0000 7f21bdfa9740 1 Processor -- start 2024-02-26T10:47:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a8d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7d920 0x558622a8d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21bdfa9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a82a90 0x558622a8da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41a6180 con 0x558622a82a90 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41a7480 con 0x558622a7cf30 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21bdfa9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41a67c0 con 0x558622a7d920 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21bdfa9740 10 client.0 _ll_register_callbacks cb 0x7f21ac00cf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a8d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21ab400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7d920 0x558622a8d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a8d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53692/0 (socket says 172.21.15.60:53692) 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21ab400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7d920 0x558622a8d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:46238/0 (socket says 172.21.15.60:46238) 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21b3e00640 1 -- 172.21.15.60:0/598663363 learned_addr learned my addr 172.21.15.60:0/598663363 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.578+0000 7f21b9000640 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a82a90 0x558622a8da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21ab400640 1 -- 172.21.15.60:0/598663363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 msgr2=0x558622a8d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21ab400640 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a8d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21ab400640 1 -- 172.21.15.60:0/598663363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a82a90 msgr2=0x558622a8da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21ab400640 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a82a90 0x558622a8da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21ab400640 1 -- 172.21.15.60:0/598663363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558622aed670 con 0x558622a7d920 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21b9000640 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a82a90 0x558622a8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21ab400640 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7d920 0x558622a8d540 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f219c0058c0 tx=0x7f219c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308087d44aa9b5fa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21a2000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21a2000640 1 -- 172.21.15.60:0/598663363 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c010cc0 con 0x558622a7d920 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.579+0000 7f21bdfa9740 1 -- 172.21.15.60:0/598663363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21a4050430 con 0x558622a7d920 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.580+0000 7f21a2000640 1 -- 172.21.15.60:0/598663363 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f219c023420 con 0x558622a7d920 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.580+0000 7f21a2000640 1 -- 172.21.15.60:0/598663363 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f219c010e60 con 0x558622a7d920 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.580+0000 7f21bdfa9740 10 client.26017 fetch_fsmap learned FSMap version 25 2024-02-26T10:47:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.580+0000 7f21bdfa9740 1 -- 172.21.15.60:0/598663363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558622aeea80 con 0x558622a7d920 2024-02-26T10:47:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.580+0000 7f21a2000640 1 -- 172.21.15.60:0/598663363 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a4050430 con 0x558622a7d920 2024-02-26T10:47:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.580+0000 7f21a2000640 1 -- 172.21.15.60:0/598663363 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f219c02d6c0 con 0x558622a7d920 2024-02-26T10:47:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.581+0000 7f21bdfa9740 10 client.26017 fetch_fsmap finished waiting for FSMap version 25 2024-02-26T10:47:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.581+0000 7f21bdfa9740 10 client.26017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.581+0000 7f21bdfa9740 20 client.26017 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.581+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x558622aef410 0x558622af1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.581+0000 7f21bdfa9740 4 client.26017 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T10:47:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.581+0000 7f21bdfa9740 1 -- 172.21.15.60:0/598663363 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5586229e5590 con 0x558622aef410 2024-02-26T10:47:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.581+0000 7f21b3e00640 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x558622aef410 0x558622af1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.581+0000 7f21b3e00640 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x558622aef410 0x558622af1830 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b8ddbef4e8de21e server_cookie=696858ab5d8b1871 in_seq=0 out_seq=0 2024-02-26T10:47:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.581+0000 7f21a2000640 10 client.26017 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.583+0000 7f21a2000640 1 -- 172.21.15.60:0/598663363 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5586229e5590 con 0x558622aef410 2024-02-26T10:47:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.583+0000 7f21a2000640 10 client.26017 handle_command_reply: tid=0 2024-02-26T10:47:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 client.26017 shutdown 2024-02-26T10:47:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 -- 172.21.15.60:0/598663363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7d920 msgr2=0x558622a8d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7d920 0x558622a8d540 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f219c0058c0 tx=0x7f219c005f90 comp rx=0 tx=0).stop 2024-02-26T10:47:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 -- 172.21.15.60:0/598663363 shutdown_connections 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x558622aef410 0x558622af1830 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558622a82a90 0x558622a8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558622a7d920 0x558622a8d540 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 --2- 172.21.15.60:0/598663363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558622a7cf30 0x558622a8d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 -- 172.21.15.60:0/598663363 >> 172.21.15.60:0/598663363 conn(0x558622a74090 msgr2=0x558622a7eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21a2000640 0 client.26017 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 -- 172.21.15.60:0/598663363 shutdown_connections 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 1 -- 172.21.15.60:0/598663363 wait complete. 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.584+0000 7f21bdfa9740 20 client.26017 trim_cache size 0 max 16384 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:47:30.579 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "scrub active (735 inodes in the stack)", 2024-02-26T10:47:30.580 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-02-26T10:47:30.580 INFO:teuthology.orchestra.run.smithi060.stdout: "b5900c1b-1dbb-41f1-8afe-0365bf0d963d": { 2024-02-26T10:47:30.580 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-02-26T10:47:30.580 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "b5900c1b-1dbb-41f1-8afe-0365bf0d963d", 2024-02-26T10:47:30.580 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-02-26T10:47:30.580 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:47:30.580 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:47:30.580 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 -- 172.21.15.60:0/3466416690 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f21980a1e00 msgr2=0x7f21980a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f21980a1e00 0x7f21980a42c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x558622a7e180 tx=0x7f219c005850 comp rx=0 tx=0).stop 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 -- 172.21.15.60:0/3466416690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b4106f60 msgr2=0x7f21b419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b4106f60 0x7f21b419eb70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00ec50 tx=0x7f21ac00c6a0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 -- 172.21.15.60:0/3466416690 shutdown_connections 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f21980a1e00 0x7f21980a42c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b4106f60 0x7f21b419eb70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b4104630 0x7f21b4197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 --2- 172.21.15.60:0/3466416690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21b4101d00 0x7f21b41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.585+0000 7f21bb000640 1 -- 172.21.15.60:0/3466416690 >> 172.21.15.60:0/3466416690 conn(0x7f21b40fb710 msgr2=0x7f21b41025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.586+0000 7f21bb000640 1 -- 172.21.15.60:0/3466416690 shutdown_connections 2024-02-26T10:47:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:47:30.586+0000 7f21bb000640 1 -- 172.21.15.60:0/3466416690 wait complete. 2024-02-26T10:47:30.598 INFO:tasks.cephfs.filesystem:scrub status for tag:b5900c1b-1dbb-41f1-8afe-0365bf0d963d - {'path': '/', 'tag': 'b5900c1b-1dbb-41f1-8afe-0365bf0d963d', 'options': 'recursive,force'} 2024-02-26T10:47:30.766 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-mon[36126]: pgmap v471: 129 pgs: 129 active+clean; 26 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 1.04k op/s 2024-02-26T10:47:30.775 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.461+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f51 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_006.c auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f51:[<0x10000000f50/uid_utils_006.c v2>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f51:[<0x10000000f50/uid_utils_006.c v2>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.775 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.461+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f52 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_002.c auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f52:[<0x10000000f50/uid_utils_002.c v4>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f52:[<0x10000000f50/uid_utils_002.c v4>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.775 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.461+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f5c [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_005.c auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f5c:[<0x10000000f50/uid_utils_005.c v24>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f5c:[<0x10000000f50/uid_utils_005.c v24>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.775 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.461+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f56 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_010.c auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f56:[<0x10000000f50/uid_utils_010.c v12>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f56:[<0x10000000f50/uid_utils_010.c v12>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.775 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.462+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f58 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_003.c auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f58:[<0x10000000f50/uid_utils_003.c v16>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f58:[<0x10000000f50/uid_utils_003.c v16>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.775 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.462+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f5d [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_011.c auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f5d:[<0x10000000f50/uid_utils_011.c v26>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f5d:[<0x10000000f50/uid_utils_011.c v26>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.775 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.462+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f5e [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_013.c auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f5e:[<0x10000000f50/uid_utils_013.c v28>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f5e:[<0x10000000f50/uid_utils_013.c v28>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.776 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.462+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f5b [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.in auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f5b:[<0x10000000f50/Makefile.in v22>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f5b:[<0x10000000f50/Makefile.in v22>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.776 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.462+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f59 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_001.c auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f59:[<0x10000000f50/uid_utils_001.c v18>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f59:[<0x10000000f50/uid_utils_001.c v18>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.776 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.463+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f53 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_009.c auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f53:[<0x10000000f50/uid_utils_009.c v6>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f53:[<0x10000000f50/uid_utils_009.c v6>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.776 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.463+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f5a [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_000.c auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f5a:[<0x10000000f50/uid_utils_000.c v20>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f5a:[<0x10000000f50/uid_utils_000.c v20>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.776 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.463+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f55 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_008.c auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f55:[<0x10000000f50/uid_utils_008.c v10>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f55:[<0x10000000f50/uid_utils_008.c v10>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.776 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.463+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f5f [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_004.c auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f5f:[<0x10000000f50/uid_utils_004.c v30>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f5f:[<0x10000000f50/uid_utils_004.c v30>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.463+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f54 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_012.c auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f54:[<0x10000000f50/uid_utils_012.c v8>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f54:[<0x10000000f50/uid_utils_012.c v8>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.464+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f91 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_045.c auth v72 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abafb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f91:[<0x10000000f6d/el_test_045.c v72>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f91:[<0x10000000f6d/el_test_045.c v72>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.464+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f60 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.am auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f60:[<0x10000000f50/Makefile.am v32>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f60:[<0x10000000f50/Makefile.am v32>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.464+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f93 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_009.c auth v76 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f93:[<0x10000000f6d/el_test_009.c v76>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f93:[<0x10000000f6d/el_test_009.c v76>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.464+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f80 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_002.c auth v38 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f80:[<0x10000000f6d/el_test_002.c v38>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f80:[<0x10000000f6d/el_test_002.c v38>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.464+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f57 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_007.c auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f57:[<0x10000000f50/uid_utils_007.c v14>,<0x10000000c85/uid v1462>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f57:[<0x10000000f50/uid_utils_007.c v14>,<0x10000000c85/uid v1594>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.465+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f83 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_025.c auth v44 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f83:[<0x10000000f6d/el_test_025.c v44>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f83:[<0x10000000f6d/el_test_025.c v44>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.465+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f9c [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_043.c auth v94 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aba8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f9c:[<0x10000000f6d/el_test_043.c v94>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f9c:[<0x10000000f6d/el_test_043.c v94>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.465+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f79 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_036.c auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aba8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f79:[<0x10000000f6d/el_test_036.c v24>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f79:[<0x10000000f6d/el_test_036.c v24>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.777 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.465+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f6f [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_017.c auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aba8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f6f:[<0x10000000f6d/el_test_017.c v4>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f6f:[<0x10000000f6d/el_test_017.c v4>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.778 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.465+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f77 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_035.c auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f77:[<0x10000000f6d/el_test_035.c v20>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f77:[<0x10000000f6d/el_test_035.c v20>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.778 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.465+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f96 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_023.c auth v82 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aba9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f96:[<0x10000000f6d/el_test_023.c v82>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f96:[<0x10000000f6d/el_test_023.c v82>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.778 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.466+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f7c [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_034.c auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aba9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f7c:[<0x10000000f6d/el_test_034.c v30>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f7c:[<0x10000000f6d/el_test_034.c v30>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.778 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.466+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f7a [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_039.c auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aba9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f7a:[<0x10000000f6d/el_test_039.c v26>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f7a:[<0x10000000f6d/el_test_039.c v26>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.778 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.466+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f7d [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_013.c auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abaa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f7d:[<0x10000000f6d/el_test_013.c v32>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f7d:[<0x10000000f6d/el_test_013.c v32>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.778 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.466+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f8c [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test.h auth v62 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f8c:[<0x10000000f6d/el_test.h v62>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f8c:[<0x10000000f6d/el_test.h v62>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.778 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.466+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f9d [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_003.c auth v96 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abaa680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f9d:[<0x10000000f6d/el_test_003.c v96>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f9d:[<0x10000000f6d/el_test_003.c v96>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.778 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.467+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f89 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_014.c auth v56 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abaac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f89:[<0x10000000f6d/el_test_014.c v56>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f89:[<0x10000000f6d/el_test_014.c v56>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.778 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.467+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f94 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_040.c auth v78 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abab180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f94:[<0x10000000f6d/el_test_040.c v78>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f94:[<0x10000000f6d/el_test_040.c v78>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.467+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f9b [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_041.c auth v92 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f9b:[<0x10000000f6d/el_test_041.c v92>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f9b:[<0x10000000f6d/el_test_041.c v92>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.467+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f71 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_011.c auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abab700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f71:[<0x10000000f6d/el_test_011.c v8>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f71:[<0x10000000f6d/el_test_011.c v8>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.467+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f97 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_018.c auth v84 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f97:[<0x10000000f6d/el_test_018.c v84>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f97:[<0x10000000f6d/el_test_018.c v84>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.467+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f76 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_032.c auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f76:[<0x10000000f6d/el_test_032.c v18>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f76:[<0x10000000f6d/el_test_032.c v18>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.468+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f82 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_028.c auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f82:[<0x10000000f6d/el_test_028.c v42>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f82:[<0x10000000f6d/el_test_028.c v42>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.468+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f8f [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.in auth v68 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafbb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f8f:[<0x10000000f6d/Makefile.in v68>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f8f:[<0x10000000f6d/Makefile.in v68>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.468+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f88 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_020.c auth v54 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f88:[<0x10000000f6d/el_test_020.c v54>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f88:[<0x10000000f6d/el_test_020.c v54>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.468+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f98 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_005.c auth v86 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f98:[<0x10000000f6d/el_test_005.c v86>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f98:[<0x10000000f6d/el_test_005.c v86>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.468+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f73 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_010.c auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f73:[<0x10000000f6d/el_test_010.c v12>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f73:[<0x10000000f6d/el_test_010.c v12>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.779 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.468+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f9e [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.am auth v98 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafd180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f9e:[<0x10000000f6d/Makefile.am v98>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f9e:[<0x10000000f6d/Makefile.am v98>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.469+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f92 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_042.c auth v74 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafd700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f92:[<0x10000000f6d/el_test_042.c v74>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f92:[<0x10000000f6d/el_test_042.c v74>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.469+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f7b [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_027.c auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aafc680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f7b:[<0x10000000f6d/el_test_027.c v28>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f7b:[<0x10000000f6d/el_test_027.c v28>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.469+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f87 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_compare.c auth v52 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac16000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f87:[<0x10000000f6d/el_compare.c v52>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f87:[<0x10000000f6d/el_compare.c v52>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.469+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f74 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_016.c auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac16580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f74:[<0x10000000f6d/el_test_016.c v14>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f74:[<0x10000000f6d/el_test_016.c v14>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.469+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f9a [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_029.c auth v90 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac16b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f9a:[<0x10000000f6d/el_test_029.c v90>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f9a:[<0x10000000f6d/el_test_029.c v90>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.470+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f8a [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_044.c auth v58 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac17600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f8a:[<0x10000000f6d/el_test_044.c v58>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f8a:[<0x10000000f6d/el_test_044.c v58>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.470+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f90 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_008.c auth v70 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac18100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f90:[<0x10000000f6d/el_test_008.c v70>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f90:[<0x10000000f6d/el_test_008.c v70>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.470+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f86 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_030.c auth v50 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac17080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f86:[<0x10000000f6d/el_test_030.c v50>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f86:[<0x10000000f6d/el_test_030.c v50>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.470+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f72 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_026.c auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac18680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f72:[<0x10000000f6d/el_test_026.c v10>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f72:[<0x10000000f6d/el_test_026.c v10>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.780 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.470+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f6e [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_024.c auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac18c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f6e:[<0x10000000f6d/el_test_024.c v2>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f6e:[<0x10000000f6d/el_test_024.c v2>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.781 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.470+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f99 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_001.c auth v88 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac17b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f99:[<0x10000000f6d/el_test_001.c v88>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f99:[<0x10000000f6d/el_test_001.c v88>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.781 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.471+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f75 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_012.c auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac19180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f75:[<0x10000000f6d/el_test_012.c v16>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f75:[<0x10000000f6d/el_test_012.c v16>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.781 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.471+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f78 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_033.c auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac19700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f78:[<0x10000000f6d/el_test_033.c v22>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f78:[<0x10000000f6d/el_test_033.c v22>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.781 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.471+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f7e [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_021.c auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f7e:[<0x10000000f6d/el_test_021.c v34>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f7e:[<0x10000000f6d/el_test_021.c v34>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.781 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.471+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f8b [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_006.c auth v60 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f8b:[<0x10000000f6d/el_test_006.c v60>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f8b:[<0x10000000f6d/el_test_006.c v60>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.781 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.471+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f95 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_015.c auth v80 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f95:[<0x10000000f6d/el_test_015.c v80>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f95:[<0x10000000f6d/el_test_015.c v80>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.781 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.471+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f8e [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_031.c auth v66 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f8e:[<0x10000000f6d/el_test_031.c v66>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f8e:[<0x10000000f6d/el_test_031.c v66>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.781 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.472+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f84 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_019.c auth v46 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f84:[<0x10000000f6d/el_test_019.c v46>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f84:[<0x10000000f6d/el_test_019.c v46>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.781 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.472+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f8d [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_037.c auth v64 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f8d:[<0x10000000f6d/el_test_037.c v64>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f8d:[<0x10000000f6d/el_test_037.c v64>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.782 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.472+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f85 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_007.c auth v48 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f85:[<0x10000000f6d/el_test_007.c v48>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f85:[<0x10000000f6d/el_test_007.c v48>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.782 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.472+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f70 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_038.c auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f70:[<0x10000000f6d/el_test_038.c v6>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f70:[<0x10000000f6d/el_test_038.c v6>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.782 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.472+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f81 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_004.c auth v40 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f81:[<0x10000000f6d/el_test_004.c v40>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f81:[<0x10000000f6d/el_test_004.c v40>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.782 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.472+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f7f [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_022.c auth v36 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f7f:[<0x10000000f6d/el_test_022.c v36>,<0x10000000c85/el v1586>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f7f:[<0x10000000f6d/el_test_022.c v36>,<0x10000000c85/el v1590>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v562>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.782 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.556+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f4c [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_utils_test.c auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f4c:[<0x10000000f41/sahpi_event_utils_test.c v22>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f4c:[<0x10000000f41/sahpi_event_utils_test.c v22>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.782 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.556+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f48 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrarea_test.c auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f48:[<0x10000000f41/sahpi_struct_idrarea_test.c v14>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f48:[<0x10000000f41/sahpi_struct_idrarea_test.c v14>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.782 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.556+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f49 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_encode_test.c auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f49:[<0x10000000f41/sahpi_event_encode_test.c v16>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f49:[<0x10000000f41/sahpi_event_encode_test.c v16>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.782 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.556+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f43 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_utils_test.c auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f43:[<0x10000000f41/sahpi_struct_utils_test.c v4>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f43:[<0x10000000f41/sahpi_struct_utils_test.c v4>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.782 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.556+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f4b [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpiatca_enum_utils_test.c auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f4b:[<0x10000000f41/sahpiatca_enum_utils_test.c v20>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f4b:[<0x10000000f41/sahpiatca_enum_utils_test.c v20>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.556+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f4e [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rdr_test.c auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f4e:[<0x10000000f41/sahpi_struct_rdr_test.c v26>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f4e:[<0x10000000f41/sahpi_struct_rdr_test.c v26>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.556+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f4a [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.in auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f4a:[<0x10000000f41/Makefile.in v18>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f4a:[<0x10000000f41/Makefile.in v18>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.556+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f46 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rptentry_test.c auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9f7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f46:[<0x10000000f41/sahpi_struct_rptentry_test.c v10>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f46:[<0x10000000f41/sahpi_struct_rptentry_test.c v10>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.557+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f44 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrfield_test.c auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f44:[<0x10000000f41/sahpi_struct_idrfield_test.c v6>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f44:[<0x10000000f41/sahpi_struct_idrfield_test.c v6>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.557+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f45 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_text_utils_test.c auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f45:[<0x10000000f41/sahpi_text_utils_test.c v8>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f45:[<0x10000000f41/sahpi_text_utils_test.c v8>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.557+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f4d [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.am auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f4d:[<0x10000000f41/Makefile.am v24>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f4d:[<0x10000000f41/Makefile.am v24>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.557+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f4f [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_enum_utils_test.c auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f4f:[<0x10000000f41/sahpi_enum_utils_test.c v28>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f4f:[<0x10000000f41/sahpi_enum_utils_test.c v28>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.557+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f47 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_time_utils_test.c auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f47:[<0x10000000f41/sahpi_time_utils_test.c v12>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f47:[<0x10000000f41/sahpi_time_utils_test.c v12>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.557+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f42 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrinfo_test.c auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abb5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f42:[<0x10000000f41/sahpi_struct_idrinfo_test.c v2>,<0x10000000c85/sahpi v1428>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f42:[<0x10000000f41/sahpi_struct_idrinfo_test.c v2>,<0x10000000c85/sahpi v1596>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.783 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.558+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f68 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_006.c auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844ac1f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f68:[<0x10000000f62/ann_test_006.c v12>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f68:[<0x10000000f62/ann_test_006.c v12>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.558+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f64 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_004.c auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f64:[<0x10000000f62/ann_test_004.c v4>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f64:[<0x10000000f62/ann_test_004.c v4>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.558+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f67 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_005.c auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f67:[<0x10000000f62/ann_test_005.c v10>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f67:[<0x10000000f62/ann_test_005.c v10>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.558+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f69 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_002.c auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f69:[<0x10000000f62/ann_test_002.c v14>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f69:[<0x10000000f62/ann_test_002.c v14>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.558+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f65 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_008.c auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f65:[<0x10000000f62/ann_test_008.c v6>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f65:[<0x10000000f62/ann_test_008.c v6>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.558+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f63 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_003.c auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f63:[<0x10000000f62/ann_test_003.c v2>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f63:[<0x10000000f62/ann_test_003.c v2>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.559+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f6a [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.in auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f6a:[<0x10000000f62/Makefile.in v16>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f6a:[<0x10000000f62/Makefile.in v16>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.559+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f6b [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_001.c auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f6b:[<0x10000000f62/ann_test_001.c v18>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f6b:[<0x10000000f62/ann_test_001.c v18>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.559+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f6c [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.am auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f6c:[<0x10000000f62/Makefile.am v20>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f6c:[<0x10000000f62/Makefile.am v20>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.559+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f66 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_007.c auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844aa8ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f66:[<0x10000000f62/ann_test_007.c v8>,<0x10000000c85/ann v1486>,<0x10000000c6b/t v70>,<0x10000000c2d/utils v126>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000f66:[<0x10000000f62/ann_test_007.c v8>,<0x10000000c85/ann v1592>,<0x10000000c6b/t v81>,<0x10000000c2d/utils v586>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.784 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.608+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000118b [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.am auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9e1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000118b:[<0x10000001187/Makefile.am v8>,<0x10000000fba/sysfs v652>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000118b:[<0x10000001187/Makefile.am v8>,<0x10000000fba/sysfs v652>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.608+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001188 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/README auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9e2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001188:[<0x10000001187/README v2>,<0x10000000fba/sysfs v652>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001188:[<0x10000001187/README v2>,<0x10000000fba/sysfs v652>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.608+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000118a [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/sysfs2hpi.c auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9e2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000118a:[<0x10000001187/sysfs2hpi.c v6>,<0x10000000fba/sysfs v652>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000118a:[<0x10000001187/sysfs2hpi.c v6>,<0x10000000fba/sysfs v652>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.608+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001189 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.in auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844a9e1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001189:[<0x10000001187/Makefile.in v4>,<0x10000000fba/sysfs v652>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001189:[<0x10000001187/Makefile.in v4>,<0x10000000fba/sysfs v652>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.771+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012fe [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/7-1.c auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abe3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012fe:[<0x100000012f4/7-1.c v20>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012fe:[<0x100000012f4/7-1.c v20>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.772+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012ff [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/4-1.c auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abe4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012ff:[<0x100000012f4/4-1.c v22>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012ff:[<0x100000012f4/4-1.c v22>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.772+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012fb [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/1-1.c auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abe3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012fb:[<0x100000012f4/1-1.c v14>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012fb:[<0x100000012f4/1-1.c v14>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.772+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012f8 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/2-1.c auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abe4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012f8:[<0x100000012f4/2-1.c v8>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012f8:[<0x100000012f4/2-1.c v8>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.772+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012fa [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/6-1.c auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abe5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012fa:[<0x100000012f4/6-1.c v12>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012fa:[<0x100000012f4/6-1.c v12>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.772+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012fc [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/8-1.c auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abe4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012fc:[<0x100000012f4/8-1.c v16>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012fc:[<0x100000012f4/8-1.c v16>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.772+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012f5 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/3-1.c auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abe5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012f5:[<0x100000012f4/3-1.c v2>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012f5:[<0x100000012f4/3-1.c v2>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.772+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012f7 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/10-1.c auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012f7:[<0x100000012f4/10-1.c v6>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012f7:[<0x100000012f4/10-1.c v6>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.785 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.773+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012f9 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/5-1.c auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abae000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012f9:[<0x100000012f4/5-1.c v10>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012f9:[<0x100000012f4/5-1.c v10>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.786 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.774+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012fd [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/assertions.xml auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abaf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012fd:[<0x100000012f4/assertions.xml v18>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012fd:[<0x100000012f4/assertions.xml v18>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.786 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.774+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001300 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/11-1.c auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abaf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001300:[<0x100000012f4/11-1.c v24>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001300:[<0x100000012f4/11-1.c v24>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.786 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:47:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:47:30.774+0000 7f075e6a6700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000012f6 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/9-1.c auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55844abaeb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000012f6:[<0x100000012f4/9-1.c v4>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v14>,<0x100000012ea/conformance v10>,<0x10000000204/open_posix_testsuite v313>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x100000012f6:[<0x100000012f4/9-1.c v4>,<0x100000012ef/mqueue_h v34>,<0x100000012ee/definitions v179>,<0x100000012ea/conformance v24>,<0x10000000204/open_posix_testsuite v362>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v16>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v31>,<0x1/volumes v30>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:30 smithi060 ceph-mon[31539]: pgmap v471: 129 pgs: 129 active+clean; 26 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 1.04k op/s 2024-02-26T10:47:30.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-mon[36051]: pgmap v471: 129 pgs: 129 active+clean; 26 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 1.04k op/s 2024-02-26T10:47:30.879 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.606+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001173 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.h auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb375700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001173:[<0x10000001166/rtas_reset.h v26>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001173:[<0x10000001166/rtas_reset.h v26>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.879 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.606+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000117b [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.c auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb375180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000117b:[<0x10000001166/rtas_indicator.c v42>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000117b:[<0x10000001166/rtas_indicator.c v42>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.879 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.606+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000116e [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.c auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb388580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000116e:[<0x10000001166/rtas_reset.c v16>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000116e:[<0x10000001166/rtas_reset.c v16>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.879 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.606+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001183 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.h auth v58 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb388b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001183:[<0x10000001166/rtas.h v58>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001183:[<0x10000001166/rtas.h v58>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.607+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000117a [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.c auth v40 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb388000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000117a:[<0x10000001166/rtas_power.c v40>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000117a:[<0x10000001166/rtas_power.c v40>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.607+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001170 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.c auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb389080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001170:[<0x10000001166/rtas_event.c v20>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001170:[<0x10000001166/rtas_event.c v20>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.607+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000116b [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.h auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb389600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000116b:[<0x10000001166/rtas_inventory.h v10>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000116b:[<0x10000001166/rtas_inventory.h v10>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.607+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001184 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.c auth v60 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb389b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001184:[<0x10000001166/rtas_control.c v60>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001184:[<0x10000001166/rtas_control.c v60>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.607+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001182 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.c auth v56 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb38a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001182:[<0x10000001166/rtas_hotswap.c v56>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001182:[<0x10000001166/rtas_hotswap.c v56>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.607+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001175 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.h auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb38ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001175:[<0x10000001166/rtas_watchdog.h v30>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001175:[<0x10000001166/rtas_watchdog.h v30>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.607+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001179 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.c auth v38 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb38b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001179:[<0x10000001166/rtas_resource.c v38>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001179:[<0x10000001166/rtas_resource.c v38>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.608+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001171 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.h auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb38b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001171:[<0x10000001166/rtas_discover.h v22>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001171:[<0x10000001166/rtas_discover.h v22>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.608+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000116d [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.h auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb38a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000116d:[<0x10000001166/rtas_utils.h v14>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000116d:[<0x10000001166/rtas_utils.h v14>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.880 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.608+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000117f [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.h auth v50 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb390580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000117f:[<0x10000001166/rtas_control.h v50>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000117f:[<0x10000001166/rtas_control.h v50>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.608+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000116c [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.c auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb390b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000116c:[<0x10000001166/rtas_watchdog.c v12>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000116c:[<0x10000001166/rtas_watchdog.c v12>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.608+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000116a [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.h auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb391600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000116a:[<0x10000001166/rtas_indicator.h v8>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000116a:[<0x10000001166/rtas_indicator.h v8>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.608+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001186 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.h auth v64 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb390000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001186:[<0x10000001166/rtas_resource.h v64>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001186:[<0x10000001166/rtas_resource.h v64>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.608+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000117e [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.c auth v48 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb391b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000117e:[<0x10000001166/rtas_annunciator.c v48>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000117e:[<0x10000001166/rtas_annunciator.c v48>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.609+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001178 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.c auth v36 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb391080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001178:[<0x10000001166/rtas_discover.c v36>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001178:[<0x10000001166/rtas_discover.c v36>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.609+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001172 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.c auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb392100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001172:[<0x10000001166/rtas_utils.c v24>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001172:[<0x10000001166/rtas_utils.c v24>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.609+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001180 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.h auth v52 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb392680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001180:[<0x10000001166/rtas_event.h v52>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001180:[<0x10000001166/rtas_event.h v52>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.609+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000117c [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.in auth v44 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb392c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000117c:[<0x10000001166/Makefile.in v44>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000117c:[<0x10000001166/Makefile.in v44>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.609+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001177 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.h auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb393180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001177:[<0x10000001166/rtas_eventlog.h v34>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001177:[<0x10000001166/rtas_eventlog.h v34>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.609+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001181 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.c auth v54 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb393700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001181:[<0x10000001166/rtas_eventlog.c v54>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001181:[<0x10000001166/rtas_eventlog.c v54>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.881 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.609+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000117d [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.h auth v46 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb39e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000117d:[<0x10000001166/rtas_power.h v46>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000117d:[<0x10000001166/rtas_power.h v46>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.610+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000116f [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.h auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb39eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000116f:[<0x10000001166/rtas_annunciator.h v18>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x1000000116f:[<0x10000001166/rtas_annunciator.h v18>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.610+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001169 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.h auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb39e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001169:[<0x10000001166/rtas_hotswap.h v6>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001169:[<0x10000001166/rtas_hotswap.h v6>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.610+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001185 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.am auth v62 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb39f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001185:[<0x10000001166/Makefile.am v62>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001185:[<0x10000001166/Makefile.am v62>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.610+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001174 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.c auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb39fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001174:[<0x10000001166/rtas_inventory.c v28>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001174:[<0x10000001166/rtas_inventory.c v28>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.610+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001176 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.h auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb39f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001176:[<0x10000001166/rtas_sensor.h v32>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001176:[<0x10000001166/rtas_sensor.h v32>,<0x10000000fba/rtas v672>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.611+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001167 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.c auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001167:[<0x10000001166/rtas.c v2>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001167:[<0x10000001166/rtas.c v2>,<0x10000000fba/rtas v672>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.611+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001168 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.c auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001168:[<0x10000001166/rtas_sensor.c v4>,<0x10000000fba/rtas v642>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000001168:[<0x10000001166/rtas_sensor.c v4>,<0x10000000fba/rtas v672>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.611+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc3 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ekeyfru.h auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3c0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc3:[<0x10000000fbb/ekeyfru.h v16>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc3:[<0x10000000fbb/ekeyfru.h v16>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.611+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fbe [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor_event.c auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3c0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fbe:[<0x10000000fbb/ipmi_sensor_event.c v6>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fbe:[<0x10000000fbb/ipmi_sensor_event.c v6>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.612+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fbf [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_close.c auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3c1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fbf:[<0x10000000fbb/ipmi_close.c v8>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fbf:[<0x10000000fbb/ipmi_close.c v8>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.882 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.612+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc1 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_control_event.c auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3c1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc1:[<0x10000000fbb/ipmi_control_event.c v12>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc1:[<0x10000000fbb/ipmi_control_event.c v12>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.612+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc0 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/hotswap.c auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3ce000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc0:[<0x10000000fbb/hotswap.c v10>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc0:[<0x10000000fbb/hotswap.c v10>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.612+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc7 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.sym auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3ce580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc7:[<0x10000000fbb/ipmi.sym v24>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc7:[<0x10000000fbb/ipmi.sym v24>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.612+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fcb [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/posix_vlog.c auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3ceb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fcb:[<0x10000000fbb/posix_vlog.c v32>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fcb:[<0x10000000fbb/posix_vlog.c v32>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.613+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc2 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_connection.c auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3cf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc2:[<0x10000000fbb/ipmi_connection.c v14>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc2:[<0x10000000fbb/ipmi_connection.c v14>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.613+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd2 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.in auth v46 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3cfb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fd2:[<0x10000000fbb/Makefile.in v46>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fd2:[<0x10000000fbb/Makefile.in v46>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.614+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc5 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.h auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3cf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc5:[<0x10000000fbb/ipmi.h v20>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc5:[<0x10000000fbb/ipmi.h v20>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.614+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fbd [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_vshm_rdrs.c auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fbd:[<0x10000000fbb/atca_vshm_rdrs.c v4>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fbd:[<0x10000000fbb/atca_vshm_rdrs.c v4>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.614+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd0 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_mc_event.c auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3d0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fd0:[<0x10000000fbb/ipmi_mc_event.c v42>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fd0:[<0x10000000fbb/ipmi_mc_event.c v42>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.614+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd3 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_fru.c auth v48 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3d0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fd3:[<0x10000000fbb/atca_shelf_fru.c v48>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fd3:[<0x10000000fbb/atca_shelf_fru.c v48>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.883 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.614+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fce [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity.c auth v38 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3d1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fce:[<0x10000000fbb/ipmi_entity.c v38>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fce:[<0x10000000fbb/ipmi_entity.c v38>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.615+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fcc [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.c auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3d1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fcc:[<0x10000000fbb/ipmi.c v34>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fcc:[<0x10000000fbb/ipmi.c v34>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.615+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fdb [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.am auth v64 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fdb:[<0x10000000fbb/Makefile.am v64>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fdb:[<0x10000000fbb/Makefile.am v64>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.615+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fbc [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity_event.c auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fbc:[<0x10000000fbb/ipmi_entity_event.c v2>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fbc:[<0x10000000fbb/ipmi_entity_event.c v2>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.615+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd4 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_rdrs.c auth v50 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fd4:[<0x10000000fbb/atca_shelf_rdrs.c v50>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fd4:[<0x10000000fbb/atca_shelf_rdrs.c v50>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.616+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fca [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory_event.c auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fca:[<0x10000000fbb/ipmi_inventory_event.c v30>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fca:[<0x10000000fbb/ipmi_inventory_event.c v30>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.616+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc6 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory.c auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc6:[<0x10000000fbb/ipmi_inventory.c v22>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc6:[<0x10000000fbb/ipmi_inventory.c v22>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.616+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd6 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_slot_rdrs.c auth v54 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fd6:[<0x10000000fbb/atca_slot_rdrs.c v54>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fd6:[<0x10000000fbb/atca_slot_rdrs.c v54>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.616+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc8 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/sync.c auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc8:[<0x10000000fbb/sync.c v26>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc8:[<0x10000000fbb/sync.c v26>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.616+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd5 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_util.c auth v52 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fd5:[<0x10000000fbb/ipmi_util.c v52>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fd5:[<0x10000000fbb/ipmi_util.c v52>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.617+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc9 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_drv.c auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc9:[<0x10000000fbb/ipmi_drv.c v28>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc9:[<0x10000000fbb/ipmi_drv.c v28>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.884 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.617+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc4 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_controls.c auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc4:[<0x10000000fbb/ipmi_controls.c v18>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fc4:[<0x10000000fbb/ipmi_controls.c v18>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.885 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.617+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fcf [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sel.c auth v40 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3a7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fcf:[<0x10000000fbb/ipmi_sel.c v40>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fcf:[<0x10000000fbb/ipmi_sel.c v40>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.885 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.617+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd1 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_fru_rdrs.c auth v44 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3b0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fd1:[<0x10000000fbb/atca_fru_rdrs.c v44>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fd1:[<0x10000000fbb/atca_fru_rdrs.c v44>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.885 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.617+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fdc [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor.c auth v66 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3b0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fdc:[<0x10000000fbb/ipmi_sensor.c v66>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fdc:[<0x10000000fbb/ipmi_sensor.c v66>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.885 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.617+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fcd [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_event.c auth v36 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3b0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fcd:[<0x10000000fbb/ipmi_event.c v36>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fcd:[<0x10000000fbb/ipmi_event.c v36>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.885 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.618+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd9 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.in auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3bf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fd9:[<0x10000000fd7/Makefile.in v4>,<0x10000000fbb/t v62>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fd9:[<0x10000000fd7/Makefile.in v4>,<0x10000000fbb/t v69>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.885 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.618+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fda [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.am auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3bf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fda:[<0x10000000fd7/Makefile.am v6>,<0x10000000fbb/t v62>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fda:[<0x10000000fd7/Makefile.am v6>,<0x10000000fbb/t v69>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.885 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.618+0000 7fb4aec02700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd8 [2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/ipmi_thres_test.c auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25387=0-4194304@1} caps={25387=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25387 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559fcb3bfb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fd8:[<0x10000000fd7/ipmi_thres_test.c v2>,<0x10000000fbb/t v62>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x10000000fd8:[<0x10000000fd7/ipmi_thres_test.c v2>,<0x10000000fbb/t v69>,<0x10000000fba/ipmi v678>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-02-26T10:47:30.885 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:47:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:47:30.621+0000 7fb4b540f700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fd7 [...2,head] /volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/ auth{0=1} v69 ap=2 RANDEPHEMERALPIN f(v0 m2024-02-26T10:47:21.378834+0000 3=3+0) n(v0 rc2024-02-26T10:47:21.381834+0000 4=3+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25387=pAsLsXsFs/pAsLsXsFsx@3} | lock=3 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 randepin 0x559fcb374100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000fd7:[<0x10000000fbb/t v62>,<0x10000000fba/ipmi v62>,<0x10000000c2d/plugins v152>,<0x10000000204/open_hpi_testsuite v295>,<0x10000000202/testcases v16>,<0x10000000007/ltp-full-20091231 v24>,<0x10000000006/fsstress v18>,<0x10000000005/tmp v8>,<0x100000001fb/client.0 v10>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v40>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000fd7:[<0x10000000fbb/t v69>,<0x10000000fba/ipmi v666>,<0x10000000c2d/plugins v582>,<0x10000000204/open_hpi_testsuite v333>,<0x10000000202/testcases v21>,<0x10000000007/ltp-full-20091231 v28>,<0x10000000006/fsstress v27>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v14>,<0x100000001fa/eb7728f5-bef9-4e6c-a969-ea3331134b72 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/qa v38>,<0x1/volumes v36>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-02-26T10:47:21.378834+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-02-26T10:47:21.378834+0000 4=3+1)","memory_value.dirstat":"f(v0 m2024-02-26T10:47:21.378834+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-02-26T10:47:21.381834+0000 4=3+1)","error_str":""},"return_code":0} 2024-02-26T10:47:31.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub queued for path: / 2024-02-26T10:47:31.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f51 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_006.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f52 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_002.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f5c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_005.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f56 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_010.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f58 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_003.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f5d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_011.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.833 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f5e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_013.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f5b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f59 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_001.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f53 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_009.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f5a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_000.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f55 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_008.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f5f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_004.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f54 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_012.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f91 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_045.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f60 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f93 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_009.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f80 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_002.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f57 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_007.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.834 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f83 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_025.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f9c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_043.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f79 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_036.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f6f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_017.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f77 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_035.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f96 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_023.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f7c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_034.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f7a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_039.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f7d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_013.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f8c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test.h) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f9d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_003.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f89 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_014.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f94 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_040.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f9b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_041.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f71 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_011.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.835 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f97 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_018.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f76 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_032.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f82 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_028.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f8f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f88 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_020.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f98 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_005.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f73 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_010.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f9e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f92 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_042.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f7b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_027.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f87 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_compare.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f74 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_016.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f9a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_029.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f8a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_044.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.836 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f90 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_008.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f86 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_030.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f72 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_026.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f6e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_024.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f99 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_001.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f75 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_012.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f78 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_033.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f7e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_021.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f8b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_006.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f95 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_015.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f8e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_031.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f84 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_019.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f8d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_037.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f85 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_007.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.837 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f70 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_038.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f81 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_004.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f7f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_022.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f4c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f48 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrarea_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f49 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_encode_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f43 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.838 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f4b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpiatca_enum_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f4e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rdr_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f4a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f46 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rptentry_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f44 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrfield_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f45 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_text_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f4d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f4f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_enum_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f47 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_time_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f42 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrinfo_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f68 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_006.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.840 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f64 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_004.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f67 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_005.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f69 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_002.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f65 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_008.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f63 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_003.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f6a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f6b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_001.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f6c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000f66 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_007.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001173 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000117b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000116e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001183 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.841 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000117a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001170 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000116b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001184 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001182 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001175 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001179 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001171 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000116d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000117f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000116c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000116a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001186 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.842 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000117e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001178 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000118b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001172 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001180 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001188 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/README) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000117c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.in) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001177 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000118a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/sysfs2hpi.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001181 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001189 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000117d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x1000000116f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.843 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001169 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001185 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.am) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001174 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001176 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001167 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001168 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc3 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ekeyfru.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fbe (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fbf (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_close.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc1 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_control_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc0 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/hotswap.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc7 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.sym) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fcb (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/posix_vlog.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc2 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_connection.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.844 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd2 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.in) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc5 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fbd (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_vshm_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd0 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_mc_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd3 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_fru.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fce (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fcc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fdb (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.am) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fbc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd4 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fca (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc6 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd6 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_slot_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc8 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/sync.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd5 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_util.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.845 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc9 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_drv.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fc4 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_controls.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fcf (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sel.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd1 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_fru_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fdc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fcd (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd9 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.in) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fda (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.am) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd8 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/ipmi_thres_test.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000000fd7 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.846 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012fe (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/7-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012ff (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/4-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012fb (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/1-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012f8 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/2-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012fa (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/6-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012fc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/8-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012f5 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/3-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012f7 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/10-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012f9 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/5-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.847 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012fd (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/assertions.xml) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x10000001300 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/11-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: Scrub error on inode 0x100000012f6 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/9-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:31.848 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:31 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub queued for path: / 2024-02-26T10:47:31.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f51 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_006.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f52 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_002.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f5c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_005.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f56 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_010.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f58 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_003.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f5d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_011.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f5e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_013.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f5b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f59 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_001.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f53 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_009.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f5a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_000.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f55 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_008.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f5f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_004.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f54 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_012.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f91 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_045.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f60 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f93 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_009.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f80 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_002.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f57 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_007.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f83 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_025.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f9c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_043.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f79 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_036.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f6f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_017.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f77 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_035.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f96 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_023.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f7c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_034.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f7a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_039.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f7d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_013.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f8c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test.h) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f9d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_003.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f89 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_014.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f94 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_040.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f9b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_041.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f71 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_011.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f97 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_018.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f76 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_032.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f82 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_028.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f8f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f88 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_020.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f98 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_005.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f73 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_010.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f9e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f92 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_042.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f7b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_027.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f87 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_compare.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f74 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_016.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f9a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_029.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f8a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_044.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f90 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_008.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f86 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_030.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f72 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_026.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f6e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_024.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f99 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_001.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f75 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_012.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f78 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_033.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f7e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_021.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f8b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_006.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f95 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_015.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f8e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_031.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f84 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_019.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f8d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_037.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f85 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_007.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f70 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_038.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f81 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_004.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f7f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_022.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f4c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f48 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrarea_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f49 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_encode_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f43 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f4b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpiatca_enum_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f4e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rdr_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f4a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f46 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rptentry_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f44 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrfield_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f45 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_text_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f4d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f4f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_enum_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f47 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_time_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f42 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrinfo_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f68 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_006.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f64 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_004.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f67 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_005.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f69 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_002.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f65 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_008.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f63 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_003.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f6a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f6b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_001.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f6c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000f66 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_007.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001173 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000117b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000116e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001183 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000117a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001170 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000116b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001184 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001182 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001175 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.877 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001179 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.877 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001171 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000116d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000117f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000116c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000116a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001186 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000117e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001178 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000118b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001172 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001180 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001188 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/README) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000117c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.in) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001177 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000118a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/sysfs2hpi.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001181 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001189 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000117d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x1000000116f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001169 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001185 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.am) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001174 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001176 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001167 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001168 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc3 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ekeyfru.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fbe (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fbf (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_close.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc1 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_control_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc0 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/hotswap.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc7 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.sym) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fcb (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/posix_vlog.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc2 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_connection.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd2 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.in) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc5 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fbd (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_vshm_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd0 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_mc_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd3 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_fru.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fce (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fcc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fdb (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.am) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fbc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd4 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fca (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc6 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd6 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_slot_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc8 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/sync.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd5 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_util.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc9 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_drv.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fc4 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_controls.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fcf (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sel.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd1 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_fru_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fdc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fcd (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd9 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.in) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fda (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.am) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd8 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/ipmi_thres_test.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000000fd7 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012fe (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/7-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012ff (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/4-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012fb (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/1-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012f8 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/2-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012fa (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/6-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012fc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/8-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012f5 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/3-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012f7 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/10-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012f9 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/5-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012fd (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/assertions.xml) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x10000001300 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/11-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: Scrub error on inode 0x100000012f6 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/9-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:31.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:31 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub queued for path: / 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f51 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_006.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f52 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_002.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f5c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_005.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f56 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_010.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f58 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_003.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f5d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_011.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f5e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_013.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f5b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.886 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f59 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_001.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f53 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_009.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f5a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_000.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f55 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_008.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f5f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_004.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f54 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_012.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f91 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_045.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f60 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f93 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_009.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f80 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_002.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f57 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_007.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f83 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_025.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f9c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_043.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f79 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_036.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.887 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f6f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_017.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f77 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_035.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f96 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_023.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f7c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_034.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f7a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_039.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f7d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_013.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f8c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test.h) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f9d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_003.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f89 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_014.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f94 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_040.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f9b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_041.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f71 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_011.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f97 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_018.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f76 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_032.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f82 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_028.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f8f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f88 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_020.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f98 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_005.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f73 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_010.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f9e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f92 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_042.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f7b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_027.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f87 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_compare.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f74 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_016.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f9a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_029.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f8a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_044.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f90 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_008.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f86 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_030.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f72 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_026.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f6e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_024.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f99 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_001.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f75 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_012.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f78 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_033.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f7e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_021.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f8b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_006.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f95 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_015.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f8e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_031.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f84 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_019.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f8d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_037.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f85 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_007.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f70 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_038.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f81 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_004.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f7f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_022.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f4c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f48 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrarea_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f49 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_encode_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f43 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f4b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpiatca_enum_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f4e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rdr_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f4a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f46 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rptentry_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.891 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f44 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrfield_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f45 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_text_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f4d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f4f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_enum_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f47 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_time_utils_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f42 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrinfo_test.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f68 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_006.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f64 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_004.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f67 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_005.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f69 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_002.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f65 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_008.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f63 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_003.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f6a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f6b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_001.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f6c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000f66 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_007.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001173 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000117b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000116e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001183 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000117a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001170 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000116b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001184 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001182 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001175 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001179 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001171 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000116d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000117f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000116c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000116a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001186 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000117e (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001178 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000118b (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.am) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001172 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001180 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001188 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/README) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000117c (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.in) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001177 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000118a (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/sysfs2hpi.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001181 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001189 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.in) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000117d (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x1000000116f (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001169 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001185 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.am) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001174 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001176 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001167 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001168 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc3 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ekeyfru.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fbe (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fbf (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_close.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc1 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_control_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc0 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/hotswap.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc7 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.sym) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fcb (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/posix_vlog.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc2 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_connection.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd2 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.in) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc5 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.h) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fbd (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_vshm_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd0 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_mc_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.896 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd3 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_fru.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fce (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fcc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fdb (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.am) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fbc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd4 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fca (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc6 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd6 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_slot_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc8 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/sync.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd5 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_util.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc9 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_drv.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fc4 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_controls.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fcf (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sel.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd1 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_fru_rdrs.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fdc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fcd (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_event.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd9 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.in) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fda (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.am) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd8 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/ipmi_thres_test.c) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000000fd7 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t) see mds.h log and `damage ls` output for details 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.898 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012fe (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/7-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012ff (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/4-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012fb (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/1-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012f8 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/2-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012fa (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/6-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012fc (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/8-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012f5 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/3-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012f7 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/10-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012f9 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/5-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012fd (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/assertions.xml) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x10000001300 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/11-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: Scrub error on inode 0x100000012f6 (/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/9-1.c) see mds.i log and `damage ls` output for details 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:31.900 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:31 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: pgmap v472: 129 pgs: 129 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 1.04k op/s 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:32 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:32.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: pgmap v472: 129 pgs: 129 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 1.04k op/s 2024-02-26T10:47:32.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:32.876 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:32.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:32 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: pgmap v472: 129 pgs: 129 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 1.04k op/s 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.083 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:32 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:33.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:33 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:33 smithi100 ceph-mon[36051]: scrub summary: active paths [/] 2024-02-26T10:47:33.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:33 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:33.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:33 smithi100 ceph-mon[36051]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:34.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:33 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:34.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:33 smithi113 ceph-mon[36126]: scrub summary: active paths [/] 2024-02-26T10:47:34.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:33 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:34.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:33 smithi113 ceph-mon[36126]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:34.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:33 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:34.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:33 smithi060 ceph-mon[31539]: scrub summary: active paths [/] 2024-02-26T10:47:34.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:33 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:34.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:33 smithi060 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-02-26T10:47:34.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:34 smithi100 ceph-mon[36051]: pgmap v473: 129 pgs: 129 active+clean; 28 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 1.04k op/s 2024-02-26T10:47:35.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:34 smithi113 ceph-mon[36126]: pgmap v473: 129 pgs: 129 active+clean; 28 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 1.04k op/s 2024-02-26T10:47:35.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:34 smithi060 ceph-mon[31539]: pgmap v473: 129 pgs: 129 active+clean; 28 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 1.04k op/s 2024-02-26T10:47:36.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:35 smithi113 ceph-mon[36126]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T10:47:36.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:35 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] up:active 2024-02-26T10:47:36.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:35 smithi113 ceph-mon[36126]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:36.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:35 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:47:35.630+0000 7fe599800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T10:47:36.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:35 smithi060 ceph-mon[31539]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T10:47:36.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:35 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] up:active 2024-02-26T10:47:36.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:35 smithi060 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:36.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:35 smithi100 ceph-mon[36051]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T10:47:36.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:35 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] up:active 2024-02-26T10:47:36.125 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:35 smithi100 ceph-mon[36051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:37.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:36 smithi113 ceph-mon[36126]: pgmap v474: 129 pgs: 129 active+clean; 31 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 1.21k op/s 2024-02-26T10:47:37.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:36 smithi060 ceph-mon[31539]: pgmap v474: 129 pgs: 129 active+clean; 31 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 1.21k op/s 2024-02-26T10:47:37.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:36 smithi100 ceph-mon[36051]: pgmap v474: 129 pgs: 129 active+clean; 31 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 1.21k op/s 2024-02-26T10:47:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.372+0000 7fefaca00640 1 -- 172.21.15.60:0/173814937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8104460 msgr2=0x7fefa8106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.372+0000 7fefaca00640 1 --2- 172.21.15.60:0/173814937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8104460 0x7fefa8106850 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fef94009900 tx=0x7fef9402bc30 comp rx=0 tx=0).stop 2024-02-26T10:47:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.372+0000 7fefaca00640 1 -- 172.21.15.60:0/173814937 shutdown_connections 2024-02-26T10:47:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.372+0000 7fefaca00640 1 --2- 172.21.15.60:0/173814937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8104460 0x7fefa8106850 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.372+0000 7fefaca00640 1 --2- 172.21.15.60:0/173814937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefa8106d90 0x7fefa81091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.372+0000 7fefaca00640 1 --2- 172.21.15.60:0/173814937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefa8101b30 0x7fefa8103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.372+0000 7fefaca00640 1 -- 172.21.15.60:0/173814937 >> 172.21.15.60:0/173814937 conn(0x7fefa80fb540 msgr2=0x7fefa80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.372+0000 7fefaca00640 1 -- 172.21.15.60:0/173814937 shutdown_connections 2024-02-26T10:47:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.372+0000 7fefaca00640 1 -- 172.21.15.60:0/173814937 wait complete. 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.373+0000 7fefaca00640 1 Processor -- start 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.373+0000 7fefaca00640 1 -- start start 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.373+0000 7fefaca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8101b30 0x7fefa81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.373+0000 7fefaca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefa8106d90 0x7fefa8199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.373+0000 7fefaca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefa819ed80 0x7fefa81a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.373+0000 7fefaca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa810cd90 con 0x7fefa8106d90 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.373+0000 7fefaca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa810cc10 con 0x7fefa8101b30 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.373+0000 7fefaca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa810cf10 con 0x7fefa819ed80 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8101b30 0x7fefa81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8101b30 0x7fefa81997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:36094/0 (socket says 172.21.15.60:36094) 2024-02-26T10:47:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7400640 1 -- 172.21.15.60:0/2019842532 learned_addr learned my addr 172.21.15.60:0/2019842532 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa6a00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefa8106d90 0x7fefa8199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefa819ed80 0x7fefa81a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7400640 1 -- 172.21.15.60:0/2019842532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefa819ed80 msgr2=0x7fefa81a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7400640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefa819ed80 0x7fefa81a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7400640 1 -- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefa8106d90 msgr2=0x7fefa8199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7400640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefa8106d90 0x7fefa8199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7400640 1 -- 172.21.15.60:0/2019842532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa81a17e0 con 0x7fefa8101b30 2024-02-26T10:47:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefa819ed80 0x7fefa81a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:47:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fefa7400640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8101b30 0x7fefa81997e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef9800e9f0 tx=0x7fef9800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.374+0000 7fef9fe00640 1 -- 172.21.15.60:0/2019842532 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9800cd80 con 0x7fefa8101b30 2024-02-26T10:47:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.375+0000 7fef9fe00640 1 -- 172.21.15.60:0/2019842532 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef98010910 con 0x7fefa8101b30 2024-02-26T10:47:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.375+0000 7fefaca00640 1 -- 172.21.15.60:0/2019842532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa81a1ad0 con 0x7fefa8101b30 2024-02-26T10:47:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.375+0000 7fef9fe00640 1 -- 172.21.15.60:0/2019842532 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98018990 con 0x7fefa8101b30 2024-02-26T10:47:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.375+0000 7fefaca00640 1 -- 172.21.15.60:0/2019842532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa81a2040 con 0x7fefa8101b30 2024-02-26T10:47:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.378+0000 7fefb102f740 1 Processor -- start 2024-02-26T10:47:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.378+0000 7fef9fe00640 1 -- 172.21.15.60:0/2019842532 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fef98010ac0 con 0x7fefa8101b30 2024-02-26T10:47:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.378+0000 7fefb102f740 1 -- start start 2024-02-26T10:47:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefb102f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6d9e0 0x555f69e6de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefb102f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6e3d0 0x555f69e72ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefb102f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefb102f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f69dd3340 con 0x555f69e6d9e0 2024-02-26T10:47:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefb102f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f69d7c520 con 0x555f69e73540 2024-02-26T10:47:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefb102f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f69dcf3e0 con 0x555f69e6e3d0 2024-02-26T10:47:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6d9e0 0x555f69e6de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6d9e0 0x555f69e6de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59152/0 (socket says 172.21.15.60:59152) 2024-02-26T10:47:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa7e00640 1 -- 172.21.15.60:0/2598635151 learned_addr learned my addr 172.21.15.60:0/2598635151 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa7400640 1 --2- 172.21.15.60:0/2598635151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6e3d0 0x555f69e72ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa6a00640 1 -- 172.21.15.60:0/2598635151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6e3d0 msgr2=0x555f69e72ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa6a00640 1 --2- 172.21.15.60:0/2598635151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6e3d0 0x555f69e72ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa6a00640 1 -- 172.21.15.60:0/2598635151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6d9e0 msgr2=0x555f69e6de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa6a00640 1 --2- 172.21.15.60:0/2598635151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6d9e0 0x555f69e6de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.379+0000 7fefa6a00640 1 -- 172.21.15.60:0/2598635151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f69d90390 con 0x555f69e73540 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.380+0000 7fefa6a00640 1 --2- 172.21.15.60:0/2598635151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e759d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefa8068c90 tx=0x7fef94009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d633645637b20a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.380+0000 7fef9fe00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef8c0a1df0 0x7fef8c0a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.380+0000 7fef93e00640 1 -- 172.21.15.60:0/2598635151 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9402c980 con 0x555f69e73540 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.380+0000 7fef93e00640 1 -- 172.21.15.60:0/2598635151 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef9402cb20 con 0x555f69e73540 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.380+0000 7fef93e00640 1 -- 172.21.15.60:0/2598635151 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef940046c0 con 0x555f69e73540 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.380+0000 7fef9fe00640 1 -- 172.21.15.60:0/2019842532 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fef980ce510 con 0x7fefa8101b30 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.380+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef8c0a1df0 0x7fef8c0a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefb102f740 1 -- 172.21.15.60:0/2598635151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 msgr2=0x555f69e759d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefb102f740 1 --2- 172.21.15.60:0/2598635151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e759d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefa8068c90 tx=0x7fef94009560 comp rx=0 tx=0).stop 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefb102f740 1 -- 172.21.15.60:0/2598635151 shutdown_connections 2024-02-26T10:47:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefb102f740 1 --2- 172.21.15.60:0/2598635151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e759d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefb102f740 1 --2- 172.21.15.60:0/2598635151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6e3d0 0x555f69e72ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefb102f740 1 --2- 172.21.15.60:0/2598635151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6d9e0 0x555f69e6de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefb102f740 1 -- 172.21.15.60:0/2598635151 >> 172.21.15.60:0/2598635151 conn(0x555f69e68f50 msgr2=0x555f69e69350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef8c0a1df0 0x7fef8c0a42b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x555f69e6ec30 tx=0x7fefa0009790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefb102f740 1 -- 172.21.15.60:0/2598635151 shutdown_connections 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.381+0000 7fefb102f740 1 -- 172.21.15.60:0/2598635151 wait complete. 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.382+0000 7fefb102f740 1 -- start start 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.382+0000 7fefb102f740 1 Processor -- start 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.382+0000 7fefb102f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e77c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.382+0000 7fefb102f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 0x555f69e781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.382+0000 7fefb102f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.382+0000 7fefb102f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f69e76490 con 0x555f69e6e3d0 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.382+0000 7fefb102f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f69d8f980 con 0x555f69e73540 2024-02-26T10:47:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.382+0000 7fefb102f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f69e76610 con 0x555f69e6d9e0 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefb102f740 10 client.0 _ll_register_callbacks cb 0x555f69dcb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 0x555f69e781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:36110/0 (socket says 172.21.15.60:36110) 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7e00640 1 -- 172.21.15.60:0/2389618537 learned_addr learned my addr 172.21.15.60:0/2389618537 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7400640 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e77c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7e00640 1 -- 172.21.15.60:0/2389618537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 msgr2=0x555f69e77c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e77c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7e00640 1 -- 172.21.15.60:0/2389618537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 msgr2=0x555f69e781a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 0x555f69e781a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7e00640 1 -- 172.21.15.60:0/2389618537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f69ae8210 con 0x555f69e73540 2024-02-26T10:47:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7f170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fefa0005fd0 tx=0x7fefa0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db72d714b3455ca9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.383+0000 7fef90c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:47:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.384+0000 7fef90c00640 1 -- 172.21.15.60:0/2389618537 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa000fd20 con 0x555f69e73540 2024-02-26T10:47:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.384+0000 7fef90c00640 1 -- 172.21.15.60:0/2389618537 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fefa0028470 con 0x555f69e73540 2024-02-26T10:47:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.384+0000 7fefb102f740 1 -- 172.21.15.60:0/2389618537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef9402cb20 con 0x555f69e73540 2024-02-26T10:47:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.384+0000 7fef90c00640 1 -- 172.21.15.60:0/2389618537 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefa0031050 con 0x555f69e73540 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.384+0000 7fefb102f740 10 client.26024 fetch_fsmap learned FSMap version 26 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.384+0000 7fefb102f740 1 -- 172.21.15.60:0/2389618537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555f69bb4620 con 0x555f69e73540 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.384+0000 7fef90c00640 1 -- 172.21.15.60:0/2389618537 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef9402cb20 con 0x555f69e73540 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fef90c00640 1 -- 172.21.15.60:0/2389618537 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fefa001e890 con 0x555f69e73540 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fefb102f740 10 client.26024 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fefb102f740 10 client.26024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fefb102f740 20 client.26024 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fefb102f740 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x555f69e83b10 0x555f69e85f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fefb102f740 4 client.26024 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fefb102f740 1 -- 172.21.15.60:0/2389618537 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f69de0880 con 0x555f69e83b10 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fefa7400640 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x555f69e83b10 0x555f69e85f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fefa7400640 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x555f69e83b10 0x555f69e85f30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93d9a8001de4f333 server_cookie=e90f977fd90bca8d in_seq=0 out_seq=0 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.385+0000 7fef90c00640 10 client.26024 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fef90c00640 1 -- 172.21.15.60:0/2389618537 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f69de0880 con 0x555f69e83b10 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fef90c00640 10 client.26024 handle_command_reply: tid=0 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 client.26024 shutdown 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 -- 172.21.15.60:0/2389618537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 msgr2=0x555f69e7f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7f170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fefa0005fd0 tx=0x7fefa0004a40 comp rx=0 tx=0).stop 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 -- 172.21.15.60:0/2389618537 shutdown_connections 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x555f69e83b10 0x555f69e85f30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7f170 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 0x555f69e781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fef90c00640 0 client.26024 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 --2- 172.21.15.60:0/2389618537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e77c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 -- 172.21.15.60:0/2389618537 >> 172.21.15.60:0/2389618537 conn(0x555f69e68f50 msgr2=0x555f69e681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 -- 172.21.15.60:0/2389618537 shutdown_connections 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 1 -- 172.21.15.60:0/2389618537 wait complete. 2024-02-26T10:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.387+0000 7fefb102f740 20 client.26024 trim_cache size 0 max 16384 2024-02-26T10:47:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.403+0000 7fefb102f740 1 Processor -- start 2024-02-26T10:47:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.403+0000 7fefb102f740 1 -- start start 2024-02-26T10:47:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.403+0000 7fefb102f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e77c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.403+0000 7fefb102f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e6e3d0 0x555f69e781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.403+0000 7fefb102f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e73540 0x555f69e7f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefb102f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa81ae850 con 0x555f69e73540 2024-02-26T10:47:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefb102f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa81ac3f0 con 0x555f69e6e3d0 2024-02-26T10:47:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefb102f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa81aa490 con 0x555f69e6d9e0 2024-02-26T10:47:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e73540 0x555f69e7f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e6e3d0 0x555f69e781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e77c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e77c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:50506/0 (socket says 172.21.15.60:50506) 2024-02-26T10:47:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7400640 1 -- 172.21.15.60:0/513311076 learned_addr learned my addr 172.21.15.60:0/513311076 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e73540 0x555f69e7f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59158/0 (socket says 172.21.15.60:59158) 2024-02-26T10:47:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7e00640 1 -- 172.21.15.60:0/513311076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 msgr2=0x555f69e77c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7e00640 1 --2- 172.21.15.60:0/513311076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e77c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7e00640 1 -- 172.21.15.60:0/513311076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e6e3d0 msgr2=0x555f69e781a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7e00640 1 --2- 172.21.15.60:0/513311076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e6e3d0 0x555f69e781a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.404+0000 7fefa7e00640 1 -- 172.21.15.60:0/513311076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f69e6ec60 con 0x555f69e73540 2024-02-26T10:47:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.405+0000 7fefa7e00640 1 --2- 172.21.15.60:0/513311076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e73540 0x555f69e7f130 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fefa001ed30 tx=0x7fefa000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8d0fd42de1677a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.405+0000 7fef93400640 1 -- 172.21.15.60:0/513311076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0039020 con 0x555f69e73540 2024-02-26T10:47:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.405+0000 7fef93400640 1 -- 172.21.15.60:0/513311076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefa0023070 con 0x555f69e73540 2024-02-26T10:47:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.405+0000 7fef93400640 1 -- 172.21.15.60:0/513311076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa001e680 con 0x555f69e73540 2024-02-26T10:47:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 -- 172.21.15.60:0/513311076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e73540 msgr2=0x555f69e7f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 --2- 172.21.15.60:0/513311076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e73540 0x555f69e7f130 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fefa001ed30 tx=0x7fefa000f790 comp rx=0 tx=0).stop 2024-02-26T10:47:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 -- 172.21.15.60:0/513311076 shutdown_connections 2024-02-26T10:47:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 --2- 172.21.15.60:0/513311076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e6e3d0 0x555f69e781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 --2- 172.21.15.60:0/513311076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e73540 0x555f69e7f130 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 --2- 172.21.15.60:0/513311076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e77c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 -- 172.21.15.60:0/513311076 >> 172.21.15.60:0/513311076 conn(0x555f69e68f50 msgr2=0x555f69e69330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 -- 172.21.15.60:0/513311076 shutdown_connections 2024-02-26T10:47:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 -- 172.21.15.60:0/513311076 wait complete. 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.406+0000 7fefb102f740 1 -- start start 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.407+0000 7fefb102f740 1 Processor -- start 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.407+0000 7fefb102f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e7ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.407+0000 7fefb102f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 0x555f69e7d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.407+0000 7fefb102f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.407+0000 7fefb102f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa81aaa90 con 0x555f69e6e3d0 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.407+0000 7fefb102f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa81abd90 con 0x555f69e73540 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.407+0000 7fefb102f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa81ab0d0 con 0x555f69e6d9e0 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.407+0000 7fefb102f740 10 client.0 _ll_register_callbacks cb 0x7fef94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 0x555f69e7d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:36130/0 (socket says 172.21.15.60:36130) 2024-02-26T10:47:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7e00640 1 -- 172.21.15.60:0/2922340431 learned_addr learned my addr 172.21.15.60:0/2922340431 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7400640 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e7ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7e00640 1 -- 172.21.15.60:0/2922340431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 msgr2=0x555f69e7ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e7ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7e00640 1 -- 172.21.15.60:0/2922340431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 msgr2=0x555f69e7d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 0x555f69e7d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7e00640 1 -- 172.21.15.60:0/2922340431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f69e7e180 con 0x555f69e73540 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7400640 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e7ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fefa7e00640 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7da70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefa001ed00 tx=0x7fefa001efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e1fb93b70f55e40 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.408+0000 7fef90c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fef90c00640 1 -- 172.21.15.60:0/2922340431 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0039030 con 0x555f69e73540 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fef90c00640 1 -- 172.21.15.60:0/2922340431 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fefa001e420 con 0x555f69e73540 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fefb102f740 1 -- 172.21.15.60:0/2922340431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefa0023070 con 0x555f69e73540 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fef90c00640 1 -- 172.21.15.60:0/2922340431 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefa0010580 con 0x555f69e73540 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fefb102f740 10 client.26030 fetch_fsmap learned FSMap version 26 2024-02-26T10:47:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fefb102f740 1 -- 172.21.15.60:0/2922340431 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555f69edaf90 con 0x555f69e73540 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fef90c00640 1 -- 172.21.15.60:0/2922340431 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefa0023070 con 0x555f69e73540 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fef90c00640 1 -- 172.21.15.60:0/2922340431 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fefa001fcb0 con 0x555f69e73540 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fefb102f740 10 client.26030 fetch_fsmap finished waiting for FSMap version 26 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fefb102f740 10 client.26030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.409+0000 7fefb102f740 20 client.26030 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.410+0000 7fefb102f740 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x555f69edb860 0x555f69eddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.410+0000 7fefb102f740 4 client.26030 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.410+0000 7fefb102f740 1 -- 172.21.15.60:0/2922340431 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f69ddabd0 con 0x555f69edb860 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.410+0000 7fefa7400640 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x555f69edb860 0x555f69eddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.410+0000 7fefa7400640 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x555f69edb860 0x555f69eddc80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97e4209fa4da479 server_cookie=2831ec63aa65ebca in_seq=0 out_seq=0 2024-02-26T10:47:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:37.410+0000 7fef90c00640 10 client.26030 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:39.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:38 smithi113 ceph-mon[36126]: pgmap v475: 129 pgs: 129 active+clean; 35 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 793 op/s 2024-02-26T10:47:39.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:38 smithi113 ceph-mon[36126]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:39.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:38 smithi060 ceph-mon[31539]: pgmap v475: 129 pgs: 129 active+clean; 35 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 793 op/s 2024-02-26T10:47:39.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:38 smithi060 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:39.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:38 smithi100 ceph-mon[36051]: pgmap v475: 129 pgs: 129 active+clean; 35 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 793 op/s 2024-02-26T10:47:39.125 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:38 smithi100 ceph-mon[36051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:40.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:39 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] up:active 2024-02-26T10:47:40.082 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:39 smithi113 ceph-mon[36126]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:40.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:39 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] up:active 2024-02-26T10:47:40.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:39 smithi060 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:40.125 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:39 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] up:active 2024-02-26T10:47:40.125 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:39 smithi100 ceph-mon[36051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:41.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:40 smithi113 ceph-mon[36126]: pgmap v476: 129 pgs: 129 active+clean; 42 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 874 op/s 2024-02-26T10:47:41.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:40 smithi060 ceph-mon[31539]: pgmap v476: 129 pgs: 129 active+clean; 42 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 874 op/s 2024-02-26T10:47:41.125 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:40 smithi100 ceph-mon[36051]: pgmap v476: 129 pgs: 129 active+clean; 42 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 874 op/s 2024-02-26T10:47:41.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:41 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:47:41.127+0000 7fe599800700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T10:47:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.806+0000 7fef90c00640 1 -- 172.21.15.60:0/2922340431 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f69ddabd0 con 0x555f69edb860 2024-02-26T10:47:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.806+0000 7fef90c00640 10 client.26030 handle_command_reply: tid=0 2024-02-26T10:47:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.806+0000 7fefb102f740 1 client.26030 shutdown 2024-02-26T10:47:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fefb102f740 1 -- 172.21.15.60:0/2922340431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 msgr2=0x555f69e7da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fefb102f740 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7da70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefa001ed00 tx=0x7fefa001efb0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fefb102f740 1 -- 172.21.15.60:0/2922340431 shutdown_connections 2024-02-26T10:47:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fefb102f740 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x555f69edb860 0x555f69eddc80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fefb102f740 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f69e73540 0x555f69e7da70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fefb102f740 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555f69e6e3d0 0x555f69e7d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fefb102f740 1 --2- 172.21.15.60:0/2922340431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555f69e6d9e0 0x555f69e7ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fef90c00640 0 client.26030 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fefb102f740 1 -- 172.21.15.60:0/2922340431 >> 172.21.15.60:0/2922340431 conn(0x555f69e68f50 msgr2=0x555f69e728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.807+0000 7fefb102f740 1 -- 172.21.15.60:0/2922340431 shutdown_connections 2024-02-26T10:47:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.808+0000 7fefb102f740 1 -- 172.21.15.60:0/2922340431 wait complete. 2024-02-26T10:47:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.808+0000 7fefb102f740 20 client.26030 trim_cache size 0 max 16384 2024-02-26T10:47:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:47:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:47:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:47:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:47:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.809+0000 7fefaca00640 1 -- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef8c0a1df0 msgr2=0x7fef8c0a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.809+0000 7fefaca00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef8c0a1df0 0x7fef8c0a42b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x555f69e6ec30 tx=0x7fefa0009790 comp rx=0 tx=0).stop 2024-02-26T10:47:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.809+0000 7fefaca00640 1 -- 172.21.15.60:0/2019842532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8101b30 msgr2=0x7fefa81997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.809+0000 7fefaca00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8101b30 0x7fefa81997e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef9800e9f0 tx=0x7fef9800eeb0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.809+0000 7fefaca00640 1 -- 172.21.15.60:0/2019842532 shutdown_connections 2024-02-26T10:47:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.809+0000 7fefaca00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fef8c0a1df0 0x7fef8c0a42b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.809+0000 7fefaca00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefa819ed80 0x7fefa81a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.809+0000 7fefaca00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefa8106d90 0x7fefa8199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.809+0000 7fefaca00640 1 --2- 172.21.15.60:0/2019842532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefa8101b30 0x7fefa81997e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.810+0000 7fefaca00640 1 -- 172.21.15.60:0/2019842532 >> 172.21.15.60:0/2019842532 conn(0x7fefa80fb540 msgr2=0x7fefa8101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.810+0000 7fefaca00640 1 -- 172.21.15.60:0/2019842532 shutdown_connections 2024-02-26T10:47:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:41.810+0000 7fefaca00640 1 -- 172.21.15.60:0/2019842532 wait complete. 2024-02-26T10:47:42.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:41 smithi113 ceph-mon[36126]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T10:47:42.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:41 smithi113 ceph-mon[36126]: scrub summary: idle 2024-02-26T10:47:42.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:41 smithi060 ceph-mon[31539]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T10:47:42.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:41 smithi060 ceph-mon[31539]: scrub summary: idle 2024-02-26T10:47:42.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:41 smithi100 ceph-mon[36051]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-02-26T10:47:42.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:41 smithi100 ceph-mon[36051]: scrub summary: idle 2024-02-26T10:47:43.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:42 smithi113 ceph-mon[36126]: pgmap v477: 129 pgs: 129 active+clean; 48 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 1.22k op/s 2024-02-26T10:47:43.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:42 smithi113 ceph-mon[36126]: scrub summary: idle 2024-02-26T10:47:43.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:42 smithi113 ceph-mon[36126]: scrub summary: idle 2024-02-26T10:47:43.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:42 smithi060 ceph-mon[31539]: pgmap v477: 129 pgs: 129 active+clean; 48 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 1.22k op/s 2024-02-26T10:47:43.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:42 smithi060 ceph-mon[31539]: scrub summary: idle 2024-02-26T10:47:43.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:42 smithi060 ceph-mon[31539]: scrub summary: idle 2024-02-26T10:47:43.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:42 smithi100 ceph-mon[36051]: pgmap v477: 129 pgs: 129 active+clean; 48 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 1.22k op/s 2024-02-26T10:47:43.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:42 smithi100 ceph-mon[36051]: scrub summary: idle 2024-02-26T10:47:43.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:42 smithi100 ceph-mon[36051]: scrub summary: idle 2024-02-26T10:47:44.838 INFO:tasks.workunit.client.0.smithi060.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-26T10:47:44.841 INFO:tasks.workunit.client.0.smithi060.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-26T10:47:44.841 INFO:tasks.workunit.client.0.smithi060.stderr:+ make 2024-02-26T10:47:45.011 INFO:tasks.workunit.client.0.smithi060.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-26T10:47:45.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:44 smithi113 ceph-mon[36126]: pgmap v478: 129 pgs: 129 active+clean; 49 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 1.61k op/s 2024-02-26T10:47:45.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:44 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:47:45.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:44 smithi060 ceph-mon[31539]: pgmap v478: 129 pgs: 129 active+clean; 49 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 1.61k op/s 2024-02-26T10:47:45.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:44 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:47:45.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:44 smithi100 ceph-mon[36051]: pgmap v478: 129 pgs: 129 active+clean; 49 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 1.61k op/s 2024-02-26T10:47:45.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:44 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:47:45.665 INFO:tasks.workunit.client.0.smithi060.stderr:++ readlink -f fsstress 2024-02-26T10:47:45.667 INFO:tasks.workunit.client.0.smithi060.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-26T10:47:45.667 INFO:tasks.workunit.client.0.smithi060.stderr:+ popd 2024-02-26T10:47:45.667 INFO:tasks.workunit.client.0.smithi060.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-26T10:47:45.667 INFO:tasks.workunit.client.0.smithi060.stderr:+ popd 2024-02-26T10:47:45.667 INFO:tasks.workunit.client.0.smithi060.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-26T10:47:45.668 INFO:tasks.workunit.client.0.smithi060.stderr:++ mktemp -d -p . 2024-02-26T10:47:45.671 INFO:tasks.workunit.client.0.smithi060.stderr:+ T=./tmp.mIOfQxL0FK 2024-02-26T10:47:45.671 INFO:tasks.workunit.client.0.smithi060.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.mIOfQxL0FK -l 1 -n 1000 -p 10 -v 2024-02-26T10:47:45.674 INFO:tasks.workunit.client.0.smithi060.stdout:seed = 1708526344 2024-02-26T10:47:45.680 INFO:tasks.workunit.client.0.smithi060.stdout:6/0: getdents . 0 2024-02-26T10:47:45.680 INFO:tasks.workunit.client.0.smithi060.stdout:6/1: chown . 1025 1 2024-02-26T10:47:45.680 INFO:tasks.workunit.client.0.smithi060.stdout:0/0: mkdir d0 0 2024-02-26T10:47:45.697 INFO:tasks.workunit.client.0.smithi060.stdout:6/2: symlink l0 0 2024-02-26T10:47:45.698 INFO:tasks.workunit.client.0.smithi060.stdout:1/0: mknod c0 0 2024-02-26T10:47:45.698 INFO:tasks.workunit.client.0.smithi060.stdout:1/1: dwrite - no filename 2024-02-26T10:47:45.706 INFO:tasks.workunit.client.0.smithi060.stdout:1/2: chown c0 398 1 2024-02-26T10:47:45.711 INFO:tasks.workunit.client.0.smithi060.stdout:6/3: rename l0 to l1 0 2024-02-26T10:47:45.712 INFO:tasks.workunit.client.0.smithi060.stdout:6/4: read - no filename 2024-02-26T10:47:45.712 INFO:tasks.workunit.client.0.smithi060.stdout:6/5: readlink l1 0 2024-02-26T10:47:45.713 INFO:tasks.workunit.client.0.smithi060.stdout:1/3: symlink l1 0 2024-02-26T10:47:45.713 INFO:tasks.workunit.client.0.smithi060.stdout:1/4: dread - no filename 2024-02-26T10:47:45.713 INFO:tasks.workunit.client.0.smithi060.stdout:1/5: truncate - no filename 2024-02-26T10:47:45.713 INFO:tasks.workunit.client.0.smithi060.stdout:1/6: chown c0 78269 1 2024-02-26T10:47:45.713 INFO:tasks.workunit.client.0.smithi060.stdout:1/7: dread - no filename 2024-02-26T10:47:45.713 INFO:tasks.workunit.client.0.smithi060.stdout:1/8: chown l1 914 1 2024-02-26T10:47:45.714 INFO:tasks.workunit.client.0.smithi060.stdout:6/6: symlink l2 0 2024-02-26T10:47:45.714 INFO:tasks.workunit.client.0.smithi060.stdout:6/7: dwrite - no filename 2024-02-26T10:47:45.762 INFO:tasks.workunit.client.0.smithi060.stdout:1/9: unlink c0 0 2024-02-26T10:47:45.762 INFO:tasks.workunit.client.0.smithi060.stdout:6/8: symlink l3 0 2024-02-26T10:47:45.762 INFO:tasks.workunit.client.0.smithi060.stdout:1/10: dread - no filename 2024-02-26T10:47:45.763 INFO:tasks.workunit.client.0.smithi060.stdout:2/0: creat f0 x:0 0 0 2024-02-26T10:47:45.766 INFO:tasks.workunit.client.0.smithi060.stdout:3/0: write - no filename 2024-02-26T10:47:45.766 INFO:tasks.workunit.client.0.smithi060.stdout:3/1: rmdir - no directory 2024-02-26T10:47:45.766 INFO:tasks.workunit.client.0.smithi060.stdout:3/2: unlink - no file 2024-02-26T10:47:45.767 INFO:tasks.workunit.client.0.smithi060.stdout:3/3: dwrite - no filename 2024-02-26T10:47:45.767 INFO:tasks.workunit.client.0.smithi060.stdout:3/4: chown . 35845 1 2024-02-26T10:47:45.767 INFO:tasks.workunit.client.0.smithi060.stdout:6/9: creat f4 x:0 0 0 2024-02-26T10:47:45.767 INFO:tasks.workunit.client.0.smithi060.stdout:1/11: mknod c2 0 2024-02-26T10:47:45.767 INFO:tasks.workunit.client.0.smithi060.stdout:1/12: write - no filename 2024-02-26T10:47:45.771 INFO:tasks.workunit.client.0.smithi060.stdout:1/13: mkdir d3 0 2024-02-26T10:47:45.771 INFO:tasks.workunit.client.0.smithi060.stdout:5/0: chown . 140178 1 2024-02-26T10:47:45.771 INFO:tasks.workunit.client.0.smithi060.stdout:5/1: dread - no filename 2024-02-26T10:47:45.771 INFO:tasks.workunit.client.0.smithi060.stdout:1/14: dwrite - no filename 2024-02-26T10:47:45.771 INFO:tasks.workunit.client.0.smithi060.stdout:1/15: chown l1 28457 1 2024-02-26T10:47:45.771 INFO:tasks.workunit.client.0.smithi060.stdout:1/16: fsync - no filename 2024-02-26T10:47:45.772 INFO:tasks.workunit.client.0.smithi060.stdout:5/2: mkdir d0 0 2024-02-26T10:47:45.772 INFO:tasks.workunit.client.0.smithi060.stdout:5/3: fsync - no filename 2024-02-26T10:47:45.772 INFO:tasks.workunit.client.0.smithi060.stdout:5/4: read - no filename 2024-02-26T10:47:45.772 INFO:tasks.workunit.client.0.smithi060.stdout:4/0: mkdir d0 0 2024-02-26T10:47:45.772 INFO:tasks.workunit.client.0.smithi060.stdout:4/1: link - no file 2024-02-26T10:47:45.772 INFO:tasks.workunit.client.0.smithi060.stdout:4/2: dread - no filename 2024-02-26T10:47:45.772 INFO:tasks.workunit.client.0.smithi060.stdout:4/3: dread - no filename 2024-02-26T10:47:45.773 INFO:tasks.workunit.client.0.smithi060.stdout:5/5: getdents d0 0 2024-02-26T10:47:45.773 INFO:tasks.workunit.client.0.smithi060.stdout:4/4: chown d0 0 1 2024-02-26T10:47:45.774 INFO:tasks.workunit.client.0.smithi060.stdout:7/0: dwrite - no filename 2024-02-26T10:47:45.774 INFO:tasks.workunit.client.0.smithi060.stdout:7/1: truncate - no filename 2024-02-26T10:47:45.774 INFO:tasks.workunit.client.0.smithi060.stdout:7/2: truncate - no filename 2024-02-26T10:47:45.776 INFO:tasks.workunit.client.0.smithi060.stdout:1/17: unlink c2 0 2024-02-26T10:47:45.776 INFO:tasks.workunit.client.0.smithi060.stdout:1/18: fdatasync - no filename 2024-02-26T10:47:45.776 INFO:tasks.workunit.client.0.smithi060.stdout:1/19: truncate - no filename 2024-02-26T10:47:45.776 INFO:tasks.workunit.client.0.smithi060.stdout:1/20: read - no filename 2024-02-26T10:47:45.776 INFO:tasks.workunit.client.0.smithi060.stdout:1/21: write - no filename 2024-02-26T10:47:45.776 INFO:tasks.workunit.client.0.smithi060.stdout:1/22: read - no filename 2024-02-26T10:47:45.781 INFO:tasks.workunit.client.0.smithi060.stdout:4/5: symlink d0/l1 0 2024-02-26T10:47:45.781 INFO:tasks.workunit.client.0.smithi060.stdout:8/0: chown . 9225125 1 2024-02-26T10:47:45.781 INFO:tasks.workunit.client.0.smithi060.stdout:8/1: dwrite - no filename 2024-02-26T10:47:45.781 INFO:tasks.workunit.client.0.smithi060.stdout:5/6: mknod d0/c1 0 2024-02-26T10:47:45.781 INFO:tasks.workunit.client.0.smithi060.stdout:5/7: write - no filename 2024-02-26T10:47:45.781 INFO:tasks.workunit.client.0.smithi060.stdout:4/6: chown d0 314973892 1 2024-02-26T10:47:45.781 INFO:tasks.workunit.client.0.smithi060.stdout:4/7: chown d0/l1 0 1 2024-02-26T10:47:45.781 INFO:tasks.workunit.client.0.smithi060.stdout:4/8: chown d0 2 1 2024-02-26T10:47:45.781 INFO:tasks.workunit.client.0.smithi060.stdout:4/9: chown d0 3 1 2024-02-26T10:47:45.782 INFO:tasks.workunit.client.0.smithi060.stdout:7/3: creat f0 x:0 0 0 2024-02-26T10:47:45.782 INFO:tasks.workunit.client.0.smithi060.stdout:7/4: read - f0 zero size 2024-02-26T10:47:45.782 INFO:tasks.workunit.client.0.smithi060.stdout:7/5: dread - f0 zero size 2024-02-26T10:47:45.782 INFO:tasks.workunit.client.0.smithi060.stdout:1/23: creat d3/f4 x:0 0 0 2024-02-26T10:47:45.783 INFO:tasks.workunit.client.0.smithi060.stdout:1/24: dread - d3/f4 zero size 2024-02-26T10:47:45.783 INFO:tasks.workunit.client.0.smithi060.stdout:1/25: readlink l1 0 2024-02-26T10:47:45.783 INFO:tasks.workunit.client.0.smithi060.stdout:1/26: dread - d3/f4 zero size 2024-02-26T10:47:45.783 INFO:tasks.workunit.client.0.smithi060.stdout:1/27: fdatasync d3/f4 0 2024-02-26T10:47:45.783 INFO:tasks.workunit.client.0.smithi060.stdout:8/2: creat f0 x:0 0 0 2024-02-26T10:47:45.787 INFO:tasks.workunit.client.0.smithi060.stdout:5/8: rename d0/c1 to d0/c2 0 2024-02-26T10:47:45.787 INFO:tasks.workunit.client.0.smithi060.stdout:9/0: write - no filename 2024-02-26T10:47:45.787 INFO:tasks.workunit.client.0.smithi060.stdout:9/1: write - no filename 2024-02-26T10:47:45.790 INFO:tasks.workunit.client.0.smithi060.stdout:1/28: mknod d3/c5 0 2024-02-26T10:47:45.790 INFO:tasks.workunit.client.0.smithi060.stdout:1/29: read - d3/f4 zero size 2024-02-26T10:47:45.791 INFO:tasks.workunit.client.0.smithi060.stdout:9/2: symlink l0 0 2024-02-26T10:47:45.791 INFO:tasks.workunit.client.0.smithi060.stdout:7/6: link f0 f1 0 2024-02-26T10:47:45.791 INFO:tasks.workunit.client.0.smithi060.stdout:7/7: dread - f1 zero size 2024-02-26T10:47:45.792 INFO:tasks.workunit.client.0.smithi060.stdout:5/9: rmdir d0 39 2024-02-26T10:47:45.792 INFO:tasks.workunit.client.0.smithi060.stdout:5/10: dread - no filename 2024-02-26T10:47:45.792 INFO:tasks.workunit.client.0.smithi060.stdout:5/11: write - no filename 2024-02-26T10:47:45.792 INFO:tasks.workunit.client.0.smithi060.stdout:5/12: dread - no filename 2024-02-26T10:47:45.794 INFO:tasks.workunit.client.0.smithi060.stdout:9/3: link l0 l1 0 2024-02-26T10:47:45.794 INFO:tasks.workunit.client.0.smithi060.stdout:9/4: read - no filename 2024-02-26T10:47:45.794 INFO:tasks.workunit.client.0.smithi060.stdout:9/5: dread - no filename 2024-02-26T10:47:45.794 INFO:tasks.workunit.client.0.smithi060.stdout:9/6: dread - no filename 2024-02-26T10:47:45.795 INFO:tasks.workunit.client.0.smithi060.stdout:7/8: rename f1 to f2 0 2024-02-26T10:47:45.795 INFO:tasks.workunit.client.0.smithi060.stdout:7/9: write f0 [426896,79831] 0 2024-02-26T10:47:45.795 INFO:tasks.workunit.client.0.smithi060.stdout:7/10: chown f0 16334635 1 2024-02-26T10:47:45.795 INFO:tasks.workunit.client.0.smithi060.stdout:7/11: chown f2 10 1 2024-02-26T10:47:45.797 INFO:tasks.workunit.client.0.smithi060.stdout:5/13: mknod d0/c3 0 2024-02-26T10:47:45.797 INFO:tasks.workunit.client.0.smithi060.stdout:5/14: chown d0/c2 2510581 1 2024-02-26T10:47:45.797 INFO:tasks.workunit.client.0.smithi060.stdout:5/15: dread - no filename 2024-02-26T10:47:45.809 INFO:tasks.workunit.client.0.smithi060.stdout:5/16: creat d0/f4 x:0 0 0 2024-02-26T10:47:45.935 INFO:tasks.workunit.client.0.smithi060.stdout:6/10: fsync f4 0 2024-02-26T10:47:45.935 INFO:tasks.workunit.client.0.smithi060.stdout:6/11: dread - f4 zero size 2024-02-26T10:47:45.938 INFO:tasks.workunit.client.0.smithi060.stdout:6/12: creat f5 x:0 0 0 2024-02-26T10:47:45.938 INFO:tasks.workunit.client.0.smithi060.stdout:6/13: truncate f5 946710 0 2024-02-26T10:47:45.940 INFO:tasks.workunit.client.0.smithi060.stdout:6/14: mkdir d6 0 2024-02-26T10:47:45.945 INFO:tasks.workunit.client.0.smithi060.stdout:6/15: mkdir d6/d7 0 2024-02-26T10:47:45.945 INFO:tasks.workunit.client.0.smithi060.stdout:6/16: truncate f5 1794833 0 2024-02-26T10:47:45.948 INFO:tasks.workunit.client.0.smithi060.stdout:6/17: rename l2 to d6/d7/l8 0 2024-02-26T10:47:45.979 INFO:tasks.workunit.client.0.smithi060.stdout:4/10: getdents d0 0 2024-02-26T10:47:45.979 INFO:tasks.workunit.client.0.smithi060.stdout:4/11: fsync - no filename 2024-02-26T10:47:45.979 INFO:tasks.workunit.client.0.smithi060.stdout:4/12: dread - no filename 2024-02-26T10:47:45.979 INFO:tasks.workunit.client.0.smithi060.stdout:4/13: dwrite - no filename 2024-02-26T10:47:45.979 INFO:tasks.workunit.client.0.smithi060.stdout:4/14: stat d0/l1 0 2024-02-26T10:47:45.981 INFO:tasks.workunit.client.0.smithi060.stdout:1/30: fsync d3/f4 0 2024-02-26T10:47:45.987 INFO:tasks.workunit.client.0.smithi060.stdout:4/15: symlink d0/l2 0 2024-02-26T10:47:45.987 INFO:tasks.workunit.client.0.smithi060.stdout:4/16: write - no filename 2024-02-26T10:47:45.987 INFO:tasks.workunit.client.0.smithi060.stdout:4/17: dwrite - no filename 2024-02-26T10:47:45.990 INFO:tasks.workunit.client.0.smithi060.stdout:4/18: unlink d0/l2 0 2024-02-26T10:47:45.996 INFO:tasks.workunit.client.0.smithi060.stdout:4/19: creat d0/f3 x:0 0 0 2024-02-26T10:47:46.002 INFO:tasks.workunit.client.0.smithi060.stdout:4/20: mknod d0/c4 0 2024-02-26T10:47:46.002 INFO:tasks.workunit.client.0.smithi060.stdout:4/21: creat d0/f5 x:0 0 0 2024-02-26T10:47:46.008 INFO:tasks.workunit.client.0.smithi060.stdout:4/22: link d0/f3 d0/f6 0 2024-02-26T10:47:46.008 INFO:tasks.workunit.client.0.smithi060.stdout:4/23: fdatasync d0/f6 0 2024-02-26T10:47:46.009 INFO:tasks.workunit.client.0.smithi060.stdout:4/24: write d0/f3 [119612,124216] 0 2024-02-26T10:47:46.024 INFO:tasks.workunit.client.0.smithi060.stdout:2/1: dwrite f0 [0,4194304] 0 2024-02-26T10:47:46.036 INFO:tasks.workunit.client.0.smithi060.stdout:8/3: dwrite f0 [0,4194304] 0 2024-02-26T10:47:46.111 INFO:tasks.workunit.client.0.smithi060.stdout:6/18: getdents d6 0 2024-02-26T10:47:46.119 INFO:tasks.workunit.client.0.smithi060.stdout:6/19: creat d6/f9 x:0 0 0 2024-02-26T10:47:46.120 INFO:tasks.workunit.client.0.smithi060.stdout:6/20: chown f5 53 1 2024-02-26T10:47:46.120 INFO:tasks.workunit.client.0.smithi060.stdout:6/21: rename d6/d7 to d6/d7/da 22 2024-02-26T10:47:46.120 INFO:tasks.workunit.client.0.smithi060.stdout:6/22: rmdir d6/d7 39 2024-02-26T10:47:46.120 INFO:tasks.workunit.client.0.smithi060.stdout:6/23: chown d6/f9 766094 1 2024-02-26T10:47:46.129 INFO:tasks.workunit.client.0.smithi060.stdout:8/4: write f0 [5006668,91595] 0 2024-02-26T10:47:46.133 INFO:tasks.workunit.client.0.smithi060.stdout:8/5: mknod c1 0 2024-02-26T10:47:46.138 INFO:tasks.workunit.client.0.smithi060.stdout:2/2: write f0 [2092693,93979] 0 2024-02-26T10:47:46.200 INFO:tasks.workunit.client.0.smithi060.stdout:6/24: fsync d6/f9 0 2024-02-26T10:47:46.285 INFO:tasks.workunit.client.0.smithi060.stdout:1/31: dwrite d3/f4 [0,4194304] 0 2024-02-26T10:47:46.298 INFO:tasks.workunit.client.0.smithi060.stdout:8/6: write f0 [4743285,28594] 0 2024-02-26T10:47:46.309 INFO:tasks.workunit.client.0.smithi060.stdout:1/32: creat d3/f6 x:0 0 0 2024-02-26T10:47:46.310 INFO:tasks.workunit.client.0.smithi060.stdout:1/33: fdatasync d3/f4 0 2024-02-26T10:47:46.325 INFO:tasks.workunit.client.0.smithi060.stdout:8/7: creat f2 x:0 0 0 2024-02-26T10:47:46.349 INFO:tasks.workunit.client.0.smithi060.stdout:1/34: link d3/f4 d3/f7 0 2024-02-26T10:47:46.349 INFO:tasks.workunit.client.0.smithi060.stdout:1/35: truncate d3/f6 65104 0 2024-02-26T10:47:46.376 INFO:tasks.workunit.client.0.smithi060.stdout:7/12: fdatasync f2 0 2024-02-26T10:47:46.377 INFO:tasks.workunit.client.0.smithi060.stdout:7/13: stat f2 0 2024-02-26T10:47:46.385 INFO:tasks.workunit.client.0.smithi060.stdout:1/36: write d3/f4 [663180,51178] 0 2024-02-26T10:47:46.417 INFO:tasks.workunit.client.0.smithi060.stdout:6/25: dwrite d6/f9 [0,4194304] 0 2024-02-26T10:47:46.417 INFO:tasks.workunit.client.0.smithi060.stdout:6/26: dread - f4 zero size 2024-02-26T10:47:46.417 INFO:tasks.workunit.client.0.smithi060.stdout:6/27: truncate f5 1953523 0 2024-02-26T10:47:46.418 INFO:tasks.workunit.client.0.smithi060.stdout:1/37: rename d3/f4 to d3/f8 0 2024-02-26T10:47:46.431 INFO:tasks.workunit.client.0.smithi060.stdout:6/28: symlink d6/lb 0 2024-02-26T10:47:46.432 INFO:tasks.workunit.client.0.smithi060.stdout:6/29: truncate f4 1048402 0 2024-02-26T10:47:46.436 INFO:tasks.workunit.client.0.smithi060.stdout:1/38: creat d3/f9 x:0 0 0 2024-02-26T10:47:46.441 INFO:tasks.workunit.client.0.smithi060.stdout:1/39: dread d3/f6 [0,4194304] 0 2024-02-26T10:47:46.464 INFO:tasks.workunit.client.0.smithi060.stdout:6/30: rename f5 to d6/d7/fc 0 2024-02-26T10:47:46.524 INFO:tasks.workunit.client.0.smithi060.stdout:7/14: dread f0 [0,4194304] 0 2024-02-26T10:47:46.530 INFO:tasks.workunit.client.0.smithi060.stdout:8/8: dread f0 [0,4194304] 0 2024-02-26T10:47:46.533 INFO:tasks.workunit.client.0.smithi060.stdout:6/31: truncate d6/f9 2002439 0 2024-02-26T10:47:46.533 INFO:tasks.workunit.client.0.smithi060.stdout:6/32: readlink d6/lb 0 2024-02-26T10:47:46.534 INFO:tasks.workunit.client.0.smithi060.stdout:6/33: write d6/d7/fc [2385407,85727] 0 2024-02-26T10:47:46.535 INFO:tasks.workunit.client.0.smithi060.stdout:5/17: dwrite d0/f4 [0,4194304] 0 2024-02-26T10:47:46.553 INFO:tasks.workunit.client.0.smithi060.stdout:4/25: dwrite d0/f6 [0,4194304] 0 2024-02-26T10:47:46.555 INFO:tasks.workunit.client.0.smithi060.stdout:2/3: dwrite f0 [4194304,4194304] 0 2024-02-26T10:47:46.575 INFO:tasks.workunit.client.0.smithi060.stdout:7/15: symlink l3 0 2024-02-26T10:47:46.578 INFO:tasks.workunit.client.0.smithi060.stdout:8/9: creat f3 x:0 0 0 2024-02-26T10:47:46.578 INFO:tasks.workunit.client.0.smithi060.stdout:8/10: chown f3 43590559 1 2024-02-26T10:47:46.578 INFO:tasks.workunit.client.0.smithi060.stdout:8/11: readlink - no filename 2024-02-26T10:47:46.581 INFO:tasks.workunit.client.0.smithi060.stdout:6/34: creat d6/d7/fd x:0 0 0 2024-02-26T10:47:46.581 INFO:tasks.workunit.client.0.smithi060.stdout:6/35: truncate d6/d7/fd 845318 0 2024-02-26T10:47:46.584 INFO:tasks.workunit.client.0.smithi060.stdout:7/16: dread f2 [0,4194304] 0 2024-02-26T10:47:46.584 INFO:tasks.workunit.client.0.smithi060.stdout:7/17: fdatasync f2 0 2024-02-26T10:47:46.584 INFO:tasks.workunit.client.0.smithi060.stdout:7/18: fsync f2 0 2024-02-26T10:47:46.585 INFO:tasks.workunit.client.0.smithi060.stdout:5/18: link d0/f4 d0/f5 0 2024-02-26T10:47:46.587 INFO:tasks.workunit.client.0.smithi060.stdout:7/19: read f0 [100190,52920] 0 2024-02-26T10:47:46.587 INFO:tasks.workunit.client.0.smithi060.stdout:7/20: rmdir - no directory 2024-02-26T10:47:46.588 INFO:tasks.workunit.client.0.smithi060.stdout:4/26: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:46.596 INFO:tasks.workunit.client.0.smithi060.stdout:2/4: dwrite f0 [0,4194304] 0 2024-02-26T10:47:46.611 INFO:tasks.workunit.client.0.smithi060.stdout:8/12: dwrite f3 [0,4194304] 0 2024-02-26T10:47:46.612 INFO:tasks.workunit.client.0.smithi060.stdout:8/13: chown f0 68 1 2024-02-26T10:47:46.622 INFO:tasks.workunit.client.0.smithi060.stdout:6/36: creat d6/fe x:0 0 0 2024-02-26T10:47:46.625 INFO:tasks.workunit.client.0.smithi060.stdout:7/21: rename f0 to f4 0 2024-02-26T10:47:46.628 INFO:tasks.workunit.client.0.smithi060.stdout:4/27: mkdir d0/d7 0 2024-02-26T10:47:46.630 INFO:tasks.workunit.client.0.smithi060.stdout:2/5: getdents . 0 2024-02-26T10:47:46.634 INFO:tasks.workunit.client.0.smithi060.stdout:6/37: creat d6/d7/ff x:0 0 0 2024-02-26T10:47:46.635 INFO:tasks.workunit.client.0.smithi060.stdout:7/22: symlink l5 0 2024-02-26T10:47:46.636 INFO:tasks.workunit.client.0.smithi060.stdout:4/28: creat d0/d7/f8 x:0 0 0 2024-02-26T10:47:46.636 INFO:tasks.workunit.client.0.smithi060.stdout:8/14: dwrite f3 [0,4194304] 0 2024-02-26T10:47:46.638 INFO:tasks.workunit.client.0.smithi060.stdout:4/29: mknod d0/d7/c9 0 2024-02-26T10:47:46.639 INFO:tasks.workunit.client.0.smithi060.stdout:4/30: mkdir d0/da 0 2024-02-26T10:47:46.643 INFO:tasks.workunit.client.0.smithi060.stdout:7/23: dread f4 [0,4194304] 0 2024-02-26T10:47:46.644 INFO:tasks.workunit.client.0.smithi060.stdout:7/24: creat f6 x:0 0 0 2024-02-26T10:47:46.645 INFO:tasks.workunit.client.0.smithi060.stdout:7/25: creat f7 x:0 0 0 2024-02-26T10:47:46.646 INFO:tasks.workunit.client.0.smithi060.stdout:7/26: symlink l8 0 2024-02-26T10:47:46.648 INFO:tasks.workunit.client.0.smithi060.stdout:7/27: creat f9 x:0 0 0 2024-02-26T10:47:46.648 INFO:tasks.workunit.client.0.smithi060.stdout:7/28: rmdir - no directory 2024-02-26T10:47:46.650 INFO:tasks.workunit.client.0.smithi060.stdout:7/29: mkdir da 0 2024-02-26T10:47:46.651 INFO:tasks.workunit.client.0.smithi060.stdout:6/38: write d6/f9 [1111083,45054] 0 2024-02-26T10:47:46.651 INFO:tasks.workunit.client.0.smithi060.stdout:7/30: mknod da/cb 0 2024-02-26T10:47:46.651 INFO:tasks.workunit.client.0.smithi060.stdout:7/31: truncate f4 681393 0 2024-02-26T10:47:46.651 INFO:tasks.workunit.client.0.smithi060.stdout:2/6: write f0 [1790041,20618] 0 2024-02-26T10:47:46.656 INFO:tasks.workunit.client.0.smithi060.stdout:6/39: link d6/d7/fc d6/d7/f10 0 2024-02-26T10:47:46.660 INFO:tasks.workunit.client.0.smithi060.stdout:6/40: unlink d6/d7/l8 0 2024-02-26T10:47:46.660 INFO:tasks.workunit.client.0.smithi060.stdout:6/41: chown d6 2 1 2024-02-26T10:47:46.661 INFO:tasks.workunit.client.0.smithi060.stdout:8/15: dread f3 [0,4194304] 0 2024-02-26T10:47:46.664 INFO:tasks.workunit.client.0.smithi060.stdout:5/19: getdents d0 0 2024-02-26T10:47:46.678 INFO:tasks.workunit.client.0.smithi060.stdout:7/32: unlink f4 0 2024-02-26T10:47:46.680 INFO:tasks.workunit.client.0.smithi060.stdout:7/33: rename f9 to da/fc 0 2024-02-26T10:47:46.682 INFO:tasks.workunit.client.0.smithi060.stdout:7/34: creat da/fd x:0 0 0 2024-02-26T10:47:46.690 INFO:tasks.workunit.client.0.smithi060.stdout:4/31: rmdir d0/d7 39 2024-02-26T10:47:46.693 INFO:tasks.workunit.client.0.smithi060.stdout:8/16: dread f3 [0,4194304] 0 2024-02-26T10:47:46.693 INFO:tasks.workunit.client.0.smithi060.stdout:4/32: creat d0/fb x:0 0 0 2024-02-26T10:47:46.693 INFO:tasks.workunit.client.0.smithi060.stdout:4/33: stat d0/d7/c9 0 2024-02-26T10:47:46.693 INFO:tasks.workunit.client.0.smithi060.stdout:4/34: dread - d0/d7/f8 zero size 2024-02-26T10:47:46.693 INFO:tasks.workunit.client.0.smithi060.stdout:4/35: truncate d0/d7/f8 868743 0 2024-02-26T10:47:46.694 INFO:tasks.workunit.client.0.smithi060.stdout:8/17: creat f4 x:0 0 0 2024-02-26T10:47:46.694 INFO:tasks.workunit.client.0.smithi060.stdout:4/36: mkdir d0/da/dc 0 2024-02-26T10:47:46.700 INFO:tasks.workunit.client.0.smithi060.stdout:2/7: dwrite f0 [4194304,4194304] 0 2024-02-26T10:47:46.708 INFO:tasks.workunit.client.0.smithi060.stdout:7/35: truncate f2 298226 0 2024-02-26T10:47:46.712 INFO:tasks.workunit.client.0.smithi060.stdout:7/36: fsync f6 0 2024-02-26T10:47:46.719 INFO:tasks.workunit.client.0.smithi060.stdout:7/37: creat da/fe x:0 0 0 2024-02-26T10:47:46.720 INFO:tasks.workunit.client.0.smithi060.stdout:7/38: mknod da/cf 0 2024-02-26T10:47:46.721 INFO:tasks.workunit.client.0.smithi060.stdout:7/39: mknod da/c10 0 2024-02-26T10:47:46.722 INFO:tasks.workunit.client.0.smithi060.stdout:7/40: symlink da/l11 0 2024-02-26T10:47:46.722 INFO:tasks.workunit.client.0.smithi060.stdout:7/41: stat da/cb 0 2024-02-26T10:47:46.722 INFO:tasks.workunit.client.0.smithi060.stdout:7/42: chown f6 250528 1 2024-02-26T10:47:46.723 INFO:tasks.workunit.client.0.smithi060.stdout:7/43: creat da/f12 x:0 0 0 2024-02-26T10:47:46.725 INFO:tasks.workunit.client.0.smithi060.stdout:7/44: unlink da/f12 0 2024-02-26T10:47:46.727 INFO:tasks.workunit.client.0.smithi060.stdout:2/8: dread f0 [0,4194304] 0 2024-02-26T10:47:46.734 INFO:tasks.workunit.client.0.smithi060.stdout:8/18: dwrite f0 [0,4194304] 0 2024-02-26T10:47:46.734 INFO:tasks.workunit.client.0.smithi060.stdout:8/19: readlink - no filename 2024-02-26T10:47:46.736 INFO:tasks.workunit.client.0.smithi060.stdout:8/20: getdents . 0 2024-02-26T10:47:46.736 INFO:tasks.workunit.client.0.smithi060.stdout:8/21: dread - f2 zero size 2024-02-26T10:47:46.738 INFO:tasks.workunit.client.0.smithi060.stdout:4/37: dread d0/f3 [0,4194304] 0 2024-02-26T10:47:46.738 INFO:tasks.workunit.client.0.smithi060.stdout:5/20: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:46.738 INFO:tasks.workunit.client.0.smithi060.stdout:4/38: truncate d0/d7/f8 1361110 0 2024-02-26T10:47:46.738 INFO:tasks.workunit.client.0.smithi060.stdout:4/39: readlink d0/l1 0 2024-02-26T10:47:46.738 INFO:tasks.workunit.client.0.smithi060.stdout:4/40: chown d0/c4 81741779 1 2024-02-26T10:47:46.742 INFO:tasks.workunit.client.0.smithi060.stdout:5/21: mkdir d0/d6 0 2024-02-26T10:47:46.754 INFO:tasks.workunit.client.0.smithi060.stdout:2/9: write f0 [8319071,14574] 0 2024-02-26T10:47:46.758 INFO:tasks.workunit.client.0.smithi060.stdout:2/10: creat f1 x:0 0 0 2024-02-26T10:47:46.758 INFO:tasks.workunit.client.0.smithi060.stdout:2/11: dread - f1 zero size 2024-02-26T10:47:46.767 INFO:tasks.workunit.client.0.smithi060.stdout:4/41: write d0/f3 [2939774,40050] 0 2024-02-26T10:47:46.781 INFO:tasks.workunit.client.0.smithi060.stdout:8/22: dwrite f2 [0,4194304] 0 2024-02-26T10:47:46.781 INFO:tasks.workunit.client.0.smithi060.stdout:8/23: write f4 [345581,66431] 0 2024-02-26T10:47:46.785 INFO:tasks.workunit.client.0.smithi060.stdout:8/24: rename f4 to f5 0 2024-02-26T10:47:46.787 INFO:tasks.workunit.client.0.smithi060.stdout:8/25: mknod c6 0 2024-02-26T10:47:46.789 INFO:tasks.workunit.client.0.smithi060.stdout:8/26: symlink l7 0 2024-02-26T10:47:46.789 INFO:tasks.workunit.client.0.smithi060.stdout:8/27: readlink l7 0 2024-02-26T10:47:46.790 INFO:tasks.workunit.client.0.smithi060.stdout:7/45: dwrite da/fd [0,4194304] 0 2024-02-26T10:47:46.790 INFO:tasks.workunit.client.0.smithi060.stdout:7/46: chown da/l11 48 1 2024-02-26T10:47:46.793 INFO:tasks.workunit.client.0.smithi060.stdout:7/47: creat da/f13 x:0 0 0 2024-02-26T10:47:46.795 INFO:tasks.workunit.client.0.smithi060.stdout:7/48: mkdir da/d14 0 2024-02-26T10:47:46.795 INFO:tasks.workunit.client.0.smithi060.stdout:7/49: write da/f13 [680920,28637] 0 2024-02-26T10:47:46.813 INFO:tasks.workunit.client.0.smithi060.stdout:5/22: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:46.813 INFO:tasks.workunit.client.0.smithi060.stdout:5/23: fdatasync d0/f4 0 2024-02-26T10:47:46.819 INFO:tasks.workunit.client.0.smithi060.stdout:5/24: symlink d0/l7 0 2024-02-26T10:47:46.819 INFO:tasks.workunit.client.0.smithi060.stdout:5/25: chown d0/f5 39805 1 2024-02-26T10:47:46.821 INFO:tasks.workunit.client.0.smithi060.stdout:8/28: dread f5 [0,4194304] 0 2024-02-26T10:47:46.823 INFO:tasks.workunit.client.0.smithi060.stdout:5/26: creat d0/f8 x:0 0 0 2024-02-26T10:47:46.823 INFO:tasks.workunit.client.0.smithi060.stdout:2/12: dwrite f0 [0,4194304] 0 2024-02-26T10:47:46.823 INFO:tasks.workunit.client.0.smithi060.stdout:2/13: fsync f1 0 2024-02-26T10:47:46.824 INFO:tasks.workunit.client.0.smithi060.stdout:2/14: creat f2 x:0 0 0 2024-02-26T10:47:46.840 INFO:tasks.workunit.client.0.smithi060.stdout:4/42: dwrite d0/f3 [0,4194304] 0 2024-02-26T10:47:46.842 INFO:tasks.workunit.client.0.smithi060.stdout:4/43: rename d0/c4 to d0/cd 0 2024-02-26T10:47:46.843 INFO:tasks.workunit.client.0.smithi060.stdout:4/44: creat d0/da/dc/fe x:0 0 0 2024-02-26T10:47:46.857 INFO:tasks.workunit.client.0.smithi060.stdout:7/50: dwrite da/f13 [0,4194304] 0 2024-02-26T10:47:46.857 INFO:tasks.workunit.client.0.smithi060.stdout:7/51: chown da/cf 4 1 2024-02-26T10:47:46.860 INFO:tasks.workunit.client.0.smithi060.stdout:7/52: truncate f7 225293 0 2024-02-26T10:47:46.862 INFO:tasks.workunit.client.0.smithi060.stdout:7/53: rename da/d14 to da/d15 0 2024-02-26T10:47:46.865 INFO:tasks.workunit.client.0.smithi060.stdout:2/15: dwrite f1 [0,4194304] 0 2024-02-26T10:47:46.867 INFO:tasks.workunit.client.0.smithi060.stdout:8/29: write f2 [1699030,12478] 0 2024-02-26T10:47:46.867 INFO:tasks.workunit.client.0.smithi060.stdout:4/45: read d0/d7/f8 [805003,84736] 0 2024-02-26T10:47:46.869 INFO:tasks.workunit.client.0.smithi060.stdout:5/27: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:46.869 INFO:tasks.workunit.client.0.smithi060.stdout:5/28: readlink d0/l7 0 2024-02-26T10:47:46.870 INFO:tasks.workunit.client.0.smithi060.stdout:2/16: getdents . 0 2024-02-26T10:47:46.871 INFO:tasks.workunit.client.0.smithi060.stdout:2/17: truncate f2 309515 0 2024-02-26T10:47:46.872 INFO:tasks.workunit.client.0.smithi060.stdout:5/29: rmdir d0/d6 0 2024-02-26T10:47:46.874 INFO:tasks.workunit.client.0.smithi060.stdout:2/18: unlink f1 0 2024-02-26T10:47:46.874 INFO:tasks.workunit.client.0.smithi060.stdout:5/30: creat d0/f9 x:0 0 0 2024-02-26T10:47:46.876 INFO:tasks.workunit.client.0.smithi060.stdout:2/19: rename f2 to f3 0 2024-02-26T10:47:46.878 INFO:tasks.workunit.client.0.smithi060.stdout:5/31: rename d0/c2 to d0/ca 0 2024-02-26T10:47:46.879 INFO:tasks.workunit.client.0.smithi060.stdout:2/20: creat f4 x:0 0 0 2024-02-26T10:47:46.879 INFO:tasks.workunit.client.0.smithi060.stdout:2/21: truncate f4 18827 0 2024-02-26T10:47:46.880 INFO:tasks.workunit.client.0.smithi060.stdout:5/32: creat d0/fb x:0 0 0 2024-02-26T10:47:46.890 INFO:tasks.workunit.client.0.smithi060.stdout:2/22: dread f3 [0,4194304] 0 2024-02-26T10:47:46.890 INFO:tasks.workunit.client.0.smithi060.stdout:2/23: write f4 [223607,10132] 0 2024-02-26T10:47:46.891 INFO:tasks.workunit.client.0.smithi060.stdout:8/30: dread f3 [0,4194304] 0 2024-02-26T10:47:46.891 INFO:tasks.workunit.client.0.smithi060.stdout:8/31: fdatasync f0 0 2024-02-26T10:47:46.892 INFO:tasks.workunit.client.0.smithi060.stdout:2/24: link f0 f5 0 2024-02-26T10:47:46.894 INFO:tasks.workunit.client.0.smithi060.stdout:2/25: link f5 f6 0 2024-02-26T10:47:46.894 INFO:tasks.workunit.client.0.smithi060.stdout:2/26: chown f5 241830 1 2024-02-26T10:47:46.915 INFO:tasks.workunit.client.0.smithi060.stdout:8/32: write f3 [2544290,98101] 0 2024-02-26T10:47:46.915 INFO:tasks.workunit.client.0.smithi060.stdout:8/33: read f2 [7690,91579] 0 2024-02-26T10:47:46.915 INFO:tasks.workunit.client.0.smithi060.stdout:8/34: fdatasync f5 0 2024-02-26T10:47:46.923 INFO:tasks.workunit.client.0.smithi060.stdout:4/46: read d0/f6 [2193870,45356] 0 2024-02-26T10:47:46.924 INFO:tasks.workunit.client.0.smithi060.stdout:7/54: dwrite da/fc [0,4194304] 0 2024-02-26T10:47:46.924 INFO:tasks.workunit.client.0.smithi060.stdout:7/55: chown da 876597487 1 2024-02-26T10:47:46.929 INFO:tasks.workunit.client.0.smithi060.stdout:7/56: symlink da/d15/l16 0 2024-02-26T10:47:46.929 INFO:tasks.workunit.client.0.smithi060.stdout:7/57: chown da/cb 3 1 2024-02-26T10:47:46.929 INFO:tasks.workunit.client.0.smithi060.stdout:7/58: fdatasync da/fc 0 2024-02-26T10:47:46.931 INFO:tasks.workunit.client.0.smithi060.stdout:8/35: dread f0 [4194304,4194304] 0 2024-02-26T10:47:46.931 INFO:tasks.workunit.client.0.smithi060.stdout:8/36: write f2 [583307,108502] 0 2024-02-26T10:47:46.931 INFO:tasks.workunit.client.0.smithi060.stdout:8/37: readlink l7 0 2024-02-26T10:47:46.933 INFO:tasks.workunit.client.0.smithi060.stdout:5/33: dwrite d0/f5 [4194304,4194304] 0 2024-02-26T10:47:46.947 INFO:tasks.workunit.client.0.smithi060.stdout:7/59: creat da/d15/f17 x:0 0 0 2024-02-26T10:47:46.947 INFO:tasks.workunit.client.0.smithi060.stdout:8/38: getdents . 0 2024-02-26T10:47:46.949 INFO:tasks.workunit.client.0.smithi060.stdout:5/34: creat d0/fc x:0 0 0 2024-02-26T10:47:46.950 INFO:tasks.workunit.client.0.smithi060.stdout:2/27: dwrite f0 [4194304,4194304] 0 2024-02-26T10:47:46.950 INFO:tasks.workunit.client.0.smithi060.stdout:2/28: mknod c7 0 2024-02-26T10:47:46.951 INFO:tasks.workunit.client.0.smithi060.stdout:2/29: mknod c8 0 2024-02-26T10:47:46.951 INFO:tasks.workunit.client.0.smithi060.stdout:2/30: chown c7 655221396 1 2024-02-26T10:47:46.953 INFO:tasks.workunit.client.0.smithi060.stdout:2/31: symlink l9 0 2024-02-26T10:47:46.958 INFO:tasks.workunit.client.0.smithi060.stdout:2/32: getdents . 0 2024-02-26T10:47:46.972 INFO:tasks.workunit.client.0.smithi060.stdout:8/39: dread f2 [0,4194304] 0 2024-02-26T10:47:46.972 INFO:tasks.workunit.client.0.smithi060.stdout:8/40: chown c6 514754 1 2024-02-26T10:47:46.974 INFO:tasks.workunit.client.0.smithi060.stdout:5/35: dread d0/f4 [4194304,4194304] 0 2024-02-26T10:47:46.976 INFO:tasks.workunit.client.0.smithi060.stdout:5/36: rename d0/fb to d0/fd 0 2024-02-26T10:47:46.977 INFO:tasks.workunit.client.0.smithi060.stdout:5/37: write d0/f9 [209445,62222] 0 2024-02-26T10:47:46.978 INFO:tasks.workunit.client.0.smithi060.stdout:5/38: dread - d0/fd zero size 2024-02-26T10:47:46.987 INFO:tasks.workunit.client.0.smithi060.stdout:4/47: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:46.988 INFO:tasks.workunit.client.0.smithi060.stdout:4/48: unlink d0/cd 0 2024-02-26T10:47:46.989 INFO:tasks.workunit.client.0.smithi060.stdout:4/49: creat d0/ff x:0 0 0 2024-02-26T10:47:46.990 INFO:tasks.workunit.client.0.smithi060.stdout:4/50: chown d0/da 1 1 2024-02-26T10:47:46.992 INFO:tasks.workunit.client.0.smithi060.stdout:2/33: write f5 [1133633,39846] 0 2024-02-26T10:47:46.996 INFO:tasks.workunit.client.0.smithi060.stdout:4/51: truncate d0/f6 729574 0 2024-02-26T10:47:47.000 INFO:tasks.workunit.client.0.smithi060.stdout:4/52: creat d0/da/f10 x:0 0 0 2024-02-26T10:47:47.001 INFO:tasks.workunit.client.0.smithi060.stdout:2/34: dread f4 [0,4194304] 0 2024-02-26T10:47:47.001 INFO:tasks.workunit.client.0.smithi060.stdout:2/35: truncate f0 8845657 0 2024-02-26T10:47:47.001 INFO:tasks.workunit.client.0.smithi060.stdout:2/36: getdents . 0 2024-02-26T10:47:47.012 INFO:tasks.workunit.client.0.smithi060.stdout:2/37: dread f4 [0,4194304] 0 2024-02-26T10:47:47.018 INFO:tasks.workunit.client.0.smithi060.stdout:8/41: write f0 [1694044,891] 0 2024-02-26T10:47:47.018 INFO:tasks.workunit.client.0.smithi060.stdout:8/42: chown f5 33 1 2024-02-26T10:47:47.018 INFO:tasks.workunit.client.0.smithi060.stdout:7/60: dwrite da/d15/f17 [0,4194304] 0 2024-02-26T10:47:47.022 INFO:tasks.workunit.client.0.smithi060.stdout:8/43: creat f8 x:0 0 0 2024-02-26T10:47:47.022 INFO:tasks.workunit.client.0.smithi060.stdout:8/44: rmdir - no directory 2024-02-26T10:47:47.025 INFO:tasks.workunit.client.0.smithi060.stdout:8/45: mkdir d9 0 2024-02-26T10:47:47.025 INFO:tasks.workunit.client.0.smithi060.stdout:7/61: creat da/f18 x:0 0 0 2024-02-26T10:47:47.025 INFO:tasks.workunit.client.0.smithi060.stdout:8/46: write f2 [4807777,39309] 0 2024-02-26T10:47:47.027 INFO:tasks.workunit.client.0.smithi060.stdout:8/47: creat d9/fa x:0 0 0 2024-02-26T10:47:47.027 INFO:tasks.workunit.client.0.smithi060.stdout:8/48: chown f8 0 1 2024-02-26T10:47:47.029 INFO:tasks.workunit.client.0.smithi060.stdout:8/49: rename c6 to d9/cb 0 2024-02-26T10:47:47.031 INFO:tasks.workunit.client.0.smithi060.stdout:8/50: link f0 d9/fc 0 2024-02-26T10:47:47.034 INFO:tasks.workunit.client.0.smithi060.stdout:8/51: unlink f5 0 2024-02-26T10:47:47.034 INFO:tasks.workunit.client.0.smithi060.stdout:8/52: dread - f8 zero size 2024-02-26T10:47:47.034 INFO:tasks.workunit.client.0.smithi060.stdout:8/53: dread - f8 zero size 2024-02-26T10:47:47.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/54: creat d9/fd x:0 0 0 2024-02-26T10:47:47.037 INFO:tasks.workunit.client.0.smithi060.stdout:8/55: symlink d9/le 0 2024-02-26T10:47:47.037 INFO:tasks.workunit.client.0.smithi060.stdout:8/56: rename d9 to d9/df 22 2024-02-26T10:47:47.038 INFO:tasks.workunit.client.0.smithi060.stdout:8/57: readlink l7 0 2024-02-26T10:47:47.038 INFO:tasks.workunit.client.0.smithi060.stdout:8/58: creat d9/f10 x:0 0 0 2024-02-26T10:47:47.041 INFO:tasks.workunit.client.0.smithi060.stdout:2/38: dread f5 [4194304,4194304] 0 2024-02-26T10:47:47.041 INFO:tasks.workunit.client.0.smithi060.stdout:4/53: dwrite d0/fb [0,4194304] 0 2024-02-26T10:47:47.055 INFO:tasks.workunit.client.0.smithi060.stdout:2/39: creat fa x:0 0 0 2024-02-26T10:47:47.058 INFO:tasks.workunit.client.0.smithi060.stdout:7/62: dread da/d15/f17 [0,4194304] 0 2024-02-26T10:47:47.058 INFO:tasks.workunit.client.0.smithi060.stdout:7/63: write da/f18 [299402,65205] 0 2024-02-26T10:47:47.063 INFO:tasks.workunit.client.0.smithi060.stdout:7/64: creat da/d15/f19 x:0 0 0 2024-02-26T10:47:47.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:46 smithi113 ceph-mon[36126]: pgmap v479: 129 pgs: 129 active+clean; 72 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 2.26k op/s 2024-02-26T10:47:47.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:47:47.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:47:47.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:46 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:47:47.099 INFO:tasks.workunit.client.0.smithi060.stdout:8/59: dwrite d9/fd [0,4194304] 0 2024-02-26T10:47:47.099 INFO:tasks.workunit.client.0.smithi060.stdout:8/60: readlink d9/le 0 2024-02-26T10:47:47.110 INFO:tasks.workunit.client.0.smithi060.stdout:4/54: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:47.110 INFO:tasks.workunit.client.0.smithi060.stdout:4/55: read - d0/da/f10 zero size 2024-02-26T10:47:47.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:46 smithi060 ceph-mon[31539]: pgmap v479: 129 pgs: 129 active+clean; 72 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 2.26k op/s 2024-02-26T10:47:47.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:47:47.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:47:47.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:46 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:47:47.113 INFO:tasks.workunit.client.0.smithi060.stdout:4/56: mknod d0/da/dc/c11 0 2024-02-26T10:47:47.114 INFO:tasks.workunit.client.0.smithi060.stdout:4/57: chown d0/f6 153899755 1 2024-02-26T10:47:47.114 INFO:tasks.workunit.client.0.smithi060.stdout:4/58: creat d0/d7/f12 x:0 0 0 2024-02-26T10:47:47.117 INFO:tasks.workunit.client.0.smithi060.stdout:2/40: dwrite f3 [0,4194304] 0 2024-02-26T10:47:47.122 INFO:tasks.workunit.client.0.smithi060.stdout:7/65: dwrite f7 [0,4194304] 0 2024-02-26T10:47:47.124 INFO:tasks.workunit.client.0.smithi060.stdout:4/59: dread d0/f3 [0,4194304] 0 2024-02-26T10:47:47.124 INFO:tasks.workunit.client.0.smithi060.stdout:4/60: write d0/d7/f12 [992214,10872] 0 2024-02-26T10:47:47.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:46 smithi100 ceph-mon[36051]: pgmap v479: 129 pgs: 129 active+clean; 72 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 2.26k op/s 2024-02-26T10:47:47.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:47:47.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:47:47.125 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:46 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:47:47.127 INFO:tasks.workunit.client.0.smithi060.stdout:7/66: rename da/fc to da/d15/f1a 0 2024-02-26T10:47:47.127 INFO:tasks.workunit.client.0.smithi060.stdout:7/67: chown da/cb 4 1 2024-02-26T10:47:47.128 INFO:tasks.workunit.client.0.smithi060.stdout:4/61: symlink d0/d7/l13 0 2024-02-26T10:47:47.132 INFO:tasks.workunit.client.0.smithi060.stdout:7/68: getdents da/d15 0 2024-02-26T10:47:47.132 INFO:tasks.workunit.client.0.smithi060.stdout:7/69: rename da to da/d15/d1b 22 2024-02-26T10:47:47.133 INFO:tasks.workunit.client.0.smithi060.stdout:7/70: symlink da/d15/l1c 0 2024-02-26T10:47:47.137 INFO:tasks.workunit.client.0.smithi060.stdout:4/62: truncate d0/f5 3018734 0 2024-02-26T10:47:47.138 INFO:tasks.workunit.client.0.smithi060.stdout:4/63: creat d0/da/f14 x:0 0 0 2024-02-26T10:47:47.138 INFO:tasks.workunit.client.0.smithi060.stdout:4/64: write d0/f6 [470131,38009] 0 2024-02-26T10:47:47.172 INFO:tasks.workunit.client.0.smithi060.stdout:2/41: dwrite f5 [0,4194304] 0 2024-02-26T10:47:47.172 INFO:tasks.workunit.client.0.smithi060.stdout:2/42: write f4 [345990,99489] 0 2024-02-26T10:47:47.178 INFO:tasks.workunit.client.0.smithi060.stdout:4/65: dwrite d0/d7/f8 [0,4194304] 0 2024-02-26T10:47:47.180 INFO:tasks.workunit.client.0.smithi060.stdout:7/71: dwrite da/d15/f17 [0,4194304] 0 2024-02-26T10:47:47.182 INFO:tasks.workunit.client.0.smithi060.stdout:7/72: rename da/cf to da/d15/c1d 0 2024-02-26T10:47:47.182 INFO:tasks.workunit.client.0.smithi060.stdout:7/73: fsync da/d15/f17 0 2024-02-26T10:47:47.184 INFO:tasks.workunit.client.0.smithi060.stdout:7/74: readlink l8 0 2024-02-26T10:47:47.191 INFO:tasks.workunit.client.0.smithi060.stdout:4/66: dread d0/f3 [0,4194304] 0 2024-02-26T10:47:47.197 INFO:tasks.workunit.client.0.smithi060.stdout:2/43: write f5 [4129286,53963] 0 2024-02-26T10:47:47.197 INFO:tasks.workunit.client.0.smithi060.stdout:2/44: readlink l9 0 2024-02-26T10:47:47.199 INFO:tasks.workunit.client.0.smithi060.stdout:2/45: getdents . 0 2024-02-26T10:47:47.201 INFO:tasks.workunit.client.0.smithi060.stdout:2/46: link l9 lb 0 2024-02-26T10:47:47.205 INFO:tasks.workunit.client.0.smithi060.stdout:2/47: rename f6 to fc 0 2024-02-26T10:47:47.207 INFO:tasks.workunit.client.0.smithi060.stdout:2/48: mkdir dd 0 2024-02-26T10:47:47.216 INFO:tasks.workunit.client.0.smithi060.stdout:2/49: dread f4 [0,4194304] 0 2024-02-26T10:47:47.218 INFO:tasks.workunit.client.0.smithi060.stdout:2/50: mkdir dd/de 0 2024-02-26T10:47:47.219 INFO:tasks.workunit.client.0.smithi060.stdout:7/75: dwrite f7 [0,4194304] 0 2024-02-26T10:47:47.249 INFO:tasks.workunit.client.0.smithi060.stdout:7/76: write da/d15/f17 [3767149,45897] 0 2024-02-26T10:47:47.251 INFO:tasks.workunit.client.0.smithi060.stdout:7/77: symlink da/d15/l1e 0 2024-02-26T10:47:47.253 INFO:tasks.workunit.client.0.smithi060.stdout:7/78: fdatasync da/d15/f1a 0 2024-02-26T10:47:47.253 INFO:tasks.workunit.client.0.smithi060.stdout:7/79: readlink da/d15/l1c 0 2024-02-26T10:47:47.254 INFO:tasks.workunit.client.0.smithi060.stdout:7/80: rename da/d15/c1d to da/c1f 0 2024-02-26T10:47:47.256 INFO:tasks.workunit.client.0.smithi060.stdout:7/81: creat da/d15/f20 x:0 0 0 2024-02-26T10:47:47.267 INFO:tasks.workunit.client.0.smithi060.stdout:2/51: dwrite f3 [4194304,4194304] 0 2024-02-26T10:47:47.271 INFO:tasks.workunit.client.0.smithi060.stdout:2/52: fsync f5 0 2024-02-26T10:47:47.309 INFO:tasks.workunit.client.0.smithi060.stdout:7/82: dwrite da/d15/f1a [4194304,4194304] 0 2024-02-26T10:47:47.310 INFO:tasks.workunit.client.0.smithi060.stdout:7/83: write da/d15/f19 [442730,121324] 0 2024-02-26T10:47:47.310 INFO:tasks.workunit.client.0.smithi060.stdout:7/84: fsync da/f13 0 2024-02-26T10:47:47.311 INFO:tasks.workunit.client.0.smithi060.stdout:7/85: readlink da/d15/l1c 0 2024-02-26T10:47:47.313 INFO:tasks.workunit.client.0.smithi060.stdout:7/86: mkdir da/d15/d21 0 2024-02-26T10:47:47.314 INFO:tasks.workunit.client.0.smithi060.stdout:7/87: write da/fd [4804484,92374] 0 2024-02-26T10:47:47.314 INFO:tasks.workunit.client.0.smithi060.stdout:7/88: read - da/fe zero size 2024-02-26T10:47:47.319 INFO:tasks.workunit.client.0.smithi060.stdout:2/53: dwrite f3 [0,4194304] 0 2024-02-26T10:47:47.320 INFO:tasks.workunit.client.0.smithi060.stdout:2/54: truncate fa 316537 0 2024-02-26T10:47:47.400 INFO:tasks.workunit.client.0.smithi060.stdout:7/89: dwrite da/f18 [0,4194304] 0 2024-02-26T10:47:47.407 INFO:tasks.workunit.client.0.smithi060.stdout:7/90: unlink da/c10 0 2024-02-26T10:47:47.411 INFO:tasks.workunit.client.0.smithi060.stdout:2/55: dwrite f0 [0,4194304] 0 2024-02-26T10:47:47.413 INFO:tasks.workunit.client.0.smithi060.stdout:2/56: creat dd/de/ff x:0 0 0 2024-02-26T10:47:47.413 INFO:tasks.workunit.client.0.smithi060.stdout:2/57: readlink lb 0 2024-02-26T10:47:47.413 INFO:tasks.workunit.client.0.smithi060.stdout:2/58: rename dd to dd/de/d10 22 2024-02-26T10:47:47.414 INFO:tasks.workunit.client.0.smithi060.stdout:2/59: mkdir dd/d11 0 2024-02-26T10:47:47.459 INFO:tasks.workunit.client.0.smithi060.stdout:7/91: dwrite da/f13 [0,4194304] 0 2024-02-26T10:47:47.467 INFO:tasks.workunit.client.0.smithi060.stdout:2/60: dread f0 [0,4194304] 0 2024-02-26T10:47:47.467 INFO:tasks.workunit.client.0.smithi060.stdout:2/61: dread - dd/de/ff zero size 2024-02-26T10:47:47.467 INFO:tasks.workunit.client.0.smithi060.stdout:2/62: dread - dd/de/ff zero size 2024-02-26T10:47:47.507 INFO:tasks.workunit.client.0.smithi060.stdout:7/92: write da/f13 [651848,123433] 0 2024-02-26T10:47:47.538 INFO:tasks.workunit.client.0.smithi060.stdout:2/63: dwrite f3 [0,4194304] 0 2024-02-26T10:47:47.547 INFO:tasks.workunit.client.0.smithi060.stdout:7/93: read da/d15/f1a [1446363,46757] 0 2024-02-26T10:47:47.551 INFO:tasks.workunit.client.0.smithi060.stdout:7/94: creat da/d15/d21/f22 x:0 0 0 2024-02-26T10:47:47.552 INFO:tasks.workunit.client.0.smithi060.stdout:7/95: readlink da/d15/l1c 0 2024-02-26T10:47:47.552 INFO:tasks.workunit.client.0.smithi060.stdout:7/96: write f6 [726675,73317] 0 2024-02-26T10:47:47.575 INFO:tasks.workunit.client.0.smithi060.stdout:2/64: write fc [3791453,27149] 0 2024-02-26T10:47:47.645 INFO:tasks.workunit.client.0.smithi060.stdout:7/97: dwrite da/d15/f1a [4194304,4194304] 0 2024-02-26T10:47:47.646 INFO:tasks.workunit.client.0.smithi060.stdout:7/98: readlink l5 0 2024-02-26T10:47:47.647 INFO:tasks.workunit.client.0.smithi060.stdout:7/99: chown f7 79 1 2024-02-26T10:47:47.647 INFO:tasks.workunit.client.0.smithi060.stdout:7/100: chown da/fd 6662 1 2024-02-26T10:47:47.647 INFO:tasks.workunit.client.0.smithi060.stdout:7/101: fdatasync f6 0 2024-02-26T10:47:47.661 INFO:tasks.workunit.client.0.smithi060.stdout:7/102: dread f2 [0,4194304] 0 2024-02-26T10:47:47.663 INFO:tasks.workunit.client.0.smithi060.stdout:7/103: symlink da/d15/l23 0 2024-02-26T10:47:47.665 INFO:tasks.workunit.client.0.smithi060.stdout:7/104: creat da/f24 x:0 0 0 2024-02-26T10:47:47.665 INFO:tasks.workunit.client.0.smithi060.stdout:7/105: chown f6 21560 1 2024-02-26T10:47:47.686 INFO:tasks.workunit.client.0.smithi060.stdout:7/106: dread f6 [0,4194304] 0 2024-02-26T10:47:47.686 INFO:tasks.workunit.client.0.smithi060.stdout:7/107: write da/f24 [129933,52999] 0 2024-02-26T10:47:47.735 INFO:tasks.workunit.client.0.smithi060.stdout:7/108: dread da/f18 [0,4194304] 0 2024-02-26T10:47:47.737 INFO:tasks.workunit.client.0.smithi060.stdout:7/109: creat da/d15/f25 x:0 0 0 2024-02-26T10:47:47.737 INFO:tasks.workunit.client.0.smithi060.stdout:7/110: stat da/f18 0 2024-02-26T10:47:47.743 INFO:tasks.workunit.client.0.smithi060.stdout:7/111: rmdir da 39 2024-02-26T10:47:47.819 INFO:tasks.workunit.client.0.smithi060.stdout:7/112: dwrite f2 [0,4194304] 0 2024-02-26T10:47:47.823 INFO:tasks.workunit.client.0.smithi060.stdout:7/113: write da/d15/d21/f22 [447187,68511] 0 2024-02-26T10:47:47.898 INFO:tasks.workunit.client.0.smithi060.stdout:7/114: dwrite da/d15/f20 [0,4194304] 0 2024-02-26T10:47:47.900 INFO:tasks.workunit.client.0.smithi060.stdout:7/115: mknod da/d15/d21/c26 0 2024-02-26T10:47:47.900 INFO:tasks.workunit.client.0.smithi060.stdout:7/116: write da/f24 [598392,5574] 0 2024-02-26T10:47:47.901 INFO:tasks.workunit.client.0.smithi060.stdout:7/117: fsync da/d15/f25 0 2024-02-26T10:47:47.902 INFO:tasks.workunit.client.0.smithi060.stdout:7/118: symlink da/d15/l27 0 2024-02-26T10:47:47.936 INFO:tasks.workunit.client.0.smithi060.stdout:7/119: dread da/f24 [0,4194304] 0 2024-02-26T10:47:47.942 INFO:tasks.workunit.client.0.smithi060.stdout:7/120: creat da/d15/d21/f28 x:0 0 0 2024-02-26T10:47:47.942 INFO:tasks.workunit.client.0.smithi060.stdout:7/121: chown f2 710 1 2024-02-26T10:47:47.942 INFO:tasks.workunit.client.0.smithi060.stdout:7/122: stat da/f13 0 2024-02-26T10:47:47.942 INFO:tasks.workunit.client.0.smithi060.stdout:7/123: truncate da/f18 4573694 0 2024-02-26T10:47:47.943 INFO:tasks.workunit.client.0.smithi060.stdout:7/124: mkdir da/d15/d21/d29 0 2024-02-26T10:47:47.944 INFO:tasks.workunit.client.0.smithi060.stdout:7/125: symlink da/d15/d21/d29/l2a 0 2024-02-26T10:47:47.949 INFO:tasks.workunit.client.0.smithi060.stdout:7/126: getdents da/d15/d21/d29 0 2024-02-26T10:47:47.950 INFO:tasks.workunit.client.0.smithi060.stdout:7/127: mknod da/d15/d21/c2b 0 2024-02-26T10:47:47.952 INFO:tasks.workunit.client.0.smithi060.stdout:7/128: creat da/d15/d21/f2c x:0 0 0 2024-02-26T10:47:47.952 INFO:tasks.workunit.client.0.smithi060.stdout:7/129: dread - da/d15/d21/f2c zero size 2024-02-26T10:47:47.953 INFO:tasks.workunit.client.0.smithi060.stdout:7/130: getdents da 0 2024-02-26T10:47:47.974 INFO:tasks.workunit.client.0.smithi060.stdout:7/131: dread da/fd [4194304,4194304] 0 2024-02-26T10:47:47.974 INFO:tasks.workunit.client.0.smithi060.stdout:7/132: chown da/d15/l16 17701508 1 2024-02-26T10:47:47.975 INFO:tasks.workunit.client.0.smithi060.stdout:7/133: stat f2 0 2024-02-26T10:47:47.975 INFO:tasks.workunit.client.0.smithi060.stdout:7/134: write da/f24 [1386701,86523] 0 2024-02-26T10:47:47.994 INFO:tasks.workunit.client.0.smithi060.stdout:7/135: dread da/d15/f19 [0,4194304] 0 2024-02-26T10:47:47.996 INFO:tasks.workunit.client.0.smithi060.stdout:7/136: rename da/d15/d21/c2b to da/d15/d21/d29/c2d 0 2024-02-26T10:47:47.997 INFO:tasks.workunit.client.0.smithi060.stdout:7/137: creat da/d15/f2e x:0 0 0 2024-02-26T10:47:47.999 INFO:tasks.workunit.client.0.smithi060.stdout:7/138: mkdir da/d15/d21/d2f 0 2024-02-26T10:47:48.086 INFO:tasks.workunit.client.0.smithi060.stdout:7/139: dwrite f2 [0,4194304] 0 2024-02-26T10:47:48.086 INFO:tasks.workunit.client.0.smithi060.stdout:7/140: write da/fe [884116,68540] 0 2024-02-26T10:47:48.089 INFO:tasks.workunit.client.0.smithi060.stdout:7/141: link da/d15/f20 da/f30 0 2024-02-26T10:47:48.089 INFO:tasks.workunit.client.0.smithi060.stdout:7/142: stat da/d15 0 2024-02-26T10:47:48.114 INFO:tasks.workunit.client.0.smithi060.stdout:7/143: dread da/d15/d21/f22 [0,4194304] 0 2024-02-26T10:47:48.114 INFO:tasks.workunit.client.0.smithi060.stdout:7/144: stat da/d15/d21/d29 0 2024-02-26T10:47:48.116 INFO:tasks.workunit.client.0.smithi060.stdout:7/145: truncate f6 632719 0 2024-02-26T10:47:48.116 INFO:tasks.workunit.client.0.smithi060.stdout:7/146: stat da/d15/l23 0 2024-02-26T10:47:48.116 INFO:tasks.workunit.client.0.smithi060.stdout:7/147: dread - da/d15/d21/f2c zero size 2024-02-26T10:47:48.120 INFO:tasks.workunit.client.0.smithi060.stdout:7/148: rmdir da/d15 39 2024-02-26T10:47:48.125 INFO:tasks.workunit.client.0.smithi060.stdout:7/149: unlink da/d15/d21/d29/c2d 0 2024-02-26T10:47:48.125 INFO:tasks.workunit.client.0.smithi060.stdout:7/150: stat da/d15/d21/f2c 0 2024-02-26T10:47:48.126 INFO:tasks.workunit.client.0.smithi060.stdout:7/151: mknod da/d15/d21/d29/c31 0 2024-02-26T10:47:48.128 INFO:tasks.workunit.client.0.smithi060.stdout:7/152: truncate da/d15/f1a 7957047 0 2024-02-26T10:47:48.160 INFO:tasks.workunit.client.0.smithi060.stdout:7/153: dread da/fe [0,4194304] 0 2024-02-26T10:47:48.160 INFO:tasks.workunit.client.0.smithi060.stdout:7/154: write da/f13 [1497161,42308] 0 2024-02-26T10:47:48.167 INFO:tasks.workunit.client.0.smithi060.stdout:7/155: mkdir da/d15/d21/d32 0 2024-02-26T10:47:48.168 INFO:tasks.workunit.client.0.smithi060.stdout:7/156: link da/d15/d21/f28 da/d15/d21/d32/f33 0 2024-02-26T10:47:48.169 INFO:tasks.workunit.client.0.smithi060.stdout:7/157: write f6 [1272636,115951] 0 2024-02-26T10:47:48.169 INFO:tasks.workunit.client.0.smithi060.stdout:7/158: write da/d15/f2e [683597,44335] 0 2024-02-26T10:47:48.170 INFO:tasks.workunit.client.0.smithi060.stdout:7/159: chown da/d15/d21/d29 2 1 2024-02-26T10:47:48.172 INFO:tasks.workunit.client.0.smithi060.stdout:7/160: mkdir da/d15/d34 0 2024-02-26T10:47:48.191 INFO:tasks.workunit.client.0.smithi060.stdout:7/161: dread da/d15/f19 [0,4194304] 0 2024-02-26T10:47:48.249 INFO:tasks.workunit.client.0.smithi060.stdout:7/162: write da/d15/f20 [2258986,104491] 0 2024-02-26T10:47:48.256 INFO:tasks.workunit.client.0.smithi060.stdout:7/163: rename da/c1f to da/d15/c35 0 2024-02-26T10:47:48.258 INFO:tasks.workunit.client.0.smithi060.stdout:7/164: symlink da/l36 0 2024-02-26T10:47:48.259 INFO:tasks.workunit.client.0.smithi060.stdout:7/165: mkdir da/d15/d21/d2f/d37 0 2024-02-26T10:47:48.259 INFO:tasks.workunit.client.0.smithi060.stdout:7/166: chown f2 91578 1 2024-02-26T10:47:48.261 INFO:tasks.workunit.client.0.smithi060.stdout:7/167: creat da/d15/d21/d2f/d37/f38 x:0 0 0 2024-02-26T10:47:48.262 INFO:tasks.workunit.client.0.smithi060.stdout:7/168: unlink da/d15/f20 0 2024-02-26T10:47:48.263 INFO:tasks.workunit.client.0.smithi060.stdout:7/169: rmdir da/d15/d21 39 2024-02-26T10:47:48.265 INFO:tasks.workunit.client.0.smithi060.stdout:7/170: mkdir da/d15/d34/d39 0 2024-02-26T10:47:49.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:48 smithi113 ceph-mon[36126]: pgmap v480: 129 pgs: 129 active+clean; 71 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 2.10k op/s 2024-02-26T10:47:49.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:48 smithi113 ceph-mon[36126]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:49.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:48 smithi060 ceph-mon[31539]: pgmap v480: 129 pgs: 129 active+clean; 71 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 2.10k op/s 2024-02-26T10:47:49.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:48 smithi060 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:49.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:48 smithi100 ceph-mon[36051]: pgmap v480: 129 pgs: 129 active+clean; 71 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 2.10k op/s 2024-02-26T10:47:49.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:48 smithi100 ceph-mon[36051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:47:50.049 INFO:tasks.workunit.client.0.smithi060.stdout:0/1: sync 2024-02-26T10:47:50.049 INFO:tasks.workunit.client.0.smithi060.stdout:8/61: sync 2024-02-26T10:47:50.049 INFO:tasks.workunit.client.0.smithi060.stdout:5/39: sync 2024-02-26T10:47:50.049 INFO:tasks.workunit.client.0.smithi060.stdout:4/67: sync 2024-02-26T10:47:50.049 INFO:tasks.workunit.client.0.smithi060.stdout:6/42: sync 2024-02-26T10:47:50.049 INFO:tasks.workunit.client.0.smithi060.stdout:1/40: sync 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:6/43: dread - d6/fe zero size 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:7/171: sync 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:6/44: write d6/f9 [187074,29208] 0 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:3/5: sync 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:3/6: fsync - no filename 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:9/7: sync 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:9/8: truncate - no filename 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:3/7: chown . 573644 1 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:9/9: write - no filename 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:3/8: dwrite - no filename 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:9/10: write - no filename 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:3/9: dread - no filename 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:2/65: sync 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:3/10: write - no filename 2024-02-26T10:47:50.050 INFO:tasks.workunit.client.0.smithi060.stdout:3/11: read - no filename 2024-02-26T10:47:50.051 INFO:tasks.workunit.client.0.smithi060.stdout:3/12: chown . 4 1 2024-02-26T10:47:50.051 INFO:tasks.workunit.client.0.smithi060.stdout:3/13: readlink - no filename 2024-02-26T10:47:50.051 INFO:tasks.workunit.client.0.smithi060.stdout:9/11: chown l1 0 1 2024-02-26T10:47:50.051 INFO:tasks.workunit.client.0.smithi060.stdout:9/12: dwrite - no filename 2024-02-26T10:47:50.051 INFO:tasks.workunit.client.0.smithi060.stdout:3/14: chown . 68 1 2024-02-26T10:47:50.051 INFO:tasks.workunit.client.0.smithi060.stdout:3/15: fsync - no filename 2024-02-26T10:47:50.051 INFO:tasks.workunit.client.0.smithi060.stdout:8/62: unlink d9/cb 0 2024-02-26T10:47:50.051 INFO:tasks.workunit.client.0.smithi060.stdout:5/40: mkdir d0/de 0 2024-02-26T10:47:50.053 INFO:tasks.workunit.client.0.smithi060.stdout:5/41: mknod d0/de/cf 0 2024-02-26T10:47:50.054 INFO:tasks.workunit.client.0.smithi060.stdout:0/2: creat d0/f1 x:0 0 0 2024-02-26T10:47:50.054 INFO:tasks.workunit.client.0.smithi060.stdout:4/68: mknod d0/da/c15 0 2024-02-26T10:47:50.055 INFO:tasks.workunit.client.0.smithi060.stdout:1/41: rename d3/f6 to d3/fa 0 2024-02-26T10:47:50.057 INFO:tasks.workunit.client.0.smithi060.stdout:6/45: truncate d6/d7/fd 27702 0 2024-02-26T10:47:50.058 INFO:tasks.workunit.client.0.smithi060.stdout:5/42: read - d0/fc zero size 2024-02-26T10:47:50.058 INFO:tasks.workunit.client.0.smithi060.stdout:6/46: write d6/d7/ff [826344,122236] 0 2024-02-26T10:47:50.060 INFO:tasks.workunit.client.0.smithi060.stdout:9/13: link l0 l2 0 2024-02-26T10:47:50.060 INFO:tasks.workunit.client.0.smithi060.stdout:9/14: fdatasync - no filename 2024-02-26T10:47:50.061 INFO:tasks.workunit.client.0.smithi060.stdout:9/15: dread - no filename 2024-02-26T10:47:50.061 INFO:tasks.workunit.client.0.smithi060.stdout:9/16: truncate - no filename 2024-02-26T10:47:50.061 INFO:tasks.workunit.client.0.smithi060.stdout:9/17: rmdir - no directory 2024-02-26T10:47:50.061 INFO:tasks.workunit.client.0.smithi060.stdout:9/18: write - no filename 2024-02-26T10:47:50.061 INFO:tasks.workunit.client.0.smithi060.stdout:9/19: dwrite - no filename 2024-02-26T10:47:50.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/16: mknod c0 0 2024-02-26T10:47:50.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/17: dread - no filename 2024-02-26T10:47:50.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/18: dwrite - no filename 2024-02-26T10:47:50.062 INFO:tasks.workunit.client.0.smithi060.stdout:1/42: mkdir d3/db 0 2024-02-26T10:47:50.064 INFO:tasks.workunit.client.0.smithi060.stdout:0/3: creat d0/f2 x:0 0 0 2024-02-26T10:47:50.064 INFO:tasks.workunit.client.0.smithi060.stdout:0/4: dread - d0/f2 zero size 2024-02-26T10:47:50.065 INFO:tasks.workunit.client.0.smithi060.stdout:0/5: chown d0/f2 1596 1 2024-02-26T10:47:50.065 INFO:tasks.workunit.client.0.smithi060.stdout:0/6: write d0/f2 [150154,21057] 0 2024-02-26T10:47:50.066 INFO:tasks.workunit.client.0.smithi060.stdout:3/19: symlink l1 0 2024-02-26T10:47:50.066 INFO:tasks.workunit.client.0.smithi060.stdout:3/20: readlink l1 0 2024-02-26T10:47:50.066 INFO:tasks.workunit.client.0.smithi060.stdout:3/21: write - no filename 2024-02-26T10:47:50.069 INFO:tasks.workunit.client.0.smithi060.stdout:7/172: symlink da/d15/l3a 0 2024-02-26T10:47:50.069 INFO:tasks.workunit.client.0.smithi060.stdout:7/173: readlink da/d15/d21/d29/l2a 0 2024-02-26T10:47:50.069 INFO:tasks.workunit.client.0.smithi060.stdout:1/43: truncate d3/fa 57904 0 2024-02-26T10:47:50.070 INFO:tasks.workunit.client.0.smithi060.stdout:5/43: rename d0/f4 to d0/f10 0 2024-02-26T10:47:50.071 INFO:tasks.workunit.client.0.smithi060.stdout:5/44: mknod d0/de/c11 0 2024-02-26T10:47:50.074 INFO:tasks.workunit.client.0.smithi060.stdout:7/174: mkdir da/d15/d34/d3b 0 2024-02-26T10:47:50.075 INFO:tasks.workunit.client.0.smithi060.stdout:1/44: creat d3/db/fc x:0 0 0 2024-02-26T10:47:50.075 INFO:tasks.workunit.client.0.smithi060.stdout:5/45: creat d0/de/f12 x:0 0 0 2024-02-26T10:47:50.078 INFO:tasks.workunit.client.0.smithi060.stdout:5/46: symlink d0/l13 0 2024-02-26T10:47:50.078 INFO:tasks.workunit.client.0.smithi060.stdout:7/175: mknod da/d15/d34/d39/c3c 0 2024-02-26T10:47:50.078 INFO:tasks.workunit.client.0.smithi060.stdout:0/7: rename d0/f2 to d0/f3 0 2024-02-26T10:47:50.078 INFO:tasks.workunit.client.0.smithi060.stdout:3/22: creat f2 x:0 0 0 2024-02-26T10:47:50.078 INFO:tasks.workunit.client.0.smithi060.stdout:3/23: truncate f2 795146 0 2024-02-26T10:47:50.079 INFO:tasks.workunit.client.0.smithi060.stdout:5/47: creat d0/f14 x:0 0 0 2024-02-26T10:47:50.079 INFO:tasks.workunit.client.0.smithi060.stdout:0/8: creat d0/f4 x:0 0 0 2024-02-26T10:47:50.082 INFO:tasks.workunit.client.0.smithi060.stdout:0/9: chown d0/f1 63639 1 2024-02-26T10:47:50.082 INFO:tasks.workunit.client.0.smithi060.stdout:0/10: stat d0/f3 0 2024-02-26T10:47:50.082 INFO:tasks.workunit.client.0.smithi060.stdout:7/176: symlink da/l3d 0 2024-02-26T10:47:50.089 INFO:tasks.workunit.client.0.smithi060.stdout:1/45: rmdir d3 39 2024-02-26T10:47:50.091 INFO:tasks.workunit.client.0.smithi060.stdout:5/48: chown d0/f5 68829927 1 2024-02-26T10:47:50.096 INFO:tasks.workunit.client.0.smithi060.stdout:5/49: getdents d0/de 0 2024-02-26T10:47:50.097 INFO:tasks.workunit.client.0.smithi060.stdout:5/50: write d0/f14 [862821,38035] 0 2024-02-26T10:47:50.097 INFO:tasks.workunit.client.0.smithi060.stdout:5/51: dread - d0/fc zero size 2024-02-26T10:47:50.098 INFO:tasks.workunit.client.0.smithi060.stdout:5/52: mkdir d0/de/d15 0 2024-02-26T10:47:50.101 INFO:tasks.workunit.client.0.smithi060.stdout:7/177: dread da/d15/f2e [0,4194304] 0 2024-02-26T10:47:50.103 INFO:tasks.workunit.client.0.smithi060.stdout:7/178: symlink da/d15/d21/d2f/d37/l3e 0 2024-02-26T10:47:50.120 INFO:tasks.workunit.client.0.smithi060.stdout:2/66: dwrite f0 [0,4194304] 0 2024-02-26T10:47:50.120 INFO:tasks.workunit.client.0.smithi060.stdout:4/69: dread d0/d7/f12 [0,4194304] 0 2024-02-26T10:47:50.121 INFO:tasks.workunit.client.0.smithi060.stdout:2/67: mknod dd/de/c12 0 2024-02-26T10:47:50.121 INFO:tasks.workunit.client.0.smithi060.stdout:2/68: stat dd 0 2024-02-26T10:47:50.127 INFO:tasks.workunit.client.0.smithi060.stdout:2/69: mkdir dd/d11/d13 0 2024-02-26T10:47:50.138 INFO:tasks.workunit.client.0.smithi060.stdout:6/47: dwrite d6/d7/f10 [0,4194304] 0 2024-02-26T10:47:50.139 INFO:tasks.workunit.client.0.smithi060.stdout:6/48: rmdir d6/d7 39 2024-02-26T10:47:50.139 INFO:tasks.workunit.client.0.smithi060.stdout:6/49: chown d6/d7/fd 142833560 1 2024-02-26T10:47:50.140 INFO:tasks.workunit.client.0.smithi060.stdout:6/50: creat d6/f11 x:0 0 0 2024-02-26T10:47:50.141 INFO:tasks.workunit.client.0.smithi060.stdout:6/51: rename d6/d7/fc to d6/d7/f12 0 2024-02-26T10:47:50.142 INFO:tasks.workunit.client.0.smithi060.stdout:6/52: rmdir d6/d7 39 2024-02-26T10:47:50.145 INFO:tasks.workunit.client.0.smithi060.stdout:6/53: creat d6/d7/f13 x:0 0 0 2024-02-26T10:47:50.145 INFO:tasks.workunit.client.0.smithi060.stdout:6/54: fdatasync d6/d7/fd 0 2024-02-26T10:47:50.145 INFO:tasks.workunit.client.0.smithi060.stdout:6/55: dread - d6/d7/f13 zero size 2024-02-26T10:47:50.148 INFO:tasks.workunit.client.0.smithi060.stdout:0/11: dwrite d0/f3 [0,4194304] 0 2024-02-26T10:47:50.148 INFO:tasks.workunit.client.0.smithi060.stdout:0/12: dread - d0/f1 zero size 2024-02-26T10:47:50.148 INFO:tasks.workunit.client.0.smithi060.stdout:0/13: fdatasync d0/f4 0 2024-02-26T10:47:50.160 INFO:tasks.workunit.client.0.smithi060.stdout:7/179: dread f2 [0,4194304] 0 2024-02-26T10:47:50.165 INFO:tasks.workunit.client.0.smithi060.stdout:7/180: mknod da/d15/d21/c3f 0 2024-02-26T10:47:50.177 INFO:tasks.workunit.client.0.smithi060.stdout:1/46: dwrite d3/db/fc [0,4194304] 0 2024-02-26T10:47:50.178 INFO:tasks.workunit.client.0.smithi060.stdout:3/24: dwrite f2 [0,4194304] 0 2024-02-26T10:47:50.184 INFO:tasks.workunit.client.0.smithi060.stdout:0/14: write d0/f3 [2658967,457] 0 2024-02-26T10:47:50.185 INFO:tasks.workunit.client.0.smithi060.stdout:0/15: write d0/f1 [872124,13027] 0 2024-02-26T10:47:50.185 INFO:tasks.workunit.client.0.smithi060.stdout:5/53: dwrite d0/f14 [0,4194304] 0 2024-02-26T10:47:50.193 INFO:tasks.workunit.client.0.smithi060.stdout:6/56: dread d6/d7/f12 [0,4194304] 0 2024-02-26T10:47:50.193 INFO:tasks.workunit.client.0.smithi060.stdout:6/57: readlink l1 0 2024-02-26T10:47:50.193 INFO:tasks.workunit.client.0.smithi060.stdout:6/58: truncate d6/d7/f13 58263 0 2024-02-26T10:47:50.193 INFO:tasks.workunit.client.0.smithi060.stdout:6/59: fsync f4 0 2024-02-26T10:47:50.194 INFO:tasks.workunit.client.0.smithi060.stdout:6/60: chown d6/f9 5353 1 2024-02-26T10:47:50.198 INFO:tasks.workunit.client.0.smithi060.stdout:4/70: dwrite d0/f3 [0,4194304] 0 2024-02-26T10:47:50.214 INFO:tasks.workunit.client.0.smithi060.stdout:4/71: mkdir d0/da/dc/d16 0 2024-02-26T10:47:50.214 INFO:tasks.workunit.client.0.smithi060.stdout:4/72: rename d0 to d0/d17 22 2024-02-26T10:47:50.225 INFO:tasks.workunit.client.0.smithi060.stdout:4/73: dread d0/d7/f12 [0,4194304] 0 2024-02-26T10:47:50.227 INFO:tasks.workunit.client.0.smithi060.stdout:4/74: creat d0/da/dc/d16/f18 x:0 0 0 2024-02-26T10:47:50.227 INFO:tasks.workunit.client.0.smithi060.stdout:4/75: chown d0/da/dc/d16 533604224 1 2024-02-26T10:47:50.227 INFO:tasks.workunit.client.0.smithi060.stdout:4/76: dread - d0/da/f14 zero size 2024-02-26T10:47:50.228 INFO:tasks.workunit.client.0.smithi060.stdout:3/25: dwrite f2 [0,4194304] 0 2024-02-26T10:47:50.229 INFO:tasks.workunit.client.0.smithi060.stdout:4/77: rmdir d0/d7 39 2024-02-26T10:47:50.240 INFO:tasks.workunit.client.0.smithi060.stdout:5/54: dwrite d0/f8 [0,4194304] 0 2024-02-26T10:47:50.243 INFO:tasks.workunit.client.0.smithi060.stdout:5/55: read - d0/fd zero size 2024-02-26T10:47:50.243 INFO:tasks.workunit.client.0.smithi060.stdout:5/56: creat d0/de/d15/f16 x:0 0 0 2024-02-26T10:47:50.243 INFO:tasks.workunit.client.0.smithi060.stdout:5/57: creat d0/f17 x:0 0 0 2024-02-26T10:47:50.265 INFO:tasks.workunit.client.0.smithi060.stdout:5/58: dread d0/f14 [0,4194304] 0 2024-02-26T10:47:50.282 INFO:tasks.workunit.client.0.smithi060.stdout:2/70: dwrite f3 [4194304,4194304] 0 2024-02-26T10:47:50.284 INFO:tasks.workunit.client.0.smithi060.stdout:2/71: symlink dd/l14 0 2024-02-26T10:47:50.285 INFO:tasks.workunit.client.0.smithi060.stdout:3/26: truncate f2 1728711 0 2024-02-26T10:47:50.286 INFO:tasks.workunit.client.0.smithi060.stdout:2/72: creat dd/de/f15 x:0 0 0 2024-02-26T10:47:50.291 INFO:tasks.workunit.client.0.smithi060.stdout:2/73: readlink l9 0 2024-02-26T10:47:50.302 INFO:tasks.workunit.client.0.smithi060.stdout:1/47: dwrite d3/f8 [0,4194304] 0 2024-02-26T10:47:50.303 INFO:tasks.workunit.client.0.smithi060.stdout:4/78: rename d0/da/f10 to d0/da/dc/d16/f19 0 2024-02-26T10:47:50.305 INFO:tasks.workunit.client.0.smithi060.stdout:7/181: write da/fd [1760640,20647] 0 2024-02-26T10:47:50.306 INFO:tasks.workunit.client.0.smithi060.stdout:4/79: unlink d0/da/dc/fe 0 2024-02-26T10:47:50.309 INFO:tasks.workunit.client.0.smithi060.stdout:1/48: rename d3/f8 to d3/db/fd 0 2024-02-26T10:47:50.309 INFO:tasks.workunit.client.0.smithi060.stdout:1/49: fdatasync d3/fa 0 2024-02-26T10:47:50.311 INFO:tasks.workunit.client.0.smithi060.stdout:7/182: symlink da/d15/d21/d32/l40 0 2024-02-26T10:47:50.311 INFO:tasks.workunit.client.0.smithi060.stdout:7/183: chown da/d15/f19 216610 1 2024-02-26T10:47:50.311 INFO:tasks.workunit.client.0.smithi060.stdout:7/184: fsync da/d15/f25 0 2024-02-26T10:47:50.311 INFO:tasks.workunit.client.0.smithi060.stdout:6/61: dwrite d6/d7/fd [0,4194304] 0 2024-02-26T10:47:50.321 INFO:tasks.workunit.client.0.smithi060.stdout:3/27: read f2 [888194,32909] 0 2024-02-26T10:47:50.322 INFO:tasks.workunit.client.0.smithi060.stdout:3/28: creat f3 x:0 0 0 2024-02-26T10:47:50.322 INFO:tasks.workunit.client.0.smithi060.stdout:3/29: write f3 [16013,2914] 0 2024-02-26T10:47:50.324 INFO:tasks.workunit.client.0.smithi060.stdout:3/30: rename c0 to c4 0 2024-02-26T10:47:50.326 INFO:tasks.workunit.client.0.smithi060.stdout:3/31: mkdir d5 0 2024-02-26T10:47:50.326 INFO:tasks.workunit.client.0.smithi060.stdout:3/32: write f3 [526868,19175] 0 2024-02-26T10:47:50.326 INFO:tasks.workunit.client.0.smithi060.stdout:3/33: write f2 [1744294,51053] 0 2024-02-26T10:47:50.337 INFO:tasks.workunit.client.0.smithi060.stdout:8/63: sync 2024-02-26T10:47:50.339 INFO:tasks.workunit.client.0.smithi060.stdout:3/34: unlink f2 0 2024-02-26T10:47:50.339 INFO:tasks.workunit.client.0.smithi060.stdout:2/74: read f0 [6924639,128027] 0 2024-02-26T10:47:50.339 INFO:tasks.workunit.client.0.smithi060.stdout:2/75: fdatasync fa 0 2024-02-26T10:47:50.341 INFO:tasks.workunit.client.0.smithi060.stdout:8/64: mknod d9/c11 0 2024-02-26T10:47:50.341 INFO:tasks.workunit.client.0.smithi060.stdout:8/65: truncate d9/fa 331197 0 2024-02-26T10:47:50.341 INFO:tasks.workunit.client.0.smithi060.stdout:2/76: mknod dd/d11/c16 0 2024-02-26T10:47:50.342 INFO:tasks.workunit.client.0.smithi060.stdout:2/77: rmdir dd/d11 39 2024-02-26T10:47:50.350 INFO:tasks.workunit.client.0.smithi060.stdout:5/59: dwrite d0/fd [0,4194304] 0 2024-02-26T10:47:50.351 INFO:tasks.workunit.client.0.smithi060.stdout:6/62: dread f4 [0,4194304] 0 2024-02-26T10:47:50.351 INFO:tasks.workunit.client.0.smithi060.stdout:6/63: fsync d6/d7/f10 0 2024-02-26T10:47:50.353 INFO:tasks.workunit.client.0.smithi060.stdout:5/60: creat d0/de/d15/f18 x:0 0 0 2024-02-26T10:47:50.354 INFO:tasks.workunit.client.0.smithi060.stdout:6/64: symlink d6/l14 0 2024-02-26T10:47:50.363 INFO:tasks.workunit.client.0.smithi060.stdout:4/80: dwrite d0/da/dc/d16/f18 [0,4194304] 0 2024-02-26T10:47:50.363 INFO:tasks.workunit.client.0.smithi060.stdout:4/81: chown d0/f6 27640 1 2024-02-26T10:47:50.363 INFO:tasks.workunit.client.0.smithi060.stdout:4/82: chown d0/d7/f8 6 1 2024-02-26T10:47:50.374 INFO:tasks.workunit.client.0.smithi060.stdout:4/83: creat d0/da/dc/f1a x:0 0 0 2024-02-26T10:47:50.374 INFO:tasks.workunit.client.0.smithi060.stdout:4/84: write d0/f3 [5181119,96157] 0 2024-02-26T10:47:50.374 INFO:tasks.workunit.client.0.smithi060.stdout:4/85: read - d0/da/dc/d16/f19 zero size 2024-02-26T10:47:50.380 INFO:tasks.workunit.client.0.smithi060.stdout:8/66: dread d9/fc [0,4194304] 0 2024-02-26T10:47:50.380 INFO:tasks.workunit.client.0.smithi060.stdout:7/185: dwrite da/d15/d21/d2f/d37/f38 [0,4194304] 0 2024-02-26T10:47:50.381 INFO:tasks.workunit.client.0.smithi060.stdout:7/186: fsync da/d15/d21/d2f/d37/f38 0 2024-02-26T10:47:50.381 INFO:tasks.workunit.client.0.smithi060.stdout:7/187: write da/d15/d21/f2c [603980,84245] 0 2024-02-26T10:47:50.381 INFO:tasks.workunit.client.0.smithi060.stdout:7/188: write da/d15/f1a [8475950,25568] 0 2024-02-26T10:47:50.381 INFO:tasks.workunit.client.0.smithi060.stdout:7/189: write da/d15/f1a [927895,31678] 0 2024-02-26T10:47:50.382 INFO:tasks.workunit.client.0.smithi060.stdout:5/61: write d0/f14 [2064845,53448] 0 2024-02-26T10:47:50.382 INFO:tasks.workunit.client.0.smithi060.stdout:7/190: mkdir da/d15/d21/d41 0 2024-02-26T10:47:50.383 INFO:tasks.workunit.client.0.smithi060.stdout:5/62: getdents d0/de/d15 0 2024-02-26T10:47:50.386 INFO:tasks.workunit.client.0.smithi060.stdout:7/191: truncate f6 1207860 0 2024-02-26T10:47:50.391 INFO:tasks.workunit.client.0.smithi060.stdout:9/20: sync 2024-02-26T10:47:50.391 INFO:tasks.workunit.client.0.smithi060.stdout:9/21: dread - no filename 2024-02-26T10:47:50.391 INFO:tasks.workunit.client.0.smithi060.stdout:6/65: rmdir d6 39 2024-02-26T10:47:50.392 INFO:tasks.workunit.client.0.smithi060.stdout:7/192: symlink da/d15/l42 0 2024-02-26T10:47:50.393 INFO:tasks.workunit.client.0.smithi060.stdout:9/22: mkdir d3 0 2024-02-26T10:47:50.394 INFO:tasks.workunit.client.0.smithi060.stdout:6/66: creat d6/f15 x:0 0 0 2024-02-26T10:47:50.395 INFO:tasks.workunit.client.0.smithi060.stdout:7/193: truncate f7 3801706 0 2024-02-26T10:47:50.397 INFO:tasks.workunit.client.0.smithi060.stdout:1/50: dwrite d3/f7 [0,4194304] 0 2024-02-26T10:47:50.403 INFO:tasks.workunit.client.0.smithi060.stdout:9/23: rename l1 to d3/l4 0 2024-02-26T10:47:50.403 INFO:tasks.workunit.client.0.smithi060.stdout:7/194: truncate da/f24 586803 0 2024-02-26T10:47:50.404 INFO:tasks.workunit.client.0.smithi060.stdout:4/86: read d0/d7/f12 [783318,50445] 0 2024-02-26T10:47:50.406 INFO:tasks.workunit.client.0.smithi060.stdout:9/24: creat d3/f5 x:0 0 0 2024-02-26T10:47:50.406 INFO:tasks.workunit.client.0.smithi060.stdout:9/25: readlink l2 0 2024-02-26T10:47:50.408 INFO:tasks.workunit.client.0.smithi060.stdout:9/26: rename d3/f5 to d3/f6 0 2024-02-26T10:47:50.408 INFO:tasks.workunit.client.0.smithi060.stdout:2/78: dwrite f0 [0,4194304] 0 2024-02-26T10:47:50.409 INFO:tasks.workunit.client.0.smithi060.stdout:9/27: creat d3/f7 x:0 0 0 2024-02-26T10:47:50.409 INFO:tasks.workunit.client.0.smithi060.stdout:9/28: chown l0 3909 1 2024-02-26T10:47:50.410 INFO:tasks.workunit.client.0.smithi060.stdout:9/29: rename l2 to d3/l8 0 2024-02-26T10:47:50.410 INFO:tasks.workunit.client.0.smithi060.stdout:9/30: fdatasync d3/f7 0 2024-02-26T10:47:50.411 INFO:tasks.workunit.client.0.smithi060.stdout:9/31: mkdir d3/d9 0 2024-02-26T10:47:50.412 INFO:tasks.workunit.client.0.smithi060.stdout:9/32: mkdir d3/da 0 2024-02-26T10:47:50.416 INFO:tasks.workunit.client.0.smithi060.stdout:9/33: symlink d3/lb 0 2024-02-26T10:47:50.416 INFO:tasks.workunit.client.0.smithi060.stdout:9/34: dread - d3/f6 zero size 2024-02-26T10:47:50.420 INFO:tasks.workunit.client.0.smithi060.stdout:9/35: rename d3/da to d3/d9/dc 0 2024-02-26T10:47:50.420 INFO:tasks.workunit.client.0.smithi060.stdout:9/36: mknod d3/d9/dc/cd 0 2024-02-26T10:47:50.420 INFO:tasks.workunit.client.0.smithi060.stdout:9/37: truncate d3/f7 996799 0 2024-02-26T10:47:50.421 INFO:tasks.workunit.client.0.smithi060.stdout:9/38: readlink l0 0 2024-02-26T10:47:50.428 INFO:tasks.workunit.client.0.smithi060.stdout:3/35: dwrite f3 [0,4194304] 0 2024-02-26T10:47:50.429 INFO:tasks.workunit.client.0.smithi060.stdout:0/16: sync 2024-02-26T10:47:50.432 INFO:tasks.workunit.client.0.smithi060.stdout:0/17: mknod d0/c5 0 2024-02-26T10:47:50.432 INFO:tasks.workunit.client.0.smithi060.stdout:0/18: fdatasync d0/f4 0 2024-02-26T10:47:50.432 INFO:tasks.workunit.client.0.smithi060.stdout:0/19: readlink - no filename 2024-02-26T10:47:50.443 INFO:tasks.workunit.client.0.smithi060.stdout:4/87: dread d0/fb [0,4194304] 0 2024-02-26T10:47:50.443 INFO:tasks.workunit.client.0.smithi060.stdout:4/88: truncate d0/da/dc/f1a 1000224 0 2024-02-26T10:47:50.443 INFO:tasks.workunit.client.0.smithi060.stdout:4/89: write d0/ff [361505,69925] 0 2024-02-26T10:47:50.447 INFO:tasks.workunit.client.0.smithi060.stdout:2/79: dread f4 [0,4194304] 0 2024-02-26T10:47:50.447 INFO:tasks.workunit.client.0.smithi060.stdout:2/80: chown dd/de/f15 8259 1 2024-02-26T10:47:50.447 INFO:tasks.workunit.client.0.smithi060.stdout:2/81: write fa [410927,35940] 0 2024-02-26T10:47:50.448 INFO:tasks.workunit.client.0.smithi060.stdout:2/82: symlink dd/d11/l17 0 2024-02-26T10:47:50.450 INFO:tasks.workunit.client.0.smithi060.stdout:2/83: link lb dd/d11/l18 0 2024-02-26T10:47:50.454 INFO:tasks.workunit.client.0.smithi060.stdout:2/84: mknod dd/de/c19 0 2024-02-26T10:47:50.454 INFO:tasks.workunit.client.0.smithi060.stdout:2/85: truncate dd/de/ff 16198 0 2024-02-26T10:47:50.456 INFO:tasks.workunit.client.0.smithi060.stdout:0/20: rmdir d0 39 2024-02-26T10:47:50.458 INFO:tasks.workunit.client.0.smithi060.stdout:0/21: getdents d0 0 2024-02-26T10:47:50.471 INFO:tasks.workunit.client.0.smithi060.stdout:3/36: dread f3 [0,4194304] 0 2024-02-26T10:47:50.471 INFO:tasks.workunit.client.0.smithi060.stdout:0/22: dread d0/f1 [0,4194304] 0 2024-02-26T10:47:50.475 INFO:tasks.workunit.client.0.smithi060.stdout:0/23: creat d0/f6 x:0 0 0 2024-02-26T10:47:50.477 INFO:tasks.workunit.client.0.smithi060.stdout:0/24: truncate d0/f3 2256460 0 2024-02-26T10:47:50.478 INFO:tasks.workunit.client.0.smithi060.stdout:0/25: mknod d0/c7 0 2024-02-26T10:47:50.480 INFO:tasks.workunit.client.0.smithi060.stdout:0/26: symlink d0/l8 0 2024-02-26T10:47:50.480 INFO:tasks.workunit.client.0.smithi060.stdout:0/27: write d0/f4 [277143,65823] 0 2024-02-26T10:47:50.481 INFO:tasks.workunit.client.0.smithi060.stdout:0/28: creat d0/f9 x:0 0 0 2024-02-26T10:47:50.481 INFO:tasks.workunit.client.0.smithi060.stdout:0/29: truncate d0/f6 551074 0 2024-02-26T10:47:50.482 INFO:tasks.workunit.client.0.smithi060.stdout:0/30: fdatasync d0/f9 0 2024-02-26T10:47:50.482 INFO:tasks.workunit.client.0.smithi060.stdout:0/31: stat d0/f4 0 2024-02-26T10:47:50.483 INFO:tasks.workunit.client.0.smithi060.stdout:0/32: symlink d0/la 0 2024-02-26T10:47:50.489 INFO:tasks.workunit.client.0.smithi060.stdout:0/33: mknod d0/cb 0 2024-02-26T10:47:50.489 INFO:tasks.workunit.client.0.smithi060.stdout:8/67: dwrite d9/f10 [0,4194304] 0 2024-02-26T10:47:50.493 INFO:tasks.workunit.client.0.smithi060.stdout:9/39: dwrite d3/f7 [0,4194304] 0 2024-02-26T10:47:50.499 INFO:tasks.workunit.client.0.smithi060.stdout:7/195: dwrite da/f13 [0,4194304] 0 2024-02-26T10:47:50.500 INFO:tasks.workunit.client.0.smithi060.stdout:0/34: dread d0/f1 [0,4194304] 0 2024-02-26T10:47:50.509 INFO:tasks.workunit.client.0.smithi060.stdout:3/37: dread f3 [0,4194304] 0 2024-02-26T10:47:50.509 INFO:tasks.workunit.client.0.smithi060.stdout:3/38: chown f3 50013598 1 2024-02-26T10:47:50.509 INFO:tasks.workunit.client.0.smithi060.stdout:3/39: fsync f3 0 2024-02-26T10:47:50.511 INFO:tasks.workunit.client.0.smithi060.stdout:1/51: write d3/db/fc [1491627,65320] 0 2024-02-26T10:47:50.512 INFO:tasks.workunit.client.0.smithi060.stdout:5/63: dwrite d0/de/d15/f18 [0,4194304] 0 2024-02-26T10:47:50.512 INFO:tasks.workunit.client.0.smithi060.stdout:6/67: dwrite d6/f9 [0,4194304] 0 2024-02-26T10:47:50.512 INFO:tasks.workunit.client.0.smithi060.stdout:6/68: dread - d6/fe zero size 2024-02-26T10:47:50.515 INFO:tasks.workunit.client.0.smithi060.stdout:5/64: rename d0/f10 to d0/de/d15/f19 0 2024-02-26T10:47:50.516 INFO:tasks.workunit.client.0.smithi060.stdout:0/35: rename d0/c5 to d0/cc 0 2024-02-26T10:47:50.516 INFO:tasks.workunit.client.0.smithi060.stdout:0/36: fdatasync d0/f9 0 2024-02-26T10:47:50.517 INFO:tasks.workunit.client.0.smithi060.stdout:7/196: dread da/f24 [0,4194304] 0 2024-02-26T10:47:50.517 INFO:tasks.workunit.client.0.smithi060.stdout:7/197: chown da/d15/d21/d2f/d37/l3e 3 1 2024-02-26T10:47:50.519 INFO:tasks.workunit.client.0.smithi060.stdout:3/40: truncate f3 2285739 0 2024-02-26T10:47:50.519 INFO:tasks.workunit.client.0.smithi060.stdout:5/65: getdents d0/de/d15 0 2024-02-26T10:47:50.520 INFO:tasks.workunit.client.0.smithi060.stdout:1/52: mknod d3/db/ce 0 2024-02-26T10:47:50.520 INFO:tasks.workunit.client.0.smithi060.stdout:1/53: readlink l1 0 2024-02-26T10:47:50.521 INFO:tasks.workunit.client.0.smithi060.stdout:5/66: truncate d0/f5 5160542 0 2024-02-26T10:47:50.521 INFO:tasks.workunit.client.0.smithi060.stdout:5/67: chown d0 3 1 2024-02-26T10:47:50.522 INFO:tasks.workunit.client.0.smithi060.stdout:5/68: write d0/de/d15/f18 [5235683,65183] 0 2024-02-26T10:47:50.525 INFO:tasks.workunit.client.0.smithi060.stdout:3/41: unlink c4 0 2024-02-26T10:47:50.525 INFO:tasks.workunit.client.0.smithi060.stdout:1/54: rename d3/db/fd to d3/db/ff 0 2024-02-26T10:47:50.525 INFO:tasks.workunit.client.0.smithi060.stdout:1/55: chown d3/fa 20357699 1 2024-02-26T10:47:50.527 INFO:tasks.workunit.client.0.smithi060.stdout:1/56: creat d3/f10 x:0 0 0 2024-02-26T10:47:50.529 INFO:tasks.workunit.client.0.smithi060.stdout:4/90: dread d0/ff [0,4194304] 0 2024-02-26T10:47:50.533 INFO:tasks.workunit.client.0.smithi060.stdout:5/69: chown d0/ca 2350 1 2024-02-26T10:47:50.534 INFO:tasks.workunit.client.0.smithi060.stdout:5/70: creat d0/de/d15/f1a x:0 0 0 2024-02-26T10:47:50.535 INFO:tasks.workunit.client.0.smithi060.stdout:5/71: write d0/de/d15/f16 [901105,87877] 0 2024-02-26T10:47:50.538 INFO:tasks.workunit.client.0.smithi060.stdout:5/72: symlink d0/de/d15/l1b 0 2024-02-26T10:47:50.541 INFO:tasks.workunit.client.0.smithi060.stdout:1/57: link d3/db/ff d3/db/f11 0 2024-02-26T10:47:50.559 INFO:tasks.workunit.client.0.smithi060.stdout:8/68: dwrite d9/f10 [0,4194304] 0 2024-02-26T10:47:50.562 INFO:tasks.workunit.client.0.smithi060.stdout:1/58: creat d3/f12 x:0 0 0 2024-02-26T10:47:50.563 INFO:tasks.workunit.client.0.smithi060.stdout:1/59: truncate d3/f9 248871 0 2024-02-26T10:47:50.563 INFO:tasks.workunit.client.0.smithi060.stdout:4/91: write d0/f6 [2036282,35938] 0 2024-02-26T10:47:50.564 INFO:tasks.workunit.client.0.smithi060.stdout:8/69: mknod d9/c12 0 2024-02-26T10:47:50.564 INFO:tasks.workunit.client.0.smithi060.stdout:8/70: read f2 [522448,77074] 0 2024-02-26T10:47:50.564 INFO:tasks.workunit.client.0.smithi060.stdout:8/71: read f0 [2720716,8772] 0 2024-02-26T10:47:50.564 INFO:tasks.workunit.client.0.smithi060.stdout:2/86: dwrite f4 [0,4194304] 0 2024-02-26T10:47:50.565 INFO:tasks.workunit.client.0.smithi060.stdout:1/60: truncate d3/f7 2841450 0 2024-02-26T10:47:50.566 INFO:tasks.workunit.client.0.smithi060.stdout:4/92: unlink d0/l1 0 2024-02-26T10:47:50.567 INFO:tasks.workunit.client.0.smithi060.stdout:4/93: chown d0/da/dc/f1a 1561 1 2024-02-26T10:47:50.572 INFO:tasks.workunit.client.0.smithi060.stdout:0/37: dread d0/f6 [0,4194304] 0 2024-02-26T10:47:50.572 INFO:tasks.workunit.client.0.smithi060.stdout:0/38: write d0/f3 [387435,80065] 0 2024-02-26T10:47:50.572 INFO:tasks.workunit.client.0.smithi060.stdout:8/72: creat d9/f13 x:0 0 0 2024-02-26T10:47:50.572 INFO:tasks.workunit.client.0.smithi060.stdout:1/61: mknod d3/c13 0 2024-02-26T10:47:50.577 INFO:tasks.workunit.client.0.smithi060.stdout:0/39: mkdir d0/dd 0 2024-02-26T10:47:50.585 INFO:tasks.workunit.client.0.smithi060.stdout:0/40: mknod d0/ce 0 2024-02-26T10:47:50.587 INFO:tasks.workunit.client.0.smithi060.stdout:6/69: dwrite d6/d7/f10 [0,4194304] 0 2024-02-26T10:47:50.592 INFO:tasks.workunit.client.0.smithi060.stdout:3/42: dwrite f3 [0,4194304] 0 2024-02-26T10:47:50.594 INFO:tasks.workunit.client.0.smithi060.stdout:9/40: dwrite d3/f7 [4194304,4194304] 0 2024-02-26T10:47:50.594 INFO:tasks.workunit.client.0.smithi060.stdout:9/41: truncate d3/f6 805958 0 2024-02-26T10:47:50.594 INFO:tasks.workunit.client.0.smithi060.stdout:9/42: chown d3/f7 48317045 1 2024-02-26T10:47:50.595 INFO:tasks.workunit.client.0.smithi060.stdout:0/41: mknod d0/dd/cf 0 2024-02-26T10:47:50.596 INFO:tasks.workunit.client.0.smithi060.stdout:9/43: write d3/f6 [63021,54034] 0 2024-02-26T10:47:50.599 INFO:tasks.workunit.client.0.smithi060.stdout:6/70: creat d6/f16 x:0 0 0 2024-02-26T10:47:50.599 INFO:tasks.workunit.client.0.smithi060.stdout:0/42: symlink d0/dd/l10 0 2024-02-26T10:47:50.612 INFO:tasks.workunit.client.0.smithi060.stdout:3/43: truncate f3 2007811 0 2024-02-26T10:47:50.612 INFO:tasks.workunit.client.0.smithi060.stdout:1/62: dread d3/db/f11 [0,4194304] 0 2024-02-26T10:47:50.612 INFO:tasks.workunit.client.0.smithi060.stdout:3/44: stat f3 0 2024-02-26T10:47:50.615 INFO:tasks.workunit.client.0.smithi060.stdout:8/73: dread f2 [0,4194304] 0 2024-02-26T10:47:50.629 INFO:tasks.workunit.client.0.smithi060.stdout:7/198: dread da/d15/f1a [0,4194304] 0 2024-02-26T10:47:50.630 INFO:tasks.workunit.client.0.smithi060.stdout:1/63: mknod d3/db/c14 0 2024-02-26T10:47:50.630 INFO:tasks.workunit.client.0.smithi060.stdout:1/64: fsync d3/f7 0 2024-02-26T10:47:50.630 INFO:tasks.workunit.client.0.smithi060.stdout:4/94: dwrite d0/ff [0,4194304] 0 2024-02-26T10:47:50.632 INFO:tasks.workunit.client.0.smithi060.stdout:7/199: creat da/d15/d21/d32/f43 x:0 0 0 2024-02-26T10:47:50.632 INFO:tasks.workunit.client.0.smithi060.stdout:8/74: rename f0 to d9/f14 0 2024-02-26T10:47:50.632 INFO:tasks.workunit.client.0.smithi060.stdout:7/200: truncate f6 1728090 0 2024-02-26T10:47:50.632 INFO:tasks.workunit.client.0.smithi060.stdout:7/201: readlink l5 0 2024-02-26T10:47:50.633 INFO:tasks.workunit.client.0.smithi060.stdout:3/45: link f3 d5/f6 0 2024-02-26T10:47:50.633 INFO:tasks.workunit.client.0.smithi060.stdout:5/73: dwrite d0/de/d15/f1a [0,4194304] 0 2024-02-26T10:47:50.634 INFO:tasks.workunit.client.0.smithi060.stdout:1/65: rename d3/fa to d3/f15 0 2024-02-26T10:47:50.636 INFO:tasks.workunit.client.0.smithi060.stdout:7/202: symlink da/d15/d21/d2f/d37/l44 0 2024-02-26T10:47:50.639 INFO:tasks.workunit.client.0.smithi060.stdout:7/203: write da/fe [1948007,114959] 0 2024-02-26T10:47:50.639 INFO:tasks.workunit.client.0.smithi060.stdout:4/95: rmdir d0 39 2024-02-26T10:47:50.639 INFO:tasks.workunit.client.0.smithi060.stdout:3/46: rmdir d5 39 2024-02-26T10:47:50.642 INFO:tasks.workunit.client.0.smithi060.stdout:9/44: dread d3/f7 [0,4194304] 0 2024-02-26T10:47:50.642 INFO:tasks.workunit.client.0.smithi060.stdout:9/45: fdatasync d3/f7 0 2024-02-26T10:47:50.642 INFO:tasks.workunit.client.0.smithi060.stdout:7/204: symlink da/d15/d21/d2f/d37/l45 0 2024-02-26T10:47:50.642 INFO:tasks.workunit.client.0.smithi060.stdout:8/75: truncate f3 2459194 0 2024-02-26T10:47:50.643 INFO:tasks.workunit.client.0.smithi060.stdout:7/205: unlink da/d15/d21/d2f/d37/l3e 0 2024-02-26T10:47:50.645 INFO:tasks.workunit.client.0.smithi060.stdout:7/206: creat da/d15/f46 x:0 0 0 2024-02-26T10:47:50.646 INFO:tasks.workunit.client.0.smithi060.stdout:7/207: unlink l5 0 2024-02-26T10:47:50.647 INFO:tasks.workunit.client.0.smithi060.stdout:7/208: chown da/l11 0 1 2024-02-26T10:47:50.647 INFO:tasks.workunit.client.0.smithi060.stdout:7/209: readlink da/d15/l1e 0 2024-02-26T10:47:50.651 INFO:tasks.workunit.client.0.smithi060.stdout:7/210: creat da/d15/d21/d32/f47 x:0 0 0 2024-02-26T10:47:50.651 INFO:tasks.workunit.client.0.smithi060.stdout:7/211: chown da/d15/l1c 2029656587 1 2024-02-26T10:47:50.666 INFO:tasks.workunit.client.0.smithi060.stdout:3/47: write f3 [454850,18644] 0 2024-02-26T10:47:50.673 INFO:tasks.workunit.client.0.smithi060.stdout:6/71: write d6/d7/fd [3908425,53557] 0 2024-02-26T10:47:50.673 INFO:tasks.workunit.client.0.smithi060.stdout:6/72: write d6/f16 [528436,5461] 0 2024-02-26T10:47:50.673 INFO:tasks.workunit.client.0.smithi060.stdout:6/73: chown d6/d7/fd 3232923 1 2024-02-26T10:47:50.673 INFO:tasks.workunit.client.0.smithi060.stdout:6/74: readlink d6/lb 0 2024-02-26T10:47:50.673 INFO:tasks.workunit.client.0.smithi060.stdout:6/75: rename d6/d7 to d6/d7/d17 22 2024-02-26T10:47:50.673 INFO:tasks.workunit.client.0.smithi060.stdout:6/76: fsync d6/d7/f13 0 2024-02-26T10:47:50.675 INFO:tasks.workunit.client.0.smithi060.stdout:9/46: write d3/f7 [2641398,24241] 0 2024-02-26T10:47:50.680 INFO:tasks.workunit.client.0.smithi060.stdout:2/87: dwrite f4 [4194304,4194304] 0 2024-02-26T10:47:50.702 INFO:tasks.workunit.client.0.smithi060.stdout:8/76: truncate f3 1235933 0 2024-02-26T10:47:50.703 INFO:tasks.workunit.client.0.smithi060.stdout:0/43: dwrite d0/f1 [0,4194304] 0 2024-02-26T10:47:50.703 INFO:tasks.workunit.client.0.smithi060.stdout:5/74: dread d0/f8 [0,4194304] 0 2024-02-26T10:47:50.703 INFO:tasks.workunit.client.0.smithi060.stdout:5/75: rename d0/de to d0/de/d1c 22 2024-02-26T10:47:50.704 INFO:tasks.workunit.client.0.smithi060.stdout:9/47: truncate d3/f7 1873435 0 2024-02-26T10:47:50.709 INFO:tasks.workunit.client.0.smithi060.stdout:7/212: dwrite da/d15/d21/f28 [0,4194304] 0 2024-02-26T10:47:50.712 INFO:tasks.workunit.client.0.smithi060.stdout:7/213: rmdir da/d15/d21 39 2024-02-26T10:47:50.713 INFO:tasks.workunit.client.0.smithi060.stdout:0/44: write d0/f4 [49039,36907] 0 2024-02-26T10:47:50.714 INFO:tasks.workunit.client.0.smithi060.stdout:7/214: creat da/d15/d21/d2f/f48 x:0 0 0 2024-02-26T10:47:50.714 INFO:tasks.workunit.client.0.smithi060.stdout:0/45: write d0/f4 [1131986,104083] 0 2024-02-26T10:47:50.715 INFO:tasks.workunit.client.0.smithi060.stdout:0/46: mknod d0/c11 0 2024-02-26T10:47:50.715 INFO:tasks.workunit.client.0.smithi060.stdout:0/47: stat d0/c7 0 2024-02-26T10:47:50.716 INFO:tasks.workunit.client.0.smithi060.stdout:4/96: dwrite d0/fb [4194304,4194304] 0 2024-02-26T10:47:50.720 INFO:tasks.workunit.client.0.smithi060.stdout:7/215: rename da/d15/l23 to da/l49 0 2024-02-26T10:47:50.731 INFO:tasks.workunit.client.0.smithi060.stdout:2/88: dwrite fc [0,4194304] 0 2024-02-26T10:47:50.734 INFO:tasks.workunit.client.0.smithi060.stdout:9/48: write d3/f7 [2133155,53799] 0 2024-02-26T10:47:50.769 INFO:tasks.workunit.client.0.smithi060.stdout:1/66: dwrite d3/f15 [0,4194304] 0 2024-02-26T10:47:50.769 INFO:tasks.workunit.client.0.smithi060.stdout:1/67: dread - d3/f10 zero size 2024-02-26T10:47:50.769 INFO:tasks.workunit.client.0.smithi060.stdout:1/68: fsync d3/f10 0 2024-02-26T10:47:50.772 INFO:tasks.workunit.client.0.smithi060.stdout:5/76: write d0/de/d15/f1a [3734801,112114] 0 2024-02-26T10:47:50.774 INFO:tasks.workunit.client.0.smithi060.stdout:8/77: dread d9/f10 [0,4194304] 0 2024-02-26T10:47:50.781 INFO:tasks.workunit.client.0.smithi060.stdout:1/69: link d3/db/c14 d3/db/c16 0 2024-02-26T10:47:50.781 INFO:tasks.workunit.client.0.smithi060.stdout:0/48: dwrite d0/f9 [0,4194304] 0 2024-02-26T10:47:50.784 INFO:tasks.workunit.client.0.smithi060.stdout:8/78: dread d9/fa [0,4194304] 0 2024-02-26T10:47:50.789 INFO:tasks.workunit.client.0.smithi060.stdout:8/79: getdents d9 0 2024-02-26T10:47:50.791 INFO:tasks.workunit.client.0.smithi060.stdout:8/80: mkdir d9/d15 0 2024-02-26T10:47:50.791 INFO:tasks.workunit.client.0.smithi060.stdout:8/81: fdatasync d9/f14 0 2024-02-26T10:47:50.792 INFO:tasks.workunit.client.0.smithi060.stdout:8/82: rename d9/c11 to d9/c16 0 2024-02-26T10:47:50.794 INFO:tasks.workunit.client.0.smithi060.stdout:8/83: rename d9/c12 to d9/c17 0 2024-02-26T10:47:50.794 INFO:tasks.workunit.client.0.smithi060.stdout:8/84: chown c1 77947 1 2024-02-26T10:47:50.796 INFO:tasks.workunit.client.0.smithi060.stdout:6/77: dwrite d6/d7/ff [0,4194304] 0 2024-02-26T10:47:50.796 INFO:tasks.workunit.client.0.smithi060.stdout:6/78: write d6/f9 [4786430,106445] 0 2024-02-26T10:47:50.799 INFO:tasks.workunit.client.0.smithi060.stdout:5/77: dread d0/f9 [0,4194304] 0 2024-02-26T10:47:50.799 INFO:tasks.workunit.client.0.smithi060.stdout:6/79: creat d6/f18 x:0 0 0 2024-02-26T10:47:50.809 INFO:tasks.workunit.client.0.smithi060.stdout:6/80: dread d6/d7/f13 [0,4194304] 0 2024-02-26T10:47:50.811 INFO:tasks.workunit.client.0.smithi060.stdout:4/97: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:50.811 INFO:tasks.workunit.client.0.smithi060.stdout:4/98: chown d0/d7/f12 1996223 1 2024-02-26T10:47:50.811 INFO:tasks.workunit.client.0.smithi060.stdout:0/49: dread d0/f9 [0,4194304] 0 2024-02-26T10:47:50.814 INFO:tasks.workunit.client.0.smithi060.stdout:4/99: mkdir d0/da/dc/d1b 0 2024-02-26T10:47:50.815 INFO:tasks.workunit.client.0.smithi060.stdout:0/50: mkdir d0/d12 0 2024-02-26T10:47:50.815 INFO:tasks.workunit.client.0.smithi060.stdout:0/51: truncate d0/f1 216952 0 2024-02-26T10:47:50.815 INFO:tasks.workunit.client.0.smithi060.stdout:0/52: getdents d0/d12 0 2024-02-26T10:47:50.844 INFO:tasks.workunit.client.0.smithi060.stdout:1/70: dwrite d3/f9 [0,4194304] 0 2024-02-26T10:47:50.844 INFO:tasks.workunit.client.0.smithi060.stdout:1/71: fdatasync d3/db/ff 0 2024-02-26T10:47:50.846 INFO:tasks.workunit.client.0.smithi060.stdout:1/72: rmdir d3 39 2024-02-26T10:47:50.847 INFO:tasks.workunit.client.0.smithi060.stdout:1/73: symlink d3/l17 0 2024-02-26T10:47:50.847 INFO:tasks.workunit.client.0.smithi060.stdout:1/74: chown d3 126997 1 2024-02-26T10:47:50.851 INFO:tasks.workunit.client.0.smithi060.stdout:2/89: dwrite f4 [0,4194304] 0 2024-02-26T10:47:50.866 INFO:tasks.workunit.client.0.smithi060.stdout:2/90: truncate f4 5759915 0 2024-02-26T10:47:50.866 INFO:tasks.workunit.client.0.smithi060.stdout:2/91: write dd/de/f15 [25320,104334] 0 2024-02-26T10:47:50.866 INFO:tasks.workunit.client.0.smithi060.stdout:1/75: dread d3/db/fc [0,4194304] 0 2024-02-26T10:47:50.867 INFO:tasks.workunit.client.0.smithi060.stdout:5/78: dwrite d0/de/f12 [0,4194304] 0 2024-02-26T10:47:50.871 INFO:tasks.workunit.client.0.smithi060.stdout:2/92: chown c8 2620 1 2024-02-26T10:47:50.871 INFO:tasks.workunit.client.0.smithi060.stdout:1/76: rmdir d3 39 2024-02-26T10:47:50.887 INFO:tasks.workunit.client.0.smithi060.stdout:0/53: dwrite d0/f9 [0,4194304] 0 2024-02-26T10:47:50.887 INFO:tasks.workunit.client.0.smithi060.stdout:1/77: write d3/db/ff [2643904,12066] 0 2024-02-26T10:47:50.887 INFO:tasks.workunit.client.0.smithi060.stdout:4/100: dwrite d0/d7/f8 [4194304,4194304] 0 2024-02-26T10:47:50.888 INFO:tasks.workunit.client.0.smithi060.stdout:1/78: getdents d3/db 0 2024-02-26T10:47:50.889 INFO:tasks.workunit.client.0.smithi060.stdout:4/101: symlink d0/da/dc/d1b/l1c 0 2024-02-26T10:47:50.890 INFO:tasks.workunit.client.0.smithi060.stdout:1/79: rename d3/db/c14 to d3/c18 0 2024-02-26T10:47:50.890 INFO:tasks.workunit.client.0.smithi060.stdout:4/102: symlink d0/da/dc/d16/l1d 0 2024-02-26T10:47:50.893 INFO:tasks.workunit.client.0.smithi060.stdout:1/80: symlink d3/db/l19 0 2024-02-26T10:47:50.893 INFO:tasks.workunit.client.0.smithi060.stdout:0/54: dread d0/f6 [0,4194304] 0 2024-02-26T10:47:50.893 INFO:tasks.workunit.client.0.smithi060.stdout:0/55: write d0/f4 [1350554,129067] 0 2024-02-26T10:47:50.898 INFO:tasks.workunit.client.0.smithi060.stdout:8/85: dwrite f8 [0,4194304] 0 2024-02-26T10:47:50.898 INFO:tasks.workunit.client.0.smithi060.stdout:8/86: write d9/fa [448969,45508] 0 2024-02-26T10:47:50.898 INFO:tasks.workunit.client.0.smithi060.stdout:8/87: getdents d9/d15 0 2024-02-26T10:47:50.910 INFO:tasks.workunit.client.0.smithi060.stdout:6/81: dwrite d6/f9 [0,4194304] 0 2024-02-26T10:47:50.911 INFO:tasks.workunit.client.0.smithi060.stdout:6/82: mkdir d6/d7/d19 0 2024-02-26T10:47:50.920 INFO:tasks.workunit.client.0.smithi060.stdout:1/81: write d3/f9 [3127835,90085] 0 2024-02-26T10:47:50.927 INFO:tasks.workunit.client.0.smithi060.stdout:8/88: dread d9/f14 [0,4194304] 0 2024-02-26T10:47:50.929 INFO:tasks.workunit.client.0.smithi060.stdout:8/89: mknod d9/d15/c18 0 2024-02-26T10:47:50.931 INFO:tasks.workunit.client.0.smithi060.stdout:4/103: write d0/da/dc/d16/f18 [3090746,72377] 0 2024-02-26T10:47:50.931 INFO:tasks.workunit.client.0.smithi060.stdout:4/104: stat d0/f5 0 2024-02-26T10:47:50.931 INFO:tasks.workunit.client.0.smithi060.stdout:5/79: dread d0/de/d15/f19 [0,4194304] 0 2024-02-26T10:47:50.931 INFO:tasks.workunit.client.0.smithi060.stdout:4/105: write d0/ff [5005422,40608] 0 2024-02-26T10:47:50.931 INFO:tasks.workunit.client.0.smithi060.stdout:2/93: dwrite fa [0,4194304] 0 2024-02-26T10:47:50.931 INFO:tasks.workunit.client.0.smithi060.stdout:4/106: write d0/d7/f12 [1543493,104097] 0 2024-02-26T10:47:50.931 INFO:tasks.workunit.client.0.smithi060.stdout:4/107: chown d0/da/dc/f1a 1 1 2024-02-26T10:47:50.933 INFO:tasks.workunit.client.0.smithi060.stdout:8/90: mkdir d9/d19 0 2024-02-26T10:47:50.933 INFO:tasks.workunit.client.0.smithi060.stdout:8/91: fsync f8 0 2024-02-26T10:47:50.933 INFO:tasks.workunit.client.0.smithi060.stdout:8/92: fsync d9/fc 0 2024-02-26T10:47:50.935 INFO:tasks.workunit.client.0.smithi060.stdout:5/80: creat d0/f1d x:0 0 0 2024-02-26T10:47:50.953 INFO:tasks.workunit.client.0.smithi060.stdout:0/56: dwrite d0/f4 [0,4194304] 0 2024-02-26T10:47:50.954 INFO:tasks.workunit.client.0.smithi060.stdout:1/82: dread d3/f15 [0,4194304] 0 2024-02-26T10:47:50.956 INFO:tasks.workunit.client.0.smithi060.stdout:9/49: sync 2024-02-26T10:47:50.957 INFO:tasks.workunit.client.0.smithi060.stdout:7/216: sync 2024-02-26T10:47:50.957 INFO:tasks.workunit.client.0.smithi060.stdout:3/48: sync 2024-02-26T10:47:50.957 INFO:tasks.workunit.client.0.smithi060.stdout:3/49: chown d5 167239 1 2024-02-26T10:47:50.957 INFO:tasks.workunit.client.0.smithi060.stdout:3/50: write f3 [1728399,72793] 0 2024-02-26T10:47:50.960 INFO:tasks.workunit.client.0.smithi060.stdout:1/83: symlink d3/db/l1a 0 2024-02-26T10:47:50.961 INFO:tasks.workunit.client.0.smithi060.stdout:7/217: write da/f18 [5191752,94489] 0 2024-02-26T10:47:50.961 INFO:tasks.workunit.client.0.smithi060.stdout:9/50: creat d3/d9/fe x:0 0 0 2024-02-26T10:47:50.961 INFO:tasks.workunit.client.0.smithi060.stdout:9/51: fsync d3/f6 0 2024-02-26T10:47:50.963 INFO:tasks.workunit.client.0.smithi060.stdout:3/51: mknod d5/c7 0 2024-02-26T10:47:50.964 INFO:tasks.workunit.client.0.smithi060.stdout:1/84: creat d3/f1b x:0 0 0 2024-02-26T10:47:50.964 INFO:tasks.workunit.client.0.smithi060.stdout:7/218: creat da/d15/d21/d29/f4a x:0 0 0 2024-02-26T10:47:50.964 INFO:tasks.workunit.client.0.smithi060.stdout:9/52: creat d3/d9/dc/ff x:0 0 0 2024-02-26T10:47:50.968 INFO:tasks.workunit.client.0.smithi060.stdout:1/85: unlink d3/db/l19 0 2024-02-26T10:47:50.968 INFO:tasks.workunit.client.0.smithi060.stdout:7/219: rmdir da 39 2024-02-26T10:47:50.968 INFO:tasks.workunit.client.0.smithi060.stdout:7/220: chown da/fd 22808409 1 2024-02-26T10:47:50.969 INFO:tasks.workunit.client.0.smithi060.stdout:1/86: chown d3/f7 66649823 1 2024-02-26T10:47:50.969 INFO:tasks.workunit.client.0.smithi060.stdout:1/87: truncate d3/f1b 576399 0 2024-02-26T10:47:50.972 INFO:tasks.workunit.client.0.smithi060.stdout:7/221: write da/fe [1192831,128544] 0 2024-02-26T10:47:50.979 INFO:tasks.workunit.client.0.smithi060.stdout:1/88: symlink d3/l1c 0 2024-02-26T10:47:50.993 INFO:tasks.workunit.client.0.smithi060.stdout:0/57: read d0/f6 [210072,31171] 0 2024-02-26T10:47:50.995 INFO:tasks.workunit.client.0.smithi060.stdout:8/93: dwrite d9/f14 [0,4194304] 0 2024-02-26T10:47:50.995 INFO:tasks.workunit.client.0.smithi060.stdout:0/58: symlink d0/dd/l13 0 2024-02-26T10:47:50.996 INFO:tasks.workunit.client.0.smithi060.stdout:8/94: mknod d9/d15/c1a 0 2024-02-26T10:47:50.997 INFO:tasks.workunit.client.0.smithi060.stdout:0/59: symlink d0/l14 0 2024-02-26T10:47:50.998 INFO:tasks.workunit.client.0.smithi060.stdout:8/95: creat d9/d15/f1b x:0 0 0 2024-02-26T10:47:50.998 INFO:tasks.workunit.client.0.smithi060.stdout:8/96: chown d9/c17 28 1 2024-02-26T10:47:50.998 INFO:tasks.workunit.client.0.smithi060.stdout:8/97: chown f3 2258185 1 2024-02-26T10:47:50.998 INFO:tasks.workunit.client.0.smithi060.stdout:8/98: getdents d9/d19 0 2024-02-26T10:47:50.998 INFO:tasks.workunit.client.0.smithi060.stdout:0/60: creat d0/f15 x:0 0 0 2024-02-26T10:47:50.999 INFO:tasks.workunit.client.0.smithi060.stdout:4/108: dread d0/f6 [0,4194304] 0 2024-02-26T10:47:51.001 INFO:tasks.workunit.client.0.smithi060.stdout:4/109: symlink d0/da/l1e 0 2024-02-26T10:47:51.002 INFO:tasks.workunit.client.0.smithi060.stdout:4/110: mknod d0/da/dc/c1f 0 2024-02-26T10:47:51.010 INFO:tasks.workunit.client.0.smithi060.stdout:9/53: dwrite d3/f6 [0,4194304] 0 2024-02-26T10:47:51.014 INFO:tasks.workunit.client.0.smithi060.stdout:1/89: dread d3/db/f11 [0,4194304] 0 2024-02-26T10:47:51.036 INFO:tasks.workunit.client.0.smithi060.stdout:5/81: dwrite d0/de/d15/f1a [0,4194304] 0 2024-02-26T10:47:51.036 INFO:tasks.workunit.client.0.smithi060.stdout:7/222: dwrite da/d15/d21/d32/f33 [0,4194304] 0 2024-02-26T10:47:51.036 INFO:tasks.workunit.client.0.smithi060.stdout:1/90: rmdir d3 39 2024-02-26T10:47:51.036 INFO:tasks.workunit.client.0.smithi060.stdout:1/91: chown d3/db/fc 6049515 1 2024-02-26T10:47:51.036 INFO:tasks.workunit.client.0.smithi060.stdout:8/99: getdents d9/d15 0 2024-02-26T10:47:51.037 INFO:tasks.workunit.client.0.smithi060.stdout:6/83: dwrite d6/d7/ff [0,4194304] 0 2024-02-26T10:47:51.037 INFO:tasks.workunit.client.0.smithi060.stdout:0/61: rmdir d0 39 2024-02-26T10:47:51.039 INFO:tasks.workunit.client.0.smithi060.stdout:1/92: mknod d3/db/c1d 0 2024-02-26T10:47:51.039 INFO:tasks.workunit.client.0.smithi060.stdout:8/100: creat d9/d15/f1c x:0 0 0 2024-02-26T10:47:51.040 INFO:tasks.workunit.client.0.smithi060.stdout:8/101: truncate d9/f13 734305 0 2024-02-26T10:47:51.042 INFO:tasks.workunit.client.0.smithi060.stdout:0/62: rename d0/d12 to d0/dd/d16 0 2024-02-26T10:47:51.042 INFO:tasks.workunit.client.0.smithi060.stdout:0/63: truncate d0/f6 1549851 0 2024-02-26T10:47:51.043 INFO:tasks.workunit.client.0.smithi060.stdout:4/111: rmdir d0/da/dc 39 2024-02-26T10:47:51.048 INFO:tasks.workunit.client.0.smithi060.stdout:1/93: rmdir d3 39 2024-02-26T10:47:51.048 INFO:tasks.workunit.client.0.smithi060.stdout:1/94: write d3/db/ff [3505933,106001] 0 2024-02-26T10:47:51.055 INFO:tasks.workunit.client.0.smithi060.stdout:4/112: creat d0/da/dc/d16/f20 x:0 0 0 2024-02-26T10:47:51.056 INFO:tasks.workunit.client.0.smithi060.stdout:0/64: creat d0/dd/d16/f17 x:0 0 0 2024-02-26T10:47:51.058 INFO:tasks.workunit.client.0.smithi060.stdout:0/65: link d0/c11 d0/dd/c18 0 2024-02-26T10:47:51.059 INFO:tasks.workunit.client.0.smithi060.stdout:4/113: mknod d0/da/c21 0 2024-02-26T10:47:51.061 INFO:tasks.workunit.client.0.smithi060.stdout:0/66: rename d0/f4 to d0/dd/d16/f19 0 2024-02-26T10:47:51.061 INFO:tasks.workunit.client.0.smithi060.stdout:0/67: truncate d0/f6 2184281 0 2024-02-26T10:47:51.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/52: dwrite f3 [0,4194304] 0 2024-02-26T10:47:51.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/53: chown f3 52783 1 2024-02-26T10:47:51.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/54: readlink l1 0 2024-02-26T10:47:51.066 INFO:tasks.workunit.client.0.smithi060.stdout:4/114: mknod d0/c22 0 2024-02-26T10:47:51.068 INFO:tasks.workunit.client.0.smithi060.stdout:9/54: dwrite d3/d9/fe [0,4194304] 0 2024-02-26T10:47:51.068 INFO:tasks.workunit.client.0.smithi060.stdout:0/68: mknod d0/dd/d16/c1a 0 2024-02-26T10:47:51.070 INFO:tasks.workunit.client.0.smithi060.stdout:9/55: getdents d3 0 2024-02-26T10:47:51.071 INFO:tasks.workunit.client.0.smithi060.stdout:9/56: mkdir d3/d9/d10 0 2024-02-26T10:47:51.076 INFO:tasks.workunit.client.0.smithi060.stdout:0/69: dread d0/f6 [0,4194304] 0 2024-02-26T10:47:51.076 INFO:tasks.workunit.client.0.smithi060.stdout:0/70: chown d0/dd/cf 5735 1 2024-02-26T10:47:51.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:50 smithi113 ceph-mon[36126]: pgmap v481: 129 pgs: 129 active+clean; 128 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 2.18k op/s 2024-02-26T10:47:51.082 INFO:tasks.workunit.client.0.smithi060.stdout:0/71: symlink d0/dd/l1b 0 2024-02-26T10:47:51.098 INFO:tasks.workunit.client.0.smithi060.stdout:1/95: dread d3/db/f11 [0,4194304] 0 2024-02-26T10:47:51.098 INFO:tasks.workunit.client.0.smithi060.stdout:1/96: write d3/f1b [1522645,98656] 0 2024-02-26T10:47:51.098 INFO:tasks.workunit.client.0.smithi060.stdout:1/97: chown d3/f7 147699177 1 2024-02-26T10:47:51.098 INFO:tasks.workunit.client.0.smithi060.stdout:4/115: dread d0/f5 [0,4194304] 0 2024-02-26T10:47:51.098 INFO:tasks.workunit.client.0.smithi060.stdout:1/98: truncate d3/f9 3383929 0 2024-02-26T10:47:51.102 INFO:tasks.workunit.client.0.smithi060.stdout:1/99: mkdir d3/db/d1e 0 2024-02-26T10:47:51.102 INFO:tasks.workunit.client.0.smithi060.stdout:1/100: readlink d3/db/l1a 0 2024-02-26T10:47:51.102 INFO:tasks.workunit.client.0.smithi060.stdout:8/102: dread f8 [0,4194304] 0 2024-02-26T10:47:51.104 INFO:tasks.workunit.client.0.smithi060.stdout:8/103: unlink d9/fa 0 2024-02-26T10:47:51.106 INFO:tasks.workunit.client.0.smithi060.stdout:8/104: truncate d9/f14 679094 0 2024-02-26T10:47:51.107 INFO:tasks.workunit.client.0.smithi060.stdout:5/82: dwrite d0/fd [0,4194304] 0 2024-02-26T10:47:51.107 INFO:tasks.workunit.client.0.smithi060.stdout:8/105: mkdir d9/d1d 0 2024-02-26T10:47:51.107 INFO:tasks.workunit.client.0.smithi060.stdout:8/106: dread - d9/d15/f1c zero size 2024-02-26T10:47:51.108 INFO:tasks.workunit.client.0.smithi060.stdout:5/83: mkdir d0/de/d15/d1e 0 2024-02-26T10:47:51.109 INFO:tasks.workunit.client.0.smithi060.stdout:5/84: rmdir d0/de/d15/d1e 0 2024-02-26T10:47:51.114 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:50 smithi060 ceph-mon[31539]: pgmap v481: 129 pgs: 129 active+clean; 128 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 2.18k op/s 2024-02-26T10:47:51.114 INFO:tasks.workunit.client.0.smithi060.stdout:6/84: dwrite d6/f16 [0,4194304] 0 2024-02-26T10:47:51.114 INFO:tasks.workunit.client.0.smithi060.stdout:7/223: dwrite da/d15/f19 [0,4194304] 0 2024-02-26T10:47:51.115 INFO:tasks.workunit.client.0.smithi060.stdout:7/224: mkdir da/d15/d21/d4b 0 2024-02-26T10:47:51.119 INFO:tasks.workunit.client.0.smithi060.stdout:7/225: link da/d15/l27 da/d15/l4c 0 2024-02-26T10:47:51.119 INFO:tasks.workunit.client.0.smithi060.stdout:7/226: readlink l8 0 2024-02-26T10:47:51.119 INFO:tasks.workunit.client.0.smithi060.stdout:7/227: fsync f7 0 2024-02-26T10:47:51.120 INFO:tasks.workunit.client.0.smithi060.stdout:7/228: symlink da/d15/d34/d39/l4d 0 2024-02-26T10:47:51.120 INFO:tasks.workunit.client.0.smithi060.stdout:7/229: truncate f2 4695247 0 2024-02-26T10:47:51.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:50 smithi100 ceph-mon[36051]: pgmap v481: 129 pgs: 129 active+clean; 128 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 2.18k op/s 2024-02-26T10:47:51.131 INFO:tasks.workunit.client.0.smithi060.stdout:3/55: write d5/f6 [3332041,77461] 0 2024-02-26T10:47:51.137 INFO:tasks.workunit.client.0.smithi060.stdout:1/101: read d3/f9 [1251728,25322] 0 2024-02-26T10:47:51.140 INFO:tasks.workunit.client.0.smithi060.stdout:4/116: write d0/f6 [4850131,115092] 0 2024-02-26T10:47:51.144 INFO:tasks.workunit.client.0.smithi060.stdout:6/85: write d6/d7/f10 [4174441,90858] 0 2024-02-26T10:47:51.145 INFO:tasks.workunit.client.0.smithi060.stdout:6/86: chown d6/fe 1342748 1 2024-02-26T10:47:51.145 INFO:tasks.workunit.client.0.smithi060.stdout:9/57: dwrite d3/f7 [0,4194304] 0 2024-02-26T10:47:51.147 INFO:tasks.workunit.client.0.smithi060.stdout:6/87: rmdir d6/d7/d19 0 2024-02-26T10:47:51.152 INFO:tasks.workunit.client.0.smithi060.stdout:0/72: dwrite d0/dd/d16/f17 [0,4194304] 0 2024-02-26T10:47:51.153 INFO:tasks.workunit.client.0.smithi060.stdout:0/73: fdatasync d0/f1 0 2024-02-26T10:47:51.156 INFO:tasks.workunit.client.0.smithi060.stdout:7/230: rename da/d15/d34 to da/d15/d21/d32/d4e 0 2024-02-26T10:47:51.156 INFO:tasks.workunit.client.0.smithi060.stdout:4/117: dread d0/d7/f12 [0,4194304] 0 2024-02-26T10:47:51.157 INFO:tasks.workunit.client.0.smithi060.stdout:4/118: write d0/da/dc/d16/f18 [4317694,44446] 0 2024-02-26T10:47:51.157 INFO:tasks.workunit.client.0.smithi060.stdout:4/119: write d0/da/dc/f1a [1856988,38012] 0 2024-02-26T10:47:51.164 INFO:tasks.workunit.client.0.smithi060.stdout:0/74: creat d0/dd/d16/f1c x:0 0 0 2024-02-26T10:47:51.164 INFO:tasks.workunit.client.0.smithi060.stdout:0/75: stat d0/c7 0 2024-02-26T10:47:51.165 INFO:tasks.workunit.client.0.smithi060.stdout:7/231: mknod da/d15/d21/d32/d4e/d3b/c4f 0 2024-02-26T10:47:51.167 INFO:tasks.workunit.client.0.smithi060.stdout:1/102: rename d3/f10 to d3/db/d1e/f1f 0 2024-02-26T10:47:51.168 INFO:tasks.workunit.client.0.smithi060.stdout:1/103: write d3/f15 [5042141,120376] 0 2024-02-26T10:47:51.168 INFO:tasks.workunit.client.0.smithi060.stdout:1/104: chown d3/l1c 306 1 2024-02-26T10:47:51.169 INFO:tasks.workunit.client.0.smithi060.stdout:3/56: dread f3 [0,4194304] 0 2024-02-26T10:47:51.169 INFO:tasks.workunit.client.0.smithi060.stdout:0/76: creat d0/dd/f1d x:0 0 0 2024-02-26T10:47:51.170 INFO:tasks.workunit.client.0.smithi060.stdout:7/232: creat da/d15/d21/d4b/f50 x:0 0 0 2024-02-26T10:47:51.175 INFO:tasks.workunit.client.0.smithi060.stdout:6/88: write d6/f16 [2431242,105121] 0 2024-02-26T10:47:51.175 INFO:tasks.workunit.client.0.smithi060.stdout:6/89: readlink l3 0 2024-02-26T10:47:51.176 INFO:tasks.workunit.client.0.smithi060.stdout:5/85: dread d0/fd [0,4194304] 0 2024-02-26T10:47:51.176 INFO:tasks.workunit.client.0.smithi060.stdout:8/107: dwrite f8 [0,4194304] 0 2024-02-26T10:47:51.177 INFO:tasks.workunit.client.0.smithi060.stdout:1/105: rename d3/f7 to d3/f20 0 2024-02-26T10:47:51.178 INFO:tasks.workunit.client.0.smithi060.stdout:3/57: symlink d5/l8 0 2024-02-26T10:47:51.178 INFO:tasks.workunit.client.0.smithi060.stdout:3/58: stat l1 0 2024-02-26T10:47:51.178 INFO:tasks.workunit.client.0.smithi060.stdout:3/59: chown d5/f6 143 1 2024-02-26T10:47:51.179 INFO:tasks.workunit.client.0.smithi060.stdout:7/233: creat da/d15/d21/d32/d4e/f51 x:0 0 0 2024-02-26T10:47:51.179 INFO:tasks.workunit.client.0.smithi060.stdout:7/234: read da/d15/f1a [4098277,66922] 0 2024-02-26T10:47:51.180 INFO:tasks.workunit.client.0.smithi060.stdout:5/86: link d0/fd d0/de/f1f 0 2024-02-26T10:47:51.188 INFO:tasks.workunit.client.0.smithi060.stdout:8/108: link d9/d15/f1c d9/f1e 0 2024-02-26T10:47:51.188 INFO:tasks.workunit.client.0.smithi060.stdout:8/109: readlink d9/le 0 2024-02-26T10:47:51.188 INFO:tasks.workunit.client.0.smithi060.stdout:8/110: fdatasync d9/f1e 0 2024-02-26T10:47:51.188 INFO:tasks.workunit.client.0.smithi060.stdout:8/111: chown d9/d15/c1a 358838460 1 2024-02-26T10:47:51.188 INFO:tasks.workunit.client.0.smithi060.stdout:9/58: dwrite d3/f7 [0,4194304] 0 2024-02-26T10:47:51.188 INFO:tasks.workunit.client.0.smithi060.stdout:3/60: creat d5/f9 x:0 0 0 2024-02-26T10:47:51.199 INFO:tasks.workunit.client.0.smithi060.stdout:7/235: link da/d15/l16 da/l52 0 2024-02-26T10:47:51.200 INFO:tasks.workunit.client.0.smithi060.stdout:7/236: write da/d15/d21/d32/f47 [112847,76521] 0 2024-02-26T10:47:51.200 INFO:tasks.workunit.client.0.smithi060.stdout:7/237: fdatasync da/d15/d21/d32/f33 0 2024-02-26T10:47:51.201 INFO:tasks.workunit.client.0.smithi060.stdout:7/238: fsync da/d15/d21/d2f/f48 0 2024-02-26T10:47:51.207 INFO:tasks.workunit.client.0.smithi060.stdout:2/94: sync 2024-02-26T10:47:51.214 INFO:tasks.workunit.client.0.smithi060.stdout:5/87: write d0/f5 [2457915,98694] 0 2024-02-26T10:47:51.220 INFO:tasks.workunit.client.0.smithi060.stdout:4/120: rename d0/f3 to d0/da/dc/f23 0 2024-02-26T10:47:51.220 INFO:tasks.workunit.client.0.smithi060.stdout:4/121: dread - d0/da/dc/d16/f20 zero size 2024-02-26T10:47:51.220 INFO:tasks.workunit.client.0.smithi060.stdout:2/95: rename fc to dd/d11/d13/f1a 0 2024-02-26T10:47:51.220 INFO:tasks.workunit.client.0.smithi060.stdout:6/90: dread d6/d7/f10 [0,4194304] 0 2024-02-26T10:47:51.221 INFO:tasks.workunit.client.0.smithi060.stdout:2/96: symlink dd/d11/l1b 0 2024-02-26T10:47:51.222 INFO:tasks.workunit.client.0.smithi060.stdout:2/97: truncate f3 464907 0 2024-02-26T10:47:51.222 INFO:tasks.workunit.client.0.smithi060.stdout:2/98: fsync f4 0 2024-02-26T10:47:51.225 INFO:tasks.workunit.client.0.smithi060.stdout:8/112: dread d9/f10 [0,4194304] 0 2024-02-26T10:47:51.225 INFO:tasks.workunit.client.0.smithi060.stdout:8/113: rename d9/d1d to d9/d1d/d1f 22 2024-02-26T10:47:51.226 INFO:tasks.workunit.client.0.smithi060.stdout:8/114: chown d9/fc 62442 1 2024-02-26T10:47:51.227 INFO:tasks.workunit.client.0.smithi060.stdout:8/115: truncate d9/f13 570368 0 2024-02-26T10:47:51.227 INFO:tasks.workunit.client.0.smithi060.stdout:8/116: fsync d9/f14 0 2024-02-26T10:47:51.227 INFO:tasks.workunit.client.0.smithi060.stdout:8/117: write d9/f14 [1709796,50605] 0 2024-02-26T10:47:51.227 INFO:tasks.workunit.client.0.smithi060.stdout:9/59: read d3/d9/fe [4112404,31419] 0 2024-02-26T10:47:51.228 INFO:tasks.workunit.client.0.smithi060.stdout:9/60: getdents d3 0 2024-02-26T10:47:51.261 INFO:tasks.workunit.client.0.smithi060.stdout:5/88: write d0/de/f1f [551542,119561] 0 2024-02-26T10:47:51.264 INFO:tasks.workunit.client.0.smithi060.stdout:9/61: write d3/f7 [2689565,127085] 0 2024-02-26T10:47:51.264 INFO:tasks.workunit.client.0.smithi060.stdout:5/89: write d0/f5 [5406052,23252] 0 2024-02-26T10:47:51.266 INFO:tasks.workunit.client.0.smithi060.stdout:9/62: creat d3/d9/d10/f11 x:0 0 0 2024-02-26T10:47:51.266 INFO:tasks.workunit.client.0.smithi060.stdout:9/63: read - d3/d9/d10/f11 zero size 2024-02-26T10:47:51.267 INFO:tasks.workunit.client.0.smithi060.stdout:5/90: getdents d0/de/d15 0 2024-02-26T10:47:51.268 INFO:tasks.workunit.client.0.smithi060.stdout:9/64: symlink d3/d9/dc/l12 0 2024-02-26T10:47:51.268 INFO:tasks.workunit.client.0.smithi060.stdout:9/65: write d3/f7 [230350,114394] 0 2024-02-26T10:47:51.271 INFO:tasks.workunit.client.0.smithi060.stdout:9/66: mknod d3/d9/c13 0 2024-02-26T10:47:51.271 INFO:tasks.workunit.client.0.smithi060.stdout:9/67: write d3/d9/dc/ff [945031,22454] 0 2024-02-26T10:47:51.272 INFO:tasks.workunit.client.0.smithi060.stdout:0/77: dwrite d0/dd/d16/f17 [0,4194304] 0 2024-02-26T10:47:51.276 INFO:tasks.workunit.client.0.smithi060.stdout:9/68: symlink d3/d9/l14 0 2024-02-26T10:47:51.278 INFO:tasks.workunit.client.0.smithi060.stdout:0/78: rename d0/f6 to d0/dd/f1e 0 2024-02-26T10:47:51.278 INFO:tasks.workunit.client.0.smithi060.stdout:0/79: stat d0/dd/d16/f19 0 2024-02-26T10:47:51.279 INFO:tasks.workunit.client.0.smithi060.stdout:9/69: mkdir d3/d15 0 2024-02-26T10:47:51.279 INFO:tasks.workunit.client.0.smithi060.stdout:0/80: creat d0/dd/d16/f1f x:0 0 0 2024-02-26T10:47:51.281 INFO:tasks.workunit.client.0.smithi060.stdout:9/70: creat d3/f16 x:0 0 0 2024-02-26T10:47:51.291 INFO:tasks.workunit.client.0.smithi060.stdout:9/71: fdatasync d3/f7 0 2024-02-26T10:47:51.292 INFO:tasks.workunit.client.0.smithi060.stdout:1/106: dwrite d3/f20 [0,4194304] 0 2024-02-26T10:47:51.293 INFO:tasks.workunit.client.0.smithi060.stdout:4/122: dwrite d0/da/f14 [0,4194304] 0 2024-02-26T10:47:51.294 INFO:tasks.workunit.client.0.smithi060.stdout:9/72: creat d3/d15/f17 x:0 0 0 2024-02-26T10:47:51.294 INFO:tasks.workunit.client.0.smithi060.stdout:9/73: write d3/d9/d10/f11 [514072,45238] 0 2024-02-26T10:47:51.294 INFO:tasks.workunit.client.0.smithi060.stdout:9/74: write d3/d15/f17 [540965,90282] 0 2024-02-26T10:47:51.295 INFO:tasks.workunit.client.0.smithi060.stdout:4/123: chown d0/fb 295811 1 2024-02-26T10:47:51.295 INFO:tasks.workunit.client.0.smithi060.stdout:1/107: rename d3/db/ce to d3/db/c21 0 2024-02-26T10:47:51.295 INFO:tasks.workunit.client.0.smithi060.stdout:4/124: creat d0/da/dc/d16/f24 x:0 0 0 2024-02-26T10:47:51.295 INFO:tasks.workunit.client.0.smithi060.stdout:1/108: rmdir d3/db 39 2024-02-26T10:47:51.296 INFO:tasks.workunit.client.0.smithi060.stdout:4/125: truncate d0/d7/f12 362610 0 2024-02-26T10:47:51.296 INFO:tasks.workunit.client.0.smithi060.stdout:1/109: rmdir d3 39 2024-02-26T10:47:51.298 INFO:tasks.workunit.client.0.smithi060.stdout:4/126: creat d0/d7/f25 x:0 0 0 2024-02-26T10:47:51.298 INFO:tasks.workunit.client.0.smithi060.stdout:1/110: creat d3/db/d1e/f22 x:0 0 0 2024-02-26T10:47:51.301 INFO:tasks.workunit.client.0.smithi060.stdout:4/127: link d0/d7/f8 d0/d7/f26 0 2024-02-26T10:47:51.301 INFO:tasks.workunit.client.0.smithi060.stdout:4/128: write d0/da/dc/d16/f24 [368477,101829] 0 2024-02-26T10:47:51.301 INFO:tasks.workunit.client.0.smithi060.stdout:2/99: dread f4 [0,4194304] 0 2024-02-26T10:47:51.302 INFO:tasks.workunit.client.0.smithi060.stdout:6/91: dwrite d6/f15 [0,4194304] 0 2024-02-26T10:47:51.302 INFO:tasks.workunit.client.0.smithi060.stdout:3/61: dwrite d5/f6 [0,4194304] 0 2024-02-26T10:47:51.309 INFO:tasks.workunit.client.0.smithi060.stdout:0/81: truncate d0/dd/d16/f17 4111752 0 2024-02-26T10:47:51.309 INFO:tasks.workunit.client.0.smithi060.stdout:0/82: fdatasync d0/dd/f1e 0 2024-02-26T10:47:51.321 INFO:tasks.workunit.client.0.smithi060.stdout:5/91: getdents d0 0 2024-02-26T10:47:51.321 INFO:tasks.workunit.client.0.smithi060.stdout:5/92: truncate d0/f9 1120606 0 2024-02-26T10:47:51.322 INFO:tasks.workunit.client.0.smithi060.stdout:5/93: mkdir d0/de/d20 0 2024-02-26T10:47:51.324 INFO:tasks.workunit.client.0.smithi060.stdout:5/94: rename d0/de/d15/f19 to d0/de/d15/f21 0 2024-02-26T10:47:51.324 INFO:tasks.workunit.client.0.smithi060.stdout:0/83: rename d0/dd/l1b to d0/dd/d16/l20 0 2024-02-26T10:47:51.326 INFO:tasks.workunit.client.0.smithi060.stdout:5/95: creat d0/de/f22 x:0 0 0 2024-02-26T10:47:51.326 INFO:tasks.workunit.client.0.smithi060.stdout:0/84: rename d0/dd/d16/c1a to d0/dd/d16/c21 0 2024-02-26T10:47:51.326 INFO:tasks.workunit.client.0.smithi060.stdout:1/111: fsync d3/db/d1e/f22 0 2024-02-26T10:47:51.326 INFO:tasks.workunit.client.0.smithi060.stdout:5/96: write d0/de/f22 [279207,104726] 0 2024-02-26T10:47:51.327 INFO:tasks.workunit.client.0.smithi060.stdout:0/85: rename d0/dd/l10 to d0/l22 0 2024-02-26T10:47:51.327 INFO:tasks.workunit.client.0.smithi060.stdout:0/86: fdatasync d0/f15 0 2024-02-26T10:47:51.327 INFO:tasks.workunit.client.0.smithi060.stdout:0/87: stat d0/c7 0 2024-02-26T10:47:51.327 INFO:tasks.workunit.client.0.smithi060.stdout:0/88: write d0/f3 [1066502,1805] 0 2024-02-26T10:47:51.328 INFO:tasks.workunit.client.0.smithi060.stdout:1/112: creat d3/f23 x:0 0 0 2024-02-26T10:47:51.329 INFO:tasks.workunit.client.0.smithi060.stdout:0/89: link d0/dd/d16/f1c d0/f23 0 2024-02-26T10:47:51.330 INFO:tasks.workunit.client.0.smithi060.stdout:1/113: rename d3/c5 to d3/c24 0 2024-02-26T10:47:51.331 INFO:tasks.workunit.client.0.smithi060.stdout:1/114: getdents d3/db/d1e 0 2024-02-26T10:47:51.331 INFO:tasks.workunit.client.0.smithi060.stdout:1/115: truncate d3/db/d1e/f1f 176830 0 2024-02-26T10:47:51.336 INFO:tasks.workunit.client.0.smithi060.stdout:5/97: rename d0/ca to d0/de/d15/c23 0 2024-02-26T10:47:51.339 INFO:tasks.workunit.client.0.smithi060.stdout:4/129: dread d0/da/dc/f1a [0,4194304] 0 2024-02-26T10:47:51.339 INFO:tasks.workunit.client.0.smithi060.stdout:4/130: mkdir d0/d27 0 2024-02-26T10:47:51.353 INFO:tasks.workunit.client.0.smithi060.stdout:2/100: dwrite dd/de/f15 [0,4194304] 0 2024-02-26T10:47:51.359 INFO:tasks.workunit.client.0.smithi060.stdout:9/75: dwrite d3/f6 [0,4194304] 0 2024-02-26T10:47:51.359 INFO:tasks.workunit.client.0.smithi060.stdout:9/76: dread - d3/f16 zero size 2024-02-26T10:47:51.362 INFO:tasks.workunit.client.0.smithi060.stdout:9/77: unlink d3/f6 0 2024-02-26T10:47:51.363 INFO:tasks.workunit.client.0.smithi060.stdout:9/78: write d3/d9/dc/ff [1382543,96393] 0 2024-02-26T10:47:51.363 INFO:tasks.workunit.client.0.smithi060.stdout:9/79: symlink d3/d15/l18 0 2024-02-26T10:47:51.365 INFO:tasks.workunit.client.0.smithi060.stdout:6/92: dwrite d6/f11 [0,4194304] 0 2024-02-26T10:47:51.366 INFO:tasks.workunit.client.0.smithi060.stdout:9/80: creat d3/d9/f19 x:0 0 0 2024-02-26T10:47:51.366 INFO:tasks.workunit.client.0.smithi060.stdout:9/81: stat d3/d9/dc/ff 0 2024-02-26T10:47:51.366 INFO:tasks.workunit.client.0.smithi060.stdout:9/82: fsync d3/f7 0 2024-02-26T10:47:51.366 INFO:tasks.workunit.client.0.smithi060.stdout:2/101: dread dd/de/ff [0,4194304] 0 2024-02-26T10:47:51.367 INFO:tasks.workunit.client.0.smithi060.stdout:6/93: symlink d6/d7/l1a 0 2024-02-26T10:47:51.368 INFO:tasks.workunit.client.0.smithi060.stdout:2/102: creat dd/d11/f1c x:0 0 0 2024-02-26T10:47:51.369 INFO:tasks.workunit.client.0.smithi060.stdout:6/94: mkdir d6/d1b 0 2024-02-26T10:47:51.370 INFO:tasks.workunit.client.0.smithi060.stdout:8/118: dwrite d9/f14 [0,4194304] 0 2024-02-26T10:47:51.371 INFO:tasks.workunit.client.0.smithi060.stdout:9/83: fsync d3/d9/dc/ff 0 2024-02-26T10:47:51.371 INFO:tasks.workunit.client.0.smithi060.stdout:9/84: truncate d3/f16 282426 0 2024-02-26T10:47:51.371 INFO:tasks.workunit.client.0.smithi060.stdout:9/85: readlink d3/lb 0 2024-02-26T10:47:51.371 INFO:tasks.workunit.client.0.smithi060.stdout:9/86: read - d3/d9/f19 zero size 2024-02-26T10:47:51.371 INFO:tasks.workunit.client.0.smithi060.stdout:2/103: link f3 dd/f1d 0 2024-02-26T10:47:51.371 INFO:tasks.workunit.client.0.smithi060.stdout:2/104: chown dd/l14 8161 1 2024-02-26T10:47:51.372 INFO:tasks.workunit.client.0.smithi060.stdout:8/119: rmdir d9/d1d 0 2024-02-26T10:47:51.376 INFO:tasks.workunit.client.0.smithi060.stdout:9/87: write d3/d15/f17 [182885,50197] 0 2024-02-26T10:47:51.376 INFO:tasks.workunit.client.0.smithi060.stdout:9/88: fsync d3/d9/dc/ff 0 2024-02-26T10:47:51.376 INFO:tasks.workunit.client.0.smithi060.stdout:2/105: creat dd/de/f1e x:0 0 0 2024-02-26T10:47:51.376 INFO:tasks.workunit.client.0.smithi060.stdout:2/106: truncate f5 8996630 0 2024-02-26T10:47:51.380 INFO:tasks.workunit.client.0.smithi060.stdout:3/62: write f3 [1348976,92915] 0 2024-02-26T10:47:51.386 INFO:tasks.workunit.client.0.smithi060.stdout:9/89: dread d3/f16 [0,4194304] 0 2024-02-26T10:47:51.386 INFO:tasks.workunit.client.0.smithi060.stdout:4/131: dread d0/f6 [4194304,4194304] 0 2024-02-26T10:47:51.388 INFO:tasks.workunit.client.0.smithi060.stdout:9/90: symlink d3/d9/l1a 0 2024-02-26T10:47:51.388 INFO:tasks.workunit.client.0.smithi060.stdout:9/91: readlink d3/d15/l18 0 2024-02-26T10:47:51.389 INFO:tasks.workunit.client.0.smithi060.stdout:4/132: creat d0/d7/f28 x:0 0 0 2024-02-26T10:47:51.389 INFO:tasks.workunit.client.0.smithi060.stdout:4/133: chown d0/da/dc/f1a 10 1 2024-02-26T10:47:51.389 INFO:tasks.workunit.client.0.smithi060.stdout:4/134: dread - d0/da/dc/d16/f19 zero size 2024-02-26T10:47:51.391 INFO:tasks.workunit.client.0.smithi060.stdout:9/92: mkdir d3/d15/d1b 0 2024-02-26T10:47:51.391 INFO:tasks.workunit.client.0.smithi060.stdout:9/93: truncate d3/d9/fe 5122323 0 2024-02-26T10:47:51.391 INFO:tasks.workunit.client.0.smithi060.stdout:9/94: write d3/f16 [200144,33422] 0 2024-02-26T10:47:51.392 INFO:tasks.workunit.client.0.smithi060.stdout:8/120: dread d9/f13 [0,4194304] 0 2024-02-26T10:47:51.393 INFO:tasks.workunit.client.0.smithi060.stdout:9/95: symlink d3/l1c 0 2024-02-26T10:47:51.395 INFO:tasks.workunit.client.0.smithi060.stdout:3/63: fsync f3 0 2024-02-26T10:47:51.395 INFO:tasks.workunit.client.0.smithi060.stdout:0/90: dread d0/dd/d16/f17 [0,4194304] 0 2024-02-26T10:47:51.395 INFO:tasks.workunit.client.0.smithi060.stdout:0/91: fsync d0/dd/d16/f1c 0 2024-02-26T10:47:51.398 INFO:tasks.workunit.client.0.smithi060.stdout:8/121: creat d9/f20 x:0 0 0 2024-02-26T10:47:51.398 INFO:tasks.workunit.client.0.smithi060.stdout:9/96: rename d3/d9/dc/cd to d3/d15/d1b/c1d 0 2024-02-26T10:47:51.402 INFO:tasks.workunit.client.0.smithi060.stdout:9/97: mkdir d3/d1e 0 2024-02-26T10:47:51.403 INFO:tasks.workunit.client.0.smithi060.stdout:0/92: truncate d0/f9 2126184 0 2024-02-26T10:47:51.403 INFO:tasks.workunit.client.0.smithi060.stdout:9/98: creat d3/d9/f1f x:0 0 0 2024-02-26T10:47:51.406 INFO:tasks.workunit.client.0.smithi060.stdout:0/93: symlink d0/dd/d16/l24 0 2024-02-26T10:47:51.406 INFO:tasks.workunit.client.0.smithi060.stdout:0/94: chown d0/dd/d16/f1c 1716612 1 2024-02-26T10:47:51.413 INFO:tasks.workunit.client.0.smithi060.stdout:0/95: rmdir d0/dd/d16 39 2024-02-26T10:47:51.413 INFO:tasks.workunit.client.0.smithi060.stdout:0/96: fsync d0/dd/d16/f1c 0 2024-02-26T10:47:51.413 INFO:tasks.workunit.client.0.smithi060.stdout:0/97: mknod d0/dd/c25 0 2024-02-26T10:47:51.414 INFO:tasks.workunit.client.0.smithi060.stdout:9/99: rmdir d3 39 2024-02-26T10:47:51.415 INFO:tasks.workunit.client.0.smithi060.stdout:0/98: creat d0/dd/d16/f26 x:0 0 0 2024-02-26T10:47:51.418 INFO:tasks.workunit.client.0.smithi060.stdout:4/135: dread d0/da/dc/f23 [0,4194304] 0 2024-02-26T10:47:51.426 INFO:tasks.workunit.client.0.smithi060.stdout:5/98: dwrite d0/f9 [0,4194304] 0 2024-02-26T10:47:51.426 INFO:tasks.workunit.client.0.smithi060.stdout:6/95: dread d6/d7/ff [0,4194304] 0 2024-02-26T10:47:51.427 INFO:tasks.workunit.client.0.smithi060.stdout:8/122: dread d9/fc [0,4194304] 0 2024-02-26T10:47:51.430 INFO:tasks.workunit.client.0.smithi060.stdout:6/96: mkdir d6/d1b/d1c 0 2024-02-26T10:47:51.431 INFO:tasks.workunit.client.0.smithi060.stdout:5/99: readlink d0/l7 0 2024-02-26T10:47:51.431 INFO:tasks.workunit.client.0.smithi060.stdout:5/100: write d0/f1d [686840,114620] 0 2024-02-26T10:47:51.432 INFO:tasks.workunit.client.0.smithi060.stdout:6/97: creat d6/f1d x:0 0 0 2024-02-26T10:47:51.433 INFO:tasks.workunit.client.0.smithi060.stdout:5/101: creat d0/de/d20/f24 x:0 0 0 2024-02-26T10:47:51.433 INFO:tasks.workunit.client.0.smithi060.stdout:6/98: symlink d6/d7/l1e 0 2024-02-26T10:47:51.438 INFO:tasks.workunit.client.0.smithi060.stdout:4/136: unlink d0/f6 0 2024-02-26T10:47:51.439 INFO:tasks.workunit.client.0.smithi060.stdout:4/137: mkdir d0/d7/d29 0 2024-02-26T10:47:51.444 INFO:tasks.workunit.client.0.smithi060.stdout:3/64: dwrite d5/f9 [0,4194304] 0 2024-02-26T10:47:51.449 INFO:tasks.workunit.client.0.smithi060.stdout:3/65: link d5/f6 d5/fa 0 2024-02-26T10:47:51.449 INFO:tasks.workunit.client.0.smithi060.stdout:3/66: write f3 [4517755,119268] 0 2024-02-26T10:47:51.459 INFO:tasks.workunit.client.0.smithi060.stdout:4/138: dread d0/d7/f8 [0,4194304] 0 2024-02-26T10:47:51.460 INFO:tasks.workunit.client.0.smithi060.stdout:4/139: rmdir d0 39 2024-02-26T10:47:51.466 INFO:tasks.workunit.client.0.smithi060.stdout:9/100: dread d3/d9/d10/f11 [0,4194304] 0 2024-02-26T10:47:51.466 INFO:tasks.workunit.client.0.smithi060.stdout:9/101: truncate d3/d9/f19 255327 0 2024-02-26T10:47:51.467 INFO:tasks.workunit.client.0.smithi060.stdout:9/102: readlink d3/d9/dc/l12 0 2024-02-26T10:47:51.468 INFO:tasks.workunit.client.0.smithi060.stdout:3/67: dread f3 [4194304,4194304] 0 2024-02-26T10:47:51.474 INFO:tasks.workunit.client.0.smithi060.stdout:3/68: fsync f3 0 2024-02-26T10:47:51.475 INFO:tasks.workunit.client.0.smithi060.stdout:3/69: chown f3 955 1 2024-02-26T10:47:51.475 INFO:tasks.workunit.client.0.smithi060.stdout:3/70: stat d5 0 2024-02-26T10:47:51.475 INFO:tasks.workunit.client.0.smithi060.stdout:3/71: write d5/fa [4785188,117561] 0 2024-02-26T10:47:51.475 INFO:tasks.workunit.client.0.smithi060.stdout:3/72: creat d5/fb x:0 0 0 2024-02-26T10:47:51.483 INFO:tasks.workunit.client.0.smithi060.stdout:3/73: rmdir d5 39 2024-02-26T10:47:51.483 INFO:tasks.workunit.client.0.smithi060.stdout:3/74: write d5/fa [2307305,65040] 0 2024-02-26T10:47:51.483 INFO:tasks.workunit.client.0.smithi060.stdout:9/103: dread d3/f16 [0,4194304] 0 2024-02-26T10:47:51.486 INFO:tasks.workunit.client.0.smithi060.stdout:4/140: dread d0/fb [0,4194304] 0 2024-02-26T10:47:51.486 INFO:tasks.workunit.client.0.smithi060.stdout:4/141: write d0/da/dc/d16/f24 [382796,117439] 0 2024-02-26T10:47:51.486 INFO:tasks.workunit.client.0.smithi060.stdout:4/142: readlink d0/da/dc/d16/l1d 0 2024-02-26T10:47:51.487 INFO:tasks.workunit.client.0.smithi060.stdout:9/104: symlink d3/d9/l20 0 2024-02-26T10:47:51.490 INFO:tasks.workunit.client.0.smithi060.stdout:0/99: dwrite d0/dd/d16/f17 [4194304,4194304] 0 2024-02-26T10:47:51.508 INFO:tasks.workunit.client.0.smithi060.stdout:0/100: symlink d0/dd/l27 0 2024-02-26T10:47:51.513 INFO:tasks.workunit.client.0.smithi060.stdout:9/105: dread d3/f7 [0,4194304] 0 2024-02-26T10:47:51.513 INFO:tasks.workunit.client.0.smithi060.stdout:9/106: truncate d3/d9/d10/f11 720235 0 2024-02-26T10:47:51.513 INFO:tasks.workunit.client.0.smithi060.stdout:0/101: getdents d0 0 2024-02-26T10:47:51.513 INFO:tasks.workunit.client.0.smithi060.stdout:0/102: stat d0/f9 0 2024-02-26T10:47:51.513 INFO:tasks.workunit.client.0.smithi060.stdout:0/103: creat d0/dd/d16/f28 x:0 0 0 2024-02-26T10:47:51.513 INFO:tasks.workunit.client.0.smithi060.stdout:0/104: dread - d0/dd/d16/f28 zero size 2024-02-26T10:47:51.514 INFO:tasks.workunit.client.0.smithi060.stdout:0/105: symlink d0/l29 0 2024-02-26T10:47:51.515 INFO:tasks.workunit.client.0.smithi060.stdout:0/106: link d0/c7 d0/dd/c2a 0 2024-02-26T10:47:51.523 INFO:tasks.workunit.client.0.smithi060.stdout:8/123: dwrite f8 [0,4194304] 0 2024-02-26T10:47:51.536 INFO:tasks.workunit.client.0.smithi060.stdout:5/102: dwrite d0/de/d20/f24 [0,4194304] 0 2024-02-26T10:47:51.537 INFO:tasks.workunit.client.0.smithi060.stdout:3/75: dwrite d5/f9 [0,4194304] 0 2024-02-26T10:47:51.539 INFO:tasks.workunit.client.0.smithi060.stdout:4/143: dwrite d0/d7/f12 [0,4194304] 0 2024-02-26T10:47:51.539 INFO:tasks.workunit.client.0.smithi060.stdout:4/144: fdatasync d0/da/dc/d16/f18 0 2024-02-26T10:47:51.539 INFO:tasks.workunit.client.0.smithi060.stdout:4/145: chown d0/da/dc/d16/f24 1522998055 1 2024-02-26T10:47:51.542 INFO:tasks.workunit.client.0.smithi060.stdout:3/76: rename d5/c7 to d5/cc 0 2024-02-26T10:47:51.545 INFO:tasks.workunit.client.0.smithi060.stdout:3/77: creat d5/fd x:0 0 0 2024-02-26T10:47:51.545 INFO:tasks.workunit.client.0.smithi060.stdout:3/78: fsync d5/fb 0 2024-02-26T10:47:51.546 INFO:tasks.workunit.client.0.smithi060.stdout:3/79: mkdir d5/de 0 2024-02-26T10:47:51.566 INFO:tasks.workunit.client.0.smithi060.stdout:9/107: dwrite d3/f7 [0,4194304] 0 2024-02-26T10:47:51.568 INFO:tasks.workunit.client.0.smithi060.stdout:0/107: dwrite d0/dd/d16/f19 [0,4194304] 0 2024-02-26T10:47:51.569 INFO:tasks.workunit.client.0.smithi060.stdout:0/108: fsync d0/f15 0 2024-02-26T10:47:51.569 INFO:tasks.workunit.client.0.smithi060.stdout:0/109: fsync d0/f15 0 2024-02-26T10:47:51.570 INFO:tasks.workunit.client.0.smithi060.stdout:0/110: creat d0/dd/f2b x:0 0 0 2024-02-26T10:47:51.572 INFO:tasks.workunit.client.0.smithi060.stdout:8/124: write f8 [721575,77105] 0 2024-02-26T10:47:51.572 INFO:tasks.workunit.client.0.smithi060.stdout:6/99: dwrite f4 [0,4194304] 0 2024-02-26T10:47:51.580 INFO:tasks.workunit.client.0.smithi060.stdout:6/100: unlink d6/d7/f13 0 2024-02-26T10:47:51.581 INFO:tasks.workunit.client.0.smithi060.stdout:6/101: dread - d6/f1d zero size 2024-02-26T10:47:51.581 INFO:tasks.workunit.client.0.smithi060.stdout:9/108: dread d3/d9/dc/ff [0,4194304] 0 2024-02-26T10:47:51.582 INFO:tasks.workunit.client.0.smithi060.stdout:6/102: mkdir d6/d1b/d1c/d1f 0 2024-02-26T10:47:51.593 INFO:tasks.workunit.client.0.smithi060.stdout:9/109: dread d3/d15/f17 [0,4194304] 0 2024-02-26T10:47:51.594 INFO:tasks.workunit.client.0.smithi060.stdout:9/110: unlink d3/d9/fe 0 2024-02-26T10:47:51.595 INFO:tasks.workunit.client.0.smithi060.stdout:7/239: sync 2024-02-26T10:47:51.595 INFO:tasks.workunit.client.0.smithi060.stdout:7/240: write da/d15/d21/f2c [781887,51005] 0 2024-02-26T10:47:51.596 INFO:tasks.workunit.client.0.smithi060.stdout:7/241: creat da/d15/f53 x:0 0 0 2024-02-26T10:47:51.597 INFO:tasks.workunit.client.0.smithi060.stdout:7/242: creat da/d15/d21/d32/d4e/d3b/f54 x:0 0 0 2024-02-26T10:47:51.599 INFO:tasks.workunit.client.0.smithi060.stdout:7/243: symlink da/l55 0 2024-02-26T10:47:51.599 INFO:tasks.workunit.client.0.smithi060.stdout:7/244: chown da/l52 41777290 1 2024-02-26T10:47:51.599 INFO:tasks.workunit.client.0.smithi060.stdout:7/245: dread - da/d15/f25 zero size 2024-02-26T10:47:51.603 INFO:tasks.workunit.client.0.smithi060.stdout:7/246: creat da/d15/d21/d32/d4e/d39/f56 x:0 0 0 2024-02-26T10:47:51.603 INFO:tasks.workunit.client.0.smithi060.stdout:7/247: truncate da/d15/d21/d32/d4e/f51 1030640 0 2024-02-26T10:47:51.604 INFO:tasks.workunit.client.0.smithi060.stdout:7/248: read da/fd [73608,40155] 0 2024-02-26T10:47:51.604 INFO:tasks.workunit.client.0.smithi060.stdout:7/249: creat da/d15/d21/d2f/f57 x:0 0 0 2024-02-26T10:47:51.611 INFO:tasks.workunit.client.0.smithi060.stdout:3/80: dwrite f3 [4194304,4194304] 0 2024-02-26T10:47:51.612 INFO:tasks.workunit.client.0.smithi060.stdout:7/250: rmdir da/d15/d21/d32/d4e 39 2024-02-26T10:47:51.612 INFO:tasks.workunit.client.0.smithi060.stdout:7/251: chown f6 127411 1 2024-02-26T10:47:51.612 INFO:tasks.workunit.client.0.smithi060.stdout:7/252: write da/d15/f46 [654712,30905] 0 2024-02-26T10:47:51.613 INFO:tasks.workunit.client.0.smithi060.stdout:7/253: write da/d15/f19 [4824552,63890] 0 2024-02-26T10:47:51.613 INFO:tasks.workunit.client.0.smithi060.stdout:3/81: mkdir d5/de/df 0 2024-02-26T10:47:51.613 INFO:tasks.workunit.client.0.smithi060.stdout:3/82: write d5/fa [375512,76112] 0 2024-02-26T10:47:51.614 INFO:tasks.workunit.client.0.smithi060.stdout:7/254: mkdir da/d15/d21/d41/d58 0 2024-02-26T10:47:51.614 INFO:tasks.workunit.client.0.smithi060.stdout:9/111: dread d3/d9/dc/ff [0,4194304] 0 2024-02-26T10:47:51.616 INFO:tasks.workunit.client.0.smithi060.stdout:9/112: creat d3/d9/d10/f21 x:0 0 0 2024-02-26T10:47:51.616 INFO:tasks.workunit.client.0.smithi060.stdout:9/113: write d3/d15/f17 [318678,18165] 0 2024-02-26T10:47:51.619 INFO:tasks.workunit.client.0.smithi060.stdout:5/103: dwrite d0/fc [0,4194304] 0 2024-02-26T10:47:51.619 INFO:tasks.workunit.client.0.smithi060.stdout:5/104: fsync d0/de/d15/f21 0 2024-02-26T10:47:51.634 INFO:tasks.workunit.client.0.smithi060.stdout:0/111: dwrite d0/dd/f1e [0,4194304] 0 2024-02-26T10:47:51.636 INFO:tasks.workunit.client.0.smithi060.stdout:0/112: rename d0/f23 to d0/dd/f2c 0 2024-02-26T10:47:51.639 INFO:tasks.workunit.client.0.smithi060.stdout:0/113: creat d0/dd/f2d x:0 0 0 2024-02-26T10:47:51.641 INFO:tasks.workunit.client.0.smithi060.stdout:7/255: dread f6 [0,4194304] 0 2024-02-26T10:47:51.642 INFO:tasks.workunit.client.0.smithi060.stdout:6/103: write d6/d7/ff [1345368,70693] 0 2024-02-26T10:47:51.646 INFO:tasks.workunit.client.0.smithi060.stdout:4/146: dwrite d0/ff [4194304,4194304] 0 2024-02-26T10:47:51.646 INFO:tasks.workunit.client.0.smithi060.stdout:4/147: readlink d0/da/dc/d1b/l1c 0 2024-02-26T10:47:51.654 INFO:tasks.workunit.client.0.smithi060.stdout:6/104: truncate d6/f16 2991105 0 2024-02-26T10:47:51.654 INFO:tasks.workunit.client.0.smithi060.stdout:6/105: fsync d6/d7/f10 0 2024-02-26T10:47:51.654 INFO:tasks.workunit.client.0.smithi060.stdout:6/106: chown d6/d7 2 1 2024-02-26T10:47:51.654 INFO:tasks.workunit.client.0.smithi060.stdout:6/107: getdents d6/d1b/d1c/d1f 0 2024-02-26T10:47:51.654 INFO:tasks.workunit.client.0.smithi060.stdout:6/108: write d6/d7/f12 [3573125,466] 0 2024-02-26T10:47:51.691 INFO:tasks.workunit.client.0.smithi060.stdout:5/105: dwrite d0/f1d [0,4194304] 0 2024-02-26T10:47:51.693 INFO:tasks.workunit.client.0.smithi060.stdout:3/83: dwrite f3 [0,4194304] 0 2024-02-26T10:47:51.693 INFO:tasks.workunit.client.0.smithi060.stdout:3/84: dread - d5/fd zero size 2024-02-26T10:47:51.693 INFO:tasks.workunit.client.0.smithi060.stdout:3/85: fsync d5/f6 0 2024-02-26T10:47:51.702 INFO:tasks.workunit.client.0.smithi060.stdout:0/114: dwrite d0/dd/f1e [0,4194304] 0 2024-02-26T10:47:51.702 INFO:tasks.workunit.client.0.smithi060.stdout:0/115: dread - d0/dd/f1d zero size 2024-02-26T10:47:51.702 INFO:tasks.workunit.client.0.smithi060.stdout:0/116: chown d0/f3 1364 1 2024-02-26T10:47:51.702 INFO:tasks.workunit.client.0.smithi060.stdout:0/117: dread - d0/dd/f2c zero size 2024-02-26T10:47:51.702 INFO:tasks.workunit.client.0.smithi060.stdout:0/118: stat d0/ce 0 2024-02-26T10:47:51.702 INFO:tasks.workunit.client.0.smithi060.stdout:0/119: truncate d0/dd/d16/f1f 929431 0 2024-02-26T10:47:51.705 INFO:tasks.workunit.client.0.smithi060.stdout:7/256: dwrite da/d15/f53 [0,4194304] 0 2024-02-26T10:47:51.706 INFO:tasks.workunit.client.0.smithi060.stdout:9/114: dwrite d3/d9/dc/ff [0,4194304] 0 2024-02-26T10:47:51.708 INFO:tasks.workunit.client.0.smithi060.stdout:7/257: symlink da/d15/d21/d2f/l59 0 2024-02-26T10:47:51.709 INFO:tasks.workunit.client.0.smithi060.stdout:9/115: rename d3/d15/l18 to d3/d15/d1b/l22 0 2024-02-26T10:47:51.709 INFO:tasks.workunit.client.0.smithi060.stdout:6/109: dwrite f4 [0,4194304] 0 2024-02-26T10:47:51.714 INFO:tasks.workunit.client.0.smithi060.stdout:9/116: mkdir d3/d15/d1b/d23 0 2024-02-26T10:47:51.714 INFO:tasks.workunit.client.0.smithi060.stdout:9/117: read - d3/d9/f1f zero size 2024-02-26T10:47:51.714 INFO:tasks.workunit.client.0.smithi060.stdout:9/118: write d3/d9/f1f [545366,51547] 0 2024-02-26T10:47:51.714 INFO:tasks.workunit.client.0.smithi060.stdout:7/258: mkdir da/d5a 0 2024-02-26T10:47:51.716 INFO:tasks.workunit.client.0.smithi060.stdout:9/119: rename l0 to d3/d9/dc/l24 0 2024-02-26T10:47:51.717 INFO:tasks.workunit.client.0.smithi060.stdout:9/120: truncate d3/d9/d10/f11 91267 0 2024-02-26T10:47:51.721 INFO:tasks.workunit.client.0.smithi060.stdout:9/121: rmdir d3 39 2024-02-26T10:47:51.721 INFO:tasks.workunit.client.0.smithi060.stdout:4/148: dwrite d0/d7/f26 [4194304,4194304] 0 2024-02-26T10:47:51.722 INFO:tasks.workunit.client.0.smithi060.stdout:7/259: dread da/d15/d21/d32/d4e/f51 [0,4194304] 0 2024-02-26T10:47:51.749 INFO:tasks.workunit.client.0.smithi060.stdout:0/120: dwrite d0/dd/f2b [0,4194304] 0 2024-02-26T10:47:51.764 INFO:tasks.workunit.client.0.smithi060.stdout:7/260: dwrite f2 [0,4194304] 0 2024-02-26T10:47:51.765 INFO:tasks.workunit.client.0.smithi060.stdout:7/261: chown da/d15/l1c 10081 1 2024-02-26T10:47:51.765 INFO:tasks.workunit.client.0.smithi060.stdout:7/262: write da/d15/d21/d29/f4a [1031725,114167] 0 2024-02-26T10:47:51.766 INFO:tasks.workunit.client.0.smithi060.stdout:0/121: mknod d0/dd/d16/c2e 0 2024-02-26T10:47:51.766 INFO:tasks.workunit.client.0.smithi060.stdout:0/122: read - d0/f15 zero size 2024-02-26T10:47:51.766 INFO:tasks.workunit.client.0.smithi060.stdout:0/123: truncate d0/f1 224080 0 2024-02-26T10:47:51.766 INFO:tasks.workunit.client.0.smithi060.stdout:4/149: write d0/d7/f8 [4571977,14753] 0 2024-02-26T10:47:51.767 INFO:tasks.workunit.client.0.smithi060.stdout:4/150: write d0/da/dc/f23 [1939042,55756] 0 2024-02-26T10:47:51.767 INFO:tasks.workunit.client.0.smithi060.stdout:4/151: readlink d0/d7/l13 0 2024-02-26T10:47:51.767 INFO:tasks.workunit.client.0.smithi060.stdout:7/263: creat da/d15/d21/d32/d4e/f5b x:0 0 0 2024-02-26T10:47:51.768 INFO:tasks.workunit.client.0.smithi060.stdout:0/124: link d0/cb d0/dd/c2f 0 2024-02-26T10:47:51.768 INFO:tasks.workunit.client.0.smithi060.stdout:0/125: chown d0/dd/f1e 16640769 1 2024-02-26T10:47:51.769 INFO:tasks.workunit.client.0.smithi060.stdout:4/152: truncate d0/fb 6879214 0 2024-02-26T10:47:51.769 INFO:tasks.workunit.client.0.smithi060.stdout:7/264: creat da/d15/d21/d32/d4e/f5c x:0 0 0 2024-02-26T10:47:51.769 INFO:tasks.workunit.client.0.smithi060.stdout:7/265: dread - da/d15/d21/d32/d4e/f5c zero size 2024-02-26T10:47:51.770 INFO:tasks.workunit.client.0.smithi060.stdout:4/153: getdents d0/da/dc/d1b 0 2024-02-26T10:47:51.773 INFO:tasks.workunit.client.0.smithi060.stdout:9/122: dwrite d3/f7 [0,4194304] 0 2024-02-26T10:47:51.791 INFO:tasks.workunit.client.0.smithi060.stdout:6/110: dwrite d6/d7/f12 [0,4194304] 0 2024-02-26T10:47:51.791 INFO:tasks.workunit.client.0.smithi060.stdout:6/111: stat d6/d1b/d1c 0 2024-02-26T10:47:51.798 INFO:tasks.workunit.client.0.smithi060.stdout:7/266: read da/d15/d21/d32/f33 [1007375,105553] 0 2024-02-26T10:47:51.798 INFO:tasks.workunit.client.0.smithi060.stdout:7/267: stat da/d15/d21/d2f/f57 0 2024-02-26T10:47:51.799 INFO:tasks.workunit.client.0.smithi060.stdout:7/268: getdents da/d15/d21/d41 0 2024-02-26T10:47:51.803 INFO:tasks.workunit.client.0.smithi060.stdout:4/154: write d0/ff [4706120,40406] 0 2024-02-26T10:47:51.803 INFO:tasks.workunit.client.0.smithi060.stdout:4/155: truncate d0/da/dc/d16/f19 158109 0 2024-02-26T10:47:51.805 INFO:tasks.workunit.client.0.smithi060.stdout:4/156: link d0/d7/c9 d0/d7/c2a 0 2024-02-26T10:47:51.805 INFO:tasks.workunit.client.0.smithi060.stdout:4/157: write d0/fb [6892364,3805] 0 2024-02-26T10:47:51.805 INFO:tasks.workunit.client.0.smithi060.stdout:4/158: stat d0/d7/c2a 0 2024-02-26T10:47:51.815 INFO:tasks.workunit.client.0.smithi060.stdout:6/112: dread d6/f16 [0,4194304] 0 2024-02-26T10:47:51.819 INFO:tasks.workunit.client.0.smithi060.stdout:9/123: dwrite d3/d15/f17 [0,4194304] 0 2024-02-26T10:47:51.825 INFO:tasks.workunit.client.0.smithi060.stdout:4/159: link d0/d7/f28 d0/da/dc/f2b 0 2024-02-26T10:47:51.825 INFO:tasks.workunit.client.0.smithi060.stdout:0/126: dwrite d0/dd/f2b [4194304,4194304] 0 2024-02-26T10:47:51.825 INFO:tasks.workunit.client.0.smithi060.stdout:2/107: sync 2024-02-26T10:47:51.825 INFO:tasks.workunit.client.0.smithi060.stdout:1/116: sync 2024-02-26T10:47:51.825 INFO:tasks.workunit.client.0.smithi060.stdout:8/125: sync 2024-02-26T10:47:51.825 INFO:tasks.workunit.client.0.smithi060.stdout:2/108: write dd/f1d [776174,92877] 0 2024-02-26T10:47:51.825 INFO:tasks.workunit.client.0.smithi060.stdout:7/269: dread da/d15/f53 [0,4194304] 0 2024-02-26T10:47:51.831 INFO:tasks.workunit.client.0.smithi060.stdout:0/127: link d0/l29 d0/l30 0 2024-02-26T10:47:51.832 INFO:tasks.workunit.client.0.smithi060.stdout:1/117: mknod d3/db/d1e/c25 0 2024-02-26T10:47:51.835 INFO:tasks.workunit.client.0.smithi060.stdout:8/126: symlink d9/l21 0 2024-02-26T10:47:51.836 INFO:tasks.workunit.client.0.smithi060.stdout:7/270: creat da/f5d x:0 0 0 2024-02-26T10:47:51.836 INFO:tasks.workunit.client.0.smithi060.stdout:0/128: rmdir d0/dd/d16 39 2024-02-26T10:47:51.836 INFO:tasks.workunit.client.0.smithi060.stdout:0/129: dread - d0/dd/f1d zero size 2024-02-26T10:47:51.836 INFO:tasks.workunit.client.0.smithi060.stdout:0/130: truncate d0/dd/f2c 868062 0 2024-02-26T10:47:51.837 INFO:tasks.workunit.client.0.smithi060.stdout:7/271: rmdir da/d15/d21/d2f/d37 39 2024-02-26T10:47:51.838 INFO:tasks.workunit.client.0.smithi060.stdout:7/272: mknod da/d15/d21/d32/d4e/d3b/c5e 0 2024-02-26T10:47:51.845 INFO:tasks.workunit.client.0.smithi060.stdout:7/273: link f6 da/d15/d21/d32/f5f 0 2024-02-26T10:47:51.845 INFO:tasks.workunit.client.0.smithi060.stdout:5/106: sync 2024-02-26T10:47:51.845 INFO:tasks.workunit.client.0.smithi060.stdout:5/107: stat d0/de/d15/f18 0 2024-02-26T10:47:51.846 INFO:tasks.workunit.client.0.smithi060.stdout:5/108: fsync d0/de/d20/f24 0 2024-02-26T10:47:51.846 INFO:tasks.workunit.client.0.smithi060.stdout:3/86: sync 2024-02-26T10:47:51.848 INFO:tasks.workunit.client.0.smithi060.stdout:3/87: symlink d5/de/l10 0 2024-02-26T10:47:51.851 INFO:tasks.workunit.client.0.smithi060.stdout:3/88: mkdir d5/de/df/d11 0 2024-02-26T10:47:51.857 INFO:tasks.workunit.client.0.smithi060.stdout:3/89: truncate d5/fa 5021403 0 2024-02-26T10:47:51.861 INFO:tasks.workunit.client.0.smithi060.stdout:9/124: dread d3/d9/dc/ff [0,4194304] 0 2024-02-26T10:47:51.864 INFO:tasks.workunit.client.0.smithi060.stdout:9/125: mknod d3/d1e/c25 0 2024-02-26T10:47:51.865 INFO:tasks.workunit.client.0.smithi060.stdout:4/160: dread d0/ff [4194304,4194304] 0 2024-02-26T10:47:51.866 INFO:tasks.workunit.client.0.smithi060.stdout:4/161: truncate d0/da/dc/d16/f18 4519155 0 2024-02-26T10:47:51.866 INFO:tasks.workunit.client.0.smithi060.stdout:4/162: chown d0/c22 747 1 2024-02-26T10:47:51.873 INFO:tasks.workunit.client.0.smithi060.stdout:4/163: symlink d0/l2c 0 2024-02-26T10:47:51.873 INFO:tasks.workunit.client.0.smithi060.stdout:4/164: write d0/ff [6631874,102952] 0 2024-02-26T10:47:51.875 INFO:tasks.workunit.client.0.smithi060.stdout:8/127: dread f8 [0,4194304] 0 2024-02-26T10:47:51.876 INFO:tasks.workunit.client.0.smithi060.stdout:8/128: symlink d9/d19/l22 0 2024-02-26T10:47:51.876 INFO:tasks.workunit.client.0.smithi060.stdout:8/129: fdatasync d9/fc 0 2024-02-26T10:47:51.877 INFO:tasks.workunit.client.0.smithi060.stdout:9/126: dread d3/d9/f1f [0,4194304] 0 2024-02-26T10:47:51.903 INFO:tasks.workunit.client.0.smithi060.stdout:2/109: dread f0 [4194304,4194304] 0 2024-02-26T10:47:51.905 INFO:tasks.workunit.client.0.smithi060.stdout:2/110: mknod dd/d11/c1f 0 2024-02-26T10:47:51.905 INFO:tasks.workunit.client.0.smithi060.stdout:7/274: dwrite da/d15/f1a [4194304,4194304] 0 2024-02-26T10:47:51.907 INFO:tasks.workunit.client.0.smithi060.stdout:6/113: dwrite d6/f16 [0,4194304] 0 2024-02-26T10:47:51.911 INFO:tasks.workunit.client.0.smithi060.stdout:0/131: dread d0/dd/f1e [0,4194304] 0 2024-02-26T10:47:51.913 INFO:tasks.workunit.client.0.smithi060.stdout:1/118: dwrite d3/f20 [0,4194304] 0 2024-02-26T10:47:51.915 INFO:tasks.workunit.client.0.smithi060.stdout:1/119: symlink d3/db/d1e/l26 0 2024-02-26T10:47:51.915 INFO:tasks.workunit.client.0.smithi060.stdout:1/120: dread - d3/f23 zero size 2024-02-26T10:47:51.918 INFO:tasks.workunit.client.0.smithi060.stdout:1/121: truncate d3/f9 2355102 0 2024-02-26T10:47:51.918 INFO:tasks.workunit.client.0.smithi060.stdout:1/122: readlink l1 0 2024-02-26T10:47:51.920 INFO:tasks.workunit.client.0.smithi060.stdout:4/165: dwrite d0/da/dc/d16/f19 [0,4194304] 0 2024-02-26T10:47:51.927 INFO:tasks.workunit.client.0.smithi060.stdout:8/130: dread d9/f10 [0,4194304] 0 2024-02-26T10:47:51.929 INFO:tasks.workunit.client.0.smithi060.stdout:3/90: dwrite d5/f6 [0,4194304] 0 2024-02-26T10:47:51.929 INFO:tasks.workunit.client.0.smithi060.stdout:3/91: truncate d5/fd 594029 0 2024-02-26T10:47:51.931 INFO:tasks.workunit.client.0.smithi060.stdout:8/131: rename f2 to d9/f23 0 2024-02-26T10:47:51.931 INFO:tasks.workunit.client.0.smithi060.stdout:0/132: dread d0/f9 [0,4194304] 0 2024-02-26T10:47:51.932 INFO:tasks.workunit.client.0.smithi060.stdout:0/133: dread - d0/dd/f1d zero size 2024-02-26T10:47:51.935 INFO:tasks.workunit.client.0.smithi060.stdout:8/132: creat d9/f24 x:0 0 0 2024-02-26T10:47:51.935 INFO:tasks.workunit.client.0.smithi060.stdout:1/123: dread d3/db/d1e/f1f [0,4194304] 0 2024-02-26T10:47:51.937 INFO:tasks.workunit.client.0.smithi060.stdout:1/124: rename d3/db/f11 to d3/db/d1e/f27 0 2024-02-26T10:47:51.938 INFO:tasks.workunit.client.0.smithi060.stdout:1/125: mknod d3/c28 0 2024-02-26T10:47:51.939 INFO:tasks.workunit.client.0.smithi060.stdout:1/126: symlink d3/db/d1e/l29 0 2024-02-26T10:47:51.939 INFO:tasks.workunit.client.0.smithi060.stdout:1/127: chown d3/db/c16 8257824 1 2024-02-26T10:47:51.941 INFO:tasks.workunit.client.0.smithi060.stdout:1/128: rename d3/db/fc to d3/f2a 0 2024-02-26T10:47:51.944 INFO:tasks.workunit.client.0.smithi060.stdout:6/114: write d6/f9 [3893973,13213] 0 2024-02-26T10:47:51.945 INFO:tasks.workunit.client.0.smithi060.stdout:6/115: mkdir d6/d7/d20 0 2024-02-26T10:47:51.952 INFO:tasks.workunit.client.0.smithi060.stdout:8/133: dread f3 [0,4194304] 0 2024-02-26T10:47:51.952 INFO:tasks.workunit.client.0.smithi060.stdout:8/134: read - d9/d15/f1b zero size 2024-02-26T10:47:51.952 INFO:tasks.workunit.client.0.smithi060.stdout:4/166: dread d0/da/dc/d16/f19 [0,4194304] 0 2024-02-26T10:47:51.953 INFO:tasks.workunit.client.0.smithi060.stdout:4/167: creat d0/da/dc/d16/f2d x:0 0 0 2024-02-26T10:47:51.954 INFO:tasks.workunit.client.0.smithi060.stdout:4/168: mknod d0/da/dc/d1b/c2e 0 2024-02-26T10:47:51.957 INFO:tasks.workunit.client.0.smithi060.stdout:9/127: read d3/f7 [719131,48206] 0 2024-02-26T10:47:51.963 INFO:tasks.workunit.client.0.smithi060.stdout:1/129: fsync d3/db/ff 0 2024-02-26T10:47:51.971 INFO:tasks.workunit.client.0.smithi060.stdout:7/275: dread da/d15/f19 [0,4194304] 0 2024-02-26T10:47:51.971 INFO:tasks.workunit.client.0.smithi060.stdout:0/134: write d0/dd/d16/f17 [7565140,129806] 0 2024-02-26T10:47:51.981 INFO:tasks.workunit.client.0.smithi060.stdout:3/92: dread d5/f9 [0,4194304] 0 2024-02-26T10:47:51.981 INFO:tasks.workunit.client.0.smithi060.stdout:3/93: truncate d5/fb 1005091 0 2024-02-26T10:47:51.984 INFO:tasks.workunit.client.0.smithi060.stdout:1/130: write d3/db/d1e/f27 [1136436,114012] 0 2024-02-26T10:47:51.984 INFO:tasks.workunit.client.0.smithi060.stdout:1/131: readlink d3/l1c 0 2024-02-26T10:47:51.984 INFO:tasks.workunit.client.0.smithi060.stdout:2/111: dwrite dd/de/ff [0,4194304] 0 2024-02-26T10:47:51.984 INFO:tasks.workunit.client.0.smithi060.stdout:2/112: chown l9 42102526 1 2024-02-26T10:47:51.984 INFO:tasks.workunit.client.0.smithi060.stdout:2/113: fdatasync dd/de/f15 0 2024-02-26T10:47:51.985 INFO:tasks.workunit.client.0.smithi060.stdout:4/169: dread d0/da/dc/d16/f18 [0,4194304] 0 2024-02-26T10:47:51.986 INFO:tasks.workunit.client.0.smithi060.stdout:7/276: creat da/f60 x:0 0 0 2024-02-26T10:47:51.987 INFO:tasks.workunit.client.0.smithi060.stdout:0/135: creat d0/dd/f31 x:0 0 0 2024-02-26T10:47:51.987 INFO:tasks.workunit.client.0.smithi060.stdout:6/116: write d6/d7/f10 [848377,23954] 0 2024-02-26T10:47:51.987 INFO:tasks.workunit.client.0.smithi060.stdout:6/117: chown d6/d1b 120 1 2024-02-26T10:47:51.987 INFO:tasks.workunit.client.0.smithi060.stdout:6/118: chown d6/d1b 369694 1 2024-02-26T10:47:51.990 INFO:tasks.workunit.client.0.smithi060.stdout:1/132: creat d3/f2b x:0 0 0 2024-02-26T10:47:51.991 INFO:tasks.workunit.client.0.smithi060.stdout:2/114: symlink dd/d11/d13/l20 0 2024-02-26T10:47:51.991 INFO:tasks.workunit.client.0.smithi060.stdout:6/119: unlink d6/l14 0 2024-02-26T10:47:51.991 INFO:tasks.workunit.client.0.smithi060.stdout:0/136: creat d0/f32 x:0 0 0 2024-02-26T10:47:51.991 INFO:tasks.workunit.client.0.smithi060.stdout:0/137: chown d0/f3 1628050 1 2024-02-26T10:47:51.995 INFO:tasks.workunit.client.0.smithi060.stdout:0/138: mkdir d0/dd/d33 0 2024-02-26T10:47:51.996 INFO:tasks.workunit.client.0.smithi060.stdout:1/133: creat d3/f2c x:0 0 0 2024-02-26T10:47:51.997 INFO:tasks.workunit.client.0.smithi060.stdout:2/115: symlink dd/l21 0 2024-02-26T10:47:51.997 INFO:tasks.workunit.client.0.smithi060.stdout:0/139: creat d0/dd/d16/f34 x:0 0 0 2024-02-26T10:47:52.010 INFO:tasks.workunit.client.0.smithi060.stdout:0/140: unlink d0/dd/f1e 0 2024-02-26T10:47:52.010 INFO:tasks.workunit.client.0.smithi060.stdout:0/141: chown d0/l14 0 1 2024-02-26T10:47:52.010 INFO:tasks.workunit.client.0.smithi060.stdout:1/134: rename d3/db/d1e/l29 to d3/l2d 0 2024-02-26T10:47:52.010 INFO:tasks.workunit.client.0.smithi060.stdout:1/135: chown d3/f9 513 1 2024-02-26T10:47:52.010 INFO:tasks.workunit.client.0.smithi060.stdout:1/136: chown d3/c24 3 1 2024-02-26T10:47:52.011 INFO:tasks.workunit.client.0.smithi060.stdout:8/135: dwrite f3 [0,4194304] 0 2024-02-26T10:47:52.011 INFO:tasks.workunit.client.0.smithi060.stdout:8/136: chown d9/f13 395 1 2024-02-26T10:47:52.011 INFO:tasks.workunit.client.0.smithi060.stdout:1/137: mkdir d3/db/d2e 0 2024-02-26T10:47:52.012 INFO:tasks.workunit.client.0.smithi060.stdout:1/138: fdatasync d3/db/ff 0 2024-02-26T10:47:52.016 INFO:tasks.workunit.client.0.smithi060.stdout:7/277: write da/f18 [3842406,100161] 0 2024-02-26T10:47:52.016 INFO:tasks.workunit.client.0.smithi060.stdout:7/278: fdatasync da/d15/d21/d2f/f48 0 2024-02-26T10:47:52.016 INFO:tasks.workunit.client.0.smithi060.stdout:7/279: rename da to da/d5a/d61 22 2024-02-26T10:47:52.021 INFO:tasks.workunit.client.0.smithi060.stdout:8/137: rmdir d9/d15 39 2024-02-26T10:47:52.022 INFO:tasks.workunit.client.0.smithi060.stdout:9/128: dwrite d3/d15/f17 [0,4194304] 0 2024-02-26T10:47:52.022 INFO:tasks.workunit.client.0.smithi060.stdout:1/139: mknod d3/db/d1e/c2f 0 2024-02-26T10:47:52.025 INFO:tasks.workunit.client.0.smithi060.stdout:7/280: symlink da/d5a/l62 0 2024-02-26T10:47:52.028 INFO:tasks.workunit.client.0.smithi060.stdout:8/138: rename d9/d15/c18 to d9/d19/c25 0 2024-02-26T10:47:52.028 INFO:tasks.workunit.client.0.smithi060.stdout:8/139: write d9/f10 [4326872,129870] 0 2024-02-26T10:47:52.028 INFO:tasks.workunit.client.0.smithi060.stdout:8/140: read - d9/f24 zero size 2024-02-26T10:47:52.031 INFO:tasks.workunit.client.0.smithi060.stdout:6/120: dwrite d6/fe [0,4194304] 0 2024-02-26T10:47:52.033 INFO:tasks.workunit.client.0.smithi060.stdout:6/121: write d6/f1d [304624,75686] 0 2024-02-26T10:47:52.041 INFO:tasks.workunit.client.0.smithi060.stdout:1/140: unlink d3/f2c 0 2024-02-26T10:47:52.041 INFO:tasks.workunit.client.0.smithi060.stdout:0/142: read d0/dd/f2b [1366961,121419] 0 2024-02-26T10:47:52.041 INFO:tasks.workunit.client.0.smithi060.stdout:0/143: rename d0 to d0/dd/d35 22 2024-02-26T10:47:52.050 INFO:tasks.workunit.client.0.smithi060.stdout:1/141: getdents d3/db/d1e 0 2024-02-26T10:47:52.050 INFO:tasks.workunit.client.0.smithi060.stdout:5/109: sync 2024-02-26T10:47:52.051 INFO:tasks.workunit.client.0.smithi060.stdout:5/110: mknod d0/de/d20/c25 0 2024-02-26T10:47:52.052 INFO:tasks.workunit.client.0.smithi060.stdout:0/144: mknod d0/dd/d33/c36 0 2024-02-26T10:47:52.054 INFO:tasks.workunit.client.0.smithi060.stdout:5/111: mkdir d0/de/d15/d26 0 2024-02-26T10:47:52.060 INFO:tasks.workunit.client.0.smithi060.stdout:5/112: truncate d0/de/f12 1745040 0 2024-02-26T10:47:52.060 INFO:tasks.workunit.client.0.smithi060.stdout:5/113: write d0/f5 [2410932,86630] 0 2024-02-26T10:47:52.060 INFO:tasks.workunit.client.0.smithi060.stdout:5/114: write d0/f17 [927857,129303] 0 2024-02-26T10:47:52.067 INFO:tasks.workunit.client.0.smithi060.stdout:2/116: dwrite dd/de/ff [0,4194304] 0 2024-02-26T10:47:52.094 INFO:tasks.workunit.client.0.smithi060.stdout:7/281: dread f7 [0,4194304] 0 2024-02-26T10:47:52.094 INFO:tasks.workunit.client.0.smithi060.stdout:7/282: dread - da/d15/d21/d32/d4e/f5b zero size 2024-02-26T10:47:52.094 INFO:tasks.workunit.client.0.smithi060.stdout:7/283: fsync da/d15/f53 0 2024-02-26T10:47:52.094 INFO:tasks.workunit.client.0.smithi060.stdout:6/122: dwrite d6/f16 [0,4194304] 0 2024-02-26T10:47:52.094 INFO:tasks.workunit.client.0.smithi060.stdout:7/284: write da/f30 [406426,13309] 0 2024-02-26T10:47:52.096 INFO:tasks.workunit.client.0.smithi060.stdout:6/123: unlink d6/d7/f10 0 2024-02-26T10:47:52.098 INFO:tasks.workunit.client.0.smithi060.stdout:6/124: mknod d6/d1b/c21 0 2024-02-26T10:47:52.104 INFO:tasks.workunit.client.0.smithi060.stdout:6/125: link l1 d6/d7/l22 0 2024-02-26T10:47:52.104 INFO:tasks.workunit.client.0.smithi060.stdout:0/145: dread d0/dd/f2b [0,4194304] 0 2024-02-26T10:47:52.104 INFO:tasks.workunit.client.0.smithi060.stdout:8/141: dwrite d9/f20 [0,4194304] 0 2024-02-26T10:47:52.106 INFO:tasks.workunit.client.0.smithi060.stdout:1/142: dwrite d3/f1b [0,4194304] 0 2024-02-26T10:47:52.109 INFO:tasks.workunit.client.0.smithi060.stdout:1/143: creat d3/db/d2e/f30 x:0 0 0 2024-02-26T10:47:52.109 INFO:tasks.workunit.client.0.smithi060.stdout:1/144: stat d3/db/d2e 0 2024-02-26T10:47:52.113 INFO:tasks.workunit.client.0.smithi060.stdout:9/129: dwrite d3/d15/f17 [4194304,4194304] 0 2024-02-26T10:47:52.135 INFO:tasks.workunit.client.0.smithi060.stdout:1/145: write d3/f1b [2944166,104731] 0 2024-02-26T10:47:52.135 INFO:tasks.workunit.client.0.smithi060.stdout:1/146: chown d3/f12 4199 1 2024-02-26T10:47:52.136 INFO:tasks.workunit.client.0.smithi060.stdout:6/126: read d6/f16 [3514251,118334] 0 2024-02-26T10:47:52.136 INFO:tasks.workunit.client.0.smithi060.stdout:6/127: readlink d6/d7/l22 0 2024-02-26T10:47:52.137 INFO:tasks.workunit.client.0.smithi060.stdout:8/142: rmdir d9 39 2024-02-26T10:47:52.139 INFO:tasks.workunit.client.0.smithi060.stdout:8/143: mknod d9/d15/c26 0 2024-02-26T10:47:52.139 INFO:tasks.workunit.client.0.smithi060.stdout:8/144: read - d9/d15/f1b zero size 2024-02-26T10:47:52.140 INFO:tasks.workunit.client.0.smithi060.stdout:5/115: dwrite d0/de/d15/f21 [0,4194304] 0 2024-02-26T10:47:52.140 INFO:tasks.workunit.client.0.smithi060.stdout:5/116: readlink d0/l13 0 2024-02-26T10:47:52.142 INFO:tasks.workunit.client.0.smithi060.stdout:6/128: mkdir d6/d1b/d1c/d1f/d23 0 2024-02-26T10:47:52.144 INFO:tasks.workunit.client.0.smithi060.stdout:5/117: mknod d0/de/d15/c27 0 2024-02-26T10:47:52.145 INFO:tasks.workunit.client.0.smithi060.stdout:8/145: creat d9/d15/f27 x:0 0 0 2024-02-26T10:47:52.145 INFO:tasks.workunit.client.0.smithi060.stdout:8/146: stat d9/f20 0 2024-02-26T10:47:52.147 INFO:tasks.workunit.client.0.smithi060.stdout:2/117: creat dd/d11/f22 x:0 0 0 2024-02-26T10:47:52.148 INFO:tasks.workunit.client.0.smithi060.stdout:2/118: read - dd/d11/f1c zero size 2024-02-26T10:47:52.148 INFO:tasks.workunit.client.0.smithi060.stdout:1/147: symlink d3/db/d1e/l31 0 2024-02-26T10:47:52.149 INFO:tasks.workunit.client.0.smithi060.stdout:1/148: chown d3/c24 7783545 1 2024-02-26T10:47:52.149 INFO:tasks.workunit.client.0.smithi060.stdout:1/149: read d3/f9 [1883139,90401] 0 2024-02-26T10:47:52.149 INFO:tasks.workunit.client.0.smithi060.stdout:2/119: symlink dd/d11/l23 0 2024-02-26T10:47:52.154 INFO:tasks.workunit.client.0.smithi060.stdout:6/129: creat d6/d1b/d1c/d1f/f24 x:0 0 0 2024-02-26T10:47:52.156 INFO:tasks.workunit.client.0.smithi060.stdout:8/147: mknod d9/d15/c28 0 2024-02-26T10:47:52.156 INFO:tasks.workunit.client.0.smithi060.stdout:8/148: dread - d9/f1e zero size 2024-02-26T10:47:52.156 INFO:tasks.workunit.client.0.smithi060.stdout:1/150: mkdir d3/d32 0 2024-02-26T10:47:52.157 INFO:tasks.workunit.client.0.smithi060.stdout:1/151: getdents d3/d32 0 2024-02-26T10:47:52.157 INFO:tasks.workunit.client.0.smithi060.stdout:1/152: truncate d3/f9 3392038 0 2024-02-26T10:47:52.159 INFO:tasks.workunit.client.0.smithi060.stdout:1/153: fsync d3/db/d1e/f1f 0 2024-02-26T10:47:52.159 INFO:tasks.workunit.client.0.smithi060.stdout:1/154: write d3/f20 [968086,98731] 0 2024-02-26T10:47:52.159 INFO:tasks.workunit.client.0.smithi060.stdout:6/130: truncate d6/f9 186109 0 2024-02-26T10:47:52.162 INFO:tasks.workunit.client.0.smithi060.stdout:2/120: chown dd/f1d 533586 1 2024-02-26T10:47:52.162 INFO:tasks.workunit.client.0.smithi060.stdout:2/121: fsync f5 0 2024-02-26T10:47:52.162 INFO:tasks.workunit.client.0.smithi060.stdout:2/122: fdatasync f5 0 2024-02-26T10:47:52.162 INFO:tasks.workunit.client.0.smithi060.stdout:2/123: readlink dd/d11/d13/l20 0 2024-02-26T10:47:52.164 INFO:tasks.workunit.client.0.smithi060.stdout:2/124: creat dd/d11/f24 x:0 0 0 2024-02-26T10:47:52.171 INFO:tasks.workunit.client.0.smithi060.stdout:6/131: rename d6/d1b to d6/d7/d25 0 2024-02-26T10:47:52.172 INFO:tasks.workunit.client.0.smithi060.stdout:6/132: symlink d6/d7/d25/l26 0 2024-02-26T10:47:52.174 INFO:tasks.workunit.client.0.smithi060.stdout:5/118: dread d0/de/d15/f1a [0,4194304] 0 2024-02-26T10:47:52.174 INFO:tasks.workunit.client.0.smithi060.stdout:6/133: write d6/f9 [417256,102755] 0 2024-02-26T10:47:52.174 INFO:tasks.workunit.client.0.smithi060.stdout:6/134: rename d6/d7/d25 to d6/d7/d25/d1c/d27 22 2024-02-26T10:47:52.174 INFO:tasks.workunit.client.0.smithi060.stdout:5/119: getdents d0/de/d15/d26 0 2024-02-26T10:47:52.174 INFO:tasks.workunit.client.0.smithi060.stdout:5/120: chown d0/f9 1821 1 2024-02-26T10:47:52.176 INFO:tasks.workunit.client.0.smithi060.stdout:7/285: dwrite da/f5d [0,4194304] 0 2024-02-26T10:47:52.179 INFO:tasks.workunit.client.0.smithi060.stdout:6/135: rmdir d6/d7/d20 0 2024-02-26T10:47:52.179 INFO:tasks.workunit.client.0.smithi060.stdout:9/130: dread d3/d9/f1f [0,4194304] 0 2024-02-26T10:47:52.183 INFO:tasks.workunit.client.0.smithi060.stdout:1/155: write d3/f15 [3758559,74566] 0 2024-02-26T10:47:52.187 INFO:tasks.workunit.client.0.smithi060.stdout:7/286: unlink da/d15/d21/d29/c31 0 2024-02-26T10:47:52.187 INFO:tasks.workunit.client.0.smithi060.stdout:7/287: write da/f30 [1658643,94577] 0 2024-02-26T10:47:52.196 INFO:tasks.workunit.client.0.smithi060.stdout:4/170: sync 2024-02-26T10:47:52.197 INFO:tasks.workunit.client.0.smithi060.stdout:3/94: sync 2024-02-26T10:47:52.198 INFO:tasks.workunit.client.0.smithi060.stdout:4/171: mkdir d0/d27/d2f 0 2024-02-26T10:47:52.198 INFO:tasks.workunit.client.0.smithi060.stdout:4/172: readlink d0/da/dc/d16/l1d 0 2024-02-26T10:47:52.198 INFO:tasks.workunit.client.0.smithi060.stdout:4/173: chown d0/da/dc/d16/f20 1672070 1 2024-02-26T10:47:52.206 INFO:tasks.workunit.client.0.smithi060.stdout:8/149: dwrite d9/f20 [4194304,4194304] 0 2024-02-26T10:47:52.206 INFO:tasks.workunit.client.0.smithi060.stdout:8/150: chown d9 406172 1 2024-02-26T10:47:52.207 INFO:tasks.workunit.client.0.smithi060.stdout:5/121: write d0/de/d15/f21 [3673287,79266] 0 2024-02-26T10:47:52.207 INFO:tasks.workunit.client.0.smithi060.stdout:5/122: rename d0 to d0/de/d20/d28 22 2024-02-26T10:47:52.208 INFO:tasks.workunit.client.0.smithi060.stdout:5/123: rmdir d0/de/d15 39 2024-02-26T10:47:52.209 INFO:tasks.workunit.client.0.smithi060.stdout:5/124: creat d0/de/f29 x:0 0 0 2024-02-26T10:47:52.210 INFO:tasks.workunit.client.0.smithi060.stdout:5/125: stat d0/l13 0 2024-02-26T10:47:52.210 INFO:tasks.workunit.client.0.smithi060.stdout:5/126: chown d0/l7 1085240658 1 2024-02-26T10:47:52.212 INFO:tasks.workunit.client.0.smithi060.stdout:4/174: link d0/da/dc/d16/f2d d0/d27/f30 0 2024-02-26T10:47:52.213 INFO:tasks.workunit.client.0.smithi060.stdout:5/127: symlink d0/de/d20/l2a 0 2024-02-26T10:47:52.214 INFO:tasks.workunit.client.0.smithi060.stdout:5/128: truncate d0/f8 2724650 0 2024-02-26T10:47:52.215 INFO:tasks.workunit.client.0.smithi060.stdout:4/175: rename d0/d7/f26 to d0/d27/d2f/f31 0 2024-02-26T10:47:52.215 INFO:tasks.workunit.client.0.smithi060.stdout:5/129: mknod d0/de/d20/c2b 0 2024-02-26T10:47:52.216 INFO:tasks.workunit.client.0.smithi060.stdout:4/176: getdents d0/da/dc/d1b 0 2024-02-26T10:47:52.218 INFO:tasks.workunit.client.0.smithi060.stdout:5/130: getdents d0/de/d20 0 2024-02-26T10:47:52.225 INFO:tasks.workunit.client.0.smithi060.stdout:3/95: dread d5/f9 [0,4194304] 0 2024-02-26T10:47:52.226 INFO:tasks.workunit.client.0.smithi060.stdout:4/177: rmdir d0/da/dc 39 2024-02-26T10:47:52.227 INFO:tasks.workunit.client.0.smithi060.stdout:4/178: creat d0/d7/d29/f32 x:0 0 0 2024-02-26T10:47:52.228 INFO:tasks.workunit.client.0.smithi060.stdout:4/179: mknod d0/d7/c33 0 2024-02-26T10:47:52.228 INFO:tasks.workunit.client.0.smithi060.stdout:4/180: write d0/d27/d2f/f31 [4249066,86838] 0 2024-02-26T10:47:52.230 INFO:tasks.workunit.client.0.smithi060.stdout:4/181: creat d0/da/dc/f34 x:0 0 0 2024-02-26T10:47:52.230 INFO:tasks.workunit.client.0.smithi060.stdout:9/131: dread d3/d9/dc/ff [0,4194304] 0 2024-02-26T10:47:52.246 INFO:tasks.workunit.client.0.smithi060.stdout:5/131: dread d0/f14 [0,4194304] 0 2024-02-26T10:47:52.246 INFO:tasks.workunit.client.0.smithi060.stdout:5/132: getdents d0/de/d20 0 2024-02-26T10:47:52.246 INFO:tasks.workunit.client.0.smithi060.stdout:5/133: readlink d0/l7 0 2024-02-26T10:47:52.246 INFO:tasks.workunit.client.0.smithi060.stdout:5/134: write d0/de/f1f [1880272,38919] 0 2024-02-26T10:47:52.264 INFO:tasks.workunit.client.0.smithi060.stdout:6/136: write d6/f15 [2413596,6324] 0 2024-02-26T10:47:52.264 INFO:tasks.workunit.client.0.smithi060.stdout:6/137: readlink d6/d7/l1e 0 2024-02-26T10:47:52.265 INFO:tasks.workunit.client.0.smithi060.stdout:6/138: write d6/f1d [395526,47517] 0 2024-02-26T10:47:52.269 INFO:tasks.workunit.client.0.smithi060.stdout:0/146: dwrite d0/dd/d16/f34 [0,4194304] 0 2024-02-26T10:47:52.270 INFO:tasks.workunit.client.0.smithi060.stdout:6/139: write d6/d7/f12 [891810,48370] 0 2024-02-26T10:47:52.287 INFO:tasks.workunit.client.0.smithi060.stdout:5/135: dread d0/de/d20/f24 [0,4194304] 0 2024-02-26T10:47:52.288 INFO:tasks.workunit.client.0.smithi060.stdout:5/136: creat d0/de/d15/f2c x:0 0 0 2024-02-26T10:47:52.288 INFO:tasks.workunit.client.0.smithi060.stdout:5/137: stat d0/de/f12 0 2024-02-26T10:47:52.289 INFO:tasks.workunit.client.0.smithi060.stdout:5/138: creat d0/de/d15/f2d x:0 0 0 2024-02-26T10:47:52.290 INFO:tasks.workunit.client.0.smithi060.stdout:3/96: dwrite d5/fa [4194304,4194304] 0 2024-02-26T10:47:52.290 INFO:tasks.workunit.client.0.smithi060.stdout:3/97: write d5/fb [1821358,105723] 0 2024-02-26T10:47:52.292 INFO:tasks.workunit.client.0.smithi060.stdout:9/132: dwrite d3/f16 [0,4194304] 0 2024-02-26T10:47:52.293 INFO:tasks.workunit.client.0.smithi060.stdout:4/182: dwrite d0/da/dc/f23 [4194304,4194304] 0 2024-02-26T10:47:52.293 INFO:tasks.workunit.client.0.smithi060.stdout:4/183: read d0/da/dc/d16/f18 [1858312,25046] 0 2024-02-26T10:47:52.293 INFO:tasks.workunit.client.0.smithi060.stdout:4/184: chown d0/d27/d2f/f31 1857603 1 2024-02-26T10:47:52.298 INFO:tasks.workunit.client.0.smithi060.stdout:3/98: truncate f3 777405 0 2024-02-26T10:47:52.298 INFO:tasks.workunit.client.0.smithi060.stdout:3/99: stat d5/f6 0 2024-02-26T10:47:52.299 INFO:tasks.workunit.client.0.smithi060.stdout:3/100: link d5/de/l10 d5/de/df/d11/l12 0 2024-02-26T10:47:52.301 INFO:tasks.workunit.client.0.smithi060.stdout:3/101: symlink d5/de/df/d11/l13 0 2024-02-26T10:47:52.309 INFO:tasks.workunit.client.0.smithi060.stdout:7/288: dwrite da/f60 [0,4194304] 0 2024-02-26T10:47:52.316 INFO:tasks.workunit.client.0.smithi060.stdout:3/102: dread f3 [0,4194304] 0 2024-02-26T10:47:52.316 INFO:tasks.workunit.client.0.smithi060.stdout:3/103: fsync d5/fd 0 2024-02-26T10:47:52.321 INFO:tasks.workunit.client.0.smithi060.stdout:9/133: rename d3/d9/l20 to d3/d15/d1b/d23/l26 0 2024-02-26T10:47:52.324 INFO:tasks.workunit.client.0.smithi060.stdout:3/104: link d5/fd d5/de/f14 0 2024-02-26T10:47:52.326 INFO:tasks.workunit.client.0.smithi060.stdout:3/105: link d5/fb d5/de/f15 0 2024-02-26T10:47:52.326 INFO:tasks.workunit.client.0.smithi060.stdout:3/106: readlink d5/de/df/d11/l13 0 2024-02-26T10:47:52.330 INFO:tasks.workunit.client.0.smithi060.stdout:7/289: dread da/d15/f19 [0,4194304] 0 2024-02-26T10:47:52.330 INFO:tasks.workunit.client.0.smithi060.stdout:7/290: truncate da/d15/d21/f2c 1841166 0 2024-02-26T10:47:52.332 INFO:tasks.workunit.client.0.smithi060.stdout:7/291: link da/d15/d21/d32/f43 da/d15/d21/d32/f63 0 2024-02-26T10:47:52.337 INFO:tasks.workunit.client.0.smithi060.stdout:7/292: mknod da/d15/d21/d32/d4e/d39/c64 0 2024-02-26T10:47:52.337 INFO:tasks.workunit.client.0.smithi060.stdout:7/293: write da/d15/d21/f22 [824936,19636] 0 2024-02-26T10:47:52.337 INFO:tasks.workunit.client.0.smithi060.stdout:7/294: write da/d15/d21/d32/d4e/f5b [619710,23944] 0 2024-02-26T10:47:52.338 INFO:tasks.workunit.client.0.smithi060.stdout:7/295: link da/l3d da/d15/d21/d32/d4e/l65 0 2024-02-26T10:47:52.338 INFO:tasks.workunit.client.0.smithi060.stdout:7/296: chown da/d15/f1a 77 1 2024-02-26T10:47:52.339 INFO:tasks.workunit.client.0.smithi060.stdout:7/297: unlink da/d15/f1a 0 2024-02-26T10:47:52.340 INFO:tasks.workunit.client.0.smithi060.stdout:4/185: fsync d0/d27/d2f/f31 0 2024-02-26T10:47:52.341 INFO:tasks.workunit.client.0.smithi060.stdout:8/151: dwrite d9/d15/f1b [0,4194304] 0 2024-02-26T10:47:52.344 INFO:tasks.workunit.client.0.smithi060.stdout:4/186: unlink d0/d7/f12 0 2024-02-26T10:47:52.347 INFO:tasks.workunit.client.0.smithi060.stdout:8/152: mkdir d9/d15/d29 0 2024-02-26T10:47:52.347 INFO:tasks.workunit.client.0.smithi060.stdout:8/153: fdatasync d9/f14 0 2024-02-26T10:47:52.356 INFO:tasks.workunit.client.0.smithi060.stdout:4/187: creat d0/d27/d2f/f35 x:0 0 0 2024-02-26T10:47:52.361 INFO:tasks.workunit.client.0.smithi060.stdout:8/154: mknod d9/d15/d29/c2a 0 2024-02-26T10:47:52.362 INFO:tasks.workunit.client.0.smithi060.stdout:8/155: write d9/f13 [798269,39580] 0 2024-02-26T10:47:52.362 INFO:tasks.workunit.client.0.smithi060.stdout:8/156: rmdir d9/d15/d29 39 2024-02-26T10:47:52.364 INFO:tasks.workunit.client.0.smithi060.stdout:9/134: dwrite d3/d9/f19 [0,4194304] 0 2024-02-26T10:47:52.365 INFO:tasks.workunit.client.0.smithi060.stdout:9/135: symlink d3/d15/l27 0 2024-02-26T10:47:52.367 INFO:tasks.workunit.client.0.smithi060.stdout:9/136: creat d3/d15/f28 x:0 0 0 2024-02-26T10:47:52.370 INFO:tasks.workunit.client.0.smithi060.stdout:4/188: rmdir d0/da/dc 39 2024-02-26T10:47:52.374 INFO:tasks.workunit.client.0.smithi060.stdout:4/189: truncate d0/da/dc/d16/f24 174337 0 2024-02-26T10:47:52.374 INFO:tasks.workunit.client.0.smithi060.stdout:4/190: dread - d0/da/dc/f34 zero size 2024-02-26T10:47:52.378 INFO:tasks.workunit.client.0.smithi060.stdout:7/298: dwrite da/d15/d21/d32/d4e/f5c [0,4194304] 0 2024-02-26T10:47:52.381 INFO:tasks.workunit.client.0.smithi060.stdout:7/299: getdents da/d15/d21/d2f/d37 0 2024-02-26T10:47:52.391 INFO:tasks.workunit.client.0.smithi060.stdout:8/157: dread d9/f23 [0,4194304] 0 2024-02-26T10:47:52.392 INFO:tasks.workunit.client.0.smithi060.stdout:7/300: mkdir da/d66 0 2024-02-26T10:47:52.394 INFO:tasks.workunit.client.0.smithi060.stdout:9/137: write d3/d9/f19 [809224,124817] 0 2024-02-26T10:47:52.394 INFO:tasks.workunit.client.0.smithi060.stdout:9/138: chown d3/d15/f17 398 1 2024-02-26T10:47:52.396 INFO:tasks.workunit.client.0.smithi060.stdout:8/158: fdatasync d9/f13 0 2024-02-26T10:47:52.411 INFO:tasks.workunit.client.0.smithi060.stdout:0/147: dwrite d0/dd/f1d [0,4194304] 0 2024-02-26T10:47:52.411 INFO:tasks.workunit.client.0.smithi060.stdout:6/140: dwrite d6/d7/fd [0,4194304] 0 2024-02-26T10:47:52.412 INFO:tasks.workunit.client.0.smithi060.stdout:4/191: dwrite d0/da/dc/d16/f18 [0,4194304] 0 2024-02-26T10:47:52.414 INFO:tasks.workunit.client.0.smithi060.stdout:0/148: mkdir d0/d37 0 2024-02-26T10:47:52.415 INFO:tasks.workunit.client.0.smithi060.stdout:6/141: getdents d6/d7/d25/d1c 0 2024-02-26T10:47:52.416 INFO:tasks.workunit.client.0.smithi060.stdout:6/142: readlink d6/d7/d25/l26 0 2024-02-26T10:47:52.416 INFO:tasks.workunit.client.0.smithi060.stdout:6/143: write d6/f18 [1002675,24004] 0 2024-02-26T10:47:52.416 INFO:tasks.workunit.client.0.smithi060.stdout:6/144: fsync d6/d7/ff 0 2024-02-26T10:47:52.417 INFO:tasks.workunit.client.0.smithi060.stdout:0/149: rmdir d0/dd/d16 39 2024-02-26T10:47:52.418 INFO:tasks.workunit.client.0.smithi060.stdout:6/145: mkdir d6/d7/d25/d1c/d1f/d23/d28 0 2024-02-26T10:47:52.418 INFO:tasks.workunit.client.0.smithi060.stdout:0/150: mknod d0/d37/c38 0 2024-02-26T10:47:52.423 INFO:tasks.workunit.client.0.smithi060.stdout:0/151: mkdir d0/dd/d39 0 2024-02-26T10:47:52.440 INFO:tasks.workunit.client.0.smithi060.stdout:7/301: dwrite f6 [0,4194304] 0 2024-02-26T10:47:52.440 INFO:tasks.workunit.client.0.smithi060.stdout:7/302: write da/d15/d21/d32/f33 [766835,117578] 0 2024-02-26T10:47:52.441 INFO:tasks.workunit.client.0.smithi060.stdout:7/303: mkdir da/d15/d21/d4b/d67 0 2024-02-26T10:47:52.443 INFO:tasks.workunit.client.0.smithi060.stdout:7/304: getdents da 0 2024-02-26T10:47:52.445 INFO:tasks.workunit.client.0.smithi060.stdout:7/305: unlink da/f5d 0 2024-02-26T10:47:52.446 INFO:tasks.workunit.client.0.smithi060.stdout:7/306: stat da/d15/d21/d4b 0 2024-02-26T10:47:52.455 INFO:tasks.workunit.client.0.smithi060.stdout:4/192: dwrite d0/d7/d29/f32 [0,4194304] 0 2024-02-26T10:47:52.457 INFO:tasks.workunit.client.0.smithi060.stdout:5/139: dwrite d0/de/d15/f1a [4194304,4194304] 0 2024-02-26T10:47:52.457 INFO:tasks.workunit.client.0.smithi060.stdout:5/140: fsync d0/de/d15/f1a 0 2024-02-26T10:47:52.457 INFO:tasks.workunit.client.0.smithi060.stdout:4/193: symlink d0/l36 0 2024-02-26T10:47:52.473 INFO:tasks.workunit.client.0.smithi060.stdout:6/146: dwrite d6/f16 [0,4194304] 0 2024-02-26T10:47:52.475 INFO:tasks.workunit.client.0.smithi060.stdout:6/147: creat d6/d7/d25/d1c/d1f/d23/d28/f29 x:0 0 0 2024-02-26T10:47:52.479 INFO:tasks.workunit.client.0.smithi060.stdout:0/152: write d0/dd/f1d [989760,77972] 0 2024-02-26T10:47:52.479 INFO:tasks.workunit.client.0.smithi060.stdout:0/153: write d0/dd/d16/f1c [1579306,45076] 0 2024-02-26T10:47:52.479 INFO:tasks.workunit.client.0.smithi060.stdout:8/159: dwrite f3 [4194304,4194304] 0 2024-02-26T10:47:52.482 INFO:tasks.workunit.client.0.smithi060.stdout:9/139: sync 2024-02-26T10:47:52.482 INFO:tasks.workunit.client.0.smithi060.stdout:3/107: sync 2024-02-26T10:47:52.482 INFO:tasks.workunit.client.0.smithi060.stdout:2/125: sync 2024-02-26T10:47:52.484 INFO:tasks.workunit.client.0.smithi060.stdout:0/154: unlink d0/f9 0 2024-02-26T10:47:52.484 INFO:tasks.workunit.client.0.smithi060.stdout:2/126: link dd/d11/f1c dd/d11/f25 0 2024-02-26T10:47:52.484 INFO:tasks.workunit.client.0.smithi060.stdout:2/127: fdatasync dd/d11/f22 0 2024-02-26T10:47:52.484 INFO:tasks.workunit.client.0.smithi060.stdout:8/160: truncate d9/f14 3643306 0 2024-02-26T10:47:52.487 INFO:tasks.workunit.client.0.smithi060.stdout:0/155: mknod d0/dd/c3a 0 2024-02-26T10:47:52.487 INFO:tasks.workunit.client.0.smithi060.stdout:1/156: sync 2024-02-26T10:47:52.488 INFO:tasks.workunit.client.0.smithi060.stdout:0/156: mknod d0/c3b 0 2024-02-26T10:47:52.489 INFO:tasks.workunit.client.0.smithi060.stdout:2/128: rename dd/de/f15 to dd/d11/f26 0 2024-02-26T10:47:52.490 INFO:tasks.workunit.client.0.smithi060.stdout:2/129: write dd/d11/d13/f1a [5777006,115612] 0 2024-02-26T10:47:52.501 INFO:tasks.workunit.client.0.smithi060.stdout:6/148: dread d6/d7/fd [0,4194304] 0 2024-02-26T10:47:52.503 INFO:tasks.workunit.client.0.smithi060.stdout:6/149: creat d6/d7/f2a x:0 0 0 2024-02-26T10:47:52.505 INFO:tasks.workunit.client.0.smithi060.stdout:6/150: mkdir d6/d7/d25/d1c/d1f/d23/d2b 0 2024-02-26T10:47:52.505 INFO:tasks.workunit.client.0.smithi060.stdout:6/151: chown d6/d7/f12 88328 1 2024-02-26T10:47:52.507 INFO:tasks.workunit.client.0.smithi060.stdout:6/152: creat d6/d7/d25/d1c/f2c x:0 0 0 2024-02-26T10:47:52.509 INFO:tasks.workunit.client.0.smithi060.stdout:6/153: creat d6/d7/d25/f2d x:0 0 0 2024-02-26T10:47:52.509 INFO:tasks.workunit.client.0.smithi060.stdout:6/154: readlink l3 0 2024-02-26T10:47:52.509 INFO:tasks.workunit.client.0.smithi060.stdout:6/155: write d6/d7/d25/d1c/f2c [928853,22297] 0 2024-02-26T10:47:52.521 INFO:tasks.workunit.client.0.smithi060.stdout:2/130: truncate dd/d11/d13/f1a 3170697 0 2024-02-26T10:47:52.522 INFO:tasks.workunit.client.0.smithi060.stdout:2/131: stat dd/d11 0 2024-02-26T10:47:52.530 INFO:tasks.workunit.client.0.smithi060.stdout:6/156: fsync d6/d7/d25/f2d 0 2024-02-26T10:47:52.531 INFO:tasks.workunit.client.0.smithi060.stdout:5/141: dwrite d0/de/f29 [0,4194304] 0 2024-02-26T10:47:52.531 INFO:tasks.workunit.client.0.smithi060.stdout:5/142: write d0/de/d15/f16 [1790529,32380] 0 2024-02-26T10:47:52.533 INFO:tasks.workunit.client.0.smithi060.stdout:6/157: unlink d6/d7/l1a 0 2024-02-26T10:47:52.542 INFO:tasks.workunit.client.0.smithi060.stdout:3/108: dwrite f3 [0,4194304] 0 2024-02-26T10:47:52.548 INFO:tasks.workunit.client.0.smithi060.stdout:9/140: dwrite d3/d9/d10/f11 [0,4194304] 0 2024-02-26T10:47:52.549 INFO:tasks.workunit.client.0.smithi060.stdout:9/141: creat d3/d15/d1b/d23/f29 x:0 0 0 2024-02-26T10:47:52.551 INFO:tasks.workunit.client.0.smithi060.stdout:9/142: creat d3/d15/f2a x:0 0 0 2024-02-26T10:47:52.558 INFO:tasks.workunit.client.0.smithi060.stdout:1/157: dwrite d3/f20 [4194304,4194304] 0 2024-02-26T10:47:52.558 INFO:tasks.workunit.client.0.smithi060.stdout:1/158: chown d3/f2b 22 1 2024-02-26T10:47:52.558 INFO:tasks.workunit.client.0.smithi060.stdout:1/159: fsync d3/f23 0 2024-02-26T10:47:52.558 INFO:tasks.workunit.client.0.smithi060.stdout:1/160: write d3/f2b [438165,112319] 0 2024-02-26T10:47:52.558 INFO:tasks.workunit.client.0.smithi060.stdout:1/161: chown d3/d32 924151 1 2024-02-26T10:47:52.558 INFO:tasks.workunit.client.0.smithi060.stdout:1/162: chown l1 107 1 2024-02-26T10:47:52.558 INFO:tasks.workunit.client.0.smithi060.stdout:1/163: readlink l1 0 2024-02-26T10:47:52.559 INFO:tasks.workunit.client.0.smithi060.stdout:1/164: truncate d3/f12 614902 0 2024-02-26T10:47:52.561 INFO:tasks.workunit.client.0.smithi060.stdout:8/161: dwrite d9/d15/f1b [0,4194304] 0 2024-02-26T10:47:52.562 INFO:tasks.workunit.client.0.smithi060.stdout:0/157: dwrite d0/f3 [0,4194304] 0 2024-02-26T10:47:52.562 INFO:tasks.workunit.client.0.smithi060.stdout:0/158: dread - d0/dd/d16/f28 zero size 2024-02-26T10:47:52.563 INFO:tasks.workunit.client.0.smithi060.stdout:5/143: dread d0/f14 [0,4194304] 0 2024-02-26T10:47:52.563 INFO:tasks.workunit.client.0.smithi060.stdout:5/144: readlink d0/de/d15/l1b 0 2024-02-26T10:47:52.568 INFO:tasks.workunit.client.0.smithi060.stdout:5/145: symlink d0/de/d20/l2e 0 2024-02-26T10:47:52.568 INFO:tasks.workunit.client.0.smithi060.stdout:2/132: rename c7 to dd/d11/d13/c27 0 2024-02-26T10:47:52.568 INFO:tasks.workunit.client.0.smithi060.stdout:2/133: rename dd to dd/d11/d13/d28 22 2024-02-26T10:47:52.568 INFO:tasks.workunit.client.0.smithi060.stdout:2/134: chown f0 2594852 1 2024-02-26T10:47:52.571 INFO:tasks.workunit.client.0.smithi060.stdout:1/165: creat d3/db/d2e/f33 x:0 0 0 2024-02-26T10:47:52.571 INFO:tasks.workunit.client.0.smithi060.stdout:1/166: dread - d3/f23 zero size 2024-02-26T10:47:52.576 INFO:tasks.workunit.client.0.smithi060.stdout:0/159: mkdir d0/dd/d33/d3c 0 2024-02-26T10:47:52.580 INFO:tasks.workunit.client.0.smithi060.stdout:3/109: write d5/f9 [4023210,66837] 0 2024-02-26T10:47:52.580 INFO:tasks.workunit.client.0.smithi060.stdout:3/110: fsync d5/fa 0 2024-02-26T10:47:52.580 INFO:tasks.workunit.client.0.smithi060.stdout:1/167: rmdir d3/db 39 2024-02-26T10:47:52.581 INFO:tasks.workunit.client.0.smithi060.stdout:2/135: truncate f3 435105 0 2024-02-26T10:47:52.581 INFO:tasks.workunit.client.0.smithi060.stdout:1/168: chown d3/db/d1e/f22 15599213 1 2024-02-26T10:47:52.581 INFO:tasks.workunit.client.0.smithi060.stdout:1/169: chown d3/db 164 1 2024-02-26T10:47:52.587 INFO:tasks.workunit.client.0.smithi060.stdout:6/158: write d6/fe [2280057,11081] 0 2024-02-26T10:47:52.588 INFO:tasks.workunit.client.0.smithi060.stdout:2/136: mknod dd/d11/c29 0 2024-02-26T10:47:52.590 INFO:tasks.workunit.client.0.smithi060.stdout:2/137: symlink dd/d11/d13/l2a 0 2024-02-26T10:47:52.591 INFO:tasks.workunit.client.0.smithi060.stdout:6/159: mkdir d6/d7/d2e 0 2024-02-26T10:47:52.591 INFO:tasks.workunit.client.0.smithi060.stdout:6/160: dread - d6/d7/d25/d1c/d1f/d23/d28/f29 zero size 2024-02-26T10:47:52.592 INFO:tasks.workunit.client.0.smithi060.stdout:6/161: mknod d6/d7/d2e/c2f 0 2024-02-26T10:47:52.594 INFO:tasks.workunit.client.0.smithi060.stdout:6/162: link d6/d7/d2e/c2f d6/d7/d25/d1c/d1f/d23/c30 0 2024-02-26T10:47:52.594 INFO:tasks.workunit.client.0.smithi060.stdout:6/163: chown d6/lb 178373 1 2024-02-26T10:47:52.598 INFO:tasks.workunit.client.0.smithi060.stdout:6/164: symlink d6/d7/l31 0 2024-02-26T10:47:52.598 INFO:tasks.workunit.client.0.smithi060.stdout:3/111: read d5/fb [1681243,10972] 0 2024-02-26T10:47:52.601 INFO:tasks.workunit.client.0.smithi060.stdout:3/112: mkdir d5/de/d16 0 2024-02-26T10:47:52.601 INFO:tasks.workunit.client.0.smithi060.stdout:3/113: stat f3 0 2024-02-26T10:47:52.601 INFO:tasks.workunit.client.0.smithi060.stdout:3/114: chown d5/cc 1122852 1 2024-02-26T10:47:52.615 INFO:tasks.workunit.client.0.smithi060.stdout:0/160: dread d0/dd/d16/f34 [0,4194304] 0 2024-02-26T10:47:52.617 INFO:tasks.workunit.client.0.smithi060.stdout:0/161: mknod d0/dd/d16/c3d 0 2024-02-26T10:47:52.617 INFO:tasks.workunit.client.0.smithi060.stdout:0/162: write d0/dd/d16/f26 [432035,108] 0 2024-02-26T10:47:52.618 INFO:tasks.workunit.client.0.smithi060.stdout:0/163: getdents d0/dd/d39 0 2024-02-26T10:47:52.618 INFO:tasks.workunit.client.0.smithi060.stdout:0/164: chown d0/f15 1271348 1 2024-02-26T10:47:52.620 INFO:tasks.workunit.client.0.smithi060.stdout:0/165: write d0/dd/d16/f1f [250724,105050] 0 2024-02-26T10:47:52.621 INFO:tasks.workunit.client.0.smithi060.stdout:8/162: dwrite d9/d15/f1c [0,4194304] 0 2024-02-26T10:47:52.626 INFO:tasks.workunit.client.0.smithi060.stdout:0/166: link d0/dd/cf d0/dd/c3e 0 2024-02-26T10:47:52.627 INFO:tasks.workunit.client.0.smithi060.stdout:0/167: getdents d0/dd/d33/d3c 0 2024-02-26T10:47:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:3/115: dread d5/f6 [0,4194304] 0 2024-02-26T10:47:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:3/116: fsync d5/fb 0 2024-02-26T10:47:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:3/117: mkdir d5/de/d16/d17 0 2024-02-26T10:47:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:3/118: fdatasync d5/fb 0 2024-02-26T10:47:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:3/119: mknod d5/de/df/d11/c18 0 2024-02-26T10:47:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:3/120: symlink d5/l19 0 2024-02-26T10:47:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:3/121: write d5/fd [1616307,78175] 0 2024-02-26T10:47:52.649 INFO:tasks.workunit.client.0.smithi060.stdout:6/165: dread d6/f11 [0,4194304] 0 2024-02-26T10:47:52.649 INFO:tasks.workunit.client.0.smithi060.stdout:6/166: fdatasync d6/d7/f2a 0 2024-02-26T10:47:52.655 INFO:tasks.workunit.client.0.smithi060.stdout:3/122: mknod d5/de/d16/c1a 0 2024-02-26T10:47:52.656 INFO:tasks.workunit.client.0.smithi060.stdout:9/143: dwrite d3/d15/f28 [0,4194304] 0 2024-02-26T10:47:52.656 INFO:tasks.workunit.client.0.smithi060.stdout:9/144: chown d3/d15/d1b/d23/f29 852852 1 2024-02-26T10:47:52.656 INFO:tasks.workunit.client.0.smithi060.stdout:5/146: dread d0/de/f29 [0,4194304] 0 2024-02-26T10:47:52.657 INFO:tasks.workunit.client.0.smithi060.stdout:6/167: mknod d6/d7/d25/d1c/d1f/d23/d2b/c32 0 2024-02-26T10:47:52.658 INFO:tasks.workunit.client.0.smithi060.stdout:5/147: creat d0/de/d15/f2f x:0 0 0 2024-02-26T10:47:52.660 INFO:tasks.workunit.client.0.smithi060.stdout:1/170: dwrite d3/f9 [0,4194304] 0 2024-02-26T10:47:52.663 INFO:tasks.workunit.client.0.smithi060.stdout:5/148: getdents d0/de/d15 0 2024-02-26T10:47:52.665 INFO:tasks.workunit.client.0.smithi060.stdout:3/123: creat d5/f1b x:0 0 0 2024-02-26T10:47:52.667 INFO:tasks.workunit.client.0.smithi060.stdout:9/145: symlink d3/l2b 0 2024-02-26T10:47:52.667 INFO:tasks.workunit.client.0.smithi060.stdout:3/124: rename d5/f1b to d5/de/df/d11/f1c 0 2024-02-26T10:47:52.668 INFO:tasks.workunit.client.0.smithi060.stdout:9/146: truncate d3/d15/f17 1172307 0 2024-02-26T10:47:52.669 INFO:tasks.workunit.client.0.smithi060.stdout:3/125: link d5/f9 d5/de/d16/d17/f1d 0 2024-02-26T10:47:52.673 INFO:tasks.workunit.client.0.smithi060.stdout:9/147: creat d3/d9/d10/f2c x:0 0 0 2024-02-26T10:47:52.673 INFO:tasks.workunit.client.0.smithi060.stdout:9/148: rename d3/d15/d1b to d3/d15/d1b/d2d 22 2024-02-26T10:47:52.673 INFO:tasks.workunit.client.0.smithi060.stdout:8/163: dwrite d9/f20 [0,4194304] 0 2024-02-26T10:47:52.673 INFO:tasks.workunit.client.0.smithi060.stdout:8/164: stat d9/f23 0 2024-02-26T10:47:52.681 INFO:tasks.workunit.client.0.smithi060.stdout:7/307: sync 2024-02-26T10:47:52.681 INFO:tasks.workunit.client.0.smithi060.stdout:4/194: sync 2024-02-26T10:47:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:6/168: dread d6/fe [0,4194304] 0 2024-02-26T10:47:52.690 INFO:tasks.workunit.client.0.smithi060.stdout:7/308: creat da/d15/d21/d2f/f68 x:0 0 0 2024-02-26T10:47:52.691 INFO:tasks.workunit.client.0.smithi060.stdout:4/195: creat d0/d7/f37 x:0 0 0 2024-02-26T10:47:52.691 INFO:tasks.workunit.client.0.smithi060.stdout:3/126: getdents d5/de/d16/d17 0 2024-02-26T10:47:52.691 INFO:tasks.workunit.client.0.smithi060.stdout:6/169: creat d6/d7/d25/d1c/d1f/d23/d28/f33 x:0 0 0 2024-02-26T10:47:52.692 INFO:tasks.workunit.client.0.smithi060.stdout:7/309: mknod da/d15/d21/d32/d4e/c69 0 2024-02-26T10:47:52.692 INFO:tasks.workunit.client.0.smithi060.stdout:7/310: readlink da/d15/d21/d2f/d37/l44 0 2024-02-26T10:47:52.694 INFO:tasks.workunit.client.0.smithi060.stdout:4/196: unlink d0/d7/c33 0 2024-02-26T10:47:52.694 INFO:tasks.workunit.client.0.smithi060.stdout:0/168: dwrite d0/dd/f31 [0,4194304] 0 2024-02-26T10:47:52.694 INFO:tasks.workunit.client.0.smithi060.stdout:6/170: creat d6/d7/d25/d1c/d1f/d23/d28/f34 x:0 0 0 2024-02-26T10:47:52.694 INFO:tasks.workunit.client.0.smithi060.stdout:6/171: chown d6/d7/d25/c21 0 1 2024-02-26T10:47:52.697 INFO:tasks.workunit.client.0.smithi060.stdout:7/311: link da/d15/l1e da/d15/d21/l6a 0 2024-02-26T10:47:52.697 INFO:tasks.workunit.client.0.smithi060.stdout:4/197: truncate d0/d7/d29/f32 397007 0 2024-02-26T10:47:52.697 INFO:tasks.workunit.client.0.smithi060.stdout:5/149: dread d0/fd [0,4194304] 0 2024-02-26T10:47:52.697 INFO:tasks.workunit.client.0.smithi060.stdout:0/169: creat d0/dd/d39/f3f x:0 0 0 2024-02-26T10:47:52.699 INFO:tasks.workunit.client.0.smithi060.stdout:4/198: creat d0/d7/d29/f38 x:0 0 0 2024-02-26T10:47:52.700 INFO:tasks.workunit.client.0.smithi060.stdout:4/199: fdatasync d0/d27/f30 0 2024-02-26T10:47:52.700 INFO:tasks.workunit.client.0.smithi060.stdout:7/312: rename da/d15/d21/d32/d4e/d3b/c4f to da/d15/d21/d41/d58/c6b 0 2024-02-26T10:47:52.702 INFO:tasks.workunit.client.0.smithi060.stdout:4/200: creat d0/da/dc/d16/f39 x:0 0 0 2024-02-26T10:47:52.702 INFO:tasks.workunit.client.0.smithi060.stdout:0/170: rename d0/dd/d16 to d0/dd/d39/d40 0 2024-02-26T10:47:52.704 INFO:tasks.workunit.client.0.smithi060.stdout:0/171: rename d0/dd/c2f to d0/dd/d33/c41 0 2024-02-26T10:47:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:1/171: dwrite d3/f2a [0,4194304] 0 2024-02-26T10:47:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:0/172: truncate d0/dd/f31 3613379 0 2024-02-26T10:47:52.714 INFO:tasks.workunit.client.0.smithi060.stdout:5/150: write d0/de/d15/f18 [4675325,32602] 0 2024-02-26T10:47:52.717 INFO:tasks.workunit.client.0.smithi060.stdout:6/172: dread d6/d7/d25/d1c/f2c [0,4194304] 0 2024-02-26T10:47:52.719 INFO:tasks.workunit.client.0.smithi060.stdout:7/313: getdents da/d15/d21/d29 0 2024-02-26T10:47:52.722 INFO:tasks.workunit.client.0.smithi060.stdout:6/173: link d6/d7/d25/f2d d6/d7/d25/d1c/d1f/f35 0 2024-02-26T10:47:52.730 INFO:tasks.workunit.client.0.smithi060.stdout:7/314: mkdir da/d15/d21/d41/d6c 0 2024-02-26T10:47:52.742 INFO:tasks.workunit.client.0.smithi060.stdout:1/172: rename d3/db/d1e/f1f to d3/f34 0 2024-02-26T10:47:52.750 INFO:tasks.workunit.client.0.smithi060.stdout:7/315: rmdir da/d5a 39 2024-02-26T10:47:52.751 INFO:tasks.workunit.client.0.smithi060.stdout:6/174: write d6/d7/fd [2383016,36978] 0 2024-02-26T10:47:52.751 INFO:tasks.workunit.client.0.smithi060.stdout:6/175: read - d6/d7/d25/f2d zero size 2024-02-26T10:47:52.753 INFO:tasks.workunit.client.0.smithi060.stdout:6/176: symlink d6/l36 0 2024-02-26T10:47:52.753 INFO:tasks.workunit.client.0.smithi060.stdout:6/177: write d6/d7/d25/d1c/d1f/f35 [271768,25265] 0 2024-02-26T10:47:52.753 INFO:tasks.workunit.client.0.smithi060.stdout:6/178: chown d6/d7/d25/d1c/d1f/d23/d28/f34 823 1 2024-02-26T10:47:52.753 INFO:tasks.workunit.client.0.smithi060.stdout:6/179: readlink d6/l36 0 2024-02-26T10:47:52.753 INFO:tasks.workunit.client.0.smithi060.stdout:6/180: chown d6/d7/d25/d1c/d1f/d23/d28/f33 49308415 1 2024-02-26T10:47:52.754 INFO:tasks.workunit.client.0.smithi060.stdout:9/149: dwrite d3/d9/d10/f11 [0,4194304] 0 2024-02-26T10:47:52.754 INFO:tasks.workunit.client.0.smithi060.stdout:6/181: creat d6/d7/d25/d1c/d1f/f37 x:0 0 0 2024-02-26T10:47:52.754 INFO:tasks.workunit.client.0.smithi060.stdout:6/182: chown d6/d7/d25/d1c/f2c 1748100 1 2024-02-26T10:47:52.767 INFO:tasks.workunit.client.0.smithi060.stdout:1/173: dread d3/f15 [0,4194304] 0 2024-02-26T10:47:52.767 INFO:tasks.workunit.client.0.smithi060.stdout:1/174: write d3/f23 [624482,9081] 0 2024-02-26T10:47:52.770 INFO:tasks.workunit.client.0.smithi060.stdout:1/175: mkdir d3/d35 0 2024-02-26T10:47:52.770 INFO:tasks.workunit.client.0.smithi060.stdout:1/176: truncate d3/f2b 1138529 0 2024-02-26T10:47:52.770 INFO:tasks.workunit.client.0.smithi060.stdout:1/177: write d3/db/d1e/f22 [676194,3387] 0 2024-02-26T10:47:52.773 INFO:tasks.workunit.client.0.smithi060.stdout:9/150: rename d3/d15/f28 to d3/d9/f2e 0 2024-02-26T10:47:52.774 INFO:tasks.workunit.client.0.smithi060.stdout:1/178: mknod d3/db/d1e/c36 0 2024-02-26T10:47:52.774 INFO:tasks.workunit.client.0.smithi060.stdout:1/179: fsync d3/f2a 0 2024-02-26T10:47:52.774 INFO:tasks.workunit.client.0.smithi060.stdout:1/180: chown d3/l17 53 1 2024-02-26T10:47:52.777 INFO:tasks.workunit.client.0.smithi060.stdout:1/181: symlink d3/db/d1e/l37 0 2024-02-26T10:47:52.779 INFO:tasks.workunit.client.0.smithi060.stdout:1/182: mkdir d3/db/d1e/d38 0 2024-02-26T10:47:52.779 INFO:tasks.workunit.client.0.smithi060.stdout:1/183: dread - d3/db/d2e/f33 zero size 2024-02-26T10:47:52.779 INFO:tasks.workunit.client.0.smithi060.stdout:1/184: fdatasync d3/f2b 0 2024-02-26T10:47:52.780 INFO:tasks.workunit.client.0.smithi060.stdout:7/316: read da/d15/d21/d2f/d37/f38 [1017087,2847] 0 2024-02-26T10:47:52.781 INFO:tasks.workunit.client.0.smithi060.stdout:4/201: dwrite d0/d7/f28 [0,4194304] 0 2024-02-26T10:47:52.781 INFO:tasks.workunit.client.0.smithi060.stdout:4/202: truncate d0/da/dc/f34 447361 0 2024-02-26T10:47:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:5/151: dwrite d0/de/d15/f18 [4194304,4194304] 0 2024-02-26T10:47:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:1/185: mkdir d3/db/d1e/d39 0 2024-02-26T10:47:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:5/152: readlink d0/de/d20/l2a 0 2024-02-26T10:47:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:4/203: creat d0/da/dc/d16/f3a x:0 0 0 2024-02-26T10:47:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:1/186: readlink d3/db/d1e/l26 0 2024-02-26T10:47:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:1/187: fsync d3/db/d1e/f22 0 2024-02-26T10:47:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:1/188: getdents d3/db/d2e 0 2024-02-26T10:47:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:7/317: mknod da/d15/d21/d29/c6d 0 2024-02-26T10:47:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:1/189: mknod d3/db/d1e/c3a 0 2024-02-26T10:47:52.789 INFO:tasks.workunit.client.0.smithi060.stdout:1/190: creat d3/db/d1e/f3b x:0 0 0 2024-02-26T10:47:52.789 INFO:tasks.workunit.client.0.smithi060.stdout:1/191: write d3/db/d1e/f3b [536361,6193] 0 2024-02-26T10:47:52.790 INFO:tasks.workunit.client.0.smithi060.stdout:5/153: read d0/f17 [163185,43238] 0 2024-02-26T10:47:52.790 INFO:tasks.workunit.client.0.smithi060.stdout:1/192: write d3/f2b [1921351,97574] 0 2024-02-26T10:47:52.792 INFO:tasks.workunit.client.0.smithi060.stdout:0/173: dwrite d0/dd/d39/d40/f1c [0,4194304] 0 2024-02-26T10:47:52.792 INFO:tasks.workunit.client.0.smithi060.stdout:1/193: mknod d3/d35/c3c 0 2024-02-26T10:47:52.794 INFO:tasks.workunit.client.0.smithi060.stdout:0/174: unlink d0/dd/c25 0 2024-02-26T10:47:52.794 INFO:tasks.workunit.client.0.smithi060.stdout:0/175: fdatasync d0/dd/f2b 0 2024-02-26T10:47:52.795 INFO:tasks.workunit.client.0.smithi060.stdout:4/204: rename d0/da/dc to d0/da/d3b 0 2024-02-26T10:47:52.795 INFO:tasks.workunit.client.0.smithi060.stdout:4/205: readlink d0/da/d3b/d1b/l1c 0 2024-02-26T10:47:52.796 INFO:tasks.workunit.client.0.smithi060.stdout:4/206: dread - d0/da/d3b/d16/f2d zero size 2024-02-26T10:47:52.797 INFO:tasks.workunit.client.0.smithi060.stdout:2/138: sync 2024-02-26T10:47:52.797 INFO:tasks.workunit.client.0.smithi060.stdout:2/139: fsync f4 0 2024-02-26T10:47:52.798 INFO:tasks.workunit.client.0.smithi060.stdout:2/140: truncate fa 618286 0 2024-02-26T10:47:52.803 INFO:tasks.workunit.client.0.smithi060.stdout:5/154: write d0/de/f12 [1437940,87735] 0 2024-02-26T10:47:52.807 INFO:tasks.workunit.client.0.smithi060.stdout:6/183: dwrite d6/d7/d25/d1c/d1f/f24 [0,4194304] 0 2024-02-26T10:47:52.807 INFO:tasks.workunit.client.0.smithi060.stdout:6/184: write d6/d7/d25/d1c/d1f/f37 [996640,64830] 0 2024-02-26T10:47:52.808 INFO:tasks.workunit.client.0.smithi060.stdout:6/185: chown d6 7 1 2024-02-26T10:47:52.808 INFO:tasks.workunit.client.0.smithi060.stdout:6/186: fdatasync d6/d7/f12 0 2024-02-26T10:47:52.811 INFO:tasks.workunit.client.0.smithi060.stdout:6/187: getdents d6/d7/d25 0 2024-02-26T10:47:52.830 INFO:tasks.workunit.client.0.smithi060.stdout:3/127: sync 2024-02-26T10:47:52.830 INFO:tasks.workunit.client.0.smithi060.stdout:8/165: sync 2024-02-26T10:47:52.830 INFO:tasks.workunit.client.0.smithi060.stdout:1/194: rename d3/db/ff to d3/db/d1e/f3d 0 2024-02-26T10:47:52.830 INFO:tasks.workunit.client.0.smithi060.stdout:3/128: write d5/de/f14 [2108556,2756] 0 2024-02-26T10:47:52.830 INFO:tasks.workunit.client.0.smithi060.stdout:1/195: write d3/db/d2e/f30 [95531,128961] 0 2024-02-26T10:47:52.830 INFO:tasks.workunit.client.0.smithi060.stdout:1/196: chown d3/f23 15831949 1 2024-02-26T10:47:52.830 INFO:tasks.workunit.client.0.smithi060.stdout:6/188: dread d6/f15 [0,4194304] 0 2024-02-26T10:47:52.831 INFO:tasks.workunit.client.0.smithi060.stdout:1/197: mknod d3/db/d1e/d38/c3e 0 2024-02-26T10:47:52.831 INFO:tasks.workunit.client.0.smithi060.stdout:1/198: write d3/f12 [651734,115405] 0 2024-02-26T10:47:52.831 INFO:tasks.workunit.client.0.smithi060.stdout:1/199: read d3/f15 [1496853,47179] 0 2024-02-26T10:47:52.834 INFO:tasks.workunit.client.0.smithi060.stdout:0/176: rename d0/dd/d33/d3c to d0/d42 0 2024-02-26T10:47:52.834 INFO:tasks.workunit.client.0.smithi060.stdout:3/129: creat d5/de/d16/d17/f1e x:0 0 0 2024-02-26T10:47:52.834 INFO:tasks.workunit.client.0.smithi060.stdout:0/177: truncate d0/dd/f31 4178133 0 2024-02-26T10:47:52.834 INFO:tasks.workunit.client.0.smithi060.stdout:0/178: dread - d0/f32 zero size 2024-02-26T10:47:52.834 INFO:tasks.workunit.client.0.smithi060.stdout:1/200: mknod d3/d35/c3f 0 2024-02-26T10:47:52.836 INFO:tasks.workunit.client.0.smithi060.stdout:3/130: creat d5/de/d16/f1f x:0 0 0 2024-02-26T10:47:52.836 INFO:tasks.workunit.client.0.smithi060.stdout:1/201: creat d3/db/f40 x:0 0 0 2024-02-26T10:47:52.842 INFO:tasks.workunit.client.0.smithi060.stdout:6/189: dread d6/d7/d25/d1c/d1f/f35 [0,4194304] 0 2024-02-26T10:47:52.843 INFO:tasks.workunit.client.0.smithi060.stdout:6/190: creat d6/f38 x:0 0 0 2024-02-26T10:47:52.843 INFO:tasks.workunit.client.0.smithi060.stdout:6/191: stat d6/d7/d25/d1c/d1f/d23/d28/f33 0 2024-02-26T10:47:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:1/202: getdents d3/db 0 2024-02-26T10:47:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:1/203: read d3/db/d2e/f30 [42069,123183] 0 2024-02-26T10:47:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:1/204: fdatasync d3/db/d1e/f22 0 2024-02-26T10:47:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:1/205: mknod d3/d32/c41 0 2024-02-26T10:47:52.854 INFO:tasks.workunit.client.0.smithi060.stdout:1/206: getdents d3/db/d1e/d38 0 2024-02-26T10:47:52.854 INFO:tasks.workunit.client.0.smithi060.stdout:4/207: dwrite d0/ff [0,4194304] 0 2024-02-26T10:47:52.862 INFO:tasks.workunit.client.0.smithi060.stdout:3/131: dread d5/f6 [0,4194304] 0 2024-02-26T10:47:52.865 INFO:tasks.workunit.client.0.smithi060.stdout:1/207: link d3/db/c21 d3/db/d1e/c42 0 2024-02-26T10:47:52.865 INFO:tasks.workunit.client.0.smithi060.stdout:1/208: chown d3/f23 76 1 2024-02-26T10:47:52.865 INFO:tasks.workunit.client.0.smithi060.stdout:1/209: fsync d3/f20 0 2024-02-26T10:47:52.867 INFO:tasks.workunit.client.0.smithi060.stdout:1/210: mknod d3/db/d1e/c43 0 2024-02-26T10:47:52.868 INFO:tasks.workunit.client.0.smithi060.stdout:1/211: symlink d3/d35/l44 0 2024-02-26T10:47:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:5/155: dwrite d0/f9 [0,4194304] 0 2024-02-26T10:47:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:4/208: write d0/da/d3b/d16/f24 [48617,74928] 0 2024-02-26T10:47:52.910 INFO:tasks.workunit.client.0.smithi060.stdout:0/179: dwrite d0/dd/d39/d40/f28 [0,4194304] 0 2024-02-26T10:47:52.910 INFO:tasks.workunit.client.0.smithi060.stdout:0/180: write d0/dd/f1d [4746691,27969] 0 2024-02-26T10:47:52.911 INFO:tasks.workunit.client.0.smithi060.stdout:0/181: creat d0/d37/f43 x:0 0 0 2024-02-26T10:47:52.912 INFO:tasks.workunit.client.0.smithi060.stdout:1/212: dwrite d3/f2a [4194304,4194304] 0 2024-02-26T10:47:52.913 INFO:tasks.workunit.client.0.smithi060.stdout:1/213: mknod d3/db/d1e/d38/c45 0 2024-02-26T10:47:52.914 INFO:tasks.workunit.client.0.smithi060.stdout:1/214: creat d3/db/d1e/f46 x:0 0 0 2024-02-26T10:47:52.923 INFO:tasks.workunit.client.0.smithi060.stdout:3/132: dwrite d5/f9 [4194304,4194304] 0 2024-02-26T10:47:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:6/192: write d6/f18 [696585,58148] 0 2024-02-26T10:47:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:8/166: dwrite d9/f1e [0,4194304] 0 2024-02-26T10:47:52.935 INFO:tasks.workunit.client.0.smithi060.stdout:3/133: rename d5/fb to d5/de/df/d11/f20 0 2024-02-26T10:47:52.935 INFO:tasks.workunit.client.0.smithi060.stdout:1/215: dread d3/f1b [0,4194304] 0 2024-02-26T10:47:52.935 INFO:tasks.workunit.client.0.smithi060.stdout:1/216: write d3/f1b [4145827,29220] 0 2024-02-26T10:47:52.936 INFO:tasks.workunit.client.0.smithi060.stdout:6/193: mkdir d6/d7/d25/d1c/d39 0 2024-02-26T10:47:52.937 INFO:tasks.workunit.client.0.smithi060.stdout:8/167: getdents d9 0 2024-02-26T10:47:52.941 INFO:tasks.workunit.client.0.smithi060.stdout:1/217: mknod d3/db/d1e/c47 0 2024-02-26T10:47:52.941 INFO:tasks.workunit.client.0.smithi060.stdout:1/218: write d3/f2a [8849601,57062] 0 2024-02-26T10:47:52.941 INFO:tasks.workunit.client.0.smithi060.stdout:6/194: truncate d6/fe 2597302 0 2024-02-26T10:47:52.956 INFO:tasks.workunit.client.0.smithi060.stdout:6/195: dread d6/d7/d25/d1c/d1f/f35 [0,4194304] 0 2024-02-26T10:47:52.958 INFO:tasks.workunit.client.0.smithi060.stdout:6/196: creat d6/d7/d25/d1c/d1f/d23/f3a x:0 0 0 2024-02-26T10:47:52.958 INFO:tasks.workunit.client.0.smithi060.stdout:6/197: getdents d6/d7/d25/d1c/d39 0 2024-02-26T10:47:52.958 INFO:tasks.workunit.client.0.smithi060.stdout:6/198: write d6/fe [2905419,110722] 0 2024-02-26T10:47:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:5/156: dwrite d0/fd [0,4194304] 0 2024-02-26T10:47:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:6/199: dread d6/d7/d25/f2d [0,4194304] 0 2024-02-26T10:47:52.975 INFO:tasks.workunit.client.0.smithi060.stdout:6/200: link d6/d7/f12 d6/d7/d25/d1c/d1f/d23/d2b/f3b 0 2024-02-26T10:47:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:7/318: sync 2024-02-26T10:47:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:3/134: dwrite d5/de/df/d11/f20 [0,4194304] 0 2024-02-26T10:47:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:9/151: sync 2024-02-26T10:47:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:3/135: creat d5/de/df/f21 x:0 0 0 2024-02-26T10:47:52.996 INFO:tasks.workunit.client.0.smithi060.stdout:0/182: write d0/dd/f31 [293852,129963] 0 2024-02-26T10:47:52.996 INFO:tasks.workunit.client.0.smithi060.stdout:0/183: chown d0/la 1 1 2024-02-26T10:47:52.998 INFO:tasks.workunit.client.0.smithi060.stdout:1/219: dwrite d3/db/d1e/f22 [0,4194304] 0 2024-02-26T10:47:52.998 INFO:tasks.workunit.client.0.smithi060.stdout:1/220: chown d3/f2a 19782494 1 2024-02-26T10:47:52.998 INFO:tasks.workunit.client.0.smithi060.stdout:1/221: read d3/f20 [1401128,25457] 0 2024-02-26T10:47:52.998 INFO:tasks.workunit.client.0.smithi060.stdout:1/222: chown d3/db/d1e/l37 510 1 2024-02-26T10:47:53.001 INFO:tasks.workunit.client.0.smithi060.stdout:5/157: rmdir d0 39 2024-02-26T10:47:53.008 INFO:tasks.workunit.client.0.smithi060.stdout:9/152: rename d3/f7 to d3/d9/f2f 0 2024-02-26T10:47:53.009 INFO:tasks.workunit.client.0.smithi060.stdout:8/168: dread d9/fd [0,4194304] 0 2024-02-26T10:47:53.010 INFO:tasks.workunit.client.0.smithi060.stdout:4/209: dwrite d0/d7/f25 [0,4194304] 0 2024-02-26T10:47:53.010 INFO:tasks.workunit.client.0.smithi060.stdout:7/319: dread da/d15/d21/d32/d4e/f5c [0,4194304] 0 2024-02-26T10:47:53.011 INFO:tasks.workunit.client.0.smithi060.stdout:9/153: mknod d3/d15/d1b/c30 0 2024-02-26T10:47:53.011 INFO:tasks.workunit.client.0.smithi060.stdout:9/154: chown d3/d15/d1b/d23 1 1 2024-02-26T10:47:53.011 INFO:tasks.workunit.client.0.smithi060.stdout:9/155: stat d3/d9/f19 0 2024-02-26T10:47:53.013 INFO:tasks.workunit.client.0.smithi060.stdout:4/210: getdents d0/d27 0 2024-02-26T10:47:53.013 INFO:tasks.workunit.client.0.smithi060.stdout:4/211: chown d0/da/d3b/d16/f20 16 1 2024-02-26T10:47:53.014 INFO:tasks.workunit.client.0.smithi060.stdout:9/156: symlink d3/d15/d1b/d23/l31 0 2024-02-26T10:47:53.014 INFO:tasks.workunit.client.0.smithi060.stdout:5/158: rename d0/de/d15/f21 to d0/f30 0 2024-02-26T10:47:53.014 INFO:tasks.workunit.client.0.smithi060.stdout:9/157: mkdir d3/d9/dc/d32 0 2024-02-26T10:47:53.015 INFO:tasks.workunit.client.0.smithi060.stdout:9/158: write d3/d15/d1b/d23/f29 [709232,73219] 0 2024-02-26T10:47:53.015 INFO:tasks.workunit.client.0.smithi060.stdout:5/159: creat d0/de/d20/f31 x:0 0 0 2024-02-26T10:47:53.017 INFO:tasks.workunit.client.0.smithi060.stdout:4/212: creat d0/f3c x:0 0 0 2024-02-26T10:47:53.018 INFO:tasks.workunit.client.0.smithi060.stdout:4/213: write d0/da/d3b/d16/f24 [277544,48414] 0 2024-02-26T10:47:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:9/159: mknod d3/d1e/c33 0 2024-02-26T10:47:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:9/160: chown d3/d9/d10/f21 213930 1 2024-02-26T10:47:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:9/161: fdatasync d3/d9/d10/f11 0 2024-02-26T10:47:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:1/223: dread d3/f12 [0,4194304] 0 2024-02-26T10:47:53.028 INFO:tasks.workunit.client.0.smithi060.stdout:0/184: dread d0/dd/d39/d40/f1c [0,4194304] 0 2024-02-26T10:47:53.028 INFO:tasks.workunit.client.0.smithi060.stdout:0/185: read d0/dd/f31 [66014,99416] 0 2024-02-26T10:47:53.028 INFO:tasks.workunit.client.0.smithi060.stdout:0/186: readlink d0/la 0 2024-02-26T10:47:53.029 INFO:tasks.workunit.client.0.smithi060.stdout:3/136: write d5/de/df/d11/f20 [1655471,2964] 0 2024-02-26T10:47:53.029 INFO:tasks.workunit.client.0.smithi060.stdout:3/137: readlink d5/de/l10 0 2024-02-26T10:47:53.032 INFO:tasks.workunit.client.0.smithi060.stdout:1/224: creat d3/db/d1e/d39/f48 x:0 0 0 2024-02-26T10:47:53.032 INFO:tasks.workunit.client.0.smithi060.stdout:1/225: readlink d3/l17 0 2024-02-26T10:47:53.033 INFO:tasks.workunit.client.0.smithi060.stdout:1/226: write d3/f1b [1824057,37819] 0 2024-02-26T10:47:53.033 INFO:tasks.workunit.client.0.smithi060.stdout:0/187: mkdir d0/d37/d44 0 2024-02-26T10:47:53.033 INFO:tasks.workunit.client.0.smithi060.stdout:0/188: fdatasync d0/f15 0 2024-02-26T10:47:53.034 INFO:tasks.workunit.client.0.smithi060.stdout:0/189: write d0/f1 [1214869,51625] 0 2024-02-26T10:47:53.034 INFO:tasks.workunit.client.0.smithi060.stdout:3/138: mknod d5/de/df/d11/c22 0 2024-02-26T10:47:53.051 INFO:tasks.workunit.client.0.smithi060.stdout:9/162: dread d3/d9/f19 [0,4194304] 0 2024-02-26T10:47:53.053 INFO:tasks.workunit.client.0.smithi060.stdout:9/163: creat d3/d15/d1b/d23/f34 x:0 0 0 2024-02-26T10:47:53.055 INFO:tasks.workunit.client.0.smithi060.stdout:9/164: mkdir d3/d15/d35 0 2024-02-26T10:47:53.068 INFO:tasks.workunit.client.0.smithi060.stdout:1/227: dread d3/db/d1e/f27 [0,4194304] 0 2024-02-26T10:47:53.070 INFO:tasks.workunit.client.0.smithi060.stdout:3/139: dread d5/fd [0,4194304] 0 2024-02-26T10:47:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:3/140: mknod d5/de/df/d11/c23 0 2024-02-26T10:47:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:7/320: dread da/d15/d21/d2f/d37/f38 [0,4194304] 0 2024-02-26T10:47:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:8/169: dwrite d9/f13 [0,4194304] 0 2024-02-26T10:47:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:8/170: chown d9/f10 301 1 2024-02-26T10:47:53.078 INFO:tasks.workunit.client.0.smithi060.stdout:5/160: dwrite d0/de/d20/f31 [0,4194304] 0 2024-02-26T10:47:53.081 INFO:tasks.workunit.client.0.smithi060.stdout:5/161: mkdir d0/de/d20/d32 0 2024-02-26T10:47:53.081 INFO:tasks.workunit.client.0.smithi060.stdout:7/321: creat da/d15/d21/d41/d6c/f6e x:0 0 0 2024-02-26T10:47:53.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:52 smithi113 ceph-mon[36126]: pgmap v482: 129 pgs: 129 active+clean; 198 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 2.04k op/s 2024-02-26T10:47:53.082 INFO:tasks.workunit.client.0.smithi060.stdout:5/162: truncate d0/f5 2996840 0 2024-02-26T10:47:53.084 INFO:tasks.workunit.client.0.smithi060.stdout:4/214: dwrite d0/da/d3b/f1a [0,4194304] 0 2024-02-26T10:47:53.084 INFO:tasks.workunit.client.0.smithi060.stdout:0/190: dwrite d0/dd/d39/d40/f19 [4194304,4194304] 0 2024-02-26T10:47:53.084 INFO:tasks.workunit.client.0.smithi060.stdout:0/191: chown d0/ce 756503676 1 2024-02-26T10:47:53.084 INFO:tasks.workunit.client.0.smithi060.stdout:7/322: creat da/d5a/f6f x:0 0 0 2024-02-26T10:47:53.087 INFO:tasks.workunit.client.0.smithi060.stdout:0/192: creat d0/d37/d44/f45 x:0 0 0 2024-02-26T10:47:53.087 INFO:tasks.workunit.client.0.smithi060.stdout:4/215: symlink d0/da/d3b/d1b/l3d 0 2024-02-26T10:47:53.091 INFO:tasks.workunit.client.0.smithi060.stdout:7/323: creat da/d15/d21/d41/d58/f70 x:0 0 0 2024-02-26T10:47:53.091 INFO:tasks.workunit.client.0.smithi060.stdout:7/324: write da/d15/f46 [1671007,55109] 0 2024-02-26T10:47:53.091 INFO:tasks.workunit.client.0.smithi060.stdout:0/193: mknod d0/dd/c46 0 2024-02-26T10:47:53.091 INFO:tasks.workunit.client.0.smithi060.stdout:0/194: fsync d0/dd/f2c 0 2024-02-26T10:47:53.091 INFO:tasks.workunit.client.0.smithi060.stdout:0/195: chown d0/d37 6 1 2024-02-26T10:47:53.094 INFO:tasks.workunit.client.0.smithi060.stdout:4/216: link d0/d7/d29/f38 d0/d7/f3e 0 2024-02-26T10:47:53.097 INFO:tasks.workunit.client.0.smithi060.stdout:3/141: write d5/fa [421758,101889] 0 2024-02-26T10:47:53.099 INFO:tasks.workunit.client.0.smithi060.stdout:9/165: write d3/d9/d10/f11 [221597,19667] 0 2024-02-26T10:47:53.099 INFO:tasks.workunit.client.0.smithi060.stdout:5/163: mkdir d0/d33 0 2024-02-26T10:47:53.100 INFO:tasks.workunit.client.0.smithi060.stdout:0/196: getdents d0/dd 0 2024-02-26T10:47:53.112 INFO:tasks.workunit.client.0.smithi060.stdout:9/166: creat d3/d15/d1b/d23/f36 x:0 0 0 2024-02-26T10:47:53.112 INFO:tasks.workunit.client.0.smithi060.stdout:4/217: rmdir d0 39 2024-02-26T10:47:53.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:52 smithi060 ceph-mon[31539]: pgmap v482: 129 pgs: 129 active+clean; 198 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 2.04k op/s 2024-02-26T10:47:53.115 INFO:tasks.workunit.client.0.smithi060.stdout:0/197: truncate d0/dd/f2b 1581827 0 2024-02-26T10:47:53.115 INFO:tasks.workunit.client.0.smithi060.stdout:8/171: dread f3 [0,4194304] 0 2024-02-26T10:47:53.115 INFO:tasks.workunit.client.0.smithi060.stdout:8/172: fsync d9/f14 0 2024-02-26T10:47:53.115 INFO:tasks.workunit.client.0.smithi060.stdout:9/167: link d3/d9/d10/f11 d3/d9/d10/f37 0 2024-02-26T10:47:53.118 INFO:tasks.workunit.client.0.smithi060.stdout:5/164: mknod d0/c34 0 2024-02-26T10:47:53.122 INFO:tasks.workunit.client.0.smithi060.stdout:4/218: creat d0/da/d3b/d16/f3f x:0 0 0 2024-02-26T10:47:53.122 INFO:tasks.workunit.client.0.smithi060.stdout:4/219: fsync d0/da/d3b/d16/f19 0 2024-02-26T10:47:53.122 INFO:tasks.workunit.client.0.smithi060.stdout:5/165: creat d0/de/f35 x:0 0 0 2024-02-26T10:47:53.122 INFO:tasks.workunit.client.0.smithi060.stdout:1/228: dwrite d3/f34 [0,4194304] 0 2024-02-26T10:47:53.122 INFO:tasks.workunit.client.0.smithi060.stdout:5/166: creat d0/f36 x:0 0 0 2024-02-26T10:47:53.123 INFO:tasks.workunit.client.0.smithi060.stdout:5/167: symlink d0/d33/l37 0 2024-02-26T10:47:53.124 INFO:tasks.workunit.client.0.smithi060.stdout:1/229: mknod d3/db/c49 0 2024-02-26T10:47:53.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:52 smithi100 ceph-mon[36051]: pgmap v482: 129 pgs: 129 active+clean; 198 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 2.04k op/s 2024-02-26T10:47:53.125 INFO:tasks.workunit.client.0.smithi060.stdout:1/230: symlink d3/d35/l4a 0 2024-02-26T10:47:53.135 INFO:tasks.workunit.client.0.smithi060.stdout:0/198: dread d0/dd/f2c [0,4194304] 0 2024-02-26T10:47:53.135 INFO:tasks.workunit.client.0.smithi060.stdout:5/168: link d0/f8 d0/de/d15/d26/f38 0 2024-02-26T10:47:53.140 INFO:tasks.workunit.client.0.smithi060.stdout:0/199: mknod d0/c47 0 2024-02-26T10:47:53.140 INFO:tasks.workunit.client.0.smithi060.stdout:0/200: chown d0/dd/d39 92 1 2024-02-26T10:47:53.140 INFO:tasks.workunit.client.0.smithi060.stdout:0/201: fdatasync d0/f32 0 2024-02-26T10:47:53.140 INFO:tasks.workunit.client.0.smithi060.stdout:2/141: sync 2024-02-26T10:47:53.147 INFO:tasks.workunit.client.0.smithi060.stdout:8/173: dread d9/f1e [0,4194304] 0 2024-02-26T10:47:53.149 INFO:tasks.workunit.client.0.smithi060.stdout:8/174: link d9/c17 d9/d15/d29/c2b 0 2024-02-26T10:47:53.149 INFO:tasks.workunit.client.0.smithi060.stdout:8/175: write f8 [4276926,22089] 0 2024-02-26T10:47:53.151 INFO:tasks.workunit.client.0.smithi060.stdout:8/176: link d9/f23 d9/d15/d29/f2c 0 2024-02-26T10:47:53.152 INFO:tasks.workunit.client.0.smithi060.stdout:8/177: creat d9/d15/f2d x:0 0 0 2024-02-26T10:47:53.153 INFO:tasks.workunit.client.0.smithi060.stdout:8/178: mknod d9/d15/c2e 0 2024-02-26T10:47:53.154 INFO:tasks.workunit.client.0.smithi060.stdout:4/220: dread d0/d7/f25 [0,4194304] 0 2024-02-26T10:47:53.154 INFO:tasks.workunit.client.0.smithi060.stdout:4/221: chown d0/d27/d2f/f31 490754227 1 2024-02-26T10:47:53.154 INFO:tasks.workunit.client.0.smithi060.stdout:8/179: mkdir d9/d15/d29/d2f 0 2024-02-26T10:47:53.155 INFO:tasks.workunit.client.0.smithi060.stdout:4/222: truncate d0/d7/f28 2922143 0 2024-02-26T10:47:53.157 INFO:tasks.workunit.client.0.smithi060.stdout:8/180: symlink d9/d19/l30 0 2024-02-26T10:47:53.158 INFO:tasks.workunit.client.0.smithi060.stdout:9/168: rename d3/d15/d1b/d23/f29 to d3/d9/f38 0 2024-02-26T10:47:53.159 INFO:tasks.workunit.client.0.smithi060.stdout:8/181: creat d9/d15/d29/d2f/f31 x:0 0 0 2024-02-26T10:47:53.160 INFO:tasks.workunit.client.0.smithi060.stdout:2/142: write f0 [2360615,2136] 0 2024-02-26T10:47:53.163 INFO:tasks.workunit.client.0.smithi060.stdout:1/231: link d3/db/d1e/c47 d3/db/c4b 0 2024-02-26T10:47:53.164 INFO:tasks.workunit.client.0.smithi060.stdout:7/325: dwrite da/d15/f19 [0,4194304] 0 2024-02-26T10:47:53.164 INFO:tasks.workunit.client.0.smithi060.stdout:6/201: sync 2024-02-26T10:47:53.164 INFO:tasks.workunit.client.0.smithi060.stdout:7/326: write da/d15/f19 [5317632,50065] 0 2024-02-26T10:47:53.165 INFO:tasks.workunit.client.0.smithi060.stdout:9/169: mknod d3/c39 0 2024-02-26T10:47:53.165 INFO:tasks.workunit.client.0.smithi060.stdout:9/170: truncate d3/d15/f17 1391151 0 2024-02-26T10:47:53.165 INFO:tasks.workunit.client.0.smithi060.stdout:9/171: stat d3/l1c 0 2024-02-26T10:47:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:5/169: write d0/de/d15/f18 [214412,75183] 0 2024-02-26T10:47:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:0/202: rename d0/dd/f1d to d0/d37/d44/f48 0 2024-02-26T10:47:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:5/170: rmdir d0/de/d15 39 2024-02-26T10:47:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:5/171: truncate d0/de/d15/f1a 8978308 0 2024-02-26T10:47:53.172 INFO:tasks.workunit.client.0.smithi060.stdout:7/327: creat da/d15/d21/d32/d4e/d3b/f71 x:0 0 0 2024-02-26T10:47:53.178 INFO:tasks.workunit.client.0.smithi060.stdout:4/223: write d0/f5 [1188152,22846] 0 2024-02-26T10:47:53.182 INFO:tasks.workunit.client.0.smithi060.stdout:0/203: symlink d0/d37/l49 0 2024-02-26T10:47:53.183 INFO:tasks.workunit.client.0.smithi060.stdout:7/328: creat da/d15/d21/f72 x:0 0 0 2024-02-26T10:47:53.186 INFO:tasks.workunit.client.0.smithi060.stdout:7/329: link da/d15/d21/d32/f5f da/d15/d21/d41/f73 0 2024-02-26T10:47:53.190 INFO:tasks.workunit.client.0.smithi060.stdout:7/330: creat da/d15/f74 x:0 0 0 2024-02-26T10:47:53.197 INFO:tasks.workunit.client.0.smithi060.stdout:6/202: write d6/d7/d25/d1c/d1f/f24 [794524,120921] 0 2024-02-26T10:47:53.198 INFO:tasks.workunit.client.0.smithi060.stdout:6/203: chown d6/lb 1909010 1 2024-02-26T10:47:53.205 INFO:tasks.workunit.client.0.smithi060.stdout:9/172: dwrite d3/d15/f17 [0,4194304] 0 2024-02-26T10:47:53.210 INFO:tasks.workunit.client.0.smithi060.stdout:0/204: write d0/dd/d39/d40/f28 [582843,85332] 0 2024-02-26T10:47:53.210 INFO:tasks.workunit.client.0.smithi060.stdout:0/205: write d0/dd/d39/d40/f34 [5036260,95642] 0 2024-02-26T10:47:53.223 INFO:tasks.workunit.client.0.smithi060.stdout:4/224: dread d0/f5 [0,4194304] 0 2024-02-26T10:47:53.238 INFO:tasks.workunit.client.0.smithi060.stdout:6/204: write d6/f11 [1570914,83107] 0 2024-02-26T10:47:53.240 INFO:tasks.workunit.client.0.smithi060.stdout:6/205: symlink d6/d7/d25/l3c 0 2024-02-26T10:47:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:6/206: mknod d6/d7/d25/d1c/d1f/c3d 0 2024-02-26T10:47:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:6/207: truncate d6/f16 4086204 0 2024-02-26T10:47:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:7/331: write f6 [846780,90753] 0 2024-02-26T10:47:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:6/208: write d6/d7/d25/d1c/d1f/f35 [528359,121483] 0 2024-02-26T10:47:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:8/182: dwrite d9/d15/d29/f2c [0,4194304] 0 2024-02-26T10:47:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:9/173: dwrite d3/d15/d1b/d23/f36 [0,4194304] 0 2024-02-26T10:47:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:6/209: mknod d6/d7/d25/c3e 0 2024-02-26T10:47:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:6/210: write d6/f1d [1203156,2732] 0 2024-02-26T10:47:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:9/174: mkdir d3/d9/dc/d32/d3a 0 2024-02-26T10:47:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:7/332: read da/d15/f46 [1130175,47873] 0 2024-02-26T10:47:53.249 INFO:tasks.workunit.client.0.smithi060.stdout:8/183: link d9/l21 d9/d15/d29/d2f/l32 0 2024-02-26T10:47:53.249 INFO:tasks.workunit.client.0.smithi060.stdout:9/175: rmdir d3/d9/dc/d32 39 2024-02-26T10:47:53.249 INFO:tasks.workunit.client.0.smithi060.stdout:9/176: stat d3/d15/d1b/c30 0 2024-02-26T10:47:53.250 INFO:tasks.workunit.client.0.smithi060.stdout:6/211: mkdir d6/d7/d25/d3f 0 2024-02-26T10:47:53.252 INFO:tasks.workunit.client.0.smithi060.stdout:8/184: truncate d9/d15/f1b 937333 0 2024-02-26T10:47:53.253 INFO:tasks.workunit.client.0.smithi060.stdout:9/177: creat d3/d9/dc/f3b x:0 0 0 2024-02-26T10:47:53.255 INFO:tasks.workunit.client.0.smithi060.stdout:9/178: mkdir d3/d9/dc/d32/d3c 0 2024-02-26T10:47:53.255 INFO:tasks.workunit.client.0.smithi060.stdout:9/179: truncate d3/d9/d10/f21 815048 0 2024-02-26T10:47:53.256 INFO:tasks.workunit.client.0.smithi060.stdout:6/212: creat d6/f40 x:0 0 0 2024-02-26T10:47:53.259 INFO:tasks.workunit.client.0.smithi060.stdout:7/333: rename da/d15/d21/d2f/f57 to da/d66/f75 0 2024-02-26T10:47:53.259 INFO:tasks.workunit.client.0.smithi060.stdout:9/180: rmdir d3/d15/d1b 39 2024-02-26T10:47:53.261 INFO:tasks.workunit.client.0.smithi060.stdout:6/213: mknod d6/d7/d25/d1c/d1f/d23/d2b/c41 0 2024-02-26T10:47:53.262 INFO:tasks.workunit.client.0.smithi060.stdout:9/181: creat d3/d9/dc/d32/f3d x:0 0 0 2024-02-26T10:47:53.262 INFO:tasks.workunit.client.0.smithi060.stdout:9/182: write d3/d9/d10/f2c [999893,10515] 0 2024-02-26T10:47:53.293 INFO:tasks.workunit.client.0.smithi060.stdout:7/334: dread da/d15/d21/d32/f33 [0,4194304] 0 2024-02-26T10:47:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:4/225: dwrite d0/da/d3b/f34 [0,4194304] 0 2024-02-26T10:47:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:4/226: write d0/d7/d29/f38 [322076,70951] 0 2024-02-26T10:47:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:4/227: creat d0/d7/f40 x:0 0 0 2024-02-26T10:47:53.311 INFO:tasks.workunit.client.0.smithi060.stdout:0/206: dwrite d0/dd/d39/d40/f28 [0,4194304] 0 2024-02-26T10:47:53.311 INFO:tasks.workunit.client.0.smithi060.stdout:0/207: getdents d0/d42 0 2024-02-26T10:47:53.313 INFO:tasks.workunit.client.0.smithi060.stdout:6/214: dwrite d6/f40 [0,4194304] 0 2024-02-26T10:47:53.315 INFO:tasks.workunit.client.0.smithi060.stdout:0/208: rename d0/d37/f43 to d0/dd/d33/f4a 0 2024-02-26T10:47:53.315 INFO:tasks.workunit.client.0.smithi060.stdout:0/209: chown d0/f1 11 1 2024-02-26T10:47:53.316 INFO:tasks.workunit.client.0.smithi060.stdout:0/210: dread - d0/f32 zero size 2024-02-26T10:47:53.316 INFO:tasks.workunit.client.0.smithi060.stdout:6/215: rmdir d6/d7/d25/d3f 0 2024-02-26T10:47:53.316 INFO:tasks.workunit.client.0.smithi060.stdout:6/216: dread - d6/d7/d25/d1c/d1f/d23/d28/f34 zero size 2024-02-26T10:47:53.318 INFO:tasks.workunit.client.0.smithi060.stdout:6/217: symlink d6/d7/d25/d1c/d1f/l42 0 2024-02-26T10:47:53.318 INFO:tasks.workunit.client.0.smithi060.stdout:6/218: chown d6/d7/d25/d1c/d1f/d23/c30 33923089 1 2024-02-26T10:47:53.326 INFO:tasks.workunit.client.0.smithi060.stdout:8/185: dwrite d9/d15/f1b [0,4194304] 0 2024-02-26T10:47:53.326 INFO:tasks.workunit.client.0.smithi060.stdout:7/335: dread da/f18 [0,4194304] 0 2024-02-26T10:47:53.328 INFO:tasks.workunit.client.0.smithi060.stdout:9/183: dwrite d3/d9/d10/f37 [0,4194304] 0 2024-02-26T10:47:53.332 INFO:tasks.workunit.client.0.smithi060.stdout:3/142: sync 2024-02-26T10:47:53.338 INFO:tasks.workunit.client.0.smithi060.stdout:7/336: mknod da/d15/d21/d2f/c76 0 2024-02-26T10:47:53.339 INFO:tasks.workunit.client.0.smithi060.stdout:7/337: fsync da/d15/f46 0 2024-02-26T10:47:53.339 INFO:tasks.workunit.client.0.smithi060.stdout:3/143: creat d5/f24 x:0 0 0 2024-02-26T10:47:53.340 INFO:tasks.workunit.client.0.smithi060.stdout:7/338: creat da/d15/d21/d4b/d67/f77 x:0 0 0 2024-02-26T10:47:53.340 INFO:tasks.workunit.client.0.smithi060.stdout:0/211: dread d0/dd/d39/d40/f28 [0,4194304] 0 2024-02-26T10:47:53.341 INFO:tasks.workunit.client.0.smithi060.stdout:7/339: truncate da/d15/d21/d32/d4e/d3b/f71 699685 0 2024-02-26T10:47:53.345 INFO:tasks.workunit.client.0.smithi060.stdout:0/212: mknod d0/c4b 0 2024-02-26T10:47:53.346 INFO:tasks.workunit.client.0.smithi060.stdout:0/213: dread - d0/d37/d44/f45 zero size 2024-02-26T10:47:53.346 INFO:tasks.workunit.client.0.smithi060.stdout:8/186: read d9/d15/f1c [589981,96465] 0 2024-02-26T10:47:53.349 INFO:tasks.workunit.client.0.smithi060.stdout:2/143: sync 2024-02-26T10:47:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/228: dwrite d0/da/d3b/f23 [4194304,4194304] 0 2024-02-26T10:47:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/229: write d0/da/d3b/d16/f3f [677623,37186] 0 2024-02-26T10:47:53.369 INFO:tasks.workunit.client.0.smithi060.stdout:2/144: write dd/f1d [1461647,69225] 0 2024-02-26T10:47:53.372 INFO:tasks.workunit.client.0.smithi060.stdout:4/230: rename d0/d7/f28 to d0/d27/f41 0 2024-02-26T10:47:53.372 INFO:tasks.workunit.client.0.smithi060.stdout:8/187: dread d9/d15/d29/f2c [0,4194304] 0 2024-02-26T10:47:53.372 INFO:tasks.workunit.client.0.smithi060.stdout:4/231: symlink d0/d27/l42 0 2024-02-26T10:47:53.373 INFO:tasks.workunit.client.0.smithi060.stdout:4/232: dread - d0/d7/f37 zero size 2024-02-26T10:47:53.374 INFO:tasks.workunit.client.0.smithi060.stdout:4/233: creat d0/d27/d2f/f43 x:0 0 0 2024-02-26T10:47:53.374 INFO:tasks.workunit.client.0.smithi060.stdout:2/145: rename dd/d11/f22 to dd/f2b 0 2024-02-26T10:47:53.375 INFO:tasks.workunit.client.0.smithi060.stdout:4/234: creat d0/f44 x:0 0 0 2024-02-26T10:47:53.376 INFO:tasks.workunit.client.0.smithi060.stdout:8/188: rename d9/c16 to d9/d15/d29/d2f/c33 0 2024-02-26T10:47:53.378 INFO:tasks.workunit.client.0.smithi060.stdout:8/189: mkdir d9/d15/d29/d2f/d34 0 2024-02-26T10:47:53.380 INFO:tasks.workunit.client.0.smithi060.stdout:8/190: symlink d9/d19/l35 0 2024-02-26T10:47:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:9/184: dread d3/d9/d10/f11 [0,4194304] 0 2024-02-26T10:47:53.391 INFO:tasks.workunit.client.0.smithi060.stdout:7/340: dwrite da/d66/f75 [0,4194304] 0 2024-02-26T10:47:53.394 INFO:tasks.workunit.client.0.smithi060.stdout:7/341: mkdir da/d15/d21/d2f/d78 0 2024-02-26T10:47:53.395 INFO:tasks.workunit.client.0.smithi060.stdout:7/342: write da/d15/d21/f2c [2278482,43910] 0 2024-02-26T10:47:53.397 INFO:tasks.workunit.client.0.smithi060.stdout:7/343: creat da/d15/d21/f79 x:0 0 0 2024-02-26T10:47:53.416 INFO:tasks.workunit.client.0.smithi060.stdout:3/144: dwrite d5/de/d16/d17/f1d [4194304,4194304] 0 2024-02-26T10:47:53.416 INFO:tasks.workunit.client.0.smithi060.stdout:3/145: chown d5/de/df/d11/c23 105840 1 2024-02-26T10:47:53.416 INFO:tasks.workunit.client.0.smithi060.stdout:8/191: dread d9/f1e [0,4194304] 0 2024-02-26T10:47:53.416 INFO:tasks.workunit.client.0.smithi060.stdout:0/214: dwrite d0/d37/d44/f48 [0,4194304] 0 2024-02-26T10:47:53.418 INFO:tasks.workunit.client.0.smithi060.stdout:0/215: rmdir d0 39 2024-02-26T10:47:53.436 INFO:tasks.workunit.client.0.smithi060.stdout:4/235: dwrite d0/f44 [0,4194304] 0 2024-02-26T10:47:53.441 INFO:tasks.workunit.client.0.smithi060.stdout:9/185: dwrite d3/d15/d1b/d23/f34 [0,4194304] 0 2024-02-26T10:47:53.444 INFO:tasks.workunit.client.0.smithi060.stdout:9/186: creat d3/d15/d35/f3e x:0 0 0 2024-02-26T10:47:53.445 INFO:tasks.workunit.client.0.smithi060.stdout:9/187: mknod d3/d9/d10/c3f 0 2024-02-26T10:47:53.445 INFO:tasks.workunit.client.0.smithi060.stdout:9/188: chown d3/d9/d10/f11 1891 1 2024-02-26T10:47:53.446 INFO:tasks.workunit.client.0.smithi060.stdout:9/189: symlink d3/d9/dc/d32/d3c/l40 0 2024-02-26T10:47:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:9/190: link d3/d15/d35/f3e d3/d15/d1b/f41 0 2024-02-26T10:47:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:9/191: getdents d3/d9/d10 0 2024-02-26T10:47:53.454 INFO:tasks.workunit.client.0.smithi060.stdout:3/146: dread d5/de/d16/d17/f1d [4194304,4194304] 0 2024-02-26T10:47:53.457 INFO:tasks.workunit.client.0.smithi060.stdout:5/172: sync 2024-02-26T10:47:53.457 INFO:tasks.workunit.client.0.smithi060.stdout:1/232: sync 2024-02-26T10:47:53.459 INFO:tasks.workunit.client.0.smithi060.stdout:1/233: mknod d3/d32/c4c 0 2024-02-26T10:47:53.459 INFO:tasks.workunit.client.0.smithi060.stdout:5/173: link d0/f9 d0/de/d15/d26/f39 0 2024-02-26T10:47:53.460 INFO:tasks.workunit.client.0.smithi060.stdout:5/174: creat d0/f3a x:0 0 0 2024-02-26T10:47:53.461 INFO:tasks.workunit.client.0.smithi060.stdout:4/236: dread d0/da/d3b/f34 [0,4194304] 0 2024-02-26T10:47:53.461 INFO:tasks.workunit.client.0.smithi060.stdout:1/234: truncate d3/f15 2475303 0 2024-02-26T10:47:53.461 INFO:tasks.workunit.client.0.smithi060.stdout:5/175: truncate d0/f5 2121307 0 2024-02-26T10:47:53.463 INFO:tasks.workunit.client.0.smithi060.stdout:5/176: creat d0/de/d15/f3b x:0 0 0 2024-02-26T10:47:53.463 INFO:tasks.workunit.client.0.smithi060.stdout:5/177: chown d0/de/d20/f31 507759 1 2024-02-26T10:47:53.463 INFO:tasks.workunit.client.0.smithi060.stdout:5/178: chown d0/fc 250586848 1 2024-02-26T10:47:53.463 INFO:tasks.workunit.client.0.smithi060.stdout:8/192: dwrite d9/d15/f1b [0,4194304] 0 2024-02-26T10:47:53.464 INFO:tasks.workunit.client.0.smithi060.stdout:8/193: dread - d9/d15/f27 zero size 2024-02-26T10:47:53.464 INFO:tasks.workunit.client.0.smithi060.stdout:8/194: chown d9/f1e 40 1 2024-02-26T10:47:53.464 INFO:tasks.workunit.client.0.smithi060.stdout:0/216: dread d0/f1 [0,4194304] 0 2024-02-26T10:47:53.464 INFO:tasks.workunit.client.0.smithi060.stdout:5/179: mknod d0/d33/c3c 0 2024-02-26T10:47:53.466 INFO:tasks.workunit.client.0.smithi060.stdout:5/180: symlink d0/d33/l3d 0 2024-02-26T10:47:53.468 INFO:tasks.workunit.client.0.smithi060.stdout:5/181: creat d0/d33/f3e x:0 0 0 2024-02-26T10:47:53.469 INFO:tasks.workunit.client.0.smithi060.stdout:5/182: mknod d0/de/d15/c3f 0 2024-02-26T10:47:53.472 INFO:tasks.workunit.client.0.smithi060.stdout:5/183: link d0/de/d20/l2e d0/l40 0 2024-02-26T10:47:53.479 INFO:tasks.workunit.client.0.smithi060.stdout:0/217: link d0/dd/d33/c36 d0/dd/c4c 0 2024-02-26T10:47:53.479 INFO:tasks.workunit.client.0.smithi060.stdout:0/218: fsync d0/dd/d39/d40/f26 0 2024-02-26T10:47:53.479 INFO:tasks.workunit.client.0.smithi060.stdout:0/219: readlink d0/l30 0 2024-02-26T10:47:53.482 INFO:tasks.workunit.client.0.smithi060.stdout:0/220: creat d0/dd/d39/d40/f4d x:0 0 0 2024-02-26T10:47:53.484 INFO:tasks.workunit.client.0.smithi060.stdout:0/221: creat d0/dd/f4e x:0 0 0 2024-02-26T10:47:53.484 INFO:tasks.workunit.client.0.smithi060.stdout:0/222: write d0/dd/f2d [812537,39982] 0 2024-02-26T10:47:53.494 INFO:tasks.workunit.client.0.smithi060.stdout:1/235: dread d3/f1b [0,4194304] 0 2024-02-26T10:47:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:1/236: creat d3/db/d2e/f4d x:0 0 0 2024-02-26T10:47:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:8/195: dread d9/f20 [0,4194304] 0 2024-02-26T10:47:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:8/196: rename d9/d15/f2d to d9/d15/f36 0 2024-02-26T10:47:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:9/192: rename d3/l2b to d3/d9/dc/d32/d3c/l42 0 2024-02-26T10:47:53.502 INFO:tasks.workunit.client.0.smithi060.stdout:9/193: symlink d3/d1e/l43 0 2024-02-26T10:47:53.502 INFO:tasks.workunit.client.0.smithi060.stdout:9/194: dread - d3/d9/dc/d32/f3d zero size 2024-02-26T10:47:53.504 INFO:tasks.workunit.client.0.smithi060.stdout:9/195: mkdir d3/d9/dc/d32/d3c/d44 0 2024-02-26T10:47:53.504 INFO:tasks.workunit.client.0.smithi060.stdout:9/196: rename d3/d9/dc to d3/d9/dc/d32/d3c/d44/d45 22 2024-02-26T10:47:53.509 INFO:tasks.workunit.client.0.smithi060.stdout:3/147: dwrite f3 [0,4194304] 0 2024-02-26T10:47:53.509 INFO:tasks.workunit.client.0.smithi060.stdout:0/223: dread d0/dd/f2d [0,4194304] 0 2024-02-26T10:47:53.512 INFO:tasks.workunit.client.0.smithi060.stdout:8/197: rmdir d9/d19 39 2024-02-26T10:47:53.512 INFO:tasks.workunit.client.0.smithi060.stdout:8/198: write d9/d15/f1c [3684466,117599] 0 2024-02-26T10:47:53.513 INFO:tasks.workunit.client.0.smithi060.stdout:8/199: chown d9/d19/l22 40680395 1 2024-02-26T10:47:53.514 INFO:tasks.workunit.client.0.smithi060.stdout:8/200: write f8 [4118226,5286] 0 2024-02-26T10:47:53.515 INFO:tasks.workunit.client.0.smithi060.stdout:8/201: symlink d9/d15/d29/d2f/l37 0 2024-02-26T10:47:53.516 INFO:tasks.workunit.client.0.smithi060.stdout:8/202: symlink d9/l38 0 2024-02-26T10:47:53.517 INFO:tasks.workunit.client.0.smithi060.stdout:8/203: creat d9/d15/d29/d2f/f39 x:0 0 0 2024-02-26T10:47:53.519 INFO:tasks.workunit.client.0.smithi060.stdout:0/224: dread d0/dd/d39/d40/f1f [0,4194304] 0 2024-02-26T10:47:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:0/225: symlink d0/dd/l4f 0 2024-02-26T10:47:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:0/226: chown d0/dd/f31 124290 1 2024-02-26T10:47:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:0/227: write d0/dd/f2b [590900,10675] 0 2024-02-26T10:47:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:0/228: write d0/dd/f2b [808900,1448] 0 2024-02-26T10:47:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:0/229: truncate d0/dd/d39/d40/f4d 119288 0 2024-02-26T10:47:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:0/230: write d0/d37/d44/f48 [5394571,11763] 0 2024-02-26T10:47:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:0/231: fdatasync d0/dd/d39/d40/f4d 0 2024-02-26T10:47:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:0/232: write d0/f1 [1694921,17049] 0 2024-02-26T10:47:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:0/233: dread - d0/f32 zero size 2024-02-26T10:47:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:0/234: creat d0/d37/f50 x:0 0 0 2024-02-26T10:47:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:0/235: write d0/dd/f4e [697467,43869] 0 2024-02-26T10:47:53.525 INFO:tasks.workunit.client.0.smithi060.stdout:0/236: symlink d0/dd/d33/l51 0 2024-02-26T10:47:53.525 INFO:tasks.workunit.client.0.smithi060.stdout:0/237: truncate d0/f32 343931 0 2024-02-26T10:47:53.527 INFO:tasks.workunit.client.0.smithi060.stdout:9/197: dread d3/f16 [0,4194304] 0 2024-02-26T10:47:53.528 INFO:tasks.workunit.client.0.smithi060.stdout:8/204: rmdir d9/d15/d29/d2f 39 2024-02-26T10:47:53.532 INFO:tasks.workunit.client.0.smithi060.stdout:4/237: rename d0/da/d3b/d16/f2d to d0/d7/d29/f45 0 2024-02-26T10:47:53.532 INFO:tasks.workunit.client.0.smithi060.stdout:4/238: readlink d0/da/d3b/d1b/l1c 0 2024-02-26T10:47:53.532 INFO:tasks.workunit.client.0.smithi060.stdout:4/239: write d0/d7/d29/f38 [1426661,45970] 0 2024-02-26T10:47:53.532 INFO:tasks.workunit.client.0.smithi060.stdout:4/240: chown d0/d7/l13 1536583 1 2024-02-26T10:47:53.532 INFO:tasks.workunit.client.0.smithi060.stdout:4/241: write d0/da/d3b/d16/f19 [4959615,53443] 0 2024-02-26T10:47:53.532 INFO:tasks.workunit.client.0.smithi060.stdout:9/198: symlink d3/d9/l46 0 2024-02-26T10:47:53.532 INFO:tasks.workunit.client.0.smithi060.stdout:9/199: write d3/d9/f19 [3555710,113474] 0 2024-02-26T10:47:53.533 INFO:tasks.workunit.client.0.smithi060.stdout:5/184: rename d0/fc to d0/de/d15/d26/f41 0 2024-02-26T10:47:53.534 INFO:tasks.workunit.client.0.smithi060.stdout:9/200: mkdir d3/d9/dc/d32/d3c/d47 0 2024-02-26T10:47:53.535 INFO:tasks.workunit.client.0.smithi060.stdout:9/201: dread - d3/d9/dc/d32/f3d zero size 2024-02-26T10:47:53.535 INFO:tasks.workunit.client.0.smithi060.stdout:9/202: write d3/d9/d10/f2c [1743065,65133] 0 2024-02-26T10:47:53.535 INFO:tasks.workunit.client.0.smithi060.stdout:9/203: dread - d3/d15/d35/f3e zero size 2024-02-26T10:47:53.537 INFO:tasks.workunit.client.0.smithi060.stdout:5/185: link d0/de/d15/c3f d0/de/d15/c42 0 2024-02-26T10:47:53.537 INFO:tasks.workunit.client.0.smithi060.stdout:5/186: truncate d0/de/d20/f31 4494910 0 2024-02-26T10:47:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:5/187: link d0/f30 d0/d33/f43 0 2024-02-26T10:47:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:5/188: chown d0/d33/f43 4853009 1 2024-02-26T10:47:53.557 INFO:tasks.workunit.client.0.smithi060.stdout:8/205: dread d9/f23 [0,4194304] 0 2024-02-26T10:47:53.558 INFO:tasks.workunit.client.0.smithi060.stdout:8/206: write d9/f1e [196075,68975] 0 2024-02-26T10:47:53.558 INFO:tasks.workunit.client.0.smithi060.stdout:8/207: dread - d9/d15/d29/d2f/f31 zero size 2024-02-26T10:47:53.559 INFO:tasks.workunit.client.0.smithi060.stdout:8/208: creat d9/d19/f3a x:0 0 0 2024-02-26T10:47:53.560 INFO:tasks.workunit.client.0.smithi060.stdout:8/209: readlink d9/l21 0 2024-02-26T10:47:53.560 INFO:tasks.workunit.client.0.smithi060.stdout:8/210: mkdir d9/d15/d29/d2f/d3b 0 2024-02-26T10:47:53.562 INFO:tasks.workunit.client.0.smithi060.stdout:8/211: creat d9/d15/d29/f3c x:0 0 0 2024-02-26T10:47:53.563 INFO:tasks.workunit.client.0.smithi060.stdout:8/212: truncate d9/d15/d29/f2c 3755787 0 2024-02-26T10:47:53.575 INFO:tasks.workunit.client.0.smithi060.stdout:0/238: write d0/f3 [2298044,63260] 0 2024-02-26T10:47:53.576 INFO:tasks.workunit.client.0.smithi060.stdout:1/237: dwrite d3/f2a [8388608,4194304] 0 2024-02-26T10:47:53.580 INFO:tasks.workunit.client.0.smithi060.stdout:1/238: unlink d3/db/d1e/f46 0 2024-02-26T10:47:53.581 INFO:tasks.workunit.client.0.smithi060.stdout:1/239: symlink d3/db/d1e/d38/l4e 0 2024-02-26T10:47:53.584 INFO:tasks.workunit.client.0.smithi060.stdout:1/240: mknod d3/db/d2e/c4f 0 2024-02-26T10:47:53.584 INFO:tasks.workunit.client.0.smithi060.stdout:1/241: fdatasync d3/db/d1e/f27 0 2024-02-26T10:47:53.584 INFO:tasks.workunit.client.0.smithi060.stdout:1/242: chown d3/d32/c41 6 1 2024-02-26T10:47:53.584 INFO:tasks.workunit.client.0.smithi060.stdout:1/243: fsync d3/db/d2e/f33 0 2024-02-26T10:47:53.585 INFO:tasks.workunit.client.0.smithi060.stdout:2/146: sync 2024-02-26T10:47:53.585 INFO:tasks.workunit.client.0.smithi060.stdout:7/344: sync 2024-02-26T10:47:53.586 INFO:tasks.workunit.client.0.smithi060.stdout:6/219: sync 2024-02-26T10:47:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:6/220: write d6/fe [1914926,99238] 0 2024-02-26T10:47:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:6/221: write d6/f15 [1413313,110024] 0 2024-02-26T10:47:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:9/204: dwrite d3/d9/f1f [0,4194304] 0 2024-02-26T10:47:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:9/205: chown d3/d1e 1 1 2024-02-26T10:47:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:9/206: creat d3/d9/dc/d32/d3c/d44/f48 x:0 0 0 2024-02-26T10:47:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:9/207: truncate d3/d9/dc/ff 652171 0 2024-02-26T10:47:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:2/147: symlink dd/l2c 0 2024-02-26T10:47:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:7/345: rmdir da/d15 39 2024-02-26T10:47:53.595 INFO:tasks.workunit.client.0.smithi060.stdout:9/208: symlink d3/l49 0 2024-02-26T10:47:53.597 INFO:tasks.workunit.client.0.smithi060.stdout:7/346: rename da/f13 to da/d15/d21/d4b/d67/f7a 0 2024-02-26T10:47:53.597 INFO:tasks.workunit.client.0.smithi060.stdout:7/347: write da/d15/f25 [214988,95990] 0 2024-02-26T10:47:53.609 INFO:tasks.workunit.client.0.smithi060.stdout:2/148: write f3 [2497438,111732] 0 2024-02-26T10:47:53.619 INFO:tasks.workunit.client.0.smithi060.stdout:5/189: dwrite d0/de/f12 [0,4194304] 0 2024-02-26T10:47:53.620 INFO:tasks.workunit.client.0.smithi060.stdout:9/209: write d3/d9/f2e [3132048,89648] 0 2024-02-26T10:47:53.620 INFO:tasks.workunit.client.0.smithi060.stdout:9/210: stat d3/d15/l27 0 2024-02-26T10:47:53.626 INFO:tasks.workunit.client.0.smithi060.stdout:8/213: dwrite f3 [0,4194304] 0 2024-02-26T10:47:53.626 INFO:tasks.workunit.client.0.smithi060.stdout:5/190: creat d0/de/d15/f44 x:0 0 0 2024-02-26T10:47:53.628 INFO:tasks.workunit.client.0.smithi060.stdout:5/191: creat d0/de/d15/d26/f45 x:0 0 0 2024-02-26T10:47:53.628 INFO:tasks.workunit.client.0.smithi060.stdout:9/211: symlink d3/d9/l4a 0 2024-02-26T10:47:53.628 INFO:tasks.workunit.client.0.smithi060.stdout:9/212: truncate d3/d9/dc/ff 1286660 0 2024-02-26T10:47:53.629 INFO:tasks.workunit.client.0.smithi060.stdout:1/244: dwrite d3/db/f40 [0,4194304] 0 2024-02-26T10:47:53.629 INFO:tasks.workunit.client.0.smithi060.stdout:1/245: write d3/f20 [1557285,15165] 0 2024-02-26T10:47:53.633 INFO:tasks.workunit.client.0.smithi060.stdout:5/192: creat d0/de/f46 x:0 0 0 2024-02-26T10:47:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:5/193: creat d0/de/d15/f47 x:0 0 0 2024-02-26T10:47:53.646 INFO:tasks.workunit.client.0.smithi060.stdout:6/222: dwrite d6/d7/d25/d1c/d1f/d23/d28/f34 [0,4194304] 0 2024-02-26T10:47:53.646 INFO:tasks.workunit.client.0.smithi060.stdout:6/223: stat d6/d7/d25/d1c/d1f/d23/d28/f29 0 2024-02-26T10:47:53.646 INFO:tasks.workunit.client.0.smithi060.stdout:6/224: truncate d6/d7/f2a 615421 0 2024-02-26T10:47:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:6/225: unlink d6/lb 0 2024-02-26T10:47:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:6/226: write d6/f38 [1007590,92902] 0 2024-02-26T10:47:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:1/246: dread d3/f2a [4194304,4194304] 0 2024-02-26T10:47:53.662 INFO:tasks.workunit.client.0.smithi060.stdout:1/247: rmdir d3/db/d1e/d39 39 2024-02-26T10:47:53.662 INFO:tasks.workunit.client.0.smithi060.stdout:1/248: chown d3/db/d1e/d38/l4e 29 1 2024-02-26T10:47:53.664 INFO:tasks.workunit.client.0.smithi060.stdout:1/249: symlink d3/db/l50 0 2024-02-26T10:47:53.664 INFO:tasks.workunit.client.0.smithi060.stdout:2/149: dwrite dd/d11/f25 [0,4194304] 0 2024-02-26T10:47:53.666 INFO:tasks.workunit.client.0.smithi060.stdout:1/250: mkdir d3/db/d2e/d51 0 2024-02-26T10:47:53.667 INFO:tasks.workunit.client.0.smithi060.stdout:6/227: getdents d6 0 2024-02-26T10:47:53.667 INFO:tasks.workunit.client.0.smithi060.stdout:6/228: readlink d6/l36 0 2024-02-26T10:47:53.668 INFO:tasks.workunit.client.0.smithi060.stdout:2/150: unlink dd/l21 0 2024-02-26T10:47:53.672 INFO:tasks.workunit.client.0.smithi060.stdout:1/251: mknod d3/c52 0 2024-02-26T10:47:53.672 INFO:tasks.workunit.client.0.smithi060.stdout:6/229: mknod d6/d7/d25/d1c/d1f/d23/d2b/c43 0 2024-02-26T10:47:53.672 INFO:tasks.workunit.client.0.smithi060.stdout:2/151: symlink dd/l2d 0 2024-02-26T10:47:53.673 INFO:tasks.workunit.client.0.smithi060.stdout:2/152: rename dd/de/c19 to dd/d11/d13/c2e 0 2024-02-26T10:47:53.673 INFO:tasks.workunit.client.0.smithi060.stdout:2/153: readlink lb 0 2024-02-26T10:47:53.675 INFO:tasks.workunit.client.0.smithi060.stdout:5/194: dread d0/de/d20/f24 [0,4194304] 0 2024-02-26T10:47:53.676 INFO:tasks.workunit.client.0.smithi060.stdout:9/213: dwrite d3/d9/dc/f3b [0,4194304] 0 2024-02-26T10:47:53.677 INFO:tasks.workunit.client.0.smithi060.stdout:5/195: creat d0/de/d15/d26/f48 x:0 0 0 2024-02-26T10:47:53.678 INFO:tasks.workunit.client.0.smithi060.stdout:2/154: rmdir dd/d11 39 2024-02-26T10:47:53.679 INFO:tasks.workunit.client.0.smithi060.stdout:5/196: mknod d0/d33/c49 0 2024-02-26T10:47:53.682 INFO:tasks.workunit.client.0.smithi060.stdout:6/230: dread d6/d7/d25/d1c/d1f/f35 [0,4194304] 0 2024-02-26T10:47:53.686 INFO:tasks.workunit.client.0.smithi060.stdout:8/214: dread d9/d15/d29/f2c [0,4194304] 0 2024-02-26T10:47:53.686 INFO:tasks.workunit.client.0.smithi060.stdout:9/214: symlink d3/d15/d35/l4b 0 2024-02-26T10:47:53.686 INFO:tasks.workunit.client.0.smithi060.stdout:3/148: sync 2024-02-26T10:47:53.688 INFO:tasks.workunit.client.0.smithi060.stdout:6/231: truncate d6/f40 1154492 0 2024-02-26T10:47:53.688 INFO:tasks.workunit.client.0.smithi060.stdout:3/149: rmdir d5/de/df 39 2024-02-26T10:47:53.689 INFO:tasks.workunit.client.0.smithi060.stdout:3/150: chown d5/de/d16 89397120 1 2024-02-26T10:47:53.689 INFO:tasks.workunit.client.0.smithi060.stdout:9/215: unlink d3/d9/dc/d32/d3c/l40 0 2024-02-26T10:47:53.694 INFO:tasks.workunit.client.0.smithi060.stdout:3/151: creat d5/f25 x:0 0 0 2024-02-26T10:47:53.697 INFO:tasks.workunit.client.0.smithi060.stdout:5/197: getdents d0 0 2024-02-26T10:47:53.719 INFO:tasks.workunit.client.0.smithi060.stdout:1/252: dwrite d3/db/d2e/f4d [0,4194304] 0 2024-02-26T10:47:53.720 INFO:tasks.workunit.client.0.smithi060.stdout:1/253: mkdir d3/d35/d53 0 2024-02-26T10:47:53.720 INFO:tasks.workunit.client.0.smithi060.stdout:1/254: readlink d3/l2d 0 2024-02-26T10:47:53.729 INFO:tasks.workunit.client.0.smithi060.stdout:6/232: dread d6/f11 [0,4194304] 0 2024-02-26T10:47:53.730 INFO:tasks.workunit.client.0.smithi060.stdout:6/233: creat d6/d7/d25/d1c/d39/f44 x:0 0 0 2024-02-26T10:47:53.741 INFO:tasks.workunit.client.0.smithi060.stdout:4/242: sync 2024-02-26T10:47:53.742 INFO:tasks.workunit.client.0.smithi060.stdout:4/243: mknod d0/da/d3b/d16/c46 0 2024-02-26T10:47:53.745 INFO:tasks.workunit.client.0.smithi060.stdout:9/216: dwrite d3/d9/d10/f2c [0,4194304] 0 2024-02-26T10:47:53.745 INFO:tasks.workunit.client.0.smithi060.stdout:9/217: write d3/d15/d35/f3e [643264,77029] 0 2024-02-26T10:47:53.745 INFO:tasks.workunit.client.0.smithi060.stdout:8/215: dwrite d9/d15/f36 [0,4194304] 0 2024-02-26T10:47:53.747 INFO:tasks.workunit.client.0.smithi060.stdout:1/255: write d3/f2a [7067188,124425] 0 2024-02-26T10:47:53.748 INFO:tasks.workunit.client.0.smithi060.stdout:8/216: creat d9/d15/d29/d2f/f3d x:0 0 0 2024-02-26T10:47:53.751 INFO:tasks.workunit.client.0.smithi060.stdout:2/155: dwrite dd/f1d [0,4194304] 0 2024-02-26T10:47:53.754 INFO:tasks.workunit.client.0.smithi060.stdout:8/217: rmdir d9/d15 39 2024-02-26T10:47:53.755 INFO:tasks.workunit.client.0.smithi060.stdout:8/218: mkdir d9/d15/d3e 0 2024-02-26T10:47:53.756 INFO:tasks.workunit.client.0.smithi060.stdout:8/219: mknod d9/d15/d29/c3f 0 2024-02-26T10:47:53.757 INFO:tasks.workunit.client.0.smithi060.stdout:8/220: mkdir d9/d15/d40 0 2024-02-26T10:47:53.760 INFO:tasks.workunit.client.0.smithi060.stdout:5/198: dread d0/fd [0,4194304] 0 2024-02-26T10:47:53.760 INFO:tasks.workunit.client.0.smithi060.stdout:5/199: chown d0/de/d15/d26/f41 118071560 1 2024-02-26T10:47:53.763 INFO:tasks.workunit.client.0.smithi060.stdout:5/200: rename d0/d33/l3d to d0/l4a 0 2024-02-26T10:47:53.766 INFO:tasks.workunit.client.0.smithi060.stdout:5/201: mkdir d0/d4b 0 2024-02-26T10:47:53.771 INFO:tasks.workunit.client.0.smithi060.stdout:6/234: dread d6/f16 [0,4194304] 0 2024-02-26T10:47:53.771 INFO:tasks.workunit.client.0.smithi060.stdout:9/218: dread d3/d9/d10/f21 [0,4194304] 0 2024-02-26T10:47:53.772 INFO:tasks.workunit.client.0.smithi060.stdout:9/219: readlink d3/d9/l46 0 2024-02-26T10:47:53.772 INFO:tasks.workunit.client.0.smithi060.stdout:6/235: mknod d6/d7/d25/d1c/d39/c45 0 2024-02-26T10:47:53.773 INFO:tasks.workunit.client.0.smithi060.stdout:6/236: creat d6/d7/d25/f46 x:0 0 0 2024-02-26T10:47:53.780 INFO:tasks.workunit.client.0.smithi060.stdout:4/244: dread d0/d7/f3e [0,4194304] 0 2024-02-26T10:47:53.780 INFO:tasks.workunit.client.0.smithi060.stdout:8/221: write d9/f13 [1018121,24667] 0 2024-02-26T10:47:53.780 INFO:tasks.workunit.client.0.smithi060.stdout:8/222: fdatasync d9/d19/f3a 0 2024-02-26T10:47:53.780 INFO:tasks.workunit.client.0.smithi060.stdout:8/223: read d9/d15/f1c [726272,97799] 0 2024-02-26T10:47:53.780 INFO:tasks.workunit.client.0.smithi060.stdout:8/224: dread - d9/d15/d29/d2f/f3d zero size 2024-02-26T10:47:53.792 INFO:tasks.workunit.client.0.smithi060.stdout:5/202: dread d0/de/d20/f31 [0,4194304] 0 2024-02-26T10:47:53.799 INFO:tasks.workunit.client.0.smithi060.stdout:1/256: dwrite d3/f2b [0,4194304] 0 2024-02-26T10:47:53.809 INFO:tasks.workunit.client.0.smithi060.stdout:0/239: sync 2024-02-26T10:47:53.809 INFO:tasks.workunit.client.0.smithi060.stdout:7/348: sync 2024-02-26T10:47:53.810 INFO:tasks.workunit.client.0.smithi060.stdout:7/349: getdents da/d66 0 2024-02-26T10:47:53.812 INFO:tasks.workunit.client.0.smithi060.stdout:7/350: symlink da/d15/d21/d41/l7b 0 2024-02-26T10:47:53.814 INFO:tasks.workunit.client.0.smithi060.stdout:7/351: rename da/d15/d21/d32/d4e/d39/l4d to da/d15/d21/d4b/l7c 0 2024-02-26T10:47:53.815 INFO:tasks.workunit.client.0.smithi060.stdout:9/220: dwrite d3/d9/d10/f21 [0,4194304] 0 2024-02-26T10:47:53.815 INFO:tasks.workunit.client.0.smithi060.stdout:9/221: fsync d3/d15/d1b/d23/f34 0 2024-02-26T10:47:53.816 INFO:tasks.workunit.client.0.smithi060.stdout:9/222: chown d3/d1e/c33 16758 1 2024-02-26T10:47:53.816 INFO:tasks.workunit.client.0.smithi060.stdout:2/156: dwrite f0 [0,4194304] 0 2024-02-26T10:47:53.816 INFO:tasks.workunit.client.0.smithi060.stdout:6/237: dread d6/d7/d25/d1c/f2c [0,4194304] 0 2024-02-26T10:47:53.816 INFO:tasks.workunit.client.0.smithi060.stdout:7/352: creat da/d15/d21/d2f/d78/f7d x:0 0 0 2024-02-26T10:47:53.817 INFO:tasks.workunit.client.0.smithi060.stdout:6/238: mkdir d6/d7/d47 0 2024-02-26T10:47:53.818 INFO:tasks.workunit.client.0.smithi060.stdout:2/157: rmdir dd/d11 39 2024-02-26T10:47:53.825 INFO:tasks.workunit.client.0.smithi060.stdout:1/257: write d3/db/d1e/f27 [5669611,48277] 0 2024-02-26T10:47:53.825 INFO:tasks.workunit.client.0.smithi060.stdout:1/258: fsync d3/db/d2e/f4d 0 2024-02-26T10:47:53.827 INFO:tasks.workunit.client.0.smithi060.stdout:1/259: mkdir d3/d54 0 2024-02-26T10:47:53.829 INFO:tasks.workunit.client.0.smithi060.stdout:1/260: mknod d3/db/d2e/d51/c55 0 2024-02-26T10:47:53.830 INFO:tasks.workunit.client.0.smithi060.stdout:1/261: creat d3/db/f56 x:0 0 0 2024-02-26T10:47:53.831 INFO:tasks.workunit.client.0.smithi060.stdout:1/262: dread - d3/db/d2e/f33 zero size 2024-02-26T10:47:53.831 INFO:tasks.workunit.client.0.smithi060.stdout:9/223: unlink d3/l8 0 2024-02-26T10:47:53.832 INFO:tasks.workunit.client.0.smithi060.stdout:9/224: creat d3/d9/dc/d32/d3a/f4c x:0 0 0 2024-02-26T10:47:53.834 INFO:tasks.workunit.client.0.smithi060.stdout:9/225: symlink d3/d9/dc/d32/d3c/l4d 0 2024-02-26T10:47:53.837 INFO:tasks.workunit.client.0.smithi060.stdout:9/226: fsync d3/d9/dc/d32/d3a/f4c 0 2024-02-26T10:47:53.844 INFO:tasks.workunit.client.0.smithi060.stdout:4/245: dwrite d0/d7/f37 [0,4194304] 0 2024-02-26T10:47:53.844 INFO:tasks.workunit.client.0.smithi060.stdout:7/353: link da/d5a/l62 da/d15/l7e 0 2024-02-26T10:47:53.844 INFO:tasks.workunit.client.0.smithi060.stdout:7/354: stat da/d15/d21/d32/d4e/d3b/c5e 0 2024-02-26T10:47:53.844 INFO:tasks.workunit.client.0.smithi060.stdout:7/355: write da/d15/f17 [5198823,52648] 0 2024-02-26T10:47:53.844 INFO:tasks.workunit.client.0.smithi060.stdout:2/158: write f5 [3919437,61554] 0 2024-02-26T10:47:53.845 INFO:tasks.workunit.client.0.smithi060.stdout:8/225: dwrite d9/d19/f3a [0,4194304] 0 2024-02-26T10:47:53.845 INFO:tasks.workunit.client.0.smithi060.stdout:6/239: dread d6/fe [0,4194304] 0 2024-02-26T10:47:53.854 INFO:tasks.workunit.client.0.smithi060.stdout:9/227: getdents d3/d15/d35 0 2024-02-26T10:47:53.854 INFO:tasks.workunit.client.0.smithi060.stdout:4/246: mknod d0/da/c47 0 2024-02-26T10:47:53.856 INFO:tasks.workunit.client.0.smithi060.stdout:7/356: symlink da/d15/d21/d41/l7f 0 2024-02-26T10:47:53.856 INFO:tasks.workunit.client.0.smithi060.stdout:7/357: chown da/d15/d21/d41/f73 0 1 2024-02-26T10:47:53.857 INFO:tasks.workunit.client.0.smithi060.stdout:8/226: symlink d9/d15/d3e/l41 0 2024-02-26T10:47:53.858 INFO:tasks.workunit.client.0.smithi060.stdout:2/159: link dd/d11/l23 dd/d11/d13/l2f 0 2024-02-26T10:47:53.859 INFO:tasks.workunit.client.0.smithi060.stdout:8/227: truncate d9/f23 3145929 0 2024-02-26T10:47:53.859 INFO:tasks.workunit.client.0.smithi060.stdout:8/228: chown d9/f20 427893 1 2024-02-26T10:47:53.859 INFO:tasks.workunit.client.0.smithi060.stdout:8/229: stat d9/d15/c28 0 2024-02-26T10:47:53.859 INFO:tasks.workunit.client.0.smithi060.stdout:8/230: write d9/d15/d29/d2f/f31 [110789,36342] 0 2024-02-26T10:47:53.859 INFO:tasks.workunit.client.0.smithi060.stdout:8/231: chown d9/d15/d29/d2f/d3b 585467951 1 2024-02-26T10:47:53.860 INFO:tasks.workunit.client.0.smithi060.stdout:2/160: link dd/d11/l23 dd/d11/d13/l30 0 2024-02-26T10:47:53.861 INFO:tasks.workunit.client.0.smithi060.stdout:4/247: rmdir d0/da/d3b 39 2024-02-26T10:47:53.861 INFO:tasks.workunit.client.0.smithi060.stdout:8/232: creat d9/d19/f42 x:0 0 0 2024-02-26T10:47:53.862 INFO:tasks.workunit.client.0.smithi060.stdout:2/161: creat dd/d11/d13/f31 x:0 0 0 2024-02-26T10:47:53.866 INFO:tasks.workunit.client.0.smithi060.stdout:8/233: write d9/d15/d29/f2c [3897966,122618] 0 2024-02-26T10:47:53.866 INFO:tasks.workunit.client.0.smithi060.stdout:2/162: symlink dd/l32 0 2024-02-26T10:47:53.866 INFO:tasks.workunit.client.0.smithi060.stdout:2/163: truncate dd/d11/f24 419014 0 2024-02-26T10:47:53.868 INFO:tasks.workunit.client.0.smithi060.stdout:8/234: link d9/d15/f1c d9/f43 0 2024-02-26T10:47:53.869 INFO:tasks.workunit.client.0.smithi060.stdout:2/164: mkdir dd/d33 0 2024-02-26T10:47:53.869 INFO:tasks.workunit.client.0.smithi060.stdout:2/165: rename dd/de to dd/de/d34 22 2024-02-26T10:47:53.874 INFO:tasks.workunit.client.0.smithi060.stdout:2/166: readlink dd/l14 0 2024-02-26T10:47:53.874 INFO:tasks.workunit.client.0.smithi060.stdout:6/240: write d6/d7/d25/d1c/f2c [24547,102669] 0 2024-02-26T10:47:53.875 INFO:tasks.workunit.client.0.smithi060.stdout:6/241: chown d6/d7/d25/f2d 104603 1 2024-02-26T10:47:53.877 INFO:tasks.workunit.client.0.smithi060.stdout:6/242: getdents d6/d7/d25/d1c/d39 0 2024-02-26T10:47:53.878 INFO:tasks.workunit.client.0.smithi060.stdout:6/243: chown d6/d7/d25/l26 469 1 2024-02-26T10:47:53.878 INFO:tasks.workunit.client.0.smithi060.stdout:1/263: dwrite d3/db/d2e/f30 [0,4194304] 0 2024-02-26T10:47:53.881 INFO:tasks.workunit.client.0.smithi060.stdout:1/264: mkdir d3/d35/d53/d57 0 2024-02-26T10:47:53.881 INFO:tasks.workunit.client.0.smithi060.stdout:2/167: read f3 [1850498,88399] 0 2024-02-26T10:47:53.882 INFO:tasks.workunit.client.0.smithi060.stdout:3/152: sync 2024-02-26T10:47:53.883 INFO:tasks.workunit.client.0.smithi060.stdout:3/153: symlink d5/l26 0 2024-02-26T10:47:53.884 INFO:tasks.workunit.client.0.smithi060.stdout:1/265: creat d3/db/d1e/d39/f58 x:0 0 0 2024-02-26T10:47:53.888 INFO:tasks.workunit.client.0.smithi060.stdout:1/266: creat d3/db/f59 x:0 0 0 2024-02-26T10:47:53.891 INFO:tasks.workunit.client.0.smithi060.stdout:2/168: rename dd/d11/c1f to dd/d11/c35 0 2024-02-26T10:47:53.892 INFO:tasks.workunit.client.0.smithi060.stdout:4/248: dread d0/f44 [0,4194304] 0 2024-02-26T10:47:53.894 INFO:tasks.workunit.client.0.smithi060.stdout:4/249: creat d0/da/d3b/d16/f48 x:0 0 0 2024-02-26T10:47:53.897 INFO:tasks.workunit.client.0.smithi060.stdout:4/250: getdents d0/da/d3b/d1b 0 2024-02-26T10:47:53.897 INFO:tasks.workunit.client.0.smithi060.stdout:4/251: write d0/d7/d29/f32 [792937,9705] 0 2024-02-26T10:47:53.902 INFO:tasks.workunit.client.0.smithi060.stdout:4/252: creat d0/d27/f49 x:0 0 0 2024-02-26T10:47:53.903 INFO:tasks.workunit.client.0.smithi060.stdout:4/253: creat d0/f4a x:0 0 0 2024-02-26T10:47:53.904 INFO:tasks.workunit.client.0.smithi060.stdout:4/254: dread - d0/da/d3b/d16/f20 zero size 2024-02-26T10:47:53.904 INFO:tasks.workunit.client.0.smithi060.stdout:4/255: dread - d0/da/d3b/d16/f48 zero size 2024-02-26T10:47:53.904 INFO:tasks.workunit.client.0.smithi060.stdout:4/256: stat d0/d27/d2f/f31 0 2024-02-26T10:47:53.904 INFO:tasks.workunit.client.0.smithi060.stdout:4/257: creat d0/f4b x:0 0 0 2024-02-26T10:47:53.906 INFO:tasks.workunit.client.0.smithi060.stdout:4/258: rename d0/d27/d2f/f43 to d0/da/f4c 0 2024-02-26T10:47:53.907 INFO:tasks.workunit.client.0.smithi060.stdout:4/259: creat d0/d27/f4d x:0 0 0 2024-02-26T10:47:53.909 INFO:tasks.workunit.client.0.smithi060.stdout:3/154: dread d5/fa [0,4194304] 0 2024-02-26T10:47:53.918 INFO:tasks.workunit.client.0.smithi060.stdout:1/267: write d3/db/d2e/f4d [2669794,89198] 0 2024-02-26T10:47:53.921 INFO:tasks.workunit.client.0.smithi060.stdout:6/244: dwrite d6/f15 [0,4194304] 0 2024-02-26T10:47:53.923 INFO:tasks.workunit.client.0.smithi060.stdout:6/245: link d6/d7/d25/d1c/d1f/d23/f3a d6/d7/d25/d1c/d1f/d23/d28/f48 0 2024-02-26T10:47:53.928 INFO:tasks.workunit.client.0.smithi060.stdout:2/169: dread f4 [0,4194304] 0 2024-02-26T10:47:53.929 INFO:tasks.workunit.client.0.smithi060.stdout:9/228: dwrite d3/d9/dc/d32/d3c/d44/f48 [0,4194304] 0 2024-02-26T10:47:53.930 INFO:tasks.workunit.client.0.smithi060.stdout:2/170: rmdir dd/d33 0 2024-02-26T10:47:53.931 INFO:tasks.workunit.client.0.smithi060.stdout:9/229: creat d3/d9/d10/f4e x:0 0 0 2024-02-26T10:47:53.934 INFO:tasks.workunit.client.0.smithi060.stdout:2/171: creat dd/d11/d13/f36 x:0 0 0 2024-02-26T10:47:53.934 INFO:tasks.workunit.client.0.smithi060.stdout:2/172: fdatasync fa 0 2024-02-26T10:47:53.935 INFO:tasks.workunit.client.0.smithi060.stdout:4/260: dread d0/da/d3b/d16/f18 [0,4194304] 0 2024-02-26T10:47:53.944 INFO:tasks.workunit.client.0.smithi060.stdout:1/268: dread d3/db/d2e/f30 [0,4194304] 0 2024-02-26T10:47:53.944 INFO:tasks.workunit.client.0.smithi060.stdout:7/358: dwrite da/fd [4194304,4194304] 0 2024-02-26T10:47:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:1/269: creat d3/d35/f5a x:0 0 0 2024-02-26T10:47:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:1/270: chown d3/db/d2e/d51/c55 171385 1 2024-02-26T10:47:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:1/271: read d3/f20 [1703107,45685] 0 2024-02-26T10:47:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:1/272: write d3/db/d2e/f4d [338245,98664] 0 2024-02-26T10:47:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:7/359: getdents da/d15/d21/d2f 0 2024-02-26T10:47:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:1/273: symlink d3/db/d1e/d38/l5b 0 2024-02-26T10:47:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:7/360: fdatasync da/d15/f17 0 2024-02-26T10:47:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:7/361: symlink da/d15/d21/d32/d4e/d39/l80 0 2024-02-26T10:47:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:7/362: readlink da/l49 0 2024-02-26T10:47:53.957 INFO:tasks.workunit.client.0.smithi060.stdout:2/173: dread dd/d11/d13/f1a [0,4194304] 0 2024-02-26T10:47:53.960 INFO:tasks.workunit.client.0.smithi060.stdout:2/174: unlink dd/d11/l1b 0 2024-02-26T10:47:53.962 INFO:tasks.workunit.client.0.smithi060.stdout:2/175: mkdir dd/d11/d13/d37 0 2024-02-26T10:47:53.969 INFO:tasks.workunit.client.0.smithi060.stdout:9/230: dwrite d3/d15/d1b/d23/f34 [0,4194304] 0 2024-02-26T10:47:53.969 INFO:tasks.workunit.client.0.smithi060.stdout:9/231: stat d3/d9/d10/f21 0 2024-02-26T10:47:53.970 INFO:tasks.workunit.client.0.smithi060.stdout:4/261: dread d0/da/d3b/f2b [0,4194304] 0 2024-02-26T10:47:53.970 INFO:tasks.workunit.client.0.smithi060.stdout:4/262: chown d0/d7/d29/f38 5102 1 2024-02-26T10:47:53.971 INFO:tasks.workunit.client.0.smithi060.stdout:4/263: link d0/f44 d0/d27/d2f/f4e 0 2024-02-26T10:47:53.974 INFO:tasks.workunit.client.0.smithi060.stdout:4/264: mknod d0/da/c4f 0 2024-02-26T10:47:53.982 INFO:tasks.workunit.client.0.smithi060.stdout:1/274: read d3/db/d1e/f22 [3902255,18251] 0 2024-02-26T10:47:53.984 INFO:tasks.workunit.client.0.smithi060.stdout:1/275: link d3/db/d2e/f33 d3/f5c 0 2024-02-26T10:47:53.987 INFO:tasks.workunit.client.0.smithi060.stdout:1/276: mknod d3/db/d1e/d39/c5d 0 2024-02-26T10:47:53.987 INFO:tasks.workunit.client.0.smithi060.stdout:2/176: write dd/d11/f1c [2609075,112837] 0 2024-02-26T10:47:53.988 INFO:tasks.workunit.client.0.smithi060.stdout:1/277: truncate d3/db/f40 3149698 0 2024-02-26T10:47:53.988 INFO:tasks.workunit.client.0.smithi060.stdout:1/278: dread - d3/db/f59 zero size 2024-02-26T10:47:53.991 INFO:tasks.workunit.client.0.smithi060.stdout:7/363: dwrite da/fd [0,4194304] 0 2024-02-26T10:47:53.991 INFO:tasks.workunit.client.0.smithi060.stdout:1/279: mkdir d3/d35/d5e 0 2024-02-26T10:47:53.992 INFO:tasks.workunit.client.0.smithi060.stdout:1/280: mknod d3/db/d2e/c5f 0 2024-02-26T10:47:53.992 INFO:tasks.workunit.client.0.smithi060.stdout:7/364: creat da/d5a/f81 x:0 0 0 2024-02-26T10:47:53.993 INFO:tasks.workunit.client.0.smithi060.stdout:7/365: creat da/d5a/f82 x:0 0 0 2024-02-26T10:47:53.994 INFO:tasks.workunit.client.0.smithi060.stdout:2/177: truncate dd/d11/f25 824141 0 2024-02-26T10:47:53.996 INFO:tasks.workunit.client.0.smithi060.stdout:2/178: write dd/f2b [1043270,75423] 0 2024-02-26T10:47:54.017 INFO:tasks.workunit.client.0.smithi060.stdout:7/366: write f2 [4050408,119059] 0 2024-02-26T10:47:54.017 INFO:tasks.workunit.client.0.smithi060.stdout:7/367: stat da/f18 0 2024-02-26T10:47:54.017 INFO:tasks.workunit.client.0.smithi060.stdout:2/179: read f4 [3251045,72064] 0 2024-02-26T10:47:54.020 INFO:tasks.workunit.client.0.smithi060.stdout:1/281: dread d3/f20 [4194304,4194304] 0 2024-02-26T10:47:54.020 INFO:tasks.workunit.client.0.smithi060.stdout:1/282: stat d3/c18 0 2024-02-26T10:47:54.020 INFO:tasks.workunit.client.0.smithi060.stdout:2/180: mkdir dd/d11/d13/d37/d38 0 2024-02-26T10:47:54.020 INFO:tasks.workunit.client.0.smithi060.stdout:7/368: mknod da/d15/d21/d29/c83 0 2024-02-26T10:47:54.022 INFO:tasks.workunit.client.0.smithi060.stdout:1/283: symlink d3/d35/l60 0 2024-02-26T10:47:54.022 INFO:tasks.workunit.client.0.smithi060.stdout:1/284: write d3/f20 [3398965,127612] 0 2024-02-26T10:47:54.022 INFO:tasks.workunit.client.0.smithi060.stdout:1/285: read d3/f20 [2865213,6833] 0 2024-02-26T10:47:54.022 INFO:tasks.workunit.client.0.smithi060.stdout:2/181: rename dd to dd/d11/d39 22 2024-02-26T10:47:54.022 INFO:tasks.workunit.client.0.smithi060.stdout:2/182: chown dd/d11/d13/d37 30974200 1 2024-02-26T10:47:54.022 INFO:tasks.workunit.client.0.smithi060.stdout:2/183: write dd/d11/f1c [330311,18978] 0 2024-02-26T10:47:54.023 INFO:tasks.workunit.client.0.smithi060.stdout:2/184: getdents dd/d11/d13/d37/d38 0 2024-02-26T10:47:54.023 INFO:tasks.workunit.client.0.smithi060.stdout:2/185: truncate f0 4789496 0 2024-02-26T10:47:54.023 INFO:tasks.workunit.client.0.smithi060.stdout:1/286: symlink d3/l61 0 2024-02-26T10:47:54.032 INFO:tasks.workunit.client.0.smithi060.stdout:2/186: dread fa [0,4194304] 0 2024-02-26T10:47:54.032 INFO:tasks.workunit.client.0.smithi060.stdout:1/287: link d3/db/d1e/c3a d3/d54/c62 0 2024-02-26T10:47:54.032 INFO:tasks.workunit.client.0.smithi060.stdout:1/288: fsync d3/db/d2e/f4d 0 2024-02-26T10:47:54.032 INFO:tasks.workunit.client.0.smithi060.stdout:1/289: chown d3/db/d2e/d51/c55 1266174 1 2024-02-26T10:47:54.032 INFO:tasks.workunit.client.0.smithi060.stdout:1/290: dread - d3/db/d2e/f33 zero size 2024-02-26T10:47:54.034 INFO:tasks.workunit.client.0.smithi060.stdout:2/187: symlink dd/l3a 0 2024-02-26T10:47:54.034 INFO:tasks.workunit.client.0.smithi060.stdout:2/188: readlink dd/d11/l17 0 2024-02-26T10:47:54.039 INFO:tasks.workunit.client.0.smithi060.stdout:4/265: dread d0/fb [0,4194304] 0 2024-02-26T10:47:54.039 INFO:tasks.workunit.client.0.smithi060.stdout:9/232: dwrite d3/d9/dc/d32/d3c/d44/f48 [0,4194304] 0 2024-02-26T10:47:54.041 INFO:tasks.workunit.client.0.smithi060.stdout:1/291: dread d3/db/d1e/f3b [0,4194304] 0 2024-02-26T10:47:54.041 INFO:tasks.workunit.client.0.smithi060.stdout:1/292: fsync d3/db/d1e/d39/f58 0 2024-02-26T10:47:54.044 INFO:tasks.workunit.client.0.smithi060.stdout:0/240: sync 2024-02-26T10:47:54.044 INFO:tasks.workunit.client.0.smithi060.stdout:5/203: sync 2024-02-26T10:47:54.044 INFO:tasks.workunit.client.0.smithi060.stdout:5/204: readlink d0/l7 0 2024-02-26T10:47:54.044 INFO:tasks.workunit.client.0.smithi060.stdout:1/293: mkdir d3/db/d2e/d51/d63 0 2024-02-26T10:47:54.045 INFO:tasks.workunit.client.0.smithi060.stdout:7/369: dread f6 [0,4194304] 0 2024-02-26T10:47:54.045 INFO:tasks.workunit.client.0.smithi060.stdout:2/189: dread fa [0,4194304] 0 2024-02-26T10:47:54.045 INFO:tasks.workunit.client.0.smithi060.stdout:7/370: write da/d15/d21/f79 [537440,105876] 0 2024-02-26T10:47:54.045 INFO:tasks.workunit.client.0.smithi060.stdout:2/190: fdatasync f0 0 2024-02-26T10:47:54.045 INFO:tasks.workunit.client.0.smithi060.stdout:7/371: read da/d15/d21/d32/f5f [3291643,56509] 0 2024-02-26T10:47:54.046 INFO:tasks.workunit.client.0.smithi060.stdout:7/372: readlink da/l49 0 2024-02-26T10:47:54.046 INFO:tasks.workunit.client.0.smithi060.stdout:5/205: mkdir d0/de/d20/d32/d4c 0 2024-02-26T10:47:54.048 INFO:tasks.workunit.client.0.smithi060.stdout:5/206: rmdir d0/de 39 2024-02-26T10:47:54.049 INFO:tasks.workunit.client.0.smithi060.stdout:5/207: write d0/de/d15/f44 [23575,110481] 0 2024-02-26T10:47:54.050 INFO:tasks.workunit.client.0.smithi060.stdout:5/208: mkdir d0/de/d20/d32/d4d 0 2024-02-26T10:47:54.050 INFO:tasks.workunit.client.0.smithi060.stdout:7/373: unlink da/l52 0 2024-02-26T10:47:54.052 INFO:tasks.workunit.client.0.smithi060.stdout:5/209: rmdir d0/de/d20 39 2024-02-26T10:47:54.052 INFO:tasks.workunit.client.0.smithi060.stdout:5/210: stat d0/de/d20 0 2024-02-26T10:47:54.052 INFO:tasks.workunit.client.0.smithi060.stdout:5/211: rename d0 to d0/de/d20/d4e 22 2024-02-26T10:47:54.055 INFO:tasks.workunit.client.0.smithi060.stdout:5/212: rename d0/l40 to d0/de/d20/d32/d4d/l4f 0 2024-02-26T10:47:54.055 INFO:tasks.workunit.client.0.smithi060.stdout:2/191: write dd/f1d [2889288,4386] 0 2024-02-26T10:47:54.055 INFO:tasks.workunit.client.0.smithi060.stdout:2/192: readlink l9 0 2024-02-26T10:47:54.056 INFO:tasks.workunit.client.0.smithi060.stdout:5/213: symlink d0/de/d20/d32/d4d/l50 0 2024-02-26T10:47:54.056 INFO:tasks.workunit.client.0.smithi060.stdout:5/214: getdents d0/de/d20/d32/d4c 0 2024-02-26T10:47:54.057 INFO:tasks.workunit.client.0.smithi060.stdout:5/215: creat d0/d4b/f51 x:0 0 0 2024-02-26T10:47:54.059 INFO:tasks.workunit.client.0.smithi060.stdout:2/193: getdents dd/d11 0 2024-02-26T10:47:54.059 INFO:tasks.workunit.client.0.smithi060.stdout:2/194: fdatasync dd/d11/f1c 0 2024-02-26T10:47:54.061 INFO:tasks.workunit.client.0.smithi060.stdout:5/216: rename d0/c3 to d0/d33/c52 0 2024-02-26T10:47:54.061 INFO:tasks.workunit.client.0.smithi060.stdout:2/195: symlink dd/l3b 0 2024-02-26T10:47:54.061 INFO:tasks.workunit.client.0.smithi060.stdout:2/196: chown f4 9617861 1 2024-02-26T10:47:54.061 INFO:tasks.workunit.client.0.smithi060.stdout:5/217: creat d0/de/d20/d32/d4d/f53 x:0 0 0 2024-02-26T10:47:54.062 INFO:tasks.workunit.client.0.smithi060.stdout:2/197: creat dd/d11/d13/d37/d38/f3c x:0 0 0 2024-02-26T10:47:54.062 INFO:tasks.workunit.client.0.smithi060.stdout:5/218: mknod d0/de/d15/c54 0 2024-02-26T10:47:54.063 INFO:tasks.workunit.client.0.smithi060.stdout:2/198: creat dd/de/f3d x:0 0 0 2024-02-26T10:47:54.063 INFO:tasks.workunit.client.0.smithi060.stdout:2/199: chown f3 5 1 2024-02-26T10:47:54.063 INFO:tasks.workunit.client.0.smithi060.stdout:2/200: readlink dd/d11/l17 0 2024-02-26T10:47:54.063 INFO:tasks.workunit.client.0.smithi060.stdout:2/201: dread - dd/de/f1e zero size 2024-02-26T10:47:54.064 INFO:tasks.workunit.client.0.smithi060.stdout:5/219: link d0/de/d15/f44 d0/de/d20/d32/d4d/f55 0 2024-02-26T10:47:54.064 INFO:tasks.workunit.client.0.smithi060.stdout:5/220: chown d0/d33/f43 404 1 2024-02-26T10:47:54.064 INFO:tasks.workunit.client.0.smithi060.stdout:2/202: creat dd/d11/f3e x:0 0 0 2024-02-26T10:47:54.064 INFO:tasks.workunit.client.0.smithi060.stdout:2/203: creat dd/de/f3f x:0 0 0 2024-02-26T10:47:54.065 INFO:tasks.workunit.client.0.smithi060.stdout:2/204: readlink lb 0 2024-02-26T10:47:54.065 INFO:tasks.workunit.client.0.smithi060.stdout:5/221: creat d0/de/d20/f56 x:0 0 0 2024-02-26T10:47:54.065 INFO:tasks.workunit.client.0.smithi060.stdout:5/222: creat d0/d4b/f57 x:0 0 0 2024-02-26T10:47:54.066 INFO:tasks.workunit.client.0.smithi060.stdout:2/205: creat dd/d11/d13/d37/d38/f40 x:0 0 0 2024-02-26T10:47:54.066 INFO:tasks.workunit.client.0.smithi060.stdout:2/206: stat dd/d11/d13/l2a 0 2024-02-26T10:47:54.068 INFO:tasks.workunit.client.0.smithi060.stdout:5/223: getdents d0/de/d20/d32 0 2024-02-26T10:47:54.069 INFO:tasks.workunit.client.0.smithi060.stdout:5/224: rename d0/de/d15/f2c to d0/de/d20/f58 0 2024-02-26T10:47:54.107 INFO:tasks.workunit.client.0.smithi060.stdout:2/207: dwrite dd/de/f3d [0,4194304] 0 2024-02-26T10:47:54.111 INFO:tasks.workunit.client.0.smithi060.stdout:9/233: dwrite d3/d9/d10/f37 [0,4194304] 0 2024-02-26T10:47:54.128 INFO:tasks.workunit.client.0.smithi060.stdout:4/266: dwrite d0/da/d3b/f2b [0,4194304] 0 2024-02-26T10:47:54.128 INFO:tasks.workunit.client.0.smithi060.stdout:4/267: chown d0/l36 26026 1 2024-02-26T10:47:54.128 INFO:tasks.workunit.client.0.smithi060.stdout:4/268: write d0/d7/d29/f45 [790143,127463] 0 2024-02-26T10:47:54.128 INFO:tasks.workunit.client.0.smithi060.stdout:0/241: dread d0/dd/d39/d40/f17 [4194304,4194304] 0 2024-02-26T10:47:54.128 INFO:tasks.workunit.client.0.smithi060.stdout:0/242: fsync d0/f15 0 2024-02-26T10:47:54.128 INFO:tasks.workunit.client.0.smithi060.stdout:0/243: chown d0/ce 1 1 2024-02-26T10:47:54.128 INFO:tasks.workunit.client.0.smithi060.stdout:0/244: fdatasync d0/dd/d39/f3f 0 2024-02-26T10:47:54.129 INFO:tasks.workunit.client.0.smithi060.stdout:0/245: mkdir d0/d37/d44/d52 0 2024-02-26T10:47:54.131 INFO:tasks.workunit.client.0.smithi060.stdout:0/246: mkdir d0/dd/d53 0 2024-02-26T10:47:54.133 INFO:tasks.workunit.client.0.smithi060.stdout:0/247: rename d0/d37/d44/f48 to d0/dd/d39/f54 0 2024-02-26T10:47:54.134 INFO:tasks.workunit.client.0.smithi060.stdout:0/248: creat d0/f55 x:0 0 0 2024-02-26T10:47:54.134 INFO:tasks.workunit.client.0.smithi060.stdout:0/249: dread - d0/d37/d44/f45 zero size 2024-02-26T10:47:54.150 INFO:tasks.workunit.client.0.smithi060.stdout:4/269: write d0/da/d3b/d16/f18 [3716360,72212] 0 2024-02-26T10:47:54.151 INFO:tasks.workunit.client.0.smithi060.stdout:2/208: dwrite dd/f1d [0,4194304] 0 2024-02-26T10:47:54.152 INFO:tasks.workunit.client.0.smithi060.stdout:4/270: mknod d0/d7/c50 0 2024-02-26T10:47:54.154 INFO:tasks.workunit.client.0.smithi060.stdout:1/294: dwrite d3/f12 [0,4194304] 0 2024-02-26T10:47:54.155 INFO:tasks.workunit.client.0.smithi060.stdout:1/295: fdatasync d3/db/d2e/f30 0 2024-02-26T10:47:54.155 INFO:tasks.workunit.client.0.smithi060.stdout:9/234: dread d3/d9/d10/f2c [0,4194304] 0 2024-02-26T10:47:54.157 INFO:tasks.workunit.client.0.smithi060.stdout:2/209: rename dd/d11/d13 to dd/de/d41 0 2024-02-26T10:47:54.157 INFO:tasks.workunit.client.0.smithi060.stdout:2/210: fdatasync dd/de/d41/f36 0 2024-02-26T10:47:54.158 INFO:tasks.workunit.client.0.smithi060.stdout:1/296: mknod d3/d35/d53/d57/c64 0 2024-02-26T10:47:54.159 INFO:tasks.workunit.client.0.smithi060.stdout:9/235: truncate d3/d15/d35/f3e 552945 0 2024-02-26T10:47:54.159 INFO:tasks.workunit.client.0.smithi060.stdout:9/236: stat d3/l49 0 2024-02-26T10:47:54.160 INFO:tasks.workunit.client.0.smithi060.stdout:9/237: stat d3/d15/d1b/d23/f36 0 2024-02-26T10:47:54.160 INFO:tasks.workunit.client.0.smithi060.stdout:9/238: readlink d3/d9/dc/l12 0 2024-02-26T10:47:54.161 INFO:tasks.workunit.client.0.smithi060.stdout:1/297: creat d3/d35/f65 x:0 0 0 2024-02-26T10:47:54.161 INFO:tasks.workunit.client.0.smithi060.stdout:2/211: truncate dd/de/d41/f1a 2168792 0 2024-02-26T10:47:54.180 INFO:tasks.workunit.client.0.smithi060.stdout:4/271: dread d0/f44 [0,4194304] 0 2024-02-26T10:47:54.181 INFO:tasks.workunit.client.0.smithi060.stdout:4/272: write d0/da/d3b/d16/f18 [1382558,53943] 0 2024-02-26T10:47:54.183 INFO:tasks.workunit.client.0.smithi060.stdout:4/273: creat d0/da/d3b/d16/f51 x:0 0 0 2024-02-26T10:47:54.184 INFO:tasks.workunit.client.0.smithi060.stdout:4/274: mkdir d0/d27/d52 0 2024-02-26T10:47:54.184 INFO:tasks.workunit.client.0.smithi060.stdout:9/239: dread d3/d15/f17 [0,4194304] 0 2024-02-26T10:47:54.185 INFO:tasks.workunit.client.0.smithi060.stdout:4/275: creat d0/da/f53 x:0 0 0 2024-02-26T10:47:54.189 INFO:tasks.workunit.client.0.smithi060.stdout:2/212: dread dd/d11/f26 [0,4194304] 0 2024-02-26T10:47:54.197 INFO:tasks.workunit.client.0.smithi060.stdout:8/235: sync 2024-02-26T10:47:54.197 INFO:tasks.workunit.client.0.smithi060.stdout:3/155: sync 2024-02-26T10:47:54.197 INFO:tasks.workunit.client.0.smithi060.stdout:6/246: sync 2024-02-26T10:47:54.197 INFO:tasks.workunit.client.0.smithi060.stdout:6/247: stat d6/d7/d25/l3c 0 2024-02-26T10:47:54.204 INFO:tasks.workunit.client.0.smithi060.stdout:6/248: stat l1 0 2024-02-26T10:47:54.205 INFO:tasks.workunit.client.0.smithi060.stdout:3/156: write d5/de/df/d11/f20 [3867740,36885] 0 2024-02-26T10:47:54.205 INFO:tasks.workunit.client.0.smithi060.stdout:3/157: dread - d5/de/d16/d17/f1e zero size 2024-02-26T10:47:54.205 INFO:tasks.workunit.client.0.smithi060.stdout:8/236: rmdir d9/d15/d29/d2f/d3b 0 2024-02-26T10:47:54.205 INFO:tasks.workunit.client.0.smithi060.stdout:3/158: getdents d5/de 0 2024-02-26T10:47:54.205 INFO:tasks.workunit.client.0.smithi060.stdout:8/237: mkdir d9/d15/d3e/d44 0 2024-02-26T10:47:54.205 INFO:tasks.workunit.client.0.smithi060.stdout:3/159: link d5/f9 d5/f27 0 2024-02-26T10:47:54.205 INFO:tasks.workunit.client.0.smithi060.stdout:3/160: write d5/de/df/f21 [982585,35385] 0 2024-02-26T10:47:54.218 INFO:tasks.workunit.client.0.smithi060.stdout:1/298: dwrite d3/db/d1e/f3d [4194304,4194304] 0 2024-02-26T10:47:54.218 INFO:tasks.workunit.client.0.smithi060.stdout:1/299: chown d3/l61 144 1 2024-02-26T10:47:54.227 INFO:tasks.workunit.client.0.smithi060.stdout:6/249: dread d6/d7/d25/d1c/d1f/f37 [0,4194304] 0 2024-02-26T10:47:54.229 INFO:tasks.workunit.client.0.smithi060.stdout:6/250: creat d6/d7/d25/f49 x:0 0 0 2024-02-26T10:47:54.229 INFO:tasks.workunit.client.0.smithi060.stdout:6/251: write d6/d7/d25/d1c/d39/f44 [1015202,47583] 0 2024-02-26T10:47:54.229 INFO:tasks.workunit.client.0.smithi060.stdout:6/252: dread - d6/d7/d25/d1c/d1f/d23/d28/f29 zero size 2024-02-26T10:47:54.236 INFO:tasks.workunit.client.0.smithi060.stdout:4/276: dwrite d0/d27/f30 [0,4194304] 0 2024-02-26T10:47:54.236 INFO:tasks.workunit.client.0.smithi060.stdout:9/240: dwrite d3/d9/f2e [0,4194304] 0 2024-02-26T10:47:54.236 INFO:tasks.workunit.client.0.smithi060.stdout:9/241: getdents d3/d15/d35 0 2024-02-26T10:47:54.238 INFO:tasks.workunit.client.0.smithi060.stdout:9/242: creat d3/d9/d10/f4f x:0 0 0 2024-02-26T10:47:54.240 INFO:tasks.workunit.client.0.smithi060.stdout:2/213: dread dd/d11/f25 [0,4194304] 0 2024-02-26T10:47:54.241 INFO:tasks.workunit.client.0.smithi060.stdout:4/277: unlink d0/d27/f41 0 2024-02-26T10:47:54.241 INFO:tasks.workunit.client.0.smithi060.stdout:2/214: write fa [923999,65291] 0 2024-02-26T10:47:54.241 INFO:tasks.workunit.client.0.smithi060.stdout:2/215: readlink dd/de/d41/l20 0 2024-02-26T10:47:54.242 INFO:tasks.workunit.client.0.smithi060.stdout:2/216: unlink dd/de/d41/d37/d38/f3c 0 2024-02-26T10:47:54.244 INFO:tasks.workunit.client.0.smithi060.stdout:2/217: getdents dd/d11 0 2024-02-26T10:47:54.244 INFO:tasks.workunit.client.0.smithi060.stdout:2/218: readlink dd/d11/l18 0 2024-02-26T10:47:54.257 INFO:tasks.workunit.client.0.smithi060.stdout:1/300: rename d3/db/d1e/f3d to d3/d35/d5e/f66 0 2024-02-26T10:47:54.257 INFO:tasks.workunit.client.0.smithi060.stdout:1/301: stat d3/l61 0 2024-02-26T10:47:54.257 INFO:tasks.workunit.client.0.smithi060.stdout:1/302: dread - d3/d35/f65 zero size 2024-02-26T10:47:54.258 INFO:tasks.workunit.client.0.smithi060.stdout:1/303: mknod d3/c67 0 2024-02-26T10:47:54.259 INFO:tasks.workunit.client.0.smithi060.stdout:1/304: mkdir d3/d35/d53/d68 0 2024-02-26T10:47:54.260 INFO:tasks.workunit.client.0.smithi060.stdout:1/305: rmdir d3/db/d2e/d51 39 2024-02-26T10:47:54.260 INFO:tasks.workunit.client.0.smithi060.stdout:1/306: chown d3/db/d2e/d51 22837928 1 2024-02-26T10:47:54.261 INFO:tasks.workunit.client.0.smithi060.stdout:1/307: creat d3/f69 x:0 0 0 2024-02-26T10:47:54.280 INFO:tasks.workunit.client.0.smithi060.stdout:8/238: dwrite d9/d19/f42 [0,4194304] 0 2024-02-26T10:47:54.282 INFO:tasks.workunit.client.0.smithi060.stdout:8/239: creat d9/d15/d40/f45 x:0 0 0 2024-02-26T10:47:54.284 INFO:tasks.workunit.client.0.smithi060.stdout:7/374: sync 2024-02-26T10:47:54.284 INFO:tasks.workunit.client.0.smithi060.stdout:5/225: sync 2024-02-26T10:47:54.284 INFO:tasks.workunit.client.0.smithi060.stdout:5/226: stat d0/de/cf 0 2024-02-26T10:47:54.290 INFO:tasks.workunit.client.0.smithi060.stdout:9/243: dwrite d3/f16 [0,4194304] 0 2024-02-26T10:47:54.290 INFO:tasks.workunit.client.0.smithi060.stdout:7/375: truncate da/d15/d21/d2f/d37/f38 3328985 0 2024-02-26T10:47:54.291 INFO:tasks.workunit.client.0.smithi060.stdout:5/227: symlink d0/de/d20/l59 0 2024-02-26T10:47:54.291 INFO:tasks.workunit.client.0.smithi060.stdout:5/228: write d0/de/d20/d32/d4d/f53 [462724,13940] 0 2024-02-26T10:47:54.291 INFO:tasks.workunit.client.0.smithi060.stdout:3/161: dwrite d5/de/d16/d17/f1d [4194304,4194304] 0 2024-02-26T10:47:54.291 INFO:tasks.workunit.client.0.smithi060.stdout:2/219: dwrite dd/f1d [0,4194304] 0 2024-02-26T10:47:54.291 INFO:tasks.workunit.client.0.smithi060.stdout:2/220: getdents dd/d11 0 2024-02-26T10:47:54.293 INFO:tasks.workunit.client.0.smithi060.stdout:5/229: creat d0/de/d15/f5a x:0 0 0 2024-02-26T10:47:54.293 INFO:tasks.workunit.client.0.smithi060.stdout:7/376: rename da/d15/d21/d32/d4e/c69 to da/d15/d21/d4b/d67/c84 0 2024-02-26T10:47:54.293 INFO:tasks.workunit.client.0.smithi060.stdout:7/377: fdatasync da/d5a/f82 0 2024-02-26T10:47:54.294 INFO:tasks.workunit.client.0.smithi060.stdout:5/230: symlink d0/de/d20/l5b 0 2024-02-26T10:47:54.295 INFO:tasks.workunit.client.0.smithi060.stdout:5/231: fsync d0/f8 0 2024-02-26T10:47:54.295 INFO:tasks.workunit.client.0.smithi060.stdout:7/378: creat da/d15/d21/d4b/d67/f85 x:0 0 0 2024-02-26T10:47:54.300 INFO:tasks.workunit.client.0.smithi060.stdout:7/379: write da/f30 [5059181,105853] 0 2024-02-26T10:47:54.300 INFO:tasks.workunit.client.0.smithi060.stdout:1/308: dread d3/db/d1e/f22 [0,4194304] 0 2024-02-26T10:47:54.300 INFO:tasks.workunit.client.0.smithi060.stdout:1/309: truncate d3/db/d1e/f3b 168291 0 2024-02-26T10:47:54.305 INFO:tasks.workunit.client.0.smithi060.stdout:2/221: dread f5 [0,4194304] 0 2024-02-26T10:47:54.309 INFO:tasks.workunit.client.0.smithi060.stdout:8/240: dread d9/d15/d29/f2c [0,4194304] 0 2024-02-26T10:47:54.309 INFO:tasks.workunit.client.0.smithi060.stdout:8/241: readlink d9/l21 0 2024-02-26T10:47:54.313 INFO:tasks.workunit.client.0.smithi060.stdout:0/250: sync 2024-02-26T10:47:54.313 INFO:tasks.workunit.client.0.smithi060.stdout:7/380: dread da/d15/d21/f22 [0,4194304] 0 2024-02-26T10:47:54.314 INFO:tasks.workunit.client.0.smithi060.stdout:0/251: symlink d0/l56 0 2024-02-26T10:47:54.316 INFO:tasks.workunit.client.0.smithi060.stdout:8/242: truncate d9/d19/f3a 4161235 0 2024-02-26T10:47:54.317 INFO:tasks.workunit.client.0.smithi060.stdout:2/222: write fa [647462,108745] 0 2024-02-26T10:47:54.317 INFO:tasks.workunit.client.0.smithi060.stdout:1/310: rename d3/d54/c62 to d3/d35/d53/c6a 0 2024-02-26T10:47:54.317 INFO:tasks.workunit.client.0.smithi060.stdout:1/311: chown d3/f23 815 1 2024-02-26T10:47:54.318 INFO:tasks.workunit.client.0.smithi060.stdout:2/223: mkdir dd/de/d41/d42 0 2024-02-26T10:47:54.321 INFO:tasks.workunit.client.0.smithi060.stdout:2/224: mkdir dd/d43 0 2024-02-26T10:47:54.323 INFO:tasks.workunit.client.0.smithi060.stdout:8/243: creat d9/d15/d40/f46 x:0 0 0 2024-02-26T10:47:54.323 INFO:tasks.workunit.client.0.smithi060.stdout:8/244: stat d9/d15/c26 0 2024-02-26T10:47:54.323 INFO:tasks.workunit.client.0.smithi060.stdout:1/312: mkdir d3/d35/d53/d68/d6b 0 2024-02-26T10:47:54.324 INFO:tasks.workunit.client.0.smithi060.stdout:1/313: mkdir d3/db/d1e/d38/d6c 0 2024-02-26T10:47:54.324 INFO:tasks.workunit.client.0.smithi060.stdout:1/314: chown d3/db/d2e/f33 691 1 2024-02-26T10:47:54.324 INFO:tasks.workunit.client.0.smithi060.stdout:1/315: dread - d3/db/d1e/d39/f48 zero size 2024-02-26T10:47:54.325 INFO:tasks.workunit.client.0.smithi060.stdout:9/244: dread d3/d9/f2f [0,4194304] 0 2024-02-26T10:47:54.328 INFO:tasks.workunit.client.0.smithi060.stdout:0/252: dread d0/dd/f2d [0,4194304] 0 2024-02-26T10:47:54.329 INFO:tasks.workunit.client.0.smithi060.stdout:0/253: write d0/dd/f2d [1308202,44116] 0 2024-02-26T10:47:54.329 INFO:tasks.workunit.client.0.smithi060.stdout:0/254: read d0/dd/d39/d40/f1f [778252,128516] 0 2024-02-26T10:47:54.332 INFO:tasks.workunit.client.0.smithi060.stdout:1/316: mkdir d3/db/d1e/d6d 0 2024-02-26T10:47:54.332 INFO:tasks.workunit.client.0.smithi060.stdout:0/255: mknod d0/c57 0 2024-02-26T10:47:54.332 INFO:tasks.workunit.client.0.smithi060.stdout:5/232: dread d0/de/f29 [0,4194304] 0 2024-02-26T10:47:54.332 INFO:tasks.workunit.client.0.smithi060.stdout:9/245: getdents d3/d9/dc/d32/d3a 0 2024-02-26T10:47:54.332 INFO:tasks.workunit.client.0.smithi060.stdout:9/246: write d3/d15/d1b/d23/f36 [4541346,118143] 0 2024-02-26T10:47:54.334 INFO:tasks.workunit.client.0.smithi060.stdout:2/225: dread dd/d11/f1c [0,4194304] 0 2024-02-26T10:47:54.334 INFO:tasks.workunit.client.0.smithi060.stdout:2/226: truncate dd/d11/f3e 402777 0 2024-02-26T10:47:54.336 INFO:tasks.workunit.client.0.smithi060.stdout:5/233: unlink d0/de/d20/c25 0 2024-02-26T10:47:54.336 INFO:tasks.workunit.client.0.smithi060.stdout:0/256: rmdir d0/dd/d39 39 2024-02-26T10:47:54.336 INFO:tasks.workunit.client.0.smithi060.stdout:0/257: truncate d0/dd/d39/f3f 427202 0 2024-02-26T10:47:54.337 INFO:tasks.workunit.client.0.smithi060.stdout:0/258: stat d0/dd/d39/d40/f28 0 2024-02-26T10:47:54.350 INFO:tasks.workunit.client.0.smithi060.stdout:5/234: dread d0/f17 [0,4194304] 0 2024-02-26T10:47:54.350 INFO:tasks.workunit.client.0.smithi060.stdout:5/235: dread - d0/de/d20/f58 zero size 2024-02-26T10:47:54.351 INFO:tasks.workunit.client.0.smithi060.stdout:5/236: truncate d0/fd 1665234 0 2024-02-26T10:47:54.352 INFO:tasks.workunit.client.0.smithi060.stdout:3/162: dwrite d5/fa [0,4194304] 0 2024-02-26T10:47:54.353 INFO:tasks.workunit.client.0.smithi060.stdout:5/237: creat d0/d33/f5c x:0 0 0 2024-02-26T10:47:54.354 INFO:tasks.workunit.client.0.smithi060.stdout:5/238: chown d0/de/d20/d32/d4d/l4f 213408036 1 2024-02-26T10:47:54.357 INFO:tasks.workunit.client.0.smithi060.stdout:8/245: dread d9/d15/f1b [0,4194304] 0 2024-02-26T10:47:54.357 INFO:tasks.workunit.client.0.smithi060.stdout:3/163: mknod d5/de/c28 0 2024-02-26T10:47:54.357 INFO:tasks.workunit.client.0.smithi060.stdout:3/164: chown d5/de/d16/d17/f1e 2163515 1 2024-02-26T10:47:54.358 INFO:tasks.workunit.client.0.smithi060.stdout:8/246: symlink d9/d15/d40/l47 0 2024-02-26T10:47:54.366 INFO:tasks.workunit.client.0.smithi060.stdout:0/259: dread d0/f3 [0,4194304] 0 2024-02-26T10:47:54.368 INFO:tasks.workunit.client.0.smithi060.stdout:0/260: rename d0/d37/d44/d52 to d0/d37/d44/d58 0 2024-02-26T10:47:54.378 INFO:tasks.workunit.client.0.smithi060.stdout:2/227: rename fa to dd/d11/f44 0 2024-02-26T10:47:54.378 INFO:tasks.workunit.client.0.smithi060.stdout:3/165: rename d5/de to d5/de/d16/d29 22 2024-02-26T10:47:54.379 INFO:tasks.workunit.client.0.smithi060.stdout:2/228: truncate dd/d11/f1c 686689 0 2024-02-26T10:47:54.380 INFO:tasks.workunit.client.0.smithi060.stdout:0/261: rename d0/l8 to d0/dd/d39/d40/l59 0 2024-02-26T10:47:54.381 INFO:tasks.workunit.client.0.smithi060.stdout:3/166: mknod d5/c2a 0 2024-02-26T10:47:54.381 INFO:tasks.workunit.client.0.smithi060.stdout:3/167: write d5/f25 [199621,80119] 0 2024-02-26T10:47:54.382 INFO:tasks.workunit.client.0.smithi060.stdout:0/262: mknod d0/d37/d44/d58/c5a 0 2024-02-26T10:47:54.382 INFO:tasks.workunit.client.0.smithi060.stdout:2/229: creat dd/de/d41/d37/d38/f45 x:0 0 0 2024-02-26T10:47:54.382 INFO:tasks.workunit.client.0.smithi060.stdout:3/168: creat d5/de/f2b x:0 0 0 2024-02-26T10:47:54.383 INFO:tasks.workunit.client.0.smithi060.stdout:0/263: creat d0/d37/d44/d58/f5b x:0 0 0 2024-02-26T10:47:54.383 INFO:tasks.workunit.client.0.smithi060.stdout:0/264: fdatasync d0/d37/d44/f45 0 2024-02-26T10:47:54.384 INFO:tasks.workunit.client.0.smithi060.stdout:3/169: symlink d5/de/l2c 0 2024-02-26T10:47:54.386 INFO:tasks.workunit.client.0.smithi060.stdout:3/170: unlink d5/de/c28 0 2024-02-26T10:47:54.390 INFO:tasks.workunit.client.0.smithi060.stdout:1/317: dwrite d3/db/d1e/f3b [0,4194304] 0 2024-02-26T10:47:54.390 INFO:tasks.workunit.client.0.smithi060.stdout:9/247: dread d3/d9/d10/f37 [0,4194304] 0 2024-02-26T10:47:54.393 INFO:tasks.workunit.client.0.smithi060.stdout:9/248: rename d3/c39 to d3/d1e/c50 0 2024-02-26T10:47:54.395 INFO:tasks.workunit.client.0.smithi060.stdout:9/249: symlink d3/d9/dc/d32/d3c/d44/l51 0 2024-02-26T10:47:54.395 INFO:tasks.workunit.client.0.smithi060.stdout:9/250: write d3/d9/dc/d32/d3a/f4c [28731,120586] 0 2024-02-26T10:47:54.396 INFO:tasks.workunit.client.0.smithi060.stdout:9/251: mkdir d3/d52 0 2024-02-26T10:47:54.401 INFO:tasks.workunit.client.0.smithi060.stdout:9/252: write d3/d9/f38 [134289,13387] 0 2024-02-26T10:47:54.401 INFO:tasks.workunit.client.0.smithi060.stdout:9/253: write d3/d9/f19 [1924108,34384] 0 2024-02-26T10:47:54.401 INFO:tasks.workunit.client.0.smithi060.stdout:9/254: chown d3/l1c 2380750 1 2024-02-26T10:47:54.401 INFO:tasks.workunit.client.0.smithi060.stdout:1/318: creat d3/db/d1e/d38/f6e x:0 0 0 2024-02-26T10:47:54.404 INFO:tasks.workunit.client.0.smithi060.stdout:7/381: dwrite da/d15/d21/d2f/f68 [0,4194304] 0 2024-02-26T10:47:54.409 INFO:tasks.workunit.client.0.smithi060.stdout:8/247: dread d9/d19/f3a [0,4194304] 0 2024-02-26T10:47:54.412 INFO:tasks.workunit.client.0.smithi060.stdout:8/248: symlink d9/d15/d40/l48 0 2024-02-26T10:47:54.423 INFO:tasks.workunit.client.0.smithi060.stdout:6/253: sync 2024-02-26T10:47:54.423 INFO:tasks.workunit.client.0.smithi060.stdout:4/278: sync 2024-02-26T10:47:54.436 INFO:tasks.workunit.client.0.smithi060.stdout:7/382: write da/d15/f53 [1130896,53172] 0 2024-02-26T10:47:54.444 INFO:tasks.workunit.client.0.smithi060.stdout:8/249: write d9/d19/f3a [1975725,72844] 0 2024-02-26T10:47:54.445 INFO:tasks.workunit.client.0.smithi060.stdout:9/255: dread d3/f16 [0,4194304] 0 2024-02-26T10:47:54.447 INFO:tasks.workunit.client.0.smithi060.stdout:8/250: creat d9/d15/d3e/d44/f49 x:0 0 0 2024-02-26T10:47:54.447 INFO:tasks.workunit.client.0.smithi060.stdout:8/251: chown d9/d15/d29/c3f 47277149 1 2024-02-26T10:47:54.447 INFO:tasks.workunit.client.0.smithi060.stdout:8/252: fsync d9/d15/d29/d2f/f31 0 2024-02-26T10:47:54.447 INFO:tasks.workunit.client.0.smithi060.stdout:8/253: chown d9/fc 0 1 2024-02-26T10:47:54.447 INFO:tasks.workunit.client.0.smithi060.stdout:9/256: mknod d3/d9/dc/d32/d3c/d47/c53 0 2024-02-26T10:47:54.447 INFO:tasks.workunit.client.0.smithi060.stdout:9/257: truncate d3/d9/dc/d32/f3d 116283 0 2024-02-26T10:47:54.448 INFO:tasks.workunit.client.0.smithi060.stdout:2/230: dwrite dd/de/d41/f36 [0,4194304] 0 2024-02-26T10:47:54.448 INFO:tasks.workunit.client.0.smithi060.stdout:4/279: dread d0/d7/d29/f38 [0,4194304] 0 2024-02-26T10:47:54.448 INFO:tasks.workunit.client.0.smithi060.stdout:4/280: readlink d0/l2c 0 2024-02-26T10:47:54.449 INFO:tasks.workunit.client.0.smithi060.stdout:9/258: unlink d3/l49 0 2024-02-26T10:47:54.449 INFO:tasks.workunit.client.0.smithi060.stdout:9/259: creat d3/d9/f54 x:0 0 0 2024-02-26T10:47:54.466 INFO:tasks.workunit.client.0.smithi060.stdout:1/319: dwrite d3/d35/f65 [0,4194304] 0 2024-02-26T10:47:54.467 INFO:tasks.workunit.client.0.smithi060.stdout:3/171: dwrite d5/fd [0,4194304] 0 2024-02-26T10:47:54.467 INFO:tasks.workunit.client.0.smithi060.stdout:0/265: dwrite d0/d37/d44/d58/f5b [0,4194304] 0 2024-02-26T10:47:54.472 INFO:tasks.workunit.client.0.smithi060.stdout:0/266: mknod d0/dd/d33/c5c 0 2024-02-26T10:47:54.472 INFO:tasks.workunit.client.0.smithi060.stdout:0/267: write d0/dd/d39/d40/f4d [611708,54447] 0 2024-02-26T10:47:54.472 INFO:tasks.workunit.client.0.smithi060.stdout:0/268: fsync d0/dd/d39/f54 0 2024-02-26T10:47:54.478 INFO:tasks.workunit.client.0.smithi060.stdout:4/281: write d0/ff [349119,50752] 0 2024-02-26T10:47:54.481 INFO:tasks.workunit.client.0.smithi060.stdout:8/254: write d9/fc [744613,79922] 0 2024-02-26T10:47:54.481 INFO:tasks.workunit.client.0.smithi060.stdout:9/260: rename d3/d1e/l43 to d3/d9/dc/d32/d3a/l55 0 2024-02-26T10:47:54.489 INFO:tasks.workunit.client.0.smithi060.stdout:4/282: rename d0/da/l1e to d0/da/l54 0 2024-02-26T10:47:54.489 INFO:tasks.workunit.client.0.smithi060.stdout:7/383: write da/d66/f75 [63066,45199] 0 2024-02-26T10:47:54.489 INFO:tasks.workunit.client.0.smithi060.stdout:7/384: dread - da/d5a/f81 zero size 2024-02-26T10:47:54.489 INFO:tasks.workunit.client.0.smithi060.stdout:4/283: write d0/f4a [45612,35371] 0 2024-02-26T10:47:54.489 INFO:tasks.workunit.client.0.smithi060.stdout:4/284: stat d0/d27/f49 0 2024-02-26T10:47:54.489 INFO:tasks.workunit.client.0.smithi060.stdout:4/285: write d0/da/d3b/f23 [1772724,3411] 0 2024-02-26T10:47:54.489 INFO:tasks.workunit.client.0.smithi060.stdout:8/255: rmdir d9/d15/d3e 39 2024-02-26T10:47:54.490 INFO:tasks.workunit.client.0.smithi060.stdout:9/261: creat d3/d15/d1b/f56 x:0 0 0 2024-02-26T10:47:54.490 INFO:tasks.workunit.client.0.smithi060.stdout:9/262: stat d3/d9/dc/d32/d3c/d44/l51 0 2024-02-26T10:47:54.491 INFO:tasks.workunit.client.0.smithi060.stdout:6/254: dwrite d6/d7/d25/f2d [0,4194304] 0 2024-02-26T10:47:54.491 INFO:tasks.workunit.client.0.smithi060.stdout:6/255: chown d6/d7/d25/f2d 0 1 2024-02-26T10:47:54.491 INFO:tasks.workunit.client.0.smithi060.stdout:6/256: chown d6/d7/d25/d1c/d1f/d23/d2b 3033 1 2024-02-26T10:47:54.491 INFO:tasks.workunit.client.0.smithi060.stdout:3/172: rename d5/f9 to d5/de/d16/f2d 0 2024-02-26T10:47:54.492 INFO:tasks.workunit.client.0.smithi060.stdout:3/173: write d5/de/f14 [5221554,86418] 0 2024-02-26T10:47:54.493 INFO:tasks.workunit.client.0.smithi060.stdout:8/256: mkdir d9/d4a 0 2024-02-26T10:47:54.493 INFO:tasks.workunit.client.0.smithi060.stdout:8/257: chown d9/d15/d40/l47 0 1 2024-02-26T10:47:54.493 INFO:tasks.workunit.client.0.smithi060.stdout:7/385: rmdir da/d15/d21 39 2024-02-26T10:47:54.494 INFO:tasks.workunit.client.0.smithi060.stdout:4/286: creat d0/d27/d2f/f55 x:0 0 0 2024-02-26T10:47:54.495 INFO:tasks.workunit.client.0.smithi060.stdout:1/320: dread d3/db/d1e/f27 [0,4194304] 0 2024-02-26T10:47:54.496 INFO:tasks.workunit.client.0.smithi060.stdout:6/257: truncate d6/d7/fd 366852 0 2024-02-26T10:47:54.496 INFO:tasks.workunit.client.0.smithi060.stdout:6/258: stat d6/d7/d25/d1c/d1f/d23/d28/f34 0 2024-02-26T10:47:54.497 INFO:tasks.workunit.client.0.smithi060.stdout:3/174: symlink d5/l2e 0 2024-02-26T10:47:54.497 INFO:tasks.workunit.client.0.smithi060.stdout:3/175: chown d5/de/f14 0 1 2024-02-26T10:47:54.498 INFO:tasks.workunit.client.0.smithi060.stdout:8/258: mknod d9/d15/d29/d2f/d34/c4b 0 2024-02-26T10:47:54.499 INFO:tasks.workunit.client.0.smithi060.stdout:7/386: mknod da/d15/d21/d2f/d37/c86 0 2024-02-26T10:47:54.500 INFO:tasks.workunit.client.0.smithi060.stdout:1/321: symlink d3/db/d2e/d51/l6f 0 2024-02-26T10:47:54.500 INFO:tasks.workunit.client.0.smithi060.stdout:8/259: creat d9/d19/f4c x:0 0 0 2024-02-26T10:47:54.500 INFO:tasks.workunit.client.0.smithi060.stdout:6/259: creat d6/d7/d25/d1c/d1f/d23/f4a x:0 0 0 2024-02-26T10:47:54.501 INFO:tasks.workunit.client.0.smithi060.stdout:3/176: link d5/l26 d5/de/df/l2f 0 2024-02-26T10:47:54.503 INFO:tasks.workunit.client.0.smithi060.stdout:2/231: dwrite dd/de/d41/d37/d38/f45 [0,4194304] 0 2024-02-26T10:47:54.504 INFO:tasks.workunit.client.0.smithi060.stdout:1/322: creat d3/d35/d5e/f70 x:0 0 0 2024-02-26T10:47:54.505 INFO:tasks.workunit.client.0.smithi060.stdout:6/260: creat d6/d7/d25/f4b x:0 0 0 2024-02-26T10:47:54.505 INFO:tasks.workunit.client.0.smithi060.stdout:2/232: symlink dd/l46 0 2024-02-26T10:47:54.507 INFO:tasks.workunit.client.0.smithi060.stdout:7/387: mknod da/c87 0 2024-02-26T10:47:54.507 INFO:tasks.workunit.client.0.smithi060.stdout:3/177: write d5/de/df/f21 [589502,108393] 0 2024-02-26T10:47:54.507 INFO:tasks.workunit.client.0.smithi060.stdout:3/178: readlink d5/de/df/l2f 0 2024-02-26T10:47:54.508 INFO:tasks.workunit.client.0.smithi060.stdout:1/323: creat d3/d35/d53/d57/f71 x:0 0 0 2024-02-26T10:47:54.508 INFO:tasks.workunit.client.0.smithi060.stdout:1/324: fsync d3/db/d2e/f4d 0 2024-02-26T10:47:54.511 INFO:tasks.workunit.client.0.smithi060.stdout:6/261: creat d6/d7/d25/d1c/d1f/d23/d28/f4c x:0 0 0 2024-02-26T10:47:54.511 INFO:tasks.workunit.client.0.smithi060.stdout:1/325: mknod d3/d54/c72 0 2024-02-26T10:47:54.512 INFO:tasks.workunit.client.0.smithi060.stdout:2/233: stat f3 0 2024-02-26T10:47:54.512 INFO:tasks.workunit.client.0.smithi060.stdout:2/234: chown lb 1839 1 2024-02-26T10:47:54.512 INFO:tasks.workunit.client.0.smithi060.stdout:2/235: chown f0 4 1 2024-02-26T10:47:54.513 INFO:tasks.workunit.client.0.smithi060.stdout:6/262: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d4d 0 2024-02-26T10:47:54.514 INFO:tasks.workunit.client.0.smithi060.stdout:1/326: creat d3/d35/d53/d57/f73 x:0 0 0 2024-02-26T10:47:54.515 INFO:tasks.workunit.client.0.smithi060.stdout:2/236: rename dd/d11/c35 to dd/de/d41/d37/d38/c47 0 2024-02-26T10:47:54.516 INFO:tasks.workunit.client.0.smithi060.stdout:1/327: creat d3/d35/d53/f74 x:0 0 0 2024-02-26T10:47:54.516 INFO:tasks.workunit.client.0.smithi060.stdout:1/328: fsync d3/db/d2e/f4d 0 2024-02-26T10:47:54.519 INFO:tasks.workunit.client.0.smithi060.stdout:2/237: mknod dd/de/d41/c48 0 2024-02-26T10:47:54.529 INFO:tasks.workunit.client.0.smithi060.stdout:8/260: dread d9/fc [0,4194304] 0 2024-02-26T10:47:54.533 INFO:tasks.workunit.client.0.smithi060.stdout:8/261: unlink d9/d15/d29/f3c 0 2024-02-26T10:47:54.552 INFO:tasks.workunit.client.0.smithi060.stdout:8/262: rename d9/f10 to d9/d15/d29/d2f/f4d 0 2024-02-26T10:47:54.552 INFO:tasks.workunit.client.0.smithi060.stdout:8/263: stat d9/d15/c26 0 2024-02-26T10:47:54.552 INFO:tasks.workunit.client.0.smithi060.stdout:1/329: link d3/db/d1e/l26 d3/d35/l75 0 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:5/239: sync 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:5/240: dread - d0/de/d20/f56 zero size 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:9/263: dread d3/d9/d10/f11 [0,4194304] 0 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:9/264: stat d3/d15/d1b/f56 0 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:5/241: write d0/de/d20/f58 [455209,27190] 0 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:9/265: dread - d3/d9/d10/f4e zero size 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:5/242: truncate d0/de/d15/f5a 677088 0 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:8/264: truncate d9/f23 3614968 0 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:8/265: write d9/fc [3987644,50050] 0 2024-02-26T10:47:54.554 INFO:tasks.workunit.client.0.smithi060.stdout:8/266: readlink d9/d19/l22 0 2024-02-26T10:47:54.555 INFO:tasks.workunit.client.0.smithi060.stdout:8/267: truncate d9/d19/f4c 923689 0 2024-02-26T10:47:54.555 INFO:tasks.workunit.client.0.smithi060.stdout:8/268: chown d9/d19/f4c 9359319 1 2024-02-26T10:47:54.555 INFO:tasks.workunit.client.0.smithi060.stdout:0/269: dwrite d0/dd/f4e [0,4194304] 0 2024-02-26T10:47:54.556 INFO:tasks.workunit.client.0.smithi060.stdout:0/270: write d0/f55 [349986,73143] 0 2024-02-26T10:47:54.558 INFO:tasks.workunit.client.0.smithi060.stdout:0/271: truncate d0/dd/d39/f3f 315086 0 2024-02-26T10:47:54.561 INFO:tasks.workunit.client.0.smithi060.stdout:0/272: creat d0/dd/f5d x:0 0 0 2024-02-26T10:47:54.561 INFO:tasks.workunit.client.0.smithi060.stdout:0/273: fdatasync d0/d37/d44/f45 0 2024-02-26T10:47:54.569 INFO:tasks.workunit.client.0.smithi060.stdout:5/243: creat d0/de/f5d x:0 0 0 2024-02-26T10:47:54.572 INFO:tasks.workunit.client.0.smithi060.stdout:3/179: dwrite f3 [0,4194304] 0 2024-02-26T10:47:54.573 INFO:tasks.workunit.client.0.smithi060.stdout:3/180: write d5/de/d16/f1f [1011673,71483] 0 2024-02-26T10:47:54.573 INFO:tasks.workunit.client.0.smithi060.stdout:3/181: write d5/de/df/d11/f1c [596965,8328] 0 2024-02-26T10:47:54.573 INFO:tasks.workunit.client.0.smithi060.stdout:3/182: fsync d5/de/d16/d17/f1e 0 2024-02-26T10:47:54.575 INFO:tasks.workunit.client.0.smithi060.stdout:6/263: write d6/d7/d25/d1c/d1f/f35 [1623772,126911] 0 2024-02-26T10:47:54.575 INFO:tasks.workunit.client.0.smithi060.stdout:7/388: dwrite da/d15/f25 [0,4194304] 0 2024-02-26T10:47:54.575 INFO:tasks.workunit.client.0.smithi060.stdout:7/389: dread - da/d15/f74 zero size 2024-02-26T10:47:54.575 INFO:tasks.workunit.client.0.smithi060.stdout:4/287: dwrite d0/d27/f30 [0,4194304] 0 2024-02-26T10:47:54.577 INFO:tasks.workunit.client.0.smithi060.stdout:2/238: dwrite dd/de/f3f [0,4194304] 0 2024-02-26T10:47:54.580 INFO:tasks.workunit.client.0.smithi060.stdout:2/239: unlink dd/d11/f1c 0 2024-02-26T10:47:54.580 INFO:tasks.workunit.client.0.smithi060.stdout:2/240: fsync dd/de/d41/f36 0 2024-02-26T10:47:54.581 INFO:tasks.workunit.client.0.smithi060.stdout:2/241: symlink dd/de/d41/d37/l49 0 2024-02-26T10:47:54.584 INFO:tasks.workunit.client.0.smithi060.stdout:2/242: unlink dd/de/ff 0 2024-02-26T10:47:54.586 INFO:tasks.workunit.client.0.smithi060.stdout:2/243: link dd/l2d dd/d11/l4a 0 2024-02-26T10:47:54.591 INFO:tasks.workunit.client.0.smithi060.stdout:9/266: read d3/d9/d10/f37 [1518985,45733] 0 2024-02-26T10:47:54.594 INFO:tasks.workunit.client.0.smithi060.stdout:0/274: write d0/dd/d39/f3f [710950,98204] 0 2024-02-26T10:47:54.600 INFO:tasks.workunit.client.0.smithi060.stdout:8/269: dread d9/d19/f42 [0,4194304] 0 2024-02-26T10:47:54.600 INFO:tasks.workunit.client.0.smithi060.stdout:4/288: getdents d0/d27/d2f 0 2024-02-26T10:47:54.600 INFO:tasks.workunit.client.0.smithi060.stdout:4/289: fsync d0/da/d3b/d16/f19 0 2024-02-26T10:47:54.601 INFO:tasks.workunit.client.0.smithi060.stdout:0/275: truncate d0/f3 3864452 0 2024-02-26T10:47:54.601 INFO:tasks.workunit.client.0.smithi060.stdout:0/276: write d0/d37/f50 [1047859,86553] 0 2024-02-26T10:47:54.601 INFO:tasks.workunit.client.0.smithi060.stdout:6/264: link d6/d7/d25/l26 d6/d7/d25/d1c/l4e 0 2024-02-26T10:47:54.601 INFO:tasks.workunit.client.0.smithi060.stdout:3/183: creat d5/de/d16/d17/f30 x:0 0 0 2024-02-26T10:47:54.604 INFO:tasks.workunit.client.0.smithi060.stdout:7/390: dread da/d15/d21/d32/d4e/f5c [0,4194304] 0 2024-02-26T10:47:54.604 INFO:tasks.workunit.client.0.smithi060.stdout:7/391: write da/d66/f75 [678101,72778] 0 2024-02-26T10:47:54.604 INFO:tasks.workunit.client.0.smithi060.stdout:7/392: fdatasync da/d15/d21/f79 0 2024-02-26T10:47:54.604 INFO:tasks.workunit.client.0.smithi060.stdout:6/265: unlink d6/d7/d25/d1c/l4e 0 2024-02-26T10:47:54.604 INFO:tasks.workunit.client.0.smithi060.stdout:3/184: mknod d5/de/c31 0 2024-02-26T10:47:54.604 INFO:tasks.workunit.client.0.smithi060.stdout:6/266: write d6/f15 [5069277,94347] 0 2024-02-26T10:47:54.606 INFO:tasks.workunit.client.0.smithi060.stdout:3/185: mknod d5/de/df/c32 0 2024-02-26T10:47:54.610 INFO:tasks.workunit.client.0.smithi060.stdout:7/393: rename da/d15/d21/d32/d4e to da/d15/d88 0 2024-02-26T10:47:54.610 INFO:tasks.workunit.client.0.smithi060.stdout:3/186: unlink d5/de/d16/f1f 0 2024-02-26T10:47:54.615 INFO:tasks.workunit.client.0.smithi060.stdout:6/267: rename d6/d7/d25/d1c/d1f/d23/c30 to d6/d7/d47/c4f 0 2024-02-26T10:47:54.615 INFO:tasks.workunit.client.0.smithi060.stdout:6/268: chown d6/d7/d25/d1c 1208008879 1 2024-02-26T10:47:54.618 INFO:tasks.workunit.client.0.smithi060.stdout:5/244: dread d0/f5 [0,4194304] 0 2024-02-26T10:47:54.618 INFO:tasks.workunit.client.0.smithi060.stdout:5/245: readlink d0/l7 0 2024-02-26T10:47:54.618 INFO:tasks.workunit.client.0.smithi060.stdout:5/246: stat d0/d33/l37 0 2024-02-26T10:47:54.662 INFO:tasks.workunit.client.0.smithi060.stdout:8/270: write f3 [6917574,87873] 0 2024-02-26T10:47:54.663 INFO:tasks.workunit.client.0.smithi060.stdout:1/330: dwrite d3/d35/d5e/f66 [0,4194304] 0 2024-02-26T10:47:54.664 INFO:tasks.workunit.client.0.smithi060.stdout:1/331: fdatasync d3/db/d1e/f3b 0 2024-02-26T10:47:54.664 INFO:tasks.workunit.client.0.smithi060.stdout:1/332: fsync d3/d35/d53/d57/f71 0 2024-02-26T10:47:54.664 INFO:tasks.workunit.client.0.smithi060.stdout:3/187: write d5/de/f14 [2856375,120853] 0 2024-02-26T10:47:54.666 INFO:tasks.workunit.client.0.smithi060.stdout:2/244: dwrite dd/de/d41/d37/d38/f45 [0,4194304] 0 2024-02-26T10:47:54.669 INFO:tasks.workunit.client.0.smithi060.stdout:2/245: symlink dd/de/d41/d37/d38/l4b 0 2024-02-26T10:47:54.669 INFO:tasks.workunit.client.0.smithi060.stdout:8/271: rmdir d9/d4a 0 2024-02-26T10:47:54.670 INFO:tasks.workunit.client.0.smithi060.stdout:1/333: unlink d3/d35/f5a 0 2024-02-26T10:47:54.671 INFO:tasks.workunit.client.0.smithi060.stdout:2/246: mkdir dd/de/d41/d37/d4c 0 2024-02-26T10:47:54.671 INFO:tasks.workunit.client.0.smithi060.stdout:2/247: fsync dd/de/d41/f31 0 2024-02-26T10:47:54.671 INFO:tasks.workunit.client.0.smithi060.stdout:2/248: fdatasync f0 0 2024-02-26T10:47:54.672 INFO:tasks.workunit.client.0.smithi060.stdout:1/334: truncate d3/d35/d53/d57/f73 920530 0 2024-02-26T10:47:54.672 INFO:tasks.workunit.client.0.smithi060.stdout:1/335: mkdir d3/d35/d76 0 2024-02-26T10:47:54.672 INFO:tasks.workunit.client.0.smithi060.stdout:2/249: symlink dd/de/l4d 0 2024-02-26T10:47:54.675 INFO:tasks.workunit.client.0.smithi060.stdout:2/250: creat dd/d11/f4e x:0 0 0 2024-02-26T10:47:54.683 INFO:tasks.workunit.client.0.smithi060.stdout:3/188: truncate d5/fa 3626425 0 2024-02-26T10:47:54.685 INFO:tasks.workunit.client.0.smithi060.stdout:1/336: truncate d3/f9 1343265 0 2024-02-26T10:47:54.687 INFO:tasks.workunit.client.0.smithi060.stdout:3/189: creat d5/de/df/f33 x:0 0 0 2024-02-26T10:47:54.687 INFO:tasks.workunit.client.0.smithi060.stdout:3/190: chown d5/l2e 143 1 2024-02-26T10:47:54.689 INFO:tasks.workunit.client.0.smithi060.stdout:3/191: creat d5/de/df/d11/f34 x:0 0 0 2024-02-26T10:47:54.689 INFO:tasks.workunit.client.0.smithi060.stdout:3/192: chown d5/de/df/d11/c18 9 1 2024-02-26T10:47:54.690 INFO:tasks.workunit.client.0.smithi060.stdout:7/394: dread da/d15/d21/d4b/d67/f7a [0,4194304] 0 2024-02-26T10:47:54.690 INFO:tasks.workunit.client.0.smithi060.stdout:7/395: dread - da/d15/d88/d3b/f54 zero size 2024-02-26T10:47:54.690 INFO:tasks.workunit.client.0.smithi060.stdout:7/396: read da/d15/d21/d2f/d37/f38 [1392935,90422] 0 2024-02-26T10:47:54.690 INFO:tasks.workunit.client.0.smithi060.stdout:7/397: dread - da/d15/d21/d4b/d67/f77 zero size 2024-02-26T10:47:54.691 INFO:tasks.workunit.client.0.smithi060.stdout:3/193: mknod d5/de/d16/c35 0 2024-02-26T10:47:54.691 INFO:tasks.workunit.client.0.smithi060.stdout:3/194: chown d5/l8 18344 1 2024-02-26T10:47:54.692 INFO:tasks.workunit.client.0.smithi060.stdout:2/251: read f3 [1018232,130092] 0 2024-02-26T10:47:54.692 INFO:tasks.workunit.client.0.smithi060.stdout:7/398: truncate da/d15/d21/d32/f5f 1149547 0 2024-02-26T10:47:54.692 INFO:tasks.workunit.client.0.smithi060.stdout:7/399: dread - da/d5a/f81 zero size 2024-02-26T10:47:54.694 INFO:tasks.workunit.client.0.smithi060.stdout:7/400: rmdir da/d15 39 2024-02-26T10:47:54.708 INFO:tasks.workunit.client.0.smithi060.stdout:2/252: write dd/f1d [2405357,47604] 0 2024-02-26T10:47:54.708 INFO:tasks.workunit.client.0.smithi060.stdout:5/247: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:54.709 INFO:tasks.workunit.client.0.smithi060.stdout:5/248: truncate d0/de/d20/f31 1576007 0 2024-02-26T10:47:54.709 INFO:tasks.workunit.client.0.smithi060.stdout:8/272: truncate f3 1417374 0 2024-02-26T10:47:54.710 INFO:tasks.workunit.client.0.smithi060.stdout:8/273: readlink d9/d15/d3e/l41 0 2024-02-26T10:47:54.710 INFO:tasks.workunit.client.0.smithi060.stdout:8/274: write d9/d15/d40/f45 [67314,66056] 0 2024-02-26T10:47:54.710 INFO:tasks.workunit.client.0.smithi060.stdout:2/253: rename dd/d11/l4a to dd/de/d41/d42/l4f 0 2024-02-26T10:47:54.710 INFO:tasks.workunit.client.0.smithi060.stdout:8/275: write d9/d15/f1c [5117192,55666] 0 2024-02-26T10:47:54.711 INFO:tasks.workunit.client.0.smithi060.stdout:4/290: dwrite d0/da/d3b/d16/f18 [0,4194304] 0 2024-02-26T10:47:54.711 INFO:tasks.workunit.client.0.smithi060.stdout:4/291: chown d0/da/d3b/d16/l1d 2983024 1 2024-02-26T10:47:54.711 INFO:tasks.workunit.client.0.smithi060.stdout:5/249: symlink d0/de/d20/d32/d4c/l5e 0 2024-02-26T10:47:54.713 INFO:tasks.workunit.client.0.smithi060.stdout:2/254: symlink dd/de/d41/d42/l50 0 2024-02-26T10:47:54.713 INFO:tasks.workunit.client.0.smithi060.stdout:6/269: dwrite d6/f11 [0,4194304] 0 2024-02-26T10:47:54.713 INFO:tasks.workunit.client.0.smithi060.stdout:6/270: truncate d6/d7/d25/f4b 710486 0 2024-02-26T10:47:54.713 INFO:tasks.workunit.client.0.smithi060.stdout:5/250: creat d0/d33/f5f x:0 0 0 2024-02-26T10:47:54.715 INFO:tasks.workunit.client.0.smithi060.stdout:8/276: symlink d9/d15/d29/l4e 0 2024-02-26T10:47:54.716 INFO:tasks.workunit.client.0.smithi060.stdout:5/251: mkdir d0/de/d20/d32/d4d/d60 0 2024-02-26T10:47:54.716 INFO:tasks.workunit.client.0.smithi060.stdout:4/292: symlink d0/d27/l56 0 2024-02-26T10:47:54.716 INFO:tasks.workunit.client.0.smithi060.stdout:4/293: readlink d0/da/d3b/d16/l1d 0 2024-02-26T10:47:54.717 INFO:tasks.workunit.client.0.smithi060.stdout:4/294: truncate d0/da/f4c 63189 0 2024-02-26T10:47:54.717 INFO:tasks.workunit.client.0.smithi060.stdout:0/277: dwrite d0/f3 [4194304,4194304] 0 2024-02-26T10:47:54.728 INFO:tasks.workunit.client.0.smithi060.stdout:2/255: rmdir dd/de 39 2024-02-26T10:47:54.732 INFO:tasks.workunit.client.0.smithi060.stdout:5/252: write d0/fd [1180296,65117] 0 2024-02-26T10:47:54.732 INFO:tasks.workunit.client.0.smithi060.stdout:5/253: fsync d0/d33/f3e 0 2024-02-26T10:47:54.732 INFO:tasks.workunit.client.0.smithi060.stdout:5/254: dread - d0/d4b/f57 zero size 2024-02-26T10:47:54.734 INFO:tasks.workunit.client.0.smithi060.stdout:5/255: symlink d0/de/d15/d26/l61 0 2024-02-26T10:47:54.737 INFO:tasks.workunit.client.0.smithi060.stdout:2/256: fdatasync dd/de/f3f 0 2024-02-26T10:47:54.738 INFO:tasks.workunit.client.0.smithi060.stdout:5/256: rmdir d0/de 39 2024-02-26T10:47:54.740 INFO:tasks.workunit.client.0.smithi060.stdout:5/257: creat d0/d33/f62 x:0 0 0 2024-02-26T10:47:54.740 INFO:tasks.workunit.client.0.smithi060.stdout:5/258: write d0/fd [44887,14949] 0 2024-02-26T10:47:54.743 INFO:tasks.workunit.client.0.smithi060.stdout:3/195: dread d5/fa [0,4194304] 0 2024-02-26T10:47:54.747 INFO:tasks.workunit.client.0.smithi060.stdout:7/401: dread da/d15/d21/f28 [0,4194304] 0 2024-02-26T10:47:54.748 INFO:tasks.workunit.client.0.smithi060.stdout:6/271: dread d6/d7/f2a [0,4194304] 0 2024-02-26T10:47:54.750 INFO:tasks.workunit.client.0.smithi060.stdout:7/402: truncate da/d15/d21/d32/f33 1218870 0 2024-02-26T10:47:54.750 INFO:tasks.workunit.client.0.smithi060.stdout:2/257: symlink dd/d11/l51 0 2024-02-26T10:47:54.756 INFO:tasks.workunit.client.0.smithi060.stdout:1/337: dwrite d3/db/d1e/f3b [0,4194304] 0 2024-02-26T10:47:54.756 INFO:tasks.workunit.client.0.smithi060.stdout:6/272: unlink d6/f38 0 2024-02-26T10:47:54.759 INFO:tasks.workunit.client.0.smithi060.stdout:7/403: read da/d15/f2e [340619,112390] 0 2024-02-26T10:47:54.759 INFO:tasks.workunit.client.0.smithi060.stdout:7/404: fdatasync da/d5a/f82 0 2024-02-26T10:47:54.761 INFO:tasks.workunit.client.0.smithi060.stdout:6/273: truncate d6/f1d 247831 0 2024-02-26T10:47:54.762 INFO:tasks.workunit.client.0.smithi060.stdout:8/277: rename d9/d19/l35 to d9/d15/d40/l4f 0 2024-02-26T10:47:54.762 INFO:tasks.workunit.client.0.smithi060.stdout:8/278: readlink d9/d19/l22 0 2024-02-26T10:47:54.762 INFO:tasks.workunit.client.0.smithi060.stdout:8/279: stat d9/d15/c2e 0 2024-02-26T10:47:54.762 INFO:tasks.workunit.client.0.smithi060.stdout:8/280: fdatasync d9/d15/d40/f45 0 2024-02-26T10:47:54.762 INFO:tasks.workunit.client.0.smithi060.stdout:8/281: chown d9/d19/f42 2040206210 1 2024-02-26T10:47:54.763 INFO:tasks.workunit.client.0.smithi060.stdout:6/274: creat d6/d7/d25/d1c/d1f/d23/f50 x:0 0 0 2024-02-26T10:47:54.764 INFO:tasks.workunit.client.0.smithi060.stdout:8/282: mkdir d9/d19/d50 0 2024-02-26T10:47:54.764 INFO:tasks.workunit.client.0.smithi060.stdout:7/405: getdents da/d15/d21/d32 0 2024-02-26T10:47:54.764 INFO:tasks.workunit.client.0.smithi060.stdout:7/406: readlink da/d15/d88/d39/l80 0 2024-02-26T10:47:54.770 INFO:tasks.workunit.client.0.smithi060.stdout:6/275: getdents d6/d7/d47 0 2024-02-26T10:47:54.785 INFO:tasks.workunit.client.0.smithi060.stdout:4/295: dwrite d0/d27/f4d [0,4194304] 0 2024-02-26T10:47:54.787 INFO:tasks.workunit.client.0.smithi060.stdout:4/296: mkdir d0/da/d57 0 2024-02-26T10:47:54.787 INFO:tasks.workunit.client.0.smithi060.stdout:4/297: chown d0/d7/d29/f45 10763 1 2024-02-26T10:47:54.788 INFO:tasks.workunit.client.0.smithi060.stdout:4/298: creat d0/da/d57/f58 x:0 0 0 2024-02-26T10:47:54.792 INFO:tasks.workunit.client.0.smithi060.stdout:4/299: dread - d0/da/d3b/d16/f39 zero size 2024-02-26T10:47:54.794 INFO:tasks.workunit.client.0.smithi060.stdout:5/259: dwrite d0/f1d [4194304,4194304] 0 2024-02-26T10:47:54.799 INFO:tasks.workunit.client.0.smithi060.stdout:3/196: dwrite d5/de/d16/d17/f30 [0,4194304] 0 2024-02-26T10:47:54.799 INFO:tasks.workunit.client.0.smithi060.stdout:6/276: rmdir d6/d7/d25 39 2024-02-26T10:47:54.799 INFO:tasks.workunit.client.0.smithi060.stdout:6/277: chown d6/d7/d25/d1c/d1f/d23/d2b 4995791 1 2024-02-26T10:47:54.801 INFO:tasks.workunit.client.0.smithi060.stdout:6/278: write d6/d7/d25/d1c/d1f/d23/d2b/f3b [1819660,24211] 0 2024-02-26T10:47:54.801 INFO:tasks.workunit.client.0.smithi060.stdout:6/279: write d6/fe [735889,52637] 0 2024-02-26T10:47:54.811 INFO:tasks.workunit.client.0.smithi060.stdout:6/280: creat d6/d7/d25/f51 x:0 0 0 2024-02-26T10:47:54.813 INFO:tasks.workunit.client.0.smithi060.stdout:6/281: creat d6/d7/f52 x:0 0 0 2024-02-26T10:47:54.813 INFO:tasks.workunit.client.0.smithi060.stdout:6/282: write d6/d7/d25/d1c/d1f/f24 [1390330,21318] 0 2024-02-26T10:47:54.814 INFO:tasks.workunit.client.0.smithi060.stdout:6/283: creat d6/d7/f53 x:0 0 0 2024-02-26T10:47:54.814 INFO:tasks.workunit.client.0.smithi060.stdout:6/284: chown d6/d7/d25/f51 14591238 1 2024-02-26T10:47:54.817 INFO:tasks.workunit.client.0.smithi060.stdout:2/258: dread f3 [0,4194304] 0 2024-02-26T10:47:54.817 INFO:tasks.workunit.client.0.smithi060.stdout:6/285: mknod d6/d7/d25/d1c/d1f/d23/d2b/c54 0 2024-02-26T10:47:54.817 INFO:tasks.workunit.client.0.smithi060.stdout:2/259: rmdir dd/de/d41/d42 39 2024-02-26T10:47:54.818 INFO:tasks.workunit.client.0.smithi060.stdout:3/197: truncate d5/de/d16/d17/f30 3655902 0 2024-02-26T10:47:54.819 INFO:tasks.workunit.client.0.smithi060.stdout:3/198: creat d5/de/df/d11/f36 x:0 0 0 2024-02-26T10:47:54.821 INFO:tasks.workunit.client.0.smithi060.stdout:3/199: mkdir d5/d37 0 2024-02-26T10:47:54.821 INFO:tasks.workunit.client.0.smithi060.stdout:3/200: write d5/de/df/f33 [886955,40593] 0 2024-02-26T10:47:54.822 INFO:tasks.workunit.client.0.smithi060.stdout:3/201: unlink d5/de/d16/d17/f1d 0 2024-02-26T10:47:54.834 INFO:tasks.workunit.client.0.smithi060.stdout:1/338: dwrite d3/d35/d5e/f70 [0,4194304] 0 2024-02-26T10:47:54.838 INFO:tasks.workunit.client.0.smithi060.stdout:6/286: fsync d6/d7/d25/d1c/d1f/f24 0 2024-02-26T10:47:54.842 INFO:tasks.workunit.client.0.smithi060.stdout:6/287: unlink d6/d7/d25/d1c/d1f/d23/d2b/c54 0 2024-02-26T10:47:54.842 INFO:tasks.workunit.client.0.smithi060.stdout:6/288: write d6/d7/d25/d1c/d1f/d23/d28/f4c [1010187,8957] 0 2024-02-26T10:47:54.843 INFO:tasks.workunit.client.0.smithi060.stdout:4/300: dwrite d0/d27/d2f/f55 [0,4194304] 0 2024-02-26T10:47:54.844 INFO:tasks.workunit.client.0.smithi060.stdout:4/301: creat d0/da/d3b/f59 x:0 0 0 2024-02-26T10:47:54.846 INFO:tasks.workunit.client.0.smithi060.stdout:2/260: dread dd/d11/f25 [0,4194304] 0 2024-02-26T10:47:54.857 INFO:tasks.workunit.client.0.smithi060.stdout:2/261: write dd/de/f3d [5071555,103663] 0 2024-02-26T10:47:54.873 INFO:tasks.workunit.client.0.smithi060.stdout:7/407: dwrite da/d15/d21/d32/f47 [0,4194304] 0 2024-02-26T10:47:54.873 INFO:tasks.workunit.client.0.smithi060.stdout:8/283: dwrite d9/d15/f1c [0,4194304] 0 2024-02-26T10:47:54.879 INFO:tasks.workunit.client.0.smithi060.stdout:7/408: mknod da/d15/d88/d3b/c89 0 2024-02-26T10:47:54.880 INFO:tasks.workunit.client.0.smithi060.stdout:7/409: rmdir da/d15/d21/d41 39 2024-02-26T10:47:54.898 INFO:tasks.workunit.client.0.smithi060.stdout:4/302: write d0/da/d3b/f34 [475575,60598] 0 2024-02-26T10:47:54.899 INFO:tasks.workunit.client.0.smithi060.stdout:4/303: chown d0/da/d3b/d16/f18 63665551 1 2024-02-26T10:47:54.899 INFO:tasks.workunit.client.0.smithi060.stdout:5/260: dwrite d0/de/d15/d26/f45 [0,4194304] 0 2024-02-26T10:47:54.900 INFO:tasks.workunit.client.0.smithi060.stdout:5/261: truncate d0/de/d15/f16 2546804 0 2024-02-26T10:47:54.901 INFO:tasks.workunit.client.0.smithi060.stdout:4/304: unlink d0/d7/l13 0 2024-02-26T10:47:54.901 INFO:tasks.workunit.client.0.smithi060.stdout:4/305: stat d0/d7/d29/f32 0 2024-02-26T10:47:54.901 INFO:tasks.workunit.client.0.smithi060.stdout:5/262: readlink d0/l4a 0 2024-02-26T10:47:54.901 INFO:tasks.workunit.client.0.smithi060.stdout:5/263: truncate d0/d4b/f51 986982 0 2024-02-26T10:47:54.902 INFO:tasks.workunit.client.0.smithi060.stdout:4/306: creat d0/d7/f5a x:0 0 0 2024-02-26T10:47:54.902 INFO:tasks.workunit.client.0.smithi060.stdout:4/307: stat d0/d27/f4d 0 2024-02-26T10:47:54.902 INFO:tasks.workunit.client.0.smithi060.stdout:4/308: write d0/d7/f3e [1939687,55438] 0 2024-02-26T10:47:54.902 INFO:tasks.workunit.client.0.smithi060.stdout:4/309: write d0/f4b [98945,117303] 0 2024-02-26T10:47:54.904 INFO:tasks.workunit.client.0.smithi060.stdout:4/310: creat d0/d7/d29/f5b x:0 0 0 2024-02-26T10:47:54.904 INFO:tasks.workunit.client.0.smithi060.stdout:4/311: fdatasync d0/da/f53 0 2024-02-26T10:47:54.917 INFO:tasks.workunit.client.0.smithi060.stdout:1/339: dwrite d3/d35/f65 [0,4194304] 0 2024-02-26T10:47:54.917 INFO:tasks.workunit.client.0.smithi060.stdout:9/267: sync 2024-02-26T10:47:54.917 INFO:tasks.workunit.client.0.smithi060.stdout:1/340: write d3/db/d1e/d39/f48 [437033,98509] 0 2024-02-26T10:47:54.918 INFO:tasks.workunit.client.0.smithi060.stdout:4/312: rmdir d0/d7 39 2024-02-26T10:47:54.918 INFO:tasks.workunit.client.0.smithi060.stdout:4/313: truncate d0/d7/f40 470915 0 2024-02-26T10:47:54.919 INFO:tasks.workunit.client.0.smithi060.stdout:9/268: mkdir d3/d1e/d57 0 2024-02-26T10:47:54.920 INFO:tasks.workunit.client.0.smithi060.stdout:9/269: getdents d3/d52 0 2024-02-26T10:47:54.920 INFO:tasks.workunit.client.0.smithi060.stdout:1/341: creat d3/d35/d53/d68/f77 x:0 0 0 2024-02-26T10:47:54.921 INFO:tasks.workunit.client.0.smithi060.stdout:1/342: mkdir d3/db/d1e/d6d/d78 0 2024-02-26T10:47:54.921 INFO:tasks.workunit.client.0.smithi060.stdout:4/314: getdents d0/d7 0 2024-02-26T10:47:54.930 INFO:tasks.workunit.client.0.smithi060.stdout:6/289: dwrite d6/d7/d25/d1c/f2c [0,4194304] 0 2024-02-26T10:47:54.940 INFO:tasks.workunit.client.0.smithi060.stdout:6/290: creat d6/d7/d47/f55 x:0 0 0 2024-02-26T10:47:54.940 INFO:tasks.workunit.client.0.smithi060.stdout:6/291: mkdir d6/d7/d2e/d56 0 2024-02-26T10:47:54.948 INFO:tasks.workunit.client.0.smithi060.stdout:6/292: mknod d6/d7/d2e/c57 0 2024-02-26T10:47:54.949 INFO:tasks.workunit.client.0.smithi060.stdout:2/262: dwrite dd/de/d41/f1a [0,4194304] 0 2024-02-26T10:47:54.949 INFO:tasks.workunit.client.0.smithi060.stdout:2/263: fsync f4 0 2024-02-26T10:47:54.954 INFO:tasks.workunit.client.0.smithi060.stdout:7/410: dwrite da/f60 [0,4194304] 0 2024-02-26T10:47:54.956 INFO:tasks.workunit.client.0.smithi060.stdout:9/270: write d3/d9/d10/f2c [1983763,127281] 0 2024-02-26T10:47:54.957 INFO:tasks.workunit.client.0.smithi060.stdout:9/271: write d3/d9/d10/f4f [157400,47963] 0 2024-02-26T10:47:54.957 INFO:tasks.workunit.client.0.smithi060.stdout:7/411: creat da/d15/d88/d3b/f8a x:0 0 0 2024-02-26T10:47:54.970 INFO:tasks.workunit.client.0.smithi060.stdout:4/315: dread d0/da/d3b/f1a [0,4194304] 0 2024-02-26T10:47:54.970 INFO:tasks.workunit.client.0.smithi060.stdout:4/316: write d0/da/d3b/d16/f51 [607473,94867] 0 2024-02-26T10:47:54.970 INFO:tasks.workunit.client.0.smithi060.stdout:4/317: fdatasync d0/da/d3b/d16/f48 0 2024-02-26T10:47:54.970 INFO:tasks.workunit.client.0.smithi060.stdout:4/318: chown d0/da/d3b/d16/f18 59480 1 2024-02-26T10:47:54.978 INFO:tasks.workunit.client.0.smithi060.stdout:1/343: dwrite d3/f15 [0,4194304] 0 2024-02-26T10:47:54.978 INFO:tasks.workunit.client.0.smithi060.stdout:1/344: fdatasync d3/d35/d5e/f66 0 2024-02-26T10:47:54.979 INFO:tasks.workunit.client.0.smithi060.stdout:1/345: write d3/f9 [2366318,60555] 0 2024-02-26T10:47:54.987 INFO:tasks.workunit.client.0.smithi060.stdout:4/319: link d0/da/d3b/c1f d0/d7/c5c 0 2024-02-26T10:47:54.987 INFO:tasks.workunit.client.0.smithi060.stdout:4/320: readlink d0/da/d3b/d1b/l1c 0 2024-02-26T10:47:54.987 INFO:tasks.workunit.client.0.smithi060.stdout:4/321: chown d0/f4a 2 1 2024-02-26T10:47:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.006+0000 7f114a200640 1 -- 172.21.15.60:0/3697321506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11441046b0 msgr2=0x7f1144104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.006+0000 7f114a200640 1 --2- 172.21.15.60:0/3697321506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11441046b0 0x7f1144104b30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1130009900 tx=0x7f113002bc40 comp rx=0 tx=0).stop 2024-02-26T10:47:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.006+0000 7f114a200640 1 -- 172.21.15.60:0/3697321506 shutdown_connections 2024-02-26T10:47:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.006+0000 7f114a200640 1 --2- 172.21.15.60:0/3697321506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11441046b0 0x7f1144104b30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.006+0000 7f114a200640 1 --2- 172.21.15.60:0/3697321506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1144105070 0x7f114410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.006+0000 7f114a200640 1 --2- 172.21.15.60:0/3697321506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11441034c0 0x7f11441038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.006+0000 7f114a200640 1 -- 172.21.15.60:0/3697321506 >> 172.21.15.60:0/3697321506 conn(0x7f11440fea50 msgr2=0x7f1144100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.006+0000 7f114a200640 1 -- 172.21.15.60:0/3697321506 shutdown_connections 2024-02-26T10:47:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.006+0000 7f114a200640 1 -- 172.21.15.60:0/3697321506 wait complete. 2024-02-26T10:47:55.001 INFO:tasks.workunit.client.0.smithi060.stdout:7/412: dwrite f7 [0,4194304] 0 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.008+0000 7f114a200640 1 Processor -- start 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.008+0000 7f114a200640 1 -- start start 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f114a200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11441034c0 0x7f114419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f114a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11441046b0 0x7f114419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f114a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1144105070 0x7f11441a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f114a200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114410f200 con 0x7f11441034c0 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f114a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114410f080 con 0x7f1144105070 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f114a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114410f380 con 0x7f11441046b0 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f1143e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1144105070 0x7f11441a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f1142a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11441046b0 0x7f114419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f1143e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1144105070 0x7f11441a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:50988/0 (socket says 172.21.15.60:50988) 2024-02-26T10:47:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f1143e00640 1 -- 172.21.15.60:0/1669312473 learned_addr learned my addr 172.21.15.60:0/1669312473 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.009+0000 7f1143400640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11441034c0 0x7f114419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.010+0000 7f1143e00640 1 -- 172.21.15.60:0/1669312473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11441046b0 msgr2=0x7f114419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.010+0000 7f1143e00640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11441046b0 0x7f114419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.010+0000 7f1143e00640 1 -- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11441034c0 msgr2=0x7f114419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.010+0000 7f1143e00640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11441034c0 0x7f114419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.010+0000 7f1143e00640 1 -- 172.21.15.60:0/1669312473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114406adb0 con 0x7f1144105070 2024-02-26T10:47:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.010+0000 7f1142a00640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11441046b0 0x7f114419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:55.005 INFO:tasks.workunit.client.0.smithi060.stdout:7/413: mknod da/d15/d21/d41/d6c/c8b 0 2024-02-26T10:47:55.005 INFO:tasks.workunit.client.0.smithi060.stdout:1/346: dread d3/db/d2e/f30 [0,4194304] 0 2024-02-26T10:47:55.006 INFO:tasks.workunit.client.0.smithi060.stdout:1/347: creat d3/db/d1e/d38/f79 x:0 0 0 2024-02-26T10:47:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.013+0000 7f1143e00640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1144105070 0x7f11441a5180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f113c00eaa0 tx=0x7f113c00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:55.007 INFO:tasks.workunit.client.0.smithi060.stdout:6/293: dwrite d6/d7/f52 [0,4194304] 0 2024-02-26T10:47:55.007 INFO:tasks.workunit.client.0.smithi060.stdout:1/348: chown d3/db/c21 821989 1 2024-02-26T10:47:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.013+0000 7f113be00640 1 -- 172.21.15.60:0/1669312473 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c004d20 con 0x7f1144105070 2024-02-26T10:47:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.014+0000 7f113be00640 1 -- 172.21.15.60:0/1669312473 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f113c005020 con 0x7f1144105070 2024-02-26T10:47:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.014+0000 7f113be00640 1 -- 172.21.15.60:0/1669312473 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c0116d0 con 0x7f1144105070 2024-02-26T10:47:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.015+0000 7f114a200640 1 -- 172.21.15.60:0/1669312473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114406b0a0 con 0x7f1144105070 2024-02-26T10:47:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.015+0000 7f114a200640 1 -- 172.21.15.60:0/1669312473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114406b570 con 0x7f1144105070 2024-02-26T10:47:55.011 INFO:tasks.workunit.client.0.smithi060.stdout:6/294: creat d6/d7/d25/d1c/d1f/d23/d2b/d4d/f58 x:0 0 0 2024-02-26T10:47:55.014 INFO:tasks.workunit.client.0.smithi060.stdout:1/349: truncate d3/d35/d5e/f70 318666 0 2024-02-26T10:47:55.014 INFO:tasks.workunit.client.0.smithi060.stdout:7/414: rmdir da 39 2024-02-26T10:47:55.014 INFO:tasks.workunit.client.0.smithi060.stdout:6/295: creat d6/d7/d25/f59 x:0 0 0 2024-02-26T10:47:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.020+0000 7f114d211740 1 Processor -- start 2024-02-26T10:47:55.016 INFO:tasks.workunit.client.0.smithi060.stdout:9/272: dwrite d3/f16 [0,4194304] 0 2024-02-26T10:47:55.018 INFO:tasks.workunit.client.0.smithi060.stdout:1/350: rmdir d3/db/d2e 39 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.021+0000 7f113be00640 1 -- 172.21.15.60:0/1669312473 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f113c0051d0 con 0x7f1144105070 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.021+0000 7f114d211740 1 -- start start 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f114d211740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69800820 0x55df69800c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f114d211740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69801210 0x55df69805d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f114d211740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 0x55df69808810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f114d211740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df69766340 con 0x55df69806380 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f114d211740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df6970f520 con 0x55df69800820 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f1142a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69800820 0x55df69800c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f1142a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69800820 0x55df69800c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:50994/0 (socket says 172.21.15.60:50994) 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f1142a00640 1 -- 172.21.15.60:0/640018672 learned_addr learned my addr 172.21.15.60:0/640018672 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f114d211740 1 -- 172.21.15.60:0/640018672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df697623e0 con 0x55df69801210 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.022+0000 7f1142a00640 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 0x55df69808810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.023+0000 7f1143400640 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69801210 0x55df69805d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.023+0000 7f1143400640 1 -- 172.21.15.60:0/640018672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69800820 msgr2=0x55df69800c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.023+0000 7f1143400640 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69800820 0x55df69800c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.023+0000 7f1143400640 1 -- 172.21.15.60:0/640018672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 msgr2=0x55df69808810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.023+0000 7f1143400640 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 0x55df69808810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.023+0000 7f1143400640 1 -- 172.21.15.60:0/640018672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df69723390 con 0x55df69801210 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.024+0000 7f1142a00640 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 0x55df69808810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.024+0000 7f1143400640 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69801210 0x55df69805d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1134009990 tx=0x7f1134009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe8e2ef457486e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.024+0000 7f112fe00640 1 -- 172.21.15.60:0/640018672 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134039070 con 0x55df69801210 2024-02-26T10:47:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.024+0000 7f112fe00640 1 -- 172.21.15.60:0/640018672 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1134035480 con 0x55df69801210 2024-02-26T10:47:55.021 INFO:tasks.workunit.client.0.smithi060.stdout:1/351: symlink d3/d35/d76/l7a 0 2024-02-26T10:47:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.026+0000 7f113be00640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11280a1ea0 0x7f11280a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.021 INFO:tasks.workunit.client.0.smithi060.stdout:6/296: truncate d6/d7/d25/f2d 858241 0 2024-02-26T10:47:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.027+0000 7f113be00640 1 -- 172.21.15.60:0/1669312473 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f113c0ce530 con 0x7f1144105070 2024-02-26T10:47:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.027+0000 7f114d211740 1 -- 172.21.15.60:0/640018672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69801210 msgr2=0x55df69805d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.027+0000 7f114d211740 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69801210 0x55df69805d10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1134009990 tx=0x7f1134009e50 comp rx=0 tx=0).stop 2024-02-26T10:47:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.028+0000 7f114d211740 1 -- 172.21.15.60:0/640018672 shutdown_connections 2024-02-26T10:47:55.022 INFO:tasks.workunit.client.0.smithi060.stdout:6/297: symlink d6/l5a 0 2024-02-26T10:47:55.023 INFO:tasks.workunit.client.0.smithi060.stdout:1/352: symlink d3/db/d1e/d38/d6c/l7b 0 2024-02-26T10:47:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.028+0000 7f114d211740 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 0x55df69808810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.028+0000 7f114d211740 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69801210 0x55df69805d10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.028+0000 7f114d211740 1 --2- 172.21.15.60:0/640018672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69800820 0x55df69800c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.028+0000 7f114d211740 1 -- 172.21.15.60:0/640018672 >> 172.21.15.60:0/640018672 conn(0x55df697fbd90 msgr2=0x55df697fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.029+0000 7f1142a00640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11280a1ea0 0x7f11280a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.029+0000 7f114d211740 1 -- 172.21.15.60:0/640018672 shutdown_connections 2024-02-26T10:47:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.030+0000 7f1142a00640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11280a1ea0 0x7f11280a4360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1130006010 tx=0x7f113002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:55.024 INFO:tasks.workunit.client.0.smithi060.stdout:6/298: mknod d6/d7/d25/d1c/d1f/c5b 0 2024-02-26T10:47:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.030+0000 7f114d211740 1 -- 172.21.15.60:0/640018672 wait complete. 2024-02-26T10:47:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.031+0000 7f114d211740 1 -- start start 2024-02-26T10:47:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:6/299: rename d6/d7/d25/f51 to d6/d7/d25/f5c 0 2024-02-26T10:47:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.032+0000 7f114d211740 1 Processor -- start 2024-02-26T10:47:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:6/300: chown d6/d7/d25/f2d 3958 1 2024-02-26T10:47:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:6/301: write d6/d7/d25/d1c/d1f/d23/d2b/d4d/f58 [977729,53677] 0 2024-02-26T10:47:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:6/302: write d6/d7/d25/d1c/d1f/d23/d28/f33 [502217,2499] 0 2024-02-26T10:47:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.033+0000 7f114d211740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df6980ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f1143400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df6980ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f114d211740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69801210 0x55df6980b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f1143400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df6980ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:32904/0 (socket says 172.21.15.60:32904) 2024-02-26T10:47:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f1143400640 1 -- 172.21.15.60:0/2912388654 learned_addr learned my addr 172.21.15.60:0/2912388654 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f114d211740 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 0x55df698122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:6/303: symlink d6/d7/d25/d1c/d1f/d23/l5d 0 2024-02-26T10:47:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:6/304: read d6/f9 [443945,117431] 0 2024-02-26T10:47:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:6/305: fdatasync d6/d7/d25/d1c/d1f/d23/d28/f34 0 2024-02-26T10:47:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:6/306: truncate d6/d7/f53 62598 0 2024-02-26T10:47:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:6/307: chown d6/d7/d25/f46 0 1 2024-02-26T10:47:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df69809380 con 0x55df69806380 2024-02-26T10:47:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df69722980 con 0x55df69801210 2024-02-26T10:47:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df69809500 con 0x55df69800820 2024-02-26T10:47:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f114d211740 10 client.0 _ll_register_callbacks cb 0x55df6975e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f1143400640 1 -- 172.21.15.60:0/2912388654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69801210 msgr2=0x55df6980b260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T10:47:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f1143400640 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69801210 0x55df6980b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f1143400640 1 -- 172.21.15.60:0/2912388654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 msgr2=0x55df698122a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f1143400640 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 0x55df698122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.034+0000 7f1143400640 1 -- 172.21.15.60:0/2912388654 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df6947b210 con 0x55df69800820 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.035+0000 7f1143400640 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df6980ad20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1134004810 tx=0x7f113400f0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cb277b79efecf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.035+0000 7f112cc00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.035+0000 7f112cc00640 1 -- 172.21.15.60:0/2912388654 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113404b060 con 0x55df69800820 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.035+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df69773880 con 0x55df69800820 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.035+0000 7f112cc00640 1 -- 172.21.15.60:0/2912388654 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f1134034800 con 0x55df69800820 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.035+0000 7f112cc00640 1 -- 172.21.15.60:0/2912388654 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1134039040 con 0x55df69800820 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.035+0000 7f114d211740 10 client.26041 fetch_fsmap learned FSMap version 29 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.035+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55df69547620 con 0x55df69800820 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.036+0000 7f112cc00640 1 -- 172.21.15.60:0/2912388654 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55df69773880 con 0x55df69800820 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.036+0000 7f112cc00640 1 -- 172.21.15.60:0/2912388654 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1134006250 con 0x55df69800820 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.036+0000 7f114d211740 10 client.26041 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:47:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.036+0000 7f114d211740 10 client.26041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.036+0000 7f114d211740 20 client.26041 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.036+0000 7f114d211740 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55df698169a0 0x55df69818df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.036+0000 7f114d211740 4 client.26041 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:47:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.036+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df6976dbd0 con 0x55df698169a0 2024-02-26T10:47:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.036+0000 7f1142a00640 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55df698169a0 0x55df69818df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.038+0000 7f1142a00640 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55df698169a0 0x55df69818df0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e3e4777dcadfc0 server_cookie=675b86203618d783 in_seq=0 out_seq=0 2024-02-26T10:47:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.038+0000 7f112cc00640 10 client.26041 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f112cc00640 1 -- 172.21.15.60:0/2912388654 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df6976dbd0 con 0x55df698169a0 2024-02-26T10:47:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f112cc00640 10 client.26041 handle_command_reply: tid=0 2024-02-26T10:47:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 client.26041 shutdown 2024-02-26T10:47:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 msgr2=0x55df6980ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df6980ad20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1134004810 tx=0x7f113400f0e0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 shutdown_connections 2024-02-26T10:47:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55df698169a0 0x55df69818df0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69801210 0x55df6980b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69806380 0x55df698122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 --2- 172.21.15.60:0/2912388654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df6980ad20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 >> 172.21.15.60:0/2912388654 conn(0x55df697fbd90 msgr2=0x55df697fafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f112cc00640 0 client.26041 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 shutdown_connections 2024-02-26T10:47:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.039+0000 7f114d211740 1 -- 172.21.15.60:0/2912388654 wait complete. 2024-02-26T10:47:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.040+0000 7f114d211740 20 client.26041 trim_cache size 0 max 16384 2024-02-26T10:47:55.034 INFO:tasks.workunit.client.0.smithi060.stdout:6/308: rename d6/d7/d25/d1c/d1f/c3d to d6/d7/d25/d1c/d1f/d23/d2b/c5e 0 2024-02-26T10:47:55.041 INFO:tasks.workunit.client.0.smithi060.stdout:9/273: write d3/d9/dc/f3b [648690,128960] 0 2024-02-26T10:47:55.041 INFO:tasks.workunit.client.0.smithi060.stdout:6/309: mknod d6/d7/d2e/d56/c5f 0 2024-02-26T10:47:55.044 INFO:tasks.workunit.client.0.smithi060.stdout:6/310: creat d6/d7/d25/d1c/f60 x:0 0 0 2024-02-26T10:47:55.045 INFO:tasks.workunit.client.0.smithi060.stdout:6/311: write d6/d7/d25/d1c/d1f/d23/d28/f33 [501771,101214] 0 2024-02-26T10:47:55.045 INFO:tasks.workunit.client.0.smithi060.stdout:1/353: link d3/db/d1e/f22 d3/d54/f7c 0 2024-02-26T10:47:55.046 INFO:tasks.workunit.client.0.smithi060.stdout:1/354: getdents d3/db/d1e/d38/d6c 0 2024-02-26T10:47:55.049 INFO:tasks.workunit.client.0.smithi060.stdout:1/355: rename d3/l1c to d3/d54/l7d 0 2024-02-26T10:47:55.049 INFO:tasks.workunit.client.0.smithi060.stdout:1/356: truncate d3/f34 4573692 0 2024-02-26T10:47:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.056+0000 7f114d211740 1 Processor -- start 2024-02-26T10:47:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.057+0000 7f114d211740 1 -- start start 2024-02-26T10:47:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.057+0000 7f114d211740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69800820 0x55df69802670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.057+0000 7f114d211740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69801210 0x55df69802bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.057+0000 7f114d211740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69806380 0x55df698121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.057+0000 7f114d211740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11441aa050 con 0x55df69800820 2024-02-26T10:47:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.057+0000 7f114d211740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11441a7b90 con 0x55df69801210 2024-02-26T10:47:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.057+0000 7f114d211740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1144073200 con 0x55df69806380 2024-02-26T10:47:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.058+0000 7f1143400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69800820 0x55df69802670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.058+0000 7f1143400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69800820 0x55df69802670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46136/0 (socket says 172.21.15.60:46136) 2024-02-26T10:47:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.058+0000 7f1143400640 1 -- 172.21.15.60:0/2576954743 learned_addr learned my addr 172.21.15.60:0/2576954743 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.058+0000 7f1143400640 1 -- 172.21.15.60:0/2576954743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69806380 msgr2=0x55df698121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:47:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.058+0000 7f1143400640 1 --2- 172.21.15.60:0/2576954743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69806380 0x55df698121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.058+0000 7f1143400640 1 -- 172.21.15.60:0/2576954743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69801210 msgr2=0x55df69802bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:47:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.058+0000 7f1143400640 1 --2- 172.21.15.60:0/2576954743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69801210 0x55df69802bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.058+0000 7f1143400640 1 -- 172.21.15.60:0/2576954743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df69808b70 con 0x55df69800820 2024-02-26T10:47:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.058+0000 7f1143400640 1 --2- 172.21.15.60:0/2576954743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69800820 0x55df69802670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1134006640 tx=0x7f1134009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953be64eca2d6475 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:55.053 INFO:tasks.workunit.client.0.smithi060.stdout:9/274: dread d3/d15/d1b/f41 [0,4194304] 0 2024-02-26T10:47:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.059+0000 7f112f400640 1 -- 172.21.15.60:0/2576954743 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113404b070 con 0x55df69800820 2024-02-26T10:47:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.059+0000 7f112f400640 1 -- 172.21.15.60:0/2576954743 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1134006200 con 0x55df69800820 2024-02-26T10:47:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.059+0000 7f112f400640 1 -- 172.21.15.60:0/2576954743 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134035480 con 0x55df69800820 2024-02-26T10:47:55.054 INFO:tasks.workunit.client.0.smithi060.stdout:0/278: sync 2024-02-26T10:47:55.054 INFO:tasks.workunit.client.0.smithi060.stdout:0/279: readlink d0/l29 0 2024-02-26T10:47:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.060+0000 7f114d211740 1 -- 172.21.15.60:0/2576954743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69800820 msgr2=0x55df69802670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.060+0000 7f114d211740 1 --2- 172.21.15.60:0/2576954743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69800820 0x55df69802670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1134006640 tx=0x7f1134009190 comp rx=0 tx=0).stop 2024-02-26T10:47:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.061+0000 7f114d211740 1 -- 172.21.15.60:0/2576954743 shutdown_connections 2024-02-26T10:47:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.061+0000 7f114d211740 1 --2- 172.21.15.60:0/2576954743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69801210 0x55df69802bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.061+0000 7f114d211740 1 --2- 172.21.15.60:0/2576954743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69806380 0x55df698121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.061+0000 7f114d211740 1 --2- 172.21.15.60:0/2576954743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69800820 0x55df69802670 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.061+0000 7f114d211740 1 -- 172.21.15.60:0/2576954743 >> 172.21.15.60:0/2576954743 conn(0x55df697fbd90 msgr2=0x55df6981d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.061+0000 7f114d211740 1 -- 172.21.15.60:0/2576954743 shutdown_connections 2024-02-26T10:47:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.061+0000 7f114d211740 1 -- 172.21.15.60:0/2576954743 wait complete. 2024-02-26T10:47:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.062+0000 7f114d211740 1 -- start start 2024-02-26T10:47:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.062+0000 7f114d211740 1 Processor -- start 2024-02-26T10:47:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.063+0000 7f114d211740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df698101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.063+0000 7f114d211740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69801210 0x55df69810710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.063+0000 7f114d211740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69806380 0x55df69811720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.063+0000 7f114d211740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11441a6250 con 0x55df69801210 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.063+0000 7f114d211740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11441a7530 con 0x55df69806380 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.063+0000 7f114d211740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11441a6870 con 0x55df69800820 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.064+0000 7f1143400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df698101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.064+0000 7f114d211740 10 client.0 _ll_register_callbacks cb 0x7f11340093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.064+0000 7f1143400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df698101d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:32926/0 (socket says 172.21.15.60:32926) 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.064+0000 7f1143400640 1 -- 172.21.15.60:0/3387219076 learned_addr learned my addr 172.21.15.60:0/3387219076 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.064+0000 7f1143400640 1 -- 172.21.15.60:0/3387219076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69806380 msgr2=0x55df69811720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.064+0000 7f1143400640 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69806380 0x55df69811720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.064+0000 7f1143400640 1 -- 172.21.15.60:0/3387219076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69801210 msgr2=0x55df69810710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:47:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.065+0000 7f1142a00640 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69801210 0x55df69810710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.065+0000 7f1143400640 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69801210 0x55df69810710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.065+0000 7f1143400640 1 -- 172.21.15.60:0/3387219076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df69811d10 con 0x55df69800820 2024-02-26T10:47:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.065+0000 7f1142a00640 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69801210 0x55df69810710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:47:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.065+0000 7f1143400640 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df698101d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1134048040 tx=0x7f1134034990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58a49b11db29dc0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:47:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.065+0000 7f112cc00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:47:55.061 INFO:tasks.workunit.client.0.smithi060.stdout:6/312: dread d6/d7/f2a [0,4194304] 0 2024-02-26T10:47:55.061 INFO:tasks.workunit.client.0.smithi060.stdout:9/275: write d3/d15/d1b/f41 [72845,104611] 0 2024-02-26T10:47:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.066+0000 7f112cc00640 1 -- 172.21.15.60:0/3387219076 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113405b020 con 0x55df69800820 2024-02-26T10:47:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.066+0000 7f112cc00640 1 -- 172.21.15.60:0/3387219076 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f1134005dd0 con 0x55df69800820 2024-02-26T10:47:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.068+0000 7f114d211740 1 -- 172.21.15.60:0/3387219076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1134006200 con 0x55df69800820 2024-02-26T10:47:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.069+0000 7f112cc00640 1 -- 172.21.15.60:0/3387219076 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f113404b040 con 0x55df69800820 2024-02-26T10:47:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.069+0000 7f114d211740 10 client.26044 fetch_fsmap learned FSMap version 29 2024-02-26T10:47:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.069+0000 7f114d211740 1 -- 172.21.15.60:0/3387219076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55df69867130 con 0x55df69800820 2024-02-26T10:47:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.069+0000 7f112cc00640 1 -- 172.21.15.60:0/3387219076 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1134006200 con 0x55df69800820 2024-02-26T10:47:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.069+0000 7f112cc00640 1 -- 172.21.15.60:0/3387219076 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f113403e450 con 0x55df69800820 2024-02-26T10:47:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.069+0000 7f114d211740 10 client.26044 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:47:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.069+0000 7f114d211740 10 client.26044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:47:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.069+0000 7f114d211740 20 client.26044 populate_metadata read hostname 'smithi060' 2024-02-26T10:47:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.069+0000 7f114d211740 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55df69867940 0x55df69869d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:47:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.070+0000 7f114d211740 4 client.26044 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:47:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.070+0000 7f114d211740 1 -- 172.21.15.60:0/3387219076 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df696ec5a0 con 0x55df69867940 2024-02-26T10:47:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.070+0000 7f1142a00640 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55df69867940 0x55df69869d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:47:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.070+0000 7f1142a00640 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55df69867940 0x55df69869d60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685ee5773adbecb3 server_cookie=bb76563f3e5a3a56 in_seq=0 out_seq=0 2024-02-26T10:47:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.070+0000 7f112cc00640 10 client.26044 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:55.072 INFO:tasks.workunit.client.0.smithi060.stdout:7/415: dwrite da/d15/d21/d4b/d67/f7a [0,4194304] 0 2024-02-26T10:47:55.073 INFO:tasks.workunit.client.0.smithi060.stdout:7/416: rmdir da/d15/d88/d39 39 2024-02-26T10:47:55.080 INFO:tasks.workunit.client.0.smithi060.stdout:0/280: read d0/dd/d39/f54 [44047,106978] 0 2024-02-26T10:47:55.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:54 smithi113 ceph-mon[36126]: pgmap v483: 129 pgs: 129 active+clean; 255 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 33 MiB/s wr, 1.37k op/s 2024-02-26T10:47:55.083 INFO:tasks.workunit.client.0.smithi060.stdout:0/281: symlink d0/d37/d44/l5e 0 2024-02-26T10:47:55.085 INFO:tasks.workunit.client.0.smithi060.stdout:0/282: rename d0/d37/d44/f45 to d0/f5f 0 2024-02-26T10:47:55.095 INFO:tasks.workunit.client.0.smithi060.stdout:1/357: dread d3/f20 [4194304,4194304] 0 2024-02-26T10:47:55.096 INFO:tasks.workunit.client.0.smithi060.stdout:1/358: mknod d3/d35/d5e/c7e 0 2024-02-26T10:47:55.096 INFO:tasks.workunit.client.0.smithi060.stdout:1/359: chown d3/c52 5631 1 2024-02-26T10:47:55.099 INFO:tasks.workunit.client.0.smithi060.stdout:6/313: dwrite d6/d7/d25/d1c/d1f/d23/d28/f34 [0,4194304] 0 2024-02-26T10:47:55.099 INFO:tasks.workunit.client.0.smithi060.stdout:9/276: dread d3/d9/d10/f11 [0,4194304] 0 2024-02-26T10:47:55.102 INFO:tasks.workunit.client.0.smithi060.stdout:7/417: dread da/d15/f17 [0,4194304] 0 2024-02-26T10:47:55.107 INFO:tasks.workunit.client.0.smithi060.stdout:1/360: dread d3/f23 [0,4194304] 0 2024-02-26T10:47:55.108 INFO:tasks.workunit.client.0.smithi060.stdout:1/361: mknod d3/d35/d5e/c7f 0 2024-02-26T10:47:55.112 INFO:tasks.workunit.client.0.smithi060.stdout:7/418: getdents da/d15/d21/d29 0 2024-02-26T10:47:55.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:54 smithi060 ceph-mon[31539]: pgmap v483: 129 pgs: 129 active+clean; 255 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 33 MiB/s wr, 1.37k op/s 2024-02-26T10:47:55.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:54 smithi100 ceph-mon[36051]: pgmap v483: 129 pgs: 129 active+clean; 255 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 33 MiB/s wr, 1.37k op/s 2024-02-26T10:47:55.125 INFO:tasks.workunit.client.0.smithi060.stdout:1/362: rename d3/f20 to d3/db/d1e/d39/f80 0 2024-02-26T10:47:55.126 INFO:tasks.workunit.client.0.smithi060.stdout:1/363: creat d3/d35/d53/d57/f81 x:0 0 0 2024-02-26T10:47:55.127 INFO:tasks.workunit.client.0.smithi060.stdout:1/364: symlink d3/d35/d53/d57/l82 0 2024-02-26T10:47:55.139 INFO:tasks.workunit.client.0.smithi060.stdout:9/277: dread d3/d9/f2e [0,4194304] 0 2024-02-26T10:47:55.140 INFO:tasks.workunit.client.0.smithi060.stdout:9/278: truncate d3/d15/d1b/d23/f36 1416357 0 2024-02-26T10:47:55.141 INFO:tasks.workunit.client.0.smithi060.stdout:9/279: creat d3/d1e/f58 x:0 0 0 2024-02-26T10:47:55.143 INFO:tasks.workunit.client.0.smithi060.stdout:9/280: getdents d3/d15/d1b/d23 0 2024-02-26T10:47:55.146 INFO:tasks.workunit.client.0.smithi060.stdout:6/314: dwrite d6/d7/d25/d1c/d1f/d23/d2b/f3b [0,4194304] 0 2024-02-26T10:47:55.154 INFO:tasks.workunit.client.0.smithi060.stdout:9/281: symlink d3/d52/l59 0 2024-02-26T10:47:55.155 INFO:tasks.workunit.client.0.smithi060.stdout:1/365: dread d3/db/d1e/f3b [0,4194304] 0 2024-02-26T10:47:55.156 INFO:tasks.workunit.client.0.smithi060.stdout:0/283: dwrite d0/dd/f2d [0,4194304] 0 2024-02-26T10:47:55.167 INFO:tasks.workunit.client.0.smithi060.stdout:6/315: write d6/d7/d25/d1c/d1f/d23/d28/f34 [1764662,103984] 0 2024-02-26T10:47:55.173 INFO:tasks.workunit.client.0.smithi060.stdout:1/366: dread d3/db/d1e/f3b [0,4194304] 0 2024-02-26T10:47:55.173 INFO:tasks.workunit.client.0.smithi060.stdout:1/367: chown d3/db/c4b 1650008 1 2024-02-26T10:47:55.174 INFO:tasks.workunit.client.0.smithi060.stdout:6/316: creat d6/d7/d2e/d56/f61 x:0 0 0 2024-02-26T10:47:55.175 INFO:tasks.workunit.client.0.smithi060.stdout:6/317: creat d6/d7/d25/f62 x:0 0 0 2024-02-26T10:47:55.176 INFO:tasks.workunit.client.0.smithi060.stdout:6/318: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d63 0 2024-02-26T10:47:55.178 INFO:tasks.workunit.client.0.smithi060.stdout:6/319: link d6/d7/d25/d1c/d1f/l42 d6/d7/d25/d1c/l64 0 2024-02-26T10:47:55.184 INFO:tasks.workunit.client.0.smithi060.stdout:6/320: dread d6/d7/f53 [0,4194304] 0 2024-02-26T10:47:55.184 INFO:tasks.workunit.client.0.smithi060.stdout:6/321: chown d6/d7/d25/d1c/d1f/f24 426097125 1 2024-02-26T10:47:55.200 INFO:tasks.workunit.client.0.smithi060.stdout:0/284: dwrite d0/f32 [0,4194304] 0 2024-02-26T10:47:55.202 INFO:tasks.workunit.client.0.smithi060.stdout:0/285: creat d0/d42/f60 x:0 0 0 2024-02-26T10:47:55.221 INFO:tasks.workunit.client.0.smithi060.stdout:6/322: dwrite d6/d7/f52 [0,4194304] 0 2024-02-26T10:47:55.221 INFO:tasks.workunit.client.0.smithi060.stdout:6/323: chown d6/f40 895 1 2024-02-26T10:47:55.225 INFO:tasks.workunit.client.0.smithi060.stdout:6/324: truncate d6/d7/ff 1274257 0 2024-02-26T10:47:55.227 INFO:tasks.workunit.client.0.smithi060.stdout:6/325: rmdir d6/d7/d25 39 2024-02-26T10:47:55.228 INFO:tasks.workunit.client.0.smithi060.stdout:6/326: symlink d6/d7/d25/d1c/d1f/d23/d2b/d63/l65 0 2024-02-26T10:47:55.253 INFO:tasks.workunit.client.0.smithi060.stdout:0/286: dwrite d0/dd/d39/d40/f1f [0,4194304] 0 2024-02-26T10:47:55.255 INFO:tasks.workunit.client.0.smithi060.stdout:0/287: creat d0/dd/d39/f61 x:0 0 0 2024-02-26T10:47:55.266 INFO:tasks.workunit.client.0.smithi060.stdout:6/327: dwrite f4 [0,4194304] 0 2024-02-26T10:47:55.268 INFO:tasks.workunit.client.0.smithi060.stdout:6/328: symlink d6/d7/d25/d1c/d1f/d23/d28/l66 0 2024-02-26T10:47:55.276 INFO:tasks.workunit.client.0.smithi060.stdout:0/288: dread d0/dd/d39/d40/f28 [0,4194304] 0 2024-02-26T10:47:55.313 INFO:tasks.workunit.client.0.smithi060.stdout:6/329: dwrite d6/d7/d25/d1c/f2c [0,4194304] 0 2024-02-26T10:47:55.313 INFO:tasks.workunit.client.0.smithi060.stdout:6/330: getdents d6/d7/d25/d1c/d39 0 2024-02-26T10:47:55.315 INFO:tasks.workunit.client.0.smithi060.stdout:6/331: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67 0 2024-02-26T10:47:55.316 INFO:tasks.workunit.client.0.smithi060.stdout:6/332: creat d6/d7/d25/d1c/f68 x:0 0 0 2024-02-26T10:47:55.317 INFO:tasks.workunit.client.0.smithi060.stdout:6/333: mkdir d6/d7/d47/d69 0 2024-02-26T10:47:55.317 INFO:tasks.workunit.client.0.smithi060.stdout:6/334: read d6/d7/d25/d1c/d1f/f35 [356783,107117] 0 2024-02-26T10:47:55.325 INFO:tasks.workunit.client.0.smithi060.stdout:6/335: dread d6/d7/d25/f4b [0,4194304] 0 2024-02-26T10:47:55.356 INFO:tasks.workunit.client.0.smithi060.stdout:6/336: unlink d6/f15 0 2024-02-26T10:47:55.360 INFO:tasks.workunit.client.0.smithi060.stdout:6/337: symlink d6/d7/d25/d1c/d1f/d23/d2b/d4d/l6a 0 2024-02-26T10:47:55.364 INFO:tasks.workunit.client.0.smithi060.stdout:6/338: mkdir d6/d7/d25/d1c/d6b 0 2024-02-26T10:47:55.739 INFO:tasks.workunit.client.0.smithi060.stdout:6/339: dread d6/f18 [0,4194304] 0 2024-02-26T10:47:55.739 INFO:tasks.workunit.client.0.smithi060.stdout:6/340: stat d6/d7/d25/d1c/f68 0 2024-02-26T10:47:55.740 INFO:tasks.workunit.client.0.smithi060.stdout:6/341: truncate d6/d7/d25/f46 858111 0 2024-02-26T10:47:55.741 INFO:tasks.workunit.client.0.smithi060.stdout:6/342: symlink d6/d7/d2e/l6c 0 2024-02-26T10:47:55.743 INFO:tasks.workunit.client.0.smithi060.stdout:6/343: creat d6/d7/d25/d1c/f6d x:0 0 0 2024-02-26T10:47:55.743 INFO:tasks.workunit.client.0.smithi060.stdout:6/344: fsync d6/d7/ff 0 2024-02-26T10:47:55.743 INFO:tasks.workunit.client.0.smithi060.stdout:6/345: chown d6/d7/d25/d1c/d1f/d23/d28/f34 17246648 1 2024-02-26T10:47:55.743 INFO:tasks.workunit.client.0.smithi060.stdout:6/346: chown d6/d7/d25/d1c/l64 13 1 2024-02-26T10:47:55.743 INFO:tasks.workunit.client.0.smithi060.stdout:6/347: fdatasync d6/f16 0 2024-02-26T10:47:55.744 INFO:tasks.workunit.client.0.smithi060.stdout:6/348: dread - d6/d7/d25/d1c/d1f/d23/f4a zero size 2024-02-26T10:47:55.747 INFO:tasks.workunit.client.0.smithi060.stdout:6/349: unlink d6/d7/f52 0 2024-02-26T10:47:55.796 INFO:tasks.workunit.client.0.smithi060.stdout:6/350: dread d6/f18 [0,4194304] 0 2024-02-26T10:47:55.796 INFO:tasks.workunit.client.0.smithi060.stdout:6/351: read - d6/d7/d25/f62 zero size 2024-02-26T10:47:55.796 INFO:tasks.workunit.client.0.smithi060.stdout:6/352: truncate d6/f18 1396562 0 2024-02-26T10:47:55.798 INFO:tasks.workunit.client.0.smithi060.stdout:6/353: mkdir d6/d6e 0 2024-02-26T10:47:55.798 INFO:tasks.workunit.client.0.smithi060.stdout:6/354: fsync d6/d7/d25/d1c/d39/f44 0 2024-02-26T10:47:55.801 INFO:tasks.workunit.client.0.smithi060.stdout:6/355: write d6/f1d [107983,128625] 0 2024-02-26T10:47:55.802 INFO:tasks.workunit.client.0.smithi060.stdout:6/356: mkdir d6/d6e/d6f 0 2024-02-26T10:47:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f112cc00640 1 -- 172.21.15.60:0/3387219076 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df696ec5a0 con 0x55df69867940 2024-02-26T10:47:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f112cc00640 10 client.26044 handle_command_reply: tid=0 2024-02-26T10:47:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f114d211740 1 client.26044 shutdown 2024-02-26T10:47:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f114d211740 1 -- 172.21.15.60:0/3387219076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 msgr2=0x55df698101d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f114d211740 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df698101d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1134048040 tx=0x7f1134034990 comp rx=0 tx=0).stop 2024-02-26T10:47:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f114d211740 1 -- 172.21.15.60:0/3387219076 shutdown_connections 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f114d211740 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55df69867940 0x55df69869d60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f114d211740 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df69806380 0x55df69811720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f114d211740 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df69801210 0x55df69810710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f114d211740 1 --2- 172.21.15.60:0/3387219076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55df69800820 0x55df698101d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f112cc00640 0 client.26044 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.867+0000 7f114d211740 1 -- 172.21.15.60:0/3387219076 >> 172.21.15.60:0/3387219076 conn(0x55df697fbd90 msgr2=0x55df69802670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.868+0000 7f114d211740 1 -- 172.21.15.60:0/3387219076 shutdown_connections 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.868+0000 7f114d211740 1 -- 172.21.15.60:0/3387219076 wait complete. 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.868+0000 7f114d211740 20 client.26044 trim_cache size 0 max 16384 2024-02-26T10:47:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:47:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:47:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:47:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:47:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.868+0000 7f114a200640 1 -- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11280a1ea0 msgr2=0x7f11280a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.868+0000 7f114a200640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11280a1ea0 0x7f11280a4360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1130006010 tx=0x7f113002cf90 comp rx=0 tx=0).stop 2024-02-26T10:47:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.868+0000 7f114a200640 1 -- 172.21.15.60:0/1669312473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1144105070 msgr2=0x7f11441a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:47:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.868+0000 7f114a200640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1144105070 0x7f11441a5180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f113c00eaa0 tx=0x7f113c00eea0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.869+0000 7f114a200640 1 -- 172.21.15.60:0/1669312473 shutdown_connections 2024-02-26T10:47:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.869+0000 7f114a200640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f11280a1ea0 0x7f11280a4360 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.869+0000 7f114a200640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1144105070 0x7f11441a5180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.869+0000 7f114a200640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11441046b0 0x7f114419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.869+0000 7f114a200640 1 --2- 172.21.15.60:0/1669312473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11441034c0 0x7f114419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:47:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.869+0000 7f114a200640 1 -- 172.21.15.60:0/1669312473 >> 172.21.15.60:0/1669312473 conn(0x7f11440fea50 msgr2=0x7f11440776b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:47:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.869+0000 7f114a200640 1 -- 172.21.15.60:0/1669312473 shutdown_connections 2024-02-26T10:47:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:47:55.869+0000 7f114a200640 1 -- 172.21.15.60:0/1669312473 wait complete. 2024-02-26T10:47:55.870 INFO:tasks.workunit.client.0.smithi060.stdout:9/282: rename d3/d9/f19 to d3/d15/d1b/f5a 0 2024-02-26T10:47:55.871 INFO:tasks.workunit.client.0.smithi060.stdout:5/264: sync 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:3/202: sync 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:2/264: sync 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:4/322: sync 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:3/203: chown d5/de/df/d11 303 1 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:5/265: chown d0/de/d15/d26/f41 0 1 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:8/284: sync 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:8/285: chown d9/f24 170 1 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:3/204: fdatasync d5/f25 0 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:7/419: sync 2024-02-26T10:47:55.872 INFO:tasks.workunit.client.0.smithi060.stdout:7/420: fdatasync da/d15/d88/d3b/f71 0 2024-02-26T10:47:55.875 INFO:tasks.workunit.client.0.smithi060.stdout:3/205: creat d5/d37/f38 x:0 0 0 2024-02-26T10:47:55.875 INFO:tasks.workunit.client.0.smithi060.stdout:8/286: creat d9/d15/f51 x:0 0 0 2024-02-26T10:47:55.875 INFO:tasks.workunit.client.0.smithi060.stdout:3/206: readlink d5/l19 0 2024-02-26T10:47:55.881 INFO:tasks.workunit.client.0.smithi060.stdout:4/323: rmdir d0/da/d3b 39 2024-02-26T10:47:55.883 INFO:tasks.workunit.client.0.smithi060.stdout:4/324: mknod d0/d27/c5d 0 2024-02-26T10:47:55.883 INFO:tasks.workunit.client.0.smithi060.stdout:4/325: readlink d0/da/d3b/d1b/l1c 0 2024-02-26T10:47:55.892 INFO:tasks.workunit.client.0.smithi060.stdout:1/368: rename d3/db/d1e/f22 to d3/d35/d53/d57/f83 0 2024-02-26T10:47:55.892 INFO:tasks.workunit.client.0.smithi060.stdout:1/369: chown d3/l61 47959 1 2024-02-26T10:47:55.893 INFO:tasks.workunit.client.0.smithi060.stdout:0/289: rename d0/d37/f50 to d0/d42/f62 0 2024-02-26T10:47:55.904 INFO:tasks.workunit.client.0.smithi060.stdout:6/357: rename d6/d7/d25/d1c/f68 to d6/d7/f70 0 2024-02-26T10:47:55.904 INFO:tasks.workunit.client.0.smithi060.stdout:1/370: getdents d3/d35/d5e 0 2024-02-26T10:47:55.904 INFO:tasks.workunit.client.0.smithi060.stdout:1/371: readlink d3/db/d1e/l26 0 2024-02-26T10:47:55.904 INFO:tasks.workunit.client.0.smithi060.stdout:0/290: mkdir d0/dd/d39/d63 0 2024-02-26T10:47:55.904 INFO:tasks.workunit.client.0.smithi060.stdout:0/291: creat d0/dd/d53/f64 x:0 0 0 2024-02-26T10:47:55.904 INFO:tasks.workunit.client.0.smithi060.stdout:7/421: write da/f60 [874222,126499] 0 2024-02-26T10:47:55.904 INFO:tasks.workunit.client.0.smithi060.stdout:7/422: truncate f6 1362904 0 2024-02-26T10:47:55.905 INFO:tasks.workunit.client.0.smithi060.stdout:0/292: rmdir d0/dd/d39/d40 39 2024-02-26T10:47:55.912 INFO:tasks.workunit.client.0.smithi060.stdout:7/423: creat da/d15/d88/d3b/f8c x:0 0 0 2024-02-26T10:47:55.912 INFO:tasks.workunit.client.0.smithi060.stdout:7/424: chown da/fd 256797114 1 2024-02-26T10:47:55.917 INFO:tasks.workunit.client.0.smithi060.stdout:5/266: dread d0/de/d20/f24 [0,4194304] 0 2024-02-26T10:47:55.917 INFO:tasks.workunit.client.0.smithi060.stdout:0/293: symlink d0/d37/d44/d58/l65 0 2024-02-26T10:47:55.918 INFO:tasks.workunit.client.0.smithi060.stdout:3/207: dread d5/f27 [0,4194304] 0 2024-02-26T10:47:55.918 INFO:tasks.workunit.client.0.smithi060.stdout:3/208: write d5/de/df/f33 [1907016,101811] 0 2024-02-26T10:47:55.919 INFO:tasks.workunit.client.0.smithi060.stdout:5/267: rename d0/de/d15/f16 to d0/d33/f63 0 2024-02-26T10:47:55.919 INFO:tasks.workunit.client.0.smithi060.stdout:5/268: fsync d0/d33/f62 0 2024-02-26T10:47:55.920 INFO:tasks.workunit.client.0.smithi060.stdout:0/294: creat d0/d37/d44/f66 x:0 0 0 2024-02-26T10:47:55.920 INFO:tasks.workunit.client.0.smithi060.stdout:0/295: chown d0/dd/d39/f3f 18535130 1 2024-02-26T10:47:55.921 INFO:tasks.workunit.client.0.smithi060.stdout:9/283: write d3/f16 [561962,78611] 0 2024-02-26T10:47:55.923 INFO:tasks.workunit.client.0.smithi060.stdout:2/265: dread dd/d11/f26 [0,4194304] 0 2024-02-26T10:47:55.927 INFO:tasks.workunit.client.0.smithi060.stdout:2/266: rmdir dd/de/d41/d42 39 2024-02-26T10:47:55.927 INFO:tasks.workunit.client.0.smithi060.stdout:3/209: symlink d5/de/d16/l39 0 2024-02-26T10:47:55.927 INFO:tasks.workunit.client.0.smithi060.stdout:5/269: symlink d0/de/l64 0 2024-02-26T10:47:55.927 INFO:tasks.workunit.client.0.smithi060.stdout:5/270: readlink d0/de/l64 0 2024-02-26T10:47:55.927 INFO:tasks.workunit.client.0.smithi060.stdout:0/296: truncate d0/dd/f2c 3075153 0 2024-02-26T10:47:55.927 INFO:tasks.workunit.client.0.smithi060.stdout:9/284: truncate d3/d9/f38 504267 0 2024-02-26T10:47:55.933 INFO:tasks.workunit.client.0.smithi060.stdout:3/210: truncate d5/f27 1188074 0 2024-02-26T10:47:55.933 INFO:tasks.workunit.client.0.smithi060.stdout:5/271: mkdir d0/de/d15/d65 0 2024-02-26T10:47:55.935 INFO:tasks.workunit.client.0.smithi060.stdout:5/272: symlink d0/de/d20/l66 0 2024-02-26T10:47:55.936 INFO:tasks.workunit.client.0.smithi060.stdout:5/273: write d0/de/f35 [645578,51104] 0 2024-02-26T10:47:55.936 INFO:tasks.workunit.client.0.smithi060.stdout:3/211: getdents d5/de/d16/d17 0 2024-02-26T10:47:55.939 INFO:tasks.workunit.client.0.smithi060.stdout:5/274: symlink d0/de/d15/d26/l67 0 2024-02-26T10:47:55.939 INFO:tasks.workunit.client.0.smithi060.stdout:3/212: unlink d5/de/f15 0 2024-02-26T10:47:55.951 INFO:tasks.workunit.client.0.smithi060.stdout:4/326: dwrite d0/f44 [0,4194304] 0 2024-02-26T10:47:55.951 INFO:tasks.workunit.client.0.smithi060.stdout:4/327: readlink d0/da/d3b/d1b/l1c 0 2024-02-26T10:47:55.952 INFO:tasks.workunit.client.0.smithi060.stdout:2/267: write dd/de/d41/f1a [166255,46748] 0 2024-02-26T10:47:55.954 INFO:tasks.workunit.client.0.smithi060.stdout:3/213: getdents d5/de/df 0 2024-02-26T10:47:55.954 INFO:tasks.workunit.client.0.smithi060.stdout:3/214: fsync d5/de/f2b 0 2024-02-26T10:47:55.954 INFO:tasks.workunit.client.0.smithi060.stdout:2/268: unlink dd/de/c12 0 2024-02-26T10:47:55.955 INFO:tasks.workunit.client.0.smithi060.stdout:3/215: creat d5/de/d16/d17/f3a x:0 0 0 2024-02-26T10:47:55.956 INFO:tasks.workunit.client.0.smithi060.stdout:2/269: mkdir dd/d52 0 2024-02-26T10:47:55.956 INFO:tasks.workunit.client.0.smithi060.stdout:0/297: rmdir d0/d37 39 2024-02-26T10:47:55.956 INFO:tasks.workunit.client.0.smithi060.stdout:2/270: write dd/de/d41/f31 [934061,128902] 0 2024-02-26T10:47:55.956 INFO:tasks.workunit.client.0.smithi060.stdout:8/287: dwrite d9/d15/d29/d2f/f3d [0,4194304] 0 2024-02-26T10:47:55.956 INFO:tasks.workunit.client.0.smithi060.stdout:3/216: creat d5/de/f3b x:0 0 0 2024-02-26T10:47:55.957 INFO:tasks.workunit.client.0.smithi060.stdout:2/271: creat dd/de/f53 x:0 0 0 2024-02-26T10:47:55.958 INFO:tasks.workunit.client.0.smithi060.stdout:2/272: symlink dd/d52/l54 0 2024-02-26T10:47:55.958 INFO:tasks.workunit.client.0.smithi060.stdout:2/273: truncate dd/f2b 2000092 0 2024-02-26T10:47:55.960 INFO:tasks.workunit.client.0.smithi060.stdout:2/274: truncate dd/d11/f3e 1001014 0 2024-02-26T10:47:55.963 INFO:tasks.workunit.client.0.smithi060.stdout:3/217: mknod d5/c3c 0 2024-02-26T10:47:55.963 INFO:tasks.workunit.client.0.smithi060.stdout:3/218: write d5/d37/f38 [667556,5305] 0 2024-02-26T10:47:55.963 INFO:tasks.workunit.client.0.smithi060.stdout:3/219: dread - d5/de/df/d11/f34 zero size 2024-02-26T10:47:55.966 INFO:tasks.workunit.client.0.smithi060.stdout:5/275: write d0/f8 [838904,74806] 0 2024-02-26T10:47:55.966 INFO:tasks.workunit.client.0.smithi060.stdout:5/276: dread - d0/de/d15/f47 zero size 2024-02-26T10:47:55.972 INFO:tasks.workunit.client.0.smithi060.stdout:1/372: write d3/d35/d5e/f66 [5899298,10504] 0 2024-02-26T10:47:55.980 INFO:tasks.workunit.client.0.smithi060.stdout:5/277: mknod d0/de/d15/c68 0 2024-02-26T10:47:55.980 INFO:tasks.workunit.client.0.smithi060.stdout:5/278: chown d0/de/d15/c27 24974 1 2024-02-26T10:47:55.983 INFO:tasks.workunit.client.0.smithi060.stdout:7/425: dwrite da/f24 [0,4194304] 0 2024-02-26T10:47:55.983 INFO:tasks.workunit.client.0.smithi060.stdout:7/426: chown da 20 1 2024-02-26T10:47:55.983 INFO:tasks.workunit.client.0.smithi060.stdout:7/427: fdatasync da/d15/d21/d32/f63 0 2024-02-26T10:47:55.983 INFO:tasks.workunit.client.0.smithi060.stdout:7/428: write da/d15/d21/f22 [1084879,38366] 0 2024-02-26T10:47:55.987 INFO:tasks.workunit.client.0.smithi060.stdout:7/429: unlink da/d15/l27 0 2024-02-26T10:47:55.989 INFO:tasks.workunit.client.0.smithi060.stdout:7/430: mknod da/d15/d21/d41/d58/c8d 0 2024-02-26T10:47:55.998 INFO:tasks.workunit.client.0.smithi060.stdout:5/279: symlink d0/l69 0 2024-02-26T10:47:56.006 INFO:tasks.workunit.client.0.smithi060.stdout:4/328: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:56.006 INFO:tasks.workunit.client.0.smithi060.stdout:4/329: chown d0/da/d3b 1114779494 1 2024-02-26T10:47:56.006 INFO:tasks.workunit.client.0.smithi060.stdout:5/280: creat d0/de/f6a x:0 0 0 2024-02-26T10:47:56.006 INFO:tasks.workunit.client.0.smithi060.stdout:5/281: chown d0/de/d15/f5a 445 1 2024-02-26T10:47:56.006 INFO:tasks.workunit.client.0.smithi060.stdout:5/282: stat d0/f30 0 2024-02-26T10:47:56.009 INFO:tasks.workunit.client.0.smithi060.stdout:4/330: creat d0/d27/f5e x:0 0 0 2024-02-26T10:47:56.010 INFO:tasks.workunit.client.0.smithi060.stdout:3/220: dwrite d5/de/d16/f2d [0,4194304] 0 2024-02-26T10:47:56.020 INFO:tasks.workunit.client.0.smithi060.stdout:2/275: dread dd/de/d41/f1a [0,4194304] 0 2024-02-26T10:47:56.025 INFO:tasks.workunit.client.0.smithi060.stdout:1/373: dread d3/f12 [0,4194304] 0 2024-02-26T10:47:56.028 INFO:tasks.workunit.client.0.smithi060.stdout:9/285: dwrite d3/d9/f38 [0,4194304] 0 2024-02-26T10:47:56.029 INFO:tasks.workunit.client.0.smithi060.stdout:7/431: dread f7 [0,4194304] 0 2024-02-26T10:47:56.029 INFO:tasks.workunit.client.0.smithi060.stdout:1/374: symlink d3/d54/l84 0 2024-02-26T10:47:56.029 INFO:tasks.workunit.client.0.smithi060.stdout:2/276: unlink dd/de/d41/d37/d38/f45 0 2024-02-26T10:47:56.030 INFO:tasks.workunit.client.0.smithi060.stdout:7/432: chown da/d15/d21/d32/f63 776698951 1 2024-02-26T10:47:56.030 INFO:tasks.workunit.client.0.smithi060.stdout:9/286: creat d3/d9/f5b x:0 0 0 2024-02-26T10:47:56.030 INFO:tasks.workunit.client.0.smithi060.stdout:9/287: write d3/d9/f2f [5158209,105296] 0 2024-02-26T10:47:56.031 INFO:tasks.workunit.client.0.smithi060.stdout:2/277: mkdir dd/d43/d55 0 2024-02-26T10:47:56.031 INFO:tasks.workunit.client.0.smithi060.stdout:5/283: rmdir d0 39 2024-02-26T10:47:56.033 INFO:tasks.workunit.client.0.smithi060.stdout:5/284: creat d0/d33/f6b x:0 0 0 2024-02-26T10:47:56.033 INFO:tasks.workunit.client.0.smithi060.stdout:5/285: dread - d0/de/d15/f47 zero size 2024-02-26T10:47:56.035 INFO:tasks.workunit.client.0.smithi060.stdout:5/286: creat d0/de/d15/d65/f6c x:0 0 0 2024-02-26T10:47:56.035 INFO:tasks.workunit.client.0.smithi060.stdout:5/287: read d0/de/d15/d26/f38 [1019264,100177] 0 2024-02-26T10:47:56.036 INFO:tasks.workunit.client.0.smithi060.stdout:5/288: mknod d0/d4b/c6d 0 2024-02-26T10:47:56.040 INFO:tasks.workunit.client.0.smithi060.stdout:5/289: rmdir d0/de/d15/d65 39 2024-02-26T10:47:56.045 INFO:tasks.workunit.client.0.smithi060.stdout:9/288: rmdir d3/d9/dc/d32 39 2024-02-26T10:47:56.045 INFO:tasks.workunit.client.0.smithi060.stdout:5/290: mknod d0/de/d20/d32/d4c/c6e 0 2024-02-26T10:47:56.045 INFO:tasks.workunit.client.0.smithi060.stdout:5/291: link d0/l69 d0/de/d20/d32/d4c/l6f 0 2024-02-26T10:47:56.045 INFO:tasks.workunit.client.0.smithi060.stdout:5/292: write d0/f14 [4607826,68919] 0 2024-02-26T10:47:56.046 INFO:tasks.workunit.client.0.smithi060.stdout:5/293: unlink d0/de/d15/f1a 0 2024-02-26T10:47:56.047 INFO:tasks.workunit.client.0.smithi060.stdout:5/294: truncate d0/de/d20/d32/d4d/f53 102468 0 2024-02-26T10:47:56.049 INFO:tasks.workunit.client.0.smithi060.stdout:5/295: mkdir d0/de/d70 0 2024-02-26T10:47:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:5/296: getdents d0/de/d20/d32/d4c 0 2024-02-26T10:47:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:9/289: dread d3/d15/d1b/f41 [0,4194304] 0 2024-02-26T10:47:56.059 INFO:tasks.workunit.client.0.smithi060.stdout:5/297: rename d0/de/d20/l5b to d0/d4b/l71 0 2024-02-26T10:47:56.059 INFO:tasks.workunit.client.0.smithi060.stdout:5/298: readlink d0/de/d20/l59 0 2024-02-26T10:47:56.059 INFO:tasks.workunit.client.0.smithi060.stdout:5/299: truncate d0/de/d15/f3b 141839 0 2024-02-26T10:47:56.059 INFO:tasks.workunit.client.0.smithi060.stdout:5/300: dread - d0/de/d15/f2d zero size 2024-02-26T10:47:56.059 INFO:tasks.workunit.client.0.smithi060.stdout:5/301: stat d0/f1d 0 2024-02-26T10:47:56.059 INFO:tasks.workunit.client.0.smithi060.stdout:5/302: dread - d0/f3a zero size 2024-02-26T10:47:56.060 INFO:tasks.workunit.client.0.smithi060.stdout:9/290: dread d3/d9/dc/d32/f3d [0,4194304] 0 2024-02-26T10:47:56.064 INFO:tasks.workunit.client.0.smithi060.stdout:4/331: dwrite d0/d7/f8 [0,4194304] 0 2024-02-26T10:47:56.064 INFO:tasks.workunit.client.0.smithi060.stdout:4/332: write d0/da/d3b/d16/f20 [438633,12890] 0 2024-02-26T10:47:56.064 INFO:tasks.workunit.client.0.smithi060.stdout:9/291: creat d3/d15/f5c x:0 0 0 2024-02-26T10:47:56.064 INFO:tasks.workunit.client.0.smithi060.stdout:4/333: truncate d0/d27/d2f/f35 564290 0 2024-02-26T10:47:56.068 INFO:tasks.workunit.client.0.smithi060.stdout:3/221: dwrite d5/de/df/d11/f20 [0,4194304] 0 2024-02-26T10:47:56.068 INFO:tasks.workunit.client.0.smithi060.stdout:3/222: fdatasync d5/de/f2b 0 2024-02-26T10:47:56.070 INFO:tasks.workunit.client.0.smithi060.stdout:3/223: rename d5/de/d16/c35 to d5/de/d16/c3d 0 2024-02-26T10:47:56.070 INFO:tasks.workunit.client.0.smithi060.stdout:3/224: fsync d5/f6 0 2024-02-26T10:47:56.079 INFO:tasks.workunit.client.0.smithi060.stdout:0/298: dwrite d0/dd/d39/d40/f28 [0,4194304] 0 2024-02-26T10:47:56.079 INFO:tasks.workunit.client.0.smithi060.stdout:1/375: dread d3/d35/d5e/f66 [0,4194304] 0 2024-02-26T10:47:56.080 INFO:tasks.workunit.client.0.smithi060.stdout:1/376: write d3/f23 [901961,62259] 0 2024-02-26T10:47:56.081 INFO:tasks.workunit.client.0.smithi060.stdout:0/299: rmdir d0/d37/d44 39 2024-02-26T10:47:56.081 INFO:tasks.workunit.client.0.smithi060.stdout:7/433: dread da/d15/d21/d32/f5f [0,4194304] 0 2024-02-26T10:47:56.082 INFO:tasks.workunit.client.0.smithi060.stdout:5/303: dread d0/f9 [0,4194304] 0 2024-02-26T10:47:56.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/304: creat d0/de/d20/d32/d4d/d60/f72 x:0 0 0 2024-02-26T10:47:56.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/305: chown d0/de/f29 1173184518 1 2024-02-26T10:47:56.085 INFO:tasks.workunit.client.0.smithi060.stdout:5/306: truncate d0/f17 398415 0 2024-02-26T10:47:56.087 INFO:tasks.workunit.client.0.smithi060.stdout:3/225: dread d5/de/df/f33 [0,4194304] 0 2024-02-26T10:47:56.088 INFO:tasks.workunit.client.0.smithi060.stdout:2/278: dwrite dd/d11/f24 [0,4194304] 0 2024-02-26T10:47:56.088 INFO:tasks.workunit.client.0.smithi060.stdout:5/307: symlink d0/d4b/l73 0 2024-02-26T10:47:56.093 INFO:tasks.workunit.client.0.smithi060.stdout:0/300: write d0/d42/f62 [1007208,9439] 0 2024-02-26T10:47:56.101 INFO:tasks.workunit.client.0.smithi060.stdout:2/279: mknod dd/c56 0 2024-02-26T10:47:56.102 INFO:tasks.workunit.client.0.smithi060.stdout:4/334: write d0/d27/d2f/f55 [1214822,69530] 0 2024-02-26T10:47:56.102 INFO:tasks.workunit.client.0.smithi060.stdout:4/335: write d0/da/d3b/d16/f51 [1666490,91034] 0 2024-02-26T10:47:56.102 INFO:tasks.workunit.client.0.smithi060.stdout:2/280: mkdir dd/de/d41/d37/d38/d57 0 2024-02-26T10:47:56.104 INFO:tasks.workunit.client.0.smithi060.stdout:2/281: symlink dd/d43/d55/l58 0 2024-02-26T10:47:56.104 INFO:tasks.workunit.client.0.smithi060.stdout:4/336: creat d0/da/d3b/f5f x:0 0 0 2024-02-26T10:47:56.105 INFO:tasks.workunit.client.0.smithi060.stdout:4/337: fdatasync d0/d7/f40 0 2024-02-26T10:47:56.106 INFO:tasks.workunit.client.0.smithi060.stdout:4/338: mkdir d0/d7/d29/d60 0 2024-02-26T10:47:56.132 INFO:tasks.workunit.client.0.smithi060.stdout:5/308: dread d0/f30 [0,4194304] 0 2024-02-26T10:47:56.135 INFO:tasks.workunit.client.0.smithi060.stdout:5/309: rename d0/d33/f6b to d0/de/d20/d32/d4c/f74 0 2024-02-26T10:47:56.135 INFO:tasks.workunit.client.0.smithi060.stdout:5/310: fdatasync d0/de/d20/f58 0 2024-02-26T10:47:56.137 INFO:tasks.workunit.client.0.smithi060.stdout:3/226: dread d5/fa [0,4194304] 0 2024-02-26T10:47:56.138 INFO:tasks.workunit.client.0.smithi060.stdout:9/292: dwrite d3/d15/f5c [0,4194304] 0 2024-02-26T10:47:56.138 INFO:tasks.workunit.client.0.smithi060.stdout:7/434: dwrite f2 [4194304,4194304] 0 2024-02-26T10:47:56.141 INFO:tasks.workunit.client.0.smithi060.stdout:9/293: mknod d3/d1e/c5d 0 2024-02-26T10:47:56.148 INFO:tasks.workunit.client.0.smithi060.stdout:9/294: rename d3/d15 to d3/d1e/d5e 0 2024-02-26T10:47:56.148 INFO:tasks.workunit.client.0.smithi060.stdout:9/295: truncate d3/d9/d10/f37 4271967 0 2024-02-26T10:47:56.155 INFO:tasks.workunit.client.0.smithi060.stdout:5/311: creat d0/de/d20/d32/d4d/d60/f75 x:0 0 0 2024-02-26T10:47:56.159 INFO:tasks.workunit.client.0.smithi060.stdout:5/312: symlink d0/de/d15/l76 0 2024-02-26T10:47:56.159 INFO:tasks.workunit.client.0.smithi060.stdout:5/313: symlink d0/d33/l77 0 2024-02-26T10:47:56.160 INFO:tasks.workunit.client.0.smithi060.stdout:5/314: truncate d0/de/d20/d32/d4d/d60/f72 641836 0 2024-02-26T10:47:56.162 INFO:tasks.workunit.client.0.smithi060.stdout:5/315: creat d0/de/d15/d26/f78 x:0 0 0 2024-02-26T10:47:56.162 INFO:tasks.workunit.client.0.smithi060.stdout:5/316: read d0/de/d15/d26/f38 [2611509,29494] 0 2024-02-26T10:47:56.162 INFO:tasks.workunit.client.0.smithi060.stdout:5/317: truncate d0/de/d20/d32/d4d/d60/f72 1316718 0 2024-02-26T10:47:56.171 INFO:tasks.workunit.client.0.smithi060.stdout:4/339: dwrite d0/d7/f5a [0,4194304] 0 2024-02-26T10:47:56.172 INFO:tasks.workunit.client.0.smithi060.stdout:4/340: chown d0/ff 231 1 2024-02-26T10:47:56.174 INFO:tasks.workunit.client.0.smithi060.stdout:4/341: mknod d0/c61 0 2024-02-26T10:47:56.177 INFO:tasks.workunit.client.0.smithi060.stdout:5/318: dread d0/d33/f63 [0,4194304] 0 2024-02-26T10:47:56.177 INFO:tasks.workunit.client.0.smithi060.stdout:5/319: chown d0/de/d20/f24 411 1 2024-02-26T10:47:56.177 INFO:tasks.workunit.client.0.smithi060.stdout:5/320: readlink d0/de/d20/l2e 0 2024-02-26T10:47:56.178 INFO:tasks.workunit.client.0.smithi060.stdout:5/321: mknod d0/d33/c79 0 2024-02-26T10:47:56.178 INFO:tasks.workunit.client.0.smithi060.stdout:5/322: fdatasync d0/f17 0 2024-02-26T10:47:56.178 INFO:tasks.workunit.client.0.smithi060.stdout:5/323: write d0/d33/f3e [589382,69382] 0 2024-02-26T10:47:56.185 INFO:tasks.workunit.client.0.smithi060.stdout:1/377: dwrite d3/f23 [0,4194304] 0 2024-02-26T10:47:56.185 INFO:tasks.workunit.client.0.smithi060.stdout:1/378: dread - d3/d35/d53/d68/f77 zero size 2024-02-26T10:47:56.185 INFO:tasks.workunit.client.0.smithi060.stdout:3/227: dwrite d5/de/df/f21 [0,4194304] 0 2024-02-26T10:47:56.185 INFO:tasks.workunit.client.0.smithi060.stdout:2/282: dread f5 [0,4194304] 0 2024-02-26T10:47:56.185 INFO:tasks.workunit.client.0.smithi060.stdout:3/228: fdatasync d5/fd 0 2024-02-26T10:47:56.195 INFO:tasks.workunit.client.0.smithi060.stdout:1/379: rename d3/d32 to d3/d35/d53/d68/d85 0 2024-02-26T10:47:56.195 INFO:tasks.workunit.client.0.smithi060.stdout:1/380: fsync d3/d54/f7c 0 2024-02-26T10:47:56.195 INFO:tasks.workunit.client.0.smithi060.stdout:1/381: readlink d3/db/d2e/d51/l6f 0 2024-02-26T10:47:56.195 INFO:tasks.workunit.client.0.smithi060.stdout:1/382: dread - d3/d35/d53/d57/f81 zero size 2024-02-26T10:47:56.196 INFO:tasks.workunit.client.0.smithi060.stdout:1/383: creat d3/db/d1e/d6d/d78/f86 x:0 0 0 2024-02-26T10:47:56.201 INFO:tasks.workunit.client.0.smithi060.stdout:9/296: dwrite d3/d9/f5b [0,4194304] 0 2024-02-26T10:47:56.201 INFO:tasks.workunit.client.0.smithi060.stdout:9/297: chown d3/d1e/d5e/d1b/d23/f34 1 1 2024-02-26T10:47:56.205 INFO:tasks.workunit.client.0.smithi060.stdout:3/229: dread d5/de/df/f33 [0,4194304] 0 2024-02-26T10:47:56.205 INFO:tasks.workunit.client.0.smithi060.stdout:3/230: fdatasync d5/de/df/d11/f36 0 2024-02-26T10:47:56.211 INFO:tasks.workunit.client.0.smithi060.stdout:4/342: dread d0/d7/f25 [0,4194304] 0 2024-02-26T10:47:56.213 INFO:tasks.workunit.client.0.smithi060.stdout:2/283: rename dd/d11/l23 to dd/d11/l59 0 2024-02-26T10:47:56.213 INFO:tasks.workunit.client.0.smithi060.stdout:2/284: chown dd/de/d41/d37 103456068 1 2024-02-26T10:47:56.213 INFO:tasks.workunit.client.0.smithi060.stdout:3/231: mknod d5/de/c3e 0 2024-02-26T10:47:56.214 INFO:tasks.workunit.client.0.smithi060.stdout:2/285: creat dd/d43/d55/f5a x:0 0 0 2024-02-26T10:47:56.215 INFO:tasks.workunit.client.0.smithi060.stdout:3/232: getdents d5 0 2024-02-26T10:47:56.238 INFO:tasks.workunit.client.0.smithi060.stdout:9/298: rename d3/d9 to d3/d1e/d5e/d1b/d5f 0 2024-02-26T10:47:56.240 INFO:tasks.workunit.client.0.smithi060.stdout:9/299: rename d3/d1e/c33 to d3/d1e/c60 0 2024-02-26T10:47:56.241 INFO:tasks.workunit.client.0.smithi060.stdout:1/384: write d3/d35/f65 [3005018,35788] 0 2024-02-26T10:47:56.241 INFO:tasks.workunit.client.0.smithi060.stdout:9/300: link d3/lb d3/d1e/d5e/d35/l61 0 2024-02-26T10:47:56.242 INFO:tasks.workunit.client.0.smithi060.stdout:1/385: creat d3/d35/d53/d57/f87 x:0 0 0 2024-02-26T10:47:56.242 INFO:tasks.workunit.client.0.smithi060.stdout:9/301: creat d3/d1e/d5e/d1b/d5f/f62 x:0 0 0 2024-02-26T10:47:56.248 INFO:tasks.workunit.client.0.smithi060.stdout:3/233: dread d5/f6 [0,4194304] 0 2024-02-26T10:47:56.248 INFO:tasks.workunit.client.0.smithi060.stdout:5/324: dwrite d0/d33/f63 [0,4194304] 0 2024-02-26T10:47:56.248 INFO:tasks.workunit.client.0.smithi060.stdout:3/234: write d5/de/df/d11/f34 [651033,55522] 0 2024-02-26T10:47:56.266 INFO:tasks.workunit.client.0.smithi060.stdout:9/302: write d3/d1e/d5e/d1b/d23/f34 [2168984,54514] 0 2024-02-26T10:47:56.266 INFO:tasks.workunit.client.0.smithi060.stdout:1/386: dread d3/d54/f7c [0,4194304] 0 2024-02-26T10:47:56.267 INFO:tasks.workunit.client.0.smithi060.stdout:1/387: write d3/f69 [147493,93238] 0 2024-02-26T10:47:56.267 INFO:tasks.workunit.client.0.smithi060.stdout:9/303: stat d3/d1e/d5e/d1b/d5f/dc/l24 0 2024-02-26T10:47:56.268 INFO:tasks.workunit.client.0.smithi060.stdout:9/304: write d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d44/f48 [4341041,130210] 0 2024-02-26T10:47:56.268 INFO:tasks.workunit.client.0.smithi060.stdout:1/388: rmdir d3/db/d1e/d38 39 2024-02-26T10:47:56.272 INFO:tasks.workunit.client.0.smithi060.stdout:1/389: creat d3/db/d1e/d38/d6c/f88 x:0 0 0 2024-02-26T10:47:56.272 INFO:tasks.workunit.client.0.smithi060.stdout:1/390: dread - d3/f5c zero size 2024-02-26T10:47:56.272 INFO:tasks.workunit.client.0.smithi060.stdout:1/391: dread - d3/db/d1e/d6d/d78/f86 zero size 2024-02-26T10:47:56.274 INFO:tasks.workunit.client.0.smithi060.stdout:1/392: creat d3/d35/d53/d68/d85/f89 x:0 0 0 2024-02-26T10:47:56.275 INFO:tasks.workunit.client.0.smithi060.stdout:1/393: mkdir d3/d35/d53/d68/d85/d8a 0 2024-02-26T10:47:56.275 INFO:tasks.workunit.client.0.smithi060.stdout:1/394: fdatasync d3/db/f59 0 2024-02-26T10:47:56.275 INFO:tasks.workunit.client.0.smithi060.stdout:1/395: chown d3/d35/d53/d57/l82 3365 1 2024-02-26T10:47:56.278 INFO:tasks.workunit.client.0.smithi060.stdout:1/396: mknod d3/db/d1e/d38/d6c/c8b 0 2024-02-26T10:47:56.278 INFO:tasks.workunit.client.0.smithi060.stdout:9/305: getdents d3/d1e/d5e/d1b/d5f 0 2024-02-26T10:47:56.279 INFO:tasks.workunit.client.0.smithi060.stdout:1/397: truncate d3/f1b 1827341 0 2024-02-26T10:47:56.280 INFO:tasks.workunit.client.0.smithi060.stdout:1/398: getdents d3/db/d1e/d39 0 2024-02-26T10:47:56.282 INFO:tasks.workunit.client.0.smithi060.stdout:1/399: getdents d3 0 2024-02-26T10:47:56.282 INFO:tasks.workunit.client.0.smithi060.stdout:3/235: dread d5/de/df/f21 [0,4194304] 0 2024-02-26T10:47:56.282 INFO:tasks.workunit.client.0.smithi060.stdout:2/286: dwrite f5 [0,4194304] 0 2024-02-26T10:47:56.283 INFO:tasks.workunit.client.0.smithi060.stdout:1/400: creat d3/d35/d53/d68/f8c x:0 0 0 2024-02-26T10:47:56.284 INFO:tasks.workunit.client.0.smithi060.stdout:9/306: creat d3/d1e/d5e/d1b/d5f/dc/d32/d3c/f63 x:0 0 0 2024-02-26T10:47:56.284 INFO:tasks.workunit.client.0.smithi060.stdout:2/287: truncate dd/de/f3d 915965 0 2024-02-26T10:47:56.285 INFO:tasks.workunit.client.0.smithi060.stdout:2/288: write dd/d43/d55/f5a [148972,51280] 0 2024-02-26T10:47:56.290 INFO:tasks.workunit.client.0.smithi060.stdout:2/289: rename dd/d43/d55/l58 to dd/de/d41/d37/d4c/l5b 0 2024-02-26T10:47:56.291 INFO:tasks.workunit.client.0.smithi060.stdout:2/290: mknod dd/de/d41/c5c 0 2024-02-26T10:47:56.293 INFO:tasks.workunit.client.0.smithi060.stdout:2/291: rename dd/l3b to dd/de/d41/d37/d38/l5d 0 2024-02-26T10:47:56.318 INFO:tasks.workunit.client.0.smithi060.stdout:5/325: dwrite d0/de/d15/d26/f39 [4194304,4194304] 0 2024-02-26T10:47:56.335 INFO:tasks.workunit.client.0.smithi060.stdout:1/401: dwrite d3/db/d1e/d39/f48 [0,4194304] 0 2024-02-26T10:47:56.335 INFO:tasks.workunit.client.0.smithi060.stdout:1/402: read - d3/d35/d53/f74 zero size 2024-02-26T10:47:56.335 INFO:tasks.workunit.client.0.smithi060.stdout:2/292: dwrite dd/d11/f4e [0,4194304] 0 2024-02-26T10:47:56.336 INFO:tasks.workunit.client.0.smithi060.stdout:1/403: mknod d3/d35/d53/d68/d6b/c8d 0 2024-02-26T10:47:56.336 INFO:tasks.workunit.client.0.smithi060.stdout:1/404: dread - d3/db/f59 zero size 2024-02-26T10:47:56.338 INFO:tasks.workunit.client.0.smithi060.stdout:2/293: mkdir dd/d43/d55/d5e 0 2024-02-26T10:47:56.345 INFO:tasks.workunit.client.0.smithi060.stdout:5/326: dread d0/de/f1f [0,4194304] 0 2024-02-26T10:47:56.345 INFO:tasks.workunit.client.0.smithi060.stdout:5/327: fdatasync d0/de/d20/f56 0 2024-02-26T10:47:56.345 INFO:tasks.workunit.client.0.smithi060.stdout:5/328: write d0/d33/f5c [122250,76221] 0 2024-02-26T10:47:56.345 INFO:tasks.workunit.client.0.smithi060.stdout:5/329: dread - d0/de/d20/d32/d4c/f74 zero size 2024-02-26T10:47:56.350 INFO:tasks.workunit.client.0.smithi060.stdout:5/330: symlink d0/de/d15/d65/l7a 0 2024-02-26T10:47:56.350 INFO:tasks.workunit.client.0.smithi060.stdout:5/331: write d0/de/d15/f2d [246624,38346] 0 2024-02-26T10:47:56.350 INFO:tasks.workunit.client.0.smithi060.stdout:2/294: mknod dd/d11/c5f 0 2024-02-26T10:47:56.350 INFO:tasks.workunit.client.0.smithi060.stdout:2/295: readlink dd/l14 0 2024-02-26T10:47:56.350 INFO:tasks.workunit.client.0.smithi060.stdout:9/307: dwrite d3/d1e/d5e/d1b/d5f/d10/f2c [0,4194304] 0 2024-02-26T10:47:56.350 INFO:tasks.workunit.client.0.smithi060.stdout:9/308: read - d3/d1e/d5e/d1b/d5f/d10/f4e zero size 2024-02-26T10:47:56.353 INFO:tasks.workunit.client.0.smithi060.stdout:2/296: rename dd/de to dd/d43/d60 0 2024-02-26T10:47:56.374 INFO:tasks.workunit.client.0.smithi060.stdout:2/297: dread dd/d11/f24 [0,4194304] 0 2024-02-26T10:47:56.374 INFO:tasks.workunit.client.0.smithi060.stdout:9/309: rename d3/d1e/d5e/d1b/d5f/dc/d32/d3a/l55 to d3/d1e/l64 0 2024-02-26T10:47:56.375 INFO:tasks.workunit.client.0.smithi060.stdout:2/298: symlink dd/d43/d60/d41/d37/d38/l61 0 2024-02-26T10:47:56.377 INFO:tasks.workunit.client.0.smithi060.stdout:9/310: truncate d3/d1e/d5e/d1b/d5f/d10/f11 3889685 0 2024-02-26T10:47:56.386 INFO:tasks.workunit.client.0.smithi060.stdout:9/311: link d3/d1e/d5e/d1b/f56 d3/d1e/d5e/d1b/d5f/dc/d32/d3c/f65 0 2024-02-26T10:47:56.390 INFO:tasks.workunit.client.0.smithi060.stdout:1/405: dwrite d3/f5c [0,4194304] 0 2024-02-26T10:47:56.391 INFO:tasks.workunit.client.0.smithi060.stdout:1/406: dread - d3/db/d1e/d39/f58 zero size 2024-02-26T10:47:56.392 INFO:tasks.workunit.client.0.smithi060.stdout:1/407: creat d3/db/d1e/d38/f8e x:0 0 0 2024-02-26T10:47:56.392 INFO:tasks.workunit.client.0.smithi060.stdout:1/408: read - d3/d35/d53/d68/f8c zero size 2024-02-26T10:47:56.393 INFO:tasks.workunit.client.0.smithi060.stdout:5/332: dwrite d0/de/d15/f18 [0,4194304] 0 2024-02-26T10:47:56.393 INFO:tasks.workunit.client.0.smithi060.stdout:9/312: getdents d3/d1e/d5e/d1b/d5f/dc/d32/d3a 0 2024-02-26T10:47:56.394 INFO:tasks.workunit.client.0.smithi060.stdout:5/333: mkdir d0/d4b/d7b 0 2024-02-26T10:47:56.394 INFO:tasks.workunit.client.0.smithi060.stdout:5/334: stat d0/d33/l77 0 2024-02-26T10:47:56.394 INFO:tasks.workunit.client.0.smithi060.stdout:9/313: getdents d3/d1e 0 2024-02-26T10:47:56.394 INFO:tasks.workunit.client.0.smithi060.stdout:9/314: chown d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/c53 7410778 1 2024-02-26T10:47:56.396 INFO:tasks.workunit.client.0.smithi060.stdout:9/315: mknod d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d44/c66 0 2024-02-26T10:47:56.396 INFO:tasks.workunit.client.0.smithi060.stdout:9/316: chown d3/d1e/d5e/d1b/d5f/dc/d32/d3c 2 1 2024-02-26T10:47:56.397 INFO:tasks.workunit.client.0.smithi060.stdout:9/317: truncate d3/d1e/d5e/d1b/d5f/f62 406209 0 2024-02-26T10:47:56.397 INFO:tasks.workunit.client.0.smithi060.stdout:9/318: write d3/d1e/f58 [616121,73468] 0 2024-02-26T10:47:56.397 INFO:tasks.workunit.client.0.smithi060.stdout:9/319: rmdir d3/d1e/d5e/d1b 39 2024-02-26T10:47:56.399 INFO:tasks.workunit.client.0.smithi060.stdout:5/335: truncate d0/de/d15/f2d 200407 0 2024-02-26T10:47:56.402 INFO:tasks.workunit.client.0.smithi060.stdout:5/336: rename d0/f9 to d0/de/d15/d65/f7c 0 2024-02-26T10:47:56.406 INFO:tasks.workunit.client.0.smithi060.stdout:5/337: rename d0/de/l64 to d0/de/d20/d32/l7d 0 2024-02-26T10:47:56.410 INFO:tasks.workunit.client.0.smithi060.stdout:9/320: dread d3/d1e/d5e/d1b/d5f/f2f [4194304,4194304] 0 2024-02-26T10:47:56.413 INFO:tasks.workunit.client.0.smithi060.stdout:9/321: unlink d3/d1e/d5e/d1b/d5f/l46 0 2024-02-26T10:47:56.414 INFO:tasks.workunit.client.0.smithi060.stdout:9/322: symlink d3/d1e/d57/l67 0 2024-02-26T10:47:56.415 INFO:tasks.workunit.client.0.smithi060.stdout:9/323: truncate d3/d1e/d5e/d1b/d5f/dc/d32/d3c/f65 996259 0 2024-02-26T10:47:56.415 INFO:tasks.workunit.client.0.smithi060.stdout:9/324: write d3/d1e/d5e/d1b/d5f/dc/f3b [2872817,98223] 0 2024-02-26T10:47:56.416 INFO:tasks.workunit.client.0.smithi060.stdout:9/325: write d3/d1e/d5e/d1b/d5f/f38 [4675883,34475] 0 2024-02-26T10:47:56.418 INFO:tasks.workunit.client.0.smithi060.stdout:2/299: dwrite f4 [0,4194304] 0 2024-02-26T10:47:56.419 INFO:tasks.workunit.client.0.smithi060.stdout:2/300: creat dd/d11/f62 x:0 0 0 2024-02-26T10:47:56.420 INFO:tasks.workunit.client.0.smithi060.stdout:2/301: fdatasync dd/d43/d60/d41/d37/d38/f40 0 2024-02-26T10:47:56.422 INFO:tasks.workunit.client.0.smithi060.stdout:2/302: rmdir dd/d43/d60/d41/d42 39 2024-02-26T10:47:56.427 INFO:tasks.workunit.client.0.smithi060.stdout:5/338: write d0/de/d15/d26/f41 [700077,117432] 0 2024-02-26T10:47:56.434 INFO:tasks.workunit.client.0.smithi060.stdout:5/339: truncate d0/de/d20/d32/d4d/d60/f75 132965 0 2024-02-26T10:47:56.435 INFO:tasks.workunit.client.0.smithi060.stdout:1/409: dwrite d3/d35/d53/d57/f87 [0,4194304] 0 2024-02-26T10:47:56.435 INFO:tasks.workunit.client.0.smithi060.stdout:1/410: readlink d3/db/d1e/d38/l5b 0 2024-02-26T10:47:56.435 INFO:tasks.workunit.client.0.smithi060.stdout:1/411: write d3/db/d1e/d38/f6e [855332,64679] 0 2024-02-26T10:47:56.437 INFO:tasks.workunit.client.0.smithi060.stdout:1/412: creat d3/db/d1e/d39/f8f x:0 0 0 2024-02-26T10:47:56.451 INFO:tasks.workunit.client.0.smithi060.stdout:5/340: write d0/de/f29 [1634959,53940] 0 2024-02-26T10:47:56.452 INFO:tasks.workunit.client.0.smithi060.stdout:5/341: symlink d0/de/l7e 0 2024-02-26T10:47:56.453 INFO:tasks.workunit.client.0.smithi060.stdout:5/342: dread - d0/de/d20/d32/d4c/f74 zero size 2024-02-26T10:47:56.460 INFO:tasks.workunit.client.0.smithi060.stdout:9/326: dwrite d3/d1e/d5e/d1b/d23/f36 [0,4194304] 0 2024-02-26T10:47:56.476 INFO:tasks.workunit.client.0.smithi060.stdout:9/327: getdents d3/d1e/d5e/d1b/d5f/dc/d32/d3c 0 2024-02-26T10:47:56.480 INFO:tasks.workunit.client.0.smithi060.stdout:1/413: dwrite d3/d54/f7c [0,4194304] 0 2024-02-26T10:47:56.482 INFO:tasks.workunit.client.0.smithi060.stdout:1/414: mkdir d3/d35/d53/d68/d90 0 2024-02-26T10:47:56.484 INFO:tasks.workunit.client.0.smithi060.stdout:1/415: mkdir d3/d35/d53/d68/d85/d8a/d91 0 2024-02-26T10:47:56.496 INFO:tasks.workunit.client.0.smithi060.stdout:5/343: dwrite d0/de/d15/f44 [0,4194304] 0 2024-02-26T10:47:56.496 INFO:tasks.workunit.client.0.smithi060.stdout:5/344: write d0/de/f35 [1495830,52864] 0 2024-02-26T10:47:56.501 INFO:tasks.workunit.client.0.smithi060.stdout:9/328: dread d3/d1e/d5e/d1b/d5f/f5b [0,4194304] 0 2024-02-26T10:47:56.503 INFO:tasks.workunit.client.0.smithi060.stdout:9/329: mkdir d3/d68 0 2024-02-26T10:47:56.516 INFO:tasks.workunit.client.0.smithi060.stdout:9/330: write d3/d1e/d5e/d1b/f56 [820743,38903] 0 2024-02-26T10:47:56.516 INFO:tasks.workunit.client.0.smithi060.stdout:5/345: dread d0/de/d15/d26/f45 [0,4194304] 0 2024-02-26T10:47:56.518 INFO:tasks.workunit.client.0.smithi060.stdout:9/331: mknod d3/d1e/d5e/d1b/d5f/dc/c69 0 2024-02-26T10:47:56.518 INFO:tasks.workunit.client.0.smithi060.stdout:9/332: rename d3/d1e/d5e/d1b/d5f/dc to d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/d6a 22 2024-02-26T10:47:56.521 INFO:tasks.workunit.client.0.smithi060.stdout:1/416: dwrite d3/d35/d53/d68/f8c [0,4194304] 0 2024-02-26T10:47:56.523 INFO:tasks.workunit.client.0.smithi060.stdout:1/417: creat d3/db/d2e/d51/d63/f92 x:0 0 0 2024-02-26T10:47:56.524 INFO:tasks.workunit.client.0.smithi060.stdout:1/418: chown d3/db/d1e/c25 4 1 2024-02-26T10:47:56.524 INFO:tasks.workunit.client.0.smithi060.stdout:1/419: dread - d3/d35/d53/d57/f71 zero size 2024-02-26T10:47:56.535 INFO:tasks.workunit.client.0.smithi060.stdout:5/346: dread d0/f8 [0,4194304] 0 2024-02-26T10:47:56.539 INFO:tasks.workunit.client.0.smithi060.stdout:9/333: dread d3/d1e/d5e/d1b/d5f/f1f [0,4194304] 0 2024-02-26T10:47:56.539 INFO:tasks.workunit.client.0.smithi060.stdout:9/334: chown d3/d1e/c60 21445 1 2024-02-26T10:47:56.540 INFO:tasks.workunit.client.0.smithi060.stdout:9/335: creat d3/f6b x:0 0 0 2024-02-26T10:47:56.548 INFO:tasks.workunit.client.0.smithi060.stdout:1/420: read d3/db/f40 [180730,84439] 0 2024-02-26T10:47:56.549 INFO:tasks.workunit.client.0.smithi060.stdout:9/336: creat d3/d1e/d5e/d1b/d5f/d10/f6c x:0 0 0 2024-02-26T10:47:56.549 INFO:tasks.workunit.client.0.smithi060.stdout:9/337: chown d3/d1e/d5e/d1b/l22 61626852 1 2024-02-26T10:47:56.551 INFO:tasks.workunit.client.0.smithi060.stdout:9/338: stat d3/d1e/d5e/d1b/c1d 0 2024-02-26T10:47:56.581 INFO:tasks.workunit.client.0.smithi060.stdout:9/339: write d3/d1e/d5e/d1b/d23/f36 [475246,35928] 0 2024-02-26T10:47:56.582 INFO:tasks.workunit.client.0.smithi060.stdout:5/347: dwrite d0/d33/f3e [0,4194304] 0 2024-02-26T10:47:56.582 INFO:tasks.workunit.client.0.smithi060.stdout:5/348: fsync d0/de/d15/f2f 0 2024-02-26T10:47:56.585 INFO:tasks.workunit.client.0.smithi060.stdout:9/340: write d3/d1e/f58 [290317,19997] 0 2024-02-26T10:47:56.591 INFO:tasks.workunit.client.0.smithi060.stdout:9/341: creat d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f6d x:0 0 0 2024-02-26T10:47:56.591 INFO:tasks.workunit.client.0.smithi060.stdout:9/342: write d3/d1e/d5e/d1b/d5f/d10/f4f [1083957,100612] 0 2024-02-26T10:47:56.593 INFO:tasks.workunit.client.0.smithi060.stdout:5/349: rmdir d0 39 2024-02-26T10:47:56.595 INFO:tasks.workunit.client.0.smithi060.stdout:5/350: rename d0/de/d15/d26/f39 to d0/de/d15/f7f 0 2024-02-26T10:47:56.595 INFO:tasks.workunit.client.0.smithi060.stdout:1/421: dwrite d3/d54/f7c [4194304,4194304] 0 2024-02-26T10:47:56.599 INFO:tasks.workunit.client.0.smithi060.stdout:1/422: rename d3/c13 to d3/d35/d53/d68/d85/d8a/d91/c93 0 2024-02-26T10:47:56.601 INFO:tasks.workunit.client.0.smithi060.stdout:5/351: unlink d0/f8 0 2024-02-26T10:47:56.601 INFO:tasks.workunit.client.0.smithi060.stdout:1/423: creat d3/d35/d53/d68/d85/f94 x:0 0 0 2024-02-26T10:47:56.601 INFO:tasks.workunit.client.0.smithi060.stdout:1/424: write d3/db/f56 [751663,93446] 0 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:7/435: sync 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:4/343: sync 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:2/303: sync 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:3/236: sync 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:0/301: sync 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:6/358: sync 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:3/237: truncate d5/f24 877955 0 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:3/238: fsync d5/de/df/d11/f20 0 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:8/288: sync 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:8/289: stat d9/d15/d3e 0 2024-02-26T10:47:56.603 INFO:tasks.workunit.client.0.smithi060.stdout:5/352: symlink d0/de/d20/d32/d4d/d60/l80 0 2024-02-26T10:47:56.607 INFO:tasks.workunit.client.0.smithi060.stdout:1/425: truncate d3/f2a 5355442 0 2024-02-26T10:47:56.607 INFO:tasks.workunit.client.0.smithi060.stdout:1/426: dread - d3/d35/d53/f74 zero size 2024-02-26T10:47:56.607 INFO:tasks.workunit.client.0.smithi060.stdout:7/436: rmdir da/d15/d21/d4b 39 2024-02-26T10:47:56.607 INFO:tasks.workunit.client.0.smithi060.stdout:0/302: creat d0/dd/d39/d63/f67 x:0 0 0 2024-02-26T10:47:56.607 INFO:tasks.workunit.client.0.smithi060.stdout:0/303: fsync d0/dd/d39/f3f 0 2024-02-26T10:47:56.608 INFO:tasks.workunit.client.0.smithi060.stdout:0/304: fsync d0/d37/d44/d58/f5b 0 2024-02-26T10:47:56.609 INFO:tasks.workunit.client.0.smithi060.stdout:8/290: creat d9/d15/f52 x:0 0 0 2024-02-26T10:47:56.609 INFO:tasks.workunit.client.0.smithi060.stdout:7/437: fsync da/d15/f17 0 2024-02-26T10:47:56.609 INFO:tasks.workunit.client.0.smithi060.stdout:1/427: creat d3/db/d2e/f95 x:0 0 0 2024-02-26T10:47:56.609 INFO:tasks.workunit.client.0.smithi060.stdout:1/428: readlink d3/d54/l84 0 2024-02-26T10:47:56.609 INFO:tasks.workunit.client.0.smithi060.stdout:1/429: read d3/db/d2e/f4d [1717856,62539] 0 2024-02-26T10:47:56.610 INFO:tasks.workunit.client.0.smithi060.stdout:0/305: mknod d0/d37/d44/d58/c68 0 2024-02-26T10:47:56.610 INFO:tasks.workunit.client.0.smithi060.stdout:0/306: chown d0/dd/f4e 94435719 1 2024-02-26T10:47:56.612 INFO:tasks.workunit.client.0.smithi060.stdout:0/307: write d0/dd/d39/d40/f34 [5498524,103824] 0 2024-02-26T10:47:56.612 INFO:tasks.workunit.client.0.smithi060.stdout:8/291: creat d9/d15/d40/f53 x:0 0 0 2024-02-26T10:47:56.612 INFO:tasks.workunit.client.0.smithi060.stdout:8/292: readlink d9/d15/d40/l48 0 2024-02-26T10:47:56.612 INFO:tasks.workunit.client.0.smithi060.stdout:8/293: readlink d9/d15/d29/l4e 0 2024-02-26T10:47:56.612 INFO:tasks.workunit.client.0.smithi060.stdout:7/438: read - da/d15/d88/d3b/f8c zero size 2024-02-26T10:47:56.612 INFO:tasks.workunit.client.0.smithi060.stdout:1/430: creat d3/f96 x:0 0 0 2024-02-26T10:47:56.612 INFO:tasks.workunit.client.0.smithi060.stdout:7/439: truncate da/d15/f46 2221877 0 2024-02-26T10:47:56.612 INFO:tasks.workunit.client.0.smithi060.stdout:7/440: fsync da/d15/d21/d32/f33 0 2024-02-26T10:47:56.612 INFO:tasks.workunit.client.0.smithi060.stdout:8/294: mknod d9/d19/d50/c54 0 2024-02-26T10:47:56.613 INFO:tasks.workunit.client.0.smithi060.stdout:1/431: creat d3/d35/d5e/f97 x:0 0 0 2024-02-26T10:47:56.613 INFO:tasks.workunit.client.0.smithi060.stdout:1/432: read - d3/db/d1e/d38/f8e zero size 2024-02-26T10:47:56.613 INFO:tasks.workunit.client.0.smithi060.stdout:7/441: creat da/d15/d21/d2f/d37/f8e x:0 0 0 2024-02-26T10:47:56.615 INFO:tasks.workunit.client.0.smithi060.stdout:5/353: dread d0/de/f22 [0,4194304] 0 2024-02-26T10:47:56.621 INFO:tasks.workunit.client.0.smithi060.stdout:9/343: read d3/d1e/d5e/f5c [2786170,31777] 0 2024-02-26T10:47:56.628 INFO:tasks.workunit.client.0.smithi060.stdout:7/442: rmdir da/d15/d21/d2f/d78 39 2024-02-26T10:47:56.630 INFO:tasks.workunit.client.0.smithi060.stdout:9/344: link d3/d1e/c60 d3/d1e/d5e/d1b/c6e 0 2024-02-26T10:47:56.630 INFO:tasks.workunit.client.0.smithi060.stdout:7/443: mkdir da/d15/d88/d39/d8f 0 2024-02-26T10:47:56.631 INFO:tasks.workunit.client.0.smithi060.stdout:9/345: mknod d3/d1e/d5e/d35/c6f 0 2024-02-26T10:47:56.632 INFO:tasks.workunit.client.0.smithi060.stdout:7/444: creat da/d66/f90 x:0 0 0 2024-02-26T10:47:56.633 INFO:tasks.workunit.client.0.smithi060.stdout:7/445: write da/d66/f75 [2357205,23157] 0 2024-02-26T10:47:56.635 INFO:tasks.workunit.client.0.smithi060.stdout:7/446: creat da/d15/d21/d2f/f91 x:0 0 0 2024-02-26T10:47:56.640 INFO:tasks.workunit.client.0.smithi060.stdout:7/447: mknod da/d15/d21/d4b/d67/c92 0 2024-02-26T10:47:56.641 INFO:tasks.workunit.client.0.smithi060.stdout:2/304: read dd/d11/f24 [3068555,46079] 0 2024-02-26T10:47:56.643 INFO:tasks.workunit.client.0.smithi060.stdout:9/346: creat d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/f70 x:0 0 0 2024-02-26T10:47:56.645 INFO:tasks.workunit.client.0.smithi060.stdout:9/347: symlink d3/d1e/d5e/d1b/d5f/dc/d32/d3c/l71 0 2024-02-26T10:47:56.645 INFO:tasks.workunit.client.0.smithi060.stdout:5/354: dread d0/de/d20/f58 [0,4194304] 0 2024-02-26T10:47:56.647 INFO:tasks.workunit.client.0.smithi060.stdout:9/348: mkdir d3/d1e/d5e/d1b/d23/d72 0 2024-02-26T10:47:56.647 INFO:tasks.workunit.client.0.smithi060.stdout:2/305: rmdir dd/d43/d60/d41/d37 39 2024-02-26T10:47:56.650 INFO:tasks.workunit.client.0.smithi060.stdout:1/433: dread d3/db/d2e/f4d [0,4194304] 0 2024-02-26T10:47:56.650 INFO:tasks.workunit.client.0.smithi060.stdout:1/434: chown d3/db/d1e/c25 0 1 2024-02-26T10:47:56.650 INFO:tasks.workunit.client.0.smithi060.stdout:9/349: rmdir d3/d1e/d5e/d1b/d23/d72 0 2024-02-26T10:47:56.652 INFO:tasks.workunit.client.0.smithi060.stdout:9/350: mknod d3/d1e/d5e/d1b/c73 0 2024-02-26T10:47:56.653 INFO:tasks.workunit.client.0.smithi060.stdout:2/306: symlink dd/d52/l63 0 2024-02-26T10:47:56.653 INFO:tasks.workunit.client.0.smithi060.stdout:2/307: chown dd/c56 54503 1 2024-02-26T10:47:56.656 INFO:tasks.workunit.client.0.smithi060.stdout:4/344: dread d0/f5 [0,4194304] 0 2024-02-26T10:47:56.656 INFO:tasks.workunit.client.0.smithi060.stdout:4/345: stat d0/d7/f5a 0 2024-02-26T10:47:56.659 INFO:tasks.workunit.client.0.smithi060.stdout:4/346: getdents d0/da 0 2024-02-26T10:47:56.660 INFO:tasks.workunit.client.0.smithi060.stdout:2/308: unlink dd/d43/d60/d41/l20 0 2024-02-26T10:47:56.660 INFO:tasks.workunit.client.0.smithi060.stdout:2/309: chown dd/d43/d60/d41/c5c 444 1 2024-02-26T10:47:56.660 INFO:tasks.workunit.client.0.smithi060.stdout:2/310: chown dd/d43/d60/d41/d37/d4c/l5b 17 1 2024-02-26T10:47:56.660 INFO:tasks.workunit.client.0.smithi060.stdout:2/311: dread - dd/d43/d60/d41/d37/d38/f40 zero size 2024-02-26T10:47:56.660 INFO:tasks.workunit.client.0.smithi060.stdout:4/347: fsync d0/d7/f5a 0 2024-02-26T10:47:56.660 INFO:tasks.workunit.client.0.smithi060.stdout:9/351: mkdir d3/d1e/d5e/d1b/d5f/d74 0 2024-02-26T10:47:56.661 INFO:tasks.workunit.client.0.smithi060.stdout:4/348: creat d0/d27/d52/f62 x:0 0 0 2024-02-26T10:47:56.664 INFO:tasks.workunit.client.0.smithi060.stdout:9/352: mknod d3/d1e/d5e/d1b/c75 0 2024-02-26T10:47:56.664 INFO:tasks.workunit.client.0.smithi060.stdout:9/353: chown d3/d1e/d5e/d1b/d5f/l4a 1599 1 2024-02-26T10:47:56.664 INFO:tasks.workunit.client.0.smithi060.stdout:4/349: unlink d0/da/c21 0 2024-02-26T10:47:56.664 INFO:tasks.workunit.client.0.smithi060.stdout:4/350: rmdir d0/da/d3b/d16 39 2024-02-26T10:47:56.664 INFO:tasks.workunit.client.0.smithi060.stdout:4/351: fsync d0/d7/f8 0 2024-02-26T10:47:56.665 INFO:tasks.workunit.client.0.smithi060.stdout:4/352: mknod d0/d7/d29/c63 0 2024-02-26T10:47:56.677 INFO:tasks.workunit.client.0.smithi060.stdout:8/295: dwrite d9/d19/f4c [0,4194304] 0 2024-02-26T10:47:56.681 INFO:tasks.workunit.client.0.smithi060.stdout:8/296: rmdir d9/d15/d40 39 2024-02-26T10:47:56.681 INFO:tasks.workunit.client.0.smithi060.stdout:8/297: write d9/d15/d40/f46 [510511,2519] 0 2024-02-26T10:47:56.681 INFO:tasks.workunit.client.0.smithi060.stdout:8/298: chown d9/d15/f51 2 1 2024-02-26T10:47:56.687 INFO:tasks.workunit.client.0.smithi060.stdout:3/239: dwrite d5/de/d16/f2d [0,4194304] 0 2024-02-26T10:47:56.689 INFO:tasks.workunit.client.0.smithi060.stdout:3/240: link d5/l2e d5/de/df/l3f 0 2024-02-26T10:47:56.690 INFO:tasks.workunit.client.0.smithi060.stdout:5/355: dread d0/de/f12 [0,4194304] 0 2024-02-26T10:47:56.691 INFO:tasks.workunit.client.0.smithi060.stdout:3/241: rmdir d5/de/d16 39 2024-02-26T10:47:56.691 INFO:tasks.workunit.client.0.smithi060.stdout:3/242: chown d5/de/l10 7185 1 2024-02-26T10:47:56.692 INFO:tasks.workunit.client.0.smithi060.stdout:5/356: unlink d0/f36 0 2024-02-26T10:47:56.702 INFO:tasks.workunit.client.0.smithi060.stdout:4/353: write d0/da/d3b/f2b [2269879,130547] 0 2024-02-26T10:47:56.709 INFO:tasks.workunit.client.0.smithi060.stdout:6/359: dwrite d6/f18 [0,4194304] 0 2024-02-26T10:47:56.714 INFO:tasks.workunit.client.0.smithi060.stdout:1/435: dwrite d3/d35/d53/d68/f8c [0,4194304] 0 2024-02-26T10:47:56.715 INFO:tasks.workunit.client.0.smithi060.stdout:9/354: dread d3/d1e/d5e/d1b/d5f/f38 [0,4194304] 0 2024-02-26T10:47:56.716 INFO:tasks.workunit.client.0.smithi060.stdout:3/243: dread d5/f24 [0,4194304] 0 2024-02-26T10:47:56.716 INFO:tasks.workunit.client.0.smithi060.stdout:3/244: chown d5/de/df/d11/c18 148416 1 2024-02-26T10:47:56.716 INFO:tasks.workunit.client.0.smithi060.stdout:9/355: mknod d3/d1e/d5e/d1b/c76 0 2024-02-26T10:47:56.719 INFO:tasks.workunit.client.0.smithi060.stdout:3/245: link d5/de/df/d11/l12 d5/de/d16/l40 0 2024-02-26T10:47:56.720 INFO:tasks.workunit.client.0.smithi060.stdout:3/246: fdatasync d5/de/df/d11/f20 0 2024-02-26T10:47:56.720 INFO:tasks.workunit.client.0.smithi060.stdout:3/247: chown d5/de/df/d11/f20 0 1 2024-02-26T10:47:56.720 INFO:tasks.workunit.client.0.smithi060.stdout:3/248: fsync d5/fd 0 2024-02-26T10:47:56.720 INFO:tasks.workunit.client.0.smithi060.stdout:3/249: read d5/f24 [622526,3120] 0 2024-02-26T10:47:56.724 INFO:tasks.workunit.client.0.smithi060.stdout:0/308: dwrite d0/dd/d39/f3f [0,4194304] 0 2024-02-26T10:47:56.724 INFO:tasks.workunit.client.0.smithi060.stdout:0/309: truncate d0/dd/d33/f4a 131318 0 2024-02-26T10:47:56.724 INFO:tasks.workunit.client.0.smithi060.stdout:7/448: dwrite da/d15/d21/f72 [0,4194304] 0 2024-02-26T10:47:56.727 INFO:tasks.workunit.client.0.smithi060.stdout:7/449: link da/d15/d21/f72 da/d15/d21/d4b/f93 0 2024-02-26T10:47:56.728 INFO:tasks.workunit.client.0.smithi060.stdout:7/450: chown da/d15/d21/d2f/f68 10490802 1 2024-02-26T10:47:56.730 INFO:tasks.workunit.client.0.smithi060.stdout:7/451: mkdir da/d94 0 2024-02-26T10:47:56.732 INFO:tasks.workunit.client.0.smithi060.stdout:7/452: creat da/d15/d21/d4b/f95 x:0 0 0 2024-02-26T10:47:56.734 INFO:tasks.workunit.client.0.smithi060.stdout:7/453: creat da/d15/d21/d2f/d78/f96 x:0 0 0 2024-02-26T10:47:56.735 INFO:tasks.workunit.client.0.smithi060.stdout:7/454: rmdir da/d15/d21/d32 39 2024-02-26T10:47:56.735 INFO:tasks.workunit.client.0.smithi060.stdout:2/312: dwrite dd/d11/f24 [0,4194304] 0 2024-02-26T10:47:56.735 INFO:tasks.workunit.client.0.smithi060.stdout:7/455: write da/d15/d88/f5c [4326791,101847] 0 2024-02-26T10:47:56.741 INFO:tasks.workunit.client.0.smithi060.stdout:4/354: rename d0/da/d3b/d16/f20 to d0/d7/d29/f64 0 2024-02-26T10:47:56.750 INFO:tasks.workunit.client.0.smithi060.stdout:8/299: write d9/f1e [3751819,116664] 0 2024-02-26T10:47:56.752 INFO:tasks.workunit.client.0.smithi060.stdout:8/300: creat d9/d15/d29/d2f/f55 x:0 0 0 2024-02-26T10:47:56.753 INFO:tasks.workunit.client.0.smithi060.stdout:8/301: unlink d9/d15/d29/d2f/d34/c4b 0 2024-02-26T10:47:56.758 INFO:tasks.workunit.client.0.smithi060.stdout:8/302: symlink d9/d15/d3e/l56 0 2024-02-26T10:47:56.758 INFO:tasks.workunit.client.0.smithi060.stdout:8/303: write f8 [1125689,53558] 0 2024-02-26T10:47:56.761 INFO:tasks.workunit.client.0.smithi060.stdout:0/310: read d0/dd/d39/d40/f17 [3282040,55235] 0 2024-02-26T10:47:56.763 INFO:tasks.workunit.client.0.smithi060.stdout:0/311: truncate d0/dd/f31 1753448 0 2024-02-26T10:47:56.763 INFO:tasks.workunit.client.0.smithi060.stdout:0/312: dread - d0/dd/d53/f64 zero size 2024-02-26T10:47:56.763 INFO:tasks.workunit.client.0.smithi060.stdout:7/456: link da/d15/d21/d29/f4a da/d15/d21/d41/f97 0 2024-02-26T10:47:56.764 INFO:tasks.workunit.client.0.smithi060.stdout:4/355: link d0/da/d3b/d16/f18 d0/da/d57/f65 0 2024-02-26T10:47:56.775 INFO:tasks.workunit.client.0.smithi060.stdout:4/356: creat d0/d27/d2f/f66 x:0 0 0 2024-02-26T10:47:56.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/356: dread d3/d1e/d5e/d1b/d23/f36 [0,4194304] 0 2024-02-26T10:47:56.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/357: fdatasync d3/d1e/d5e/d1b/d5f/f1f 0 2024-02-26T10:47:56.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/358: readlink d3/d1e/d5e/d1b/d5f/dc/l24 0 2024-02-26T10:47:56.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/359: fdatasync d3/d1e/d5e/d1b/d5f/dc/d32/d3c/f65 0 2024-02-26T10:47:56.779 INFO:tasks.workunit.client.0.smithi060.stdout:0/313: mknod d0/d37/c69 0 2024-02-26T10:47:56.779 INFO:tasks.workunit.client.0.smithi060.stdout:0/314: write d0/dd/f5d [99063,104450] 0 2024-02-26T10:47:56.779 INFO:tasks.workunit.client.0.smithi060.stdout:7/457: rmdir da/d15/d21/d41/d58 39 2024-02-26T10:47:56.788 INFO:tasks.workunit.client.0.smithi060.stdout:0/315: mkdir d0/dd/d33/d6a 0 2024-02-26T10:47:56.788 INFO:tasks.workunit.client.0.smithi060.stdout:0/316: fsync d0/dd/d39/d40/f4d 0 2024-02-26T10:47:56.791 INFO:tasks.workunit.client.0.smithi060.stdout:5/357: dwrite d0/de/d20/d32/d4d/d60/f72 [0,4194304] 0 2024-02-26T10:47:56.792 INFO:tasks.workunit.client.0.smithi060.stdout:0/317: unlink d0/f1 0 2024-02-26T10:47:56.794 INFO:tasks.workunit.client.0.smithi060.stdout:6/360: dwrite d6/f18 [0,4194304] 0 2024-02-26T10:47:56.794 INFO:tasks.workunit.client.0.smithi060.stdout:6/361: chown d6/d7/d25/d1c/d1f/d23/f50 41910 1 2024-02-26T10:47:56.797 INFO:tasks.workunit.client.0.smithi060.stdout:3/250: dwrite d5/fd [0,4194304] 0 2024-02-26T10:47:56.800 INFO:tasks.workunit.client.0.smithi060.stdout:6/362: creat d6/f71 x:0 0 0 2024-02-26T10:47:56.801 INFO:tasks.workunit.client.0.smithi060.stdout:3/251: creat d5/d37/f41 x:0 0 0 2024-02-26T10:47:56.802 INFO:tasks.workunit.client.0.smithi060.stdout:6/363: creat d6/d7/d47/f72 x:0 0 0 2024-02-26T10:47:56.810 INFO:tasks.workunit.client.0.smithi060.stdout:2/313: dwrite dd/d43/d60/f3f [4194304,4194304] 0 2024-02-26T10:47:56.812 INFO:tasks.workunit.client.0.smithi060.stdout:6/364: read d6/d7/d25/d1c/d1f/d23/d28/f33 [321421,102479] 0 2024-02-26T10:47:56.814 INFO:tasks.workunit.client.0.smithi060.stdout:2/314: unlink dd/d11/c29 0 2024-02-26T10:47:56.814 INFO:tasks.workunit.client.0.smithi060.stdout:2/315: stat dd/d43/d60/f1e 0 2024-02-26T10:47:56.814 INFO:tasks.workunit.client.0.smithi060.stdout:2/316: readlink dd/l2c 0 2024-02-26T10:47:56.822 INFO:tasks.workunit.client.0.smithi060.stdout:9/360: rename d3/d1e/d5e/d1b/d5f/l1a to d3/d68/l77 0 2024-02-26T10:47:56.822 INFO:tasks.workunit.client.0.smithi060.stdout:9/361: write d3/d1e/d5e/d1b/d5f/dc/f3b [2891141,83527] 0 2024-02-26T10:47:56.822 INFO:tasks.workunit.client.0.smithi060.stdout:9/362: write d3/d1e/d5e/d1b/d5f/dc/d32/d3c/f65 [1474039,52918] 0 2024-02-26T10:47:56.822 INFO:tasks.workunit.client.0.smithi060.stdout:9/363: write d3/d1e/f58 [380829,116673] 0 2024-02-26T10:47:56.822 INFO:tasks.workunit.client.0.smithi060.stdout:7/458: dread da/d15/d88/f5c [0,4194304] 0 2024-02-26T10:47:56.823 INFO:tasks.workunit.client.0.smithi060.stdout:9/364: symlink d3/d1e/d5e/d1b/d5f/dc/d32/l78 0 2024-02-26T10:47:56.825 INFO:tasks.workunit.client.0.smithi060.stdout:0/318: rename d0/dd/d39/d40/l24 to d0/l6b 0 2024-02-26T10:47:56.826 INFO:tasks.workunit.client.0.smithi060.stdout:7/459: truncate f6 673880 0 2024-02-26T10:47:56.826 INFO:tasks.workunit.client.0.smithi060.stdout:7/460: write da/d66/f90 [188519,2855] 0 2024-02-26T10:47:56.829 INFO:tasks.workunit.client.0.smithi060.stdout:8/304: dwrite d9/d15/f27 [0,4194304] 0 2024-02-26T10:47:56.829 INFO:tasks.workunit.client.0.smithi060.stdout:8/305: stat d9/f14 0 2024-02-26T10:47:56.829 INFO:tasks.workunit.client.0.smithi060.stdout:7/461: creat da/d94/f98 x:0 0 0 2024-02-26T10:47:56.831 INFO:tasks.workunit.client.0.smithi060.stdout:7/462: getdents da/d15/d88/d39 0 2024-02-26T10:47:56.834 INFO:tasks.workunit.client.0.smithi060.stdout:7/463: mknod da/d15/d21/c99 0 2024-02-26T10:47:56.834 INFO:tasks.workunit.client.0.smithi060.stdout:0/319: mkdir d0/dd/d33/d6a/d6c 0 2024-02-26T10:47:56.853 INFO:tasks.workunit.client.0.smithi060.stdout:4/357: dwrite d0/d27/f4d [0,4194304] 0 2024-02-26T10:47:56.869 INFO:tasks.workunit.client.0.smithi060.stdout:9/365: dwrite d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f4c [0,4194304] 0 2024-02-26T10:47:56.874 INFO:tasks.workunit.client.0.smithi060.stdout:9/366: chown d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d44/l51 9 1 2024-02-26T10:47:56.886 INFO:tasks.workunit.client.0.smithi060.stdout:0/320: dread d0/dd/d39/f3f [0,4194304] 0 2024-02-26T10:47:56.887 INFO:tasks.workunit.client.0.smithi060.stdout:8/306: read d9/d15/f36 [480614,122954] 0 2024-02-26T10:47:56.888 INFO:tasks.workunit.client.0.smithi060.stdout:3/252: dwrite d5/f27 [4194304,4194304] 0 2024-02-26T10:47:56.888 INFO:tasks.workunit.client.0.smithi060.stdout:2/317: dwrite dd/d43/d60/f1e [0,4194304] 0 2024-02-26T10:47:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:3/253: getdents d5/de/d16 0 2024-02-26T10:47:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:2/318: rename dd/l46 to dd/d43/d55/l64 0 2024-02-26T10:47:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:8/307: mknod d9/d19/c57 0 2024-02-26T10:47:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:2/319: creat dd/d11/f65 x:0 0 0 2024-02-26T10:47:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:8/308: mkdir d9/d15/d3e/d44/d58 0 2024-02-26T10:47:56.895 INFO:tasks.workunit.client.0.smithi060.stdout:0/321: write d0/dd/d39/d40/f26 [241854,103270] 0 2024-02-26T10:47:56.895 INFO:tasks.workunit.client.0.smithi060.stdout:0/322: write d0/dd/f2b [147,1495] 0 2024-02-26T10:47:56.895 INFO:tasks.workunit.client.0.smithi060.stdout:2/320: creat dd/d43/d55/f66 x:0 0 0 2024-02-26T10:47:56.895 INFO:tasks.workunit.client.0.smithi060.stdout:9/367: read d3/d1e/d5e/f17 [1294426,20999] 0 2024-02-26T10:47:56.895 INFO:tasks.workunit.client.0.smithi060.stdout:2/321: read - dd/d43/d55/f66 zero size 2024-02-26T10:47:56.896 INFO:tasks.workunit.client.0.smithi060.stdout:8/309: creat d9/d15/d29/f59 x:0 0 0 2024-02-26T10:47:56.896 INFO:tasks.workunit.client.0.smithi060.stdout:8/310: chown d9/fd 244 1 2024-02-26T10:47:56.900 INFO:tasks.workunit.client.0.smithi060.stdout:6/365: dwrite f4 [0,4194304] 0 2024-02-26T10:47:56.900 INFO:tasks.workunit.client.0.smithi060.stdout:7/464: dwrite f7 [0,4194304] 0 2024-02-26T10:47:56.900 INFO:tasks.workunit.client.0.smithi060.stdout:0/323: creat d0/d37/f6d x:0 0 0 2024-02-26T10:47:56.900 INFO:tasks.workunit.client.0.smithi060.stdout:9/368: creat d3/d1e/d57/f79 x:0 0 0 2024-02-26T10:47:56.901 INFO:tasks.workunit.client.0.smithi060.stdout:2/322: mknod dd/d43/d60/d41/d37/d38/d57/c67 0 2024-02-26T10:47:56.901 INFO:tasks.workunit.client.0.smithi060.stdout:2/323: write dd/d43/d60/f53 [260321,72753] 0 2024-02-26T10:47:56.901 INFO:tasks.workunit.client.0.smithi060.stdout:2/324: write dd/d43/d55/f66 [575020,125796] 0 2024-02-26T10:47:56.905 INFO:tasks.workunit.client.0.smithi060.stdout:6/366: mkdir d6/d7/d2e/d56/d73 0 2024-02-26T10:47:56.906 INFO:tasks.workunit.client.0.smithi060.stdout:4/358: dread d0/f44 [0,4194304] 0 2024-02-26T10:47:56.918 INFO:tasks.workunit.client.0.smithi060.stdout:8/311: write d9/d19/f42 [850059,25632] 0 2024-02-26T10:47:56.920 INFO:tasks.workunit.client.0.smithi060.stdout:1/436: sync 2024-02-26T10:47:56.924 INFO:tasks.workunit.client.0.smithi060.stdout:1/437: getdents d3/db/d1e/d6d 0 2024-02-26T10:47:56.924 INFO:tasks.workunit.client.0.smithi060.stdout:7/465: mknod da/d15/d21/d2f/d37/c9a 0 2024-02-26T10:47:56.924 INFO:tasks.workunit.client.0.smithi060.stdout:6/367: creat d6/d7/d25/d1c/d1f/d23/d2b/d63/f74 x:0 0 0 2024-02-26T10:47:56.925 INFO:tasks.workunit.client.0.smithi060.stdout:4/359: creat d0/da/d57/f67 x:0 0 0 2024-02-26T10:47:56.925 INFO:tasks.workunit.client.0.smithi060.stdout:6/368: write d6/d7/d25/d1c/d1f/d23/d28/f33 [787295,96849] 0 2024-02-26T10:47:56.925 INFO:tasks.workunit.client.0.smithi060.stdout:6/369: chown d6/d7/d47/c4f 102842 1 2024-02-26T10:47:56.926 INFO:tasks.workunit.client.0.smithi060.stdout:6/370: chown d6/d7/d25/d1c/d1f/d23/d28/f33 199861255 1 2024-02-26T10:47:56.926 INFO:tasks.workunit.client.0.smithi060.stdout:1/438: creat d3/d35/d76/f98 x:0 0 0 2024-02-26T10:47:56.927 INFO:tasks.workunit.client.0.smithi060.stdout:1/439: creat d3/db/d1e/d6d/d78/f99 x:0 0 0 2024-02-26T10:47:56.929 INFO:tasks.workunit.client.0.smithi060.stdout:6/371: creat d6/f75 x:0 0 0 2024-02-26T10:47:56.931 INFO:tasks.workunit.client.0.smithi060.stdout:7/466: mknod da/d15/c9b 0 2024-02-26T10:47:56.932 INFO:tasks.workunit.client.0.smithi060.stdout:6/372: getdents d6/d7/d47 0 2024-02-26T10:47:56.932 INFO:tasks.workunit.client.0.smithi060.stdout:6/373: write d6/f9 [627221,55736] 0 2024-02-26T10:47:56.932 INFO:tasks.workunit.client.0.smithi060.stdout:6/374: stat d6/d7/f2a 0 2024-02-26T10:47:56.938 INFO:tasks.workunit.client.0.smithi060.stdout:7/467: creat da/d15/d21/d2f/d37/f9c x:0 0 0 2024-02-26T10:47:56.938 INFO:tasks.workunit.client.0.smithi060.stdout:5/358: sync 2024-02-26T10:47:56.938 INFO:tasks.workunit.client.0.smithi060.stdout:1/440: unlink d3/d35/d53/d68/f77 0 2024-02-26T10:47:56.946 INFO:tasks.workunit.client.0.smithi060.stdout:3/254: dwrite f3 [0,4194304] 0 2024-02-26T10:47:56.951 INFO:tasks.workunit.client.0.smithi060.stdout:3/255: link d5/l8 d5/de/df/d11/l42 0 2024-02-26T10:47:56.957 INFO:tasks.workunit.client.0.smithi060.stdout:5/359: dread d0/fd [0,4194304] 0 2024-02-26T10:47:56.959 INFO:tasks.workunit.client.0.smithi060.stdout:5/360: unlink d0/de/f1f 0 2024-02-26T10:47:56.959 INFO:tasks.workunit.client.0.smithi060.stdout:5/361: truncate d0/d33/f62 639772 0 2024-02-26T10:47:56.960 INFO:tasks.workunit.client.0.smithi060.stdout:5/362: creat d0/f81 x:0 0 0 2024-02-26T10:47:56.961 INFO:tasks.workunit.client.0.smithi060.stdout:5/363: mkdir d0/de/d20/d82 0 2024-02-26T10:47:56.963 INFO:tasks.workunit.client.0.smithi060.stdout:7/468: read da/fd [961865,20968] 0 2024-02-26T10:47:56.963 INFO:tasks.workunit.client.0.smithi060.stdout:5/364: write d0/d33/f62 [616753,35759] 0 2024-02-26T10:47:56.963 INFO:tasks.workunit.client.0.smithi060.stdout:5/365: stat d0/de/d15/d65/l7a 0 2024-02-26T10:47:56.964 INFO:tasks.workunit.client.0.smithi060.stdout:5/366: link d0/l4a d0/de/d20/l83 0 2024-02-26T10:47:56.968 INFO:tasks.workunit.client.0.smithi060.stdout:7/469: rmdir da 39 2024-02-26T10:47:56.968 INFO:tasks.workunit.client.0.smithi060.stdout:1/441: write d3/f5c [1230397,16962] 0 2024-02-26T10:47:56.968 INFO:tasks.workunit.client.0.smithi060.stdout:1/442: chown d3/db/d1e 104327 1 2024-02-26T10:47:56.975 INFO:tasks.workunit.client.0.smithi060.stdout:7/470: unlink da/d15/d21/d32/f33 0 2024-02-26T10:47:56.975 INFO:tasks.workunit.client.0.smithi060.stdout:7/471: truncate da/d66/f75 4886630 0 2024-02-26T10:47:56.975 INFO:tasks.workunit.client.0.smithi060.stdout:0/324: dwrite d0/f5f [0,4194304] 0 2024-02-26T10:47:56.976 INFO:tasks.workunit.client.0.smithi060.stdout:0/325: stat d0/f3 0 2024-02-26T10:47:56.976 INFO:tasks.workunit.client.0.smithi060.stdout:0/326: stat d0/dd/d39/d40/c2e 0 2024-02-26T10:47:56.976 INFO:tasks.workunit.client.0.smithi060.stdout:0/327: chown d0/dd/d33/d6a/d6c 1 1 2024-02-26T10:47:56.976 INFO:tasks.workunit.client.0.smithi060.stdout:0/328: fdatasync d0/dd/f5d 0 2024-02-26T10:47:56.976 INFO:tasks.workunit.client.0.smithi060.stdout:0/329: stat d0/dd/d39/f61 0 2024-02-26T10:47:56.977 INFO:tasks.workunit.client.0.smithi060.stdout:0/330: write d0/dd/f5d [713233,38831] 0 2024-02-26T10:47:56.977 INFO:tasks.workunit.client.0.smithi060.stdout:0/331: mknod d0/d37/c6e 0 2024-02-26T10:47:56.977 INFO:tasks.workunit.client.0.smithi060.stdout:6/375: dread d6/f11 [0,4194304] 0 2024-02-26T10:47:56.977 INFO:tasks.workunit.client.0.smithi060.stdout:0/332: link d0/l14 d0/dd/d33/l6f 0 2024-02-26T10:47:56.977 INFO:tasks.workunit.client.0.smithi060.stdout:1/443: dread d3/d35/d53/d57/f73 [0,4194304] 0 2024-02-26T10:47:56.977 INFO:tasks.workunit.client.0.smithi060.stdout:0/333: getdents d0/dd/d39/d63 0 2024-02-26T10:47:56.977 INFO:tasks.workunit.client.0.smithi060.stdout:6/376: chown d6/d7/d25/d1c/d1f/d23/d28/l66 5265 1 2024-02-26T10:47:56.978 INFO:tasks.workunit.client.0.smithi060.stdout:6/377: fdatasync d6/d7/f53 0 2024-02-26T10:47:56.980 INFO:tasks.workunit.client.0.smithi060.stdout:1/444: getdents d3/d35/d53/d68/d85 0 2024-02-26T10:47:56.980 INFO:tasks.workunit.client.0.smithi060.stdout:6/378: symlink d6/d7/d2e/l76 0 2024-02-26T10:47:56.994 INFO:tasks.workunit.client.0.smithi060.stdout:6/379: dread d6/d7/ff [0,4194304] 0 2024-02-26T10:47:56.999 INFO:tasks.workunit.client.0.smithi060.stdout:4/360: dread d0/d7/f40 [0,4194304] 0 2024-02-26T10:47:57.000 INFO:tasks.workunit.client.0.smithi060.stdout:9/369: dread d3/d1e/d5e/d1b/f5a [0,4194304] 0 2024-02-26T10:47:57.009 INFO:tasks.workunit.client.0.smithi060.stdout:3/256: dread d5/de/df/d11/f20 [0,4194304] 0 2024-02-26T10:47:57.010 INFO:tasks.workunit.client.0.smithi060.stdout:4/361: dread d0/da/d3b/d16/f24 [0,4194304] 0 2024-02-26T10:47:57.011 INFO:tasks.workunit.client.0.smithi060.stdout:4/362: readlink d0/l36 0 2024-02-26T10:47:57.011 INFO:tasks.workunit.client.0.smithi060.stdout:4/363: dread - d0/da/d57/f58 zero size 2024-02-26T10:47:57.011 INFO:tasks.workunit.client.0.smithi060.stdout:3/257: mknod d5/de/df/d11/c43 0 2024-02-26T10:47:57.012 INFO:tasks.workunit.client.0.smithi060.stdout:3/258: symlink d5/l44 0 2024-02-26T10:47:57.017 INFO:tasks.workunit.client.0.smithi060.stdout:1/445: write d3/f9 [1558919,10723] 0 2024-02-26T10:47:57.028 INFO:tasks.workunit.client.0.smithi060.stdout:3/259: dread d5/de/df/f33 [0,4194304] 0 2024-02-26T10:47:57.028 INFO:tasks.workunit.client.0.smithi060.stdout:8/312: dwrite d9/d19/f3a [0,4194304] 0 2024-02-26T10:47:57.029 INFO:tasks.workunit.client.0.smithi060.stdout:8/313: fdatasync d9/d15/d29/d2f/f39 0 2024-02-26T10:47:57.029 INFO:tasks.workunit.client.0.smithi060.stdout:0/334: dread d0/f55 [0,4194304] 0 2024-02-26T10:47:57.029 INFO:tasks.workunit.client.0.smithi060.stdout:7/472: write da/d15/d21/d4b/d67/f7a [4018538,94054] 0 2024-02-26T10:47:57.030 INFO:tasks.workunit.client.0.smithi060.stdout:3/260: mkdir d5/d45 0 2024-02-26T10:47:57.030 INFO:tasks.workunit.client.0.smithi060.stdout:3/261: write d5/de/d16/d17/f1e [917951,94215] 0 2024-02-26T10:47:57.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/314: rename d9/d15/d3e/d44 to d9/d5a 0 2024-02-26T10:47:57.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/315: write d9/d5a/f49 [83335,5063] 0 2024-02-26T10:47:57.043 INFO:tasks.workunit.client.0.smithi060.stdout:3/262: dread d5/de/df/d11/f34 [0,4194304] 0 2024-02-26T10:47:57.044 INFO:tasks.workunit.client.0.smithi060.stdout:3/263: creat d5/de/f46 x:0 0 0 2024-02-26T10:47:57.044 INFO:tasks.workunit.client.0.smithi060.stdout:3/264: fsync d5/f6 0 2024-02-26T10:47:57.061 INFO:tasks.workunit.client.0.smithi060.stdout:9/370: dwrite d3/d1e/d5e/d1b/f56 [0,4194304] 0 2024-02-26T10:47:57.061 INFO:tasks.workunit.client.0.smithi060.stdout:9/371: chown d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d44 1860 1 2024-02-26T10:47:57.061 INFO:tasks.workunit.client.0.smithi060.stdout:9/372: fdatasync d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f4c 0 2024-02-26T10:47:57.066 INFO:tasks.workunit.client.0.smithi060.stdout:8/316: write d9/d19/f3a [1381754,91715] 0 2024-02-26T10:47:57.066 INFO:tasks.workunit.client.0.smithi060.stdout:9/373: mkdir d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/d7a 0 2024-02-26T10:47:57.066 INFO:tasks.workunit.client.0.smithi060.stdout:8/317: fdatasync f8 0 2024-02-26T10:47:57.069 INFO:tasks.workunit.client.0.smithi060.stdout:1/446: dwrite d3/d35/d76/f98 [0,4194304] 0 2024-02-26T10:47:57.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:56 smithi113 ceph-mon[36126]: pgmap v484: 129 pgs: 129 active+clean; 549 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s rd, 80 MiB/s wr, 1.05k op/s 2024-02-26T10:47:57.084 INFO:tasks.workunit.client.0.smithi060.stdout:8/318: unlink d9/f43 0 2024-02-26T10:47:57.094 INFO:tasks.workunit.client.0.smithi060.stdout:8/319: dread f3 [0,4194304] 0 2024-02-26T10:47:57.096 INFO:tasks.workunit.client.0.smithi060.stdout:8/320: mknod d9/d15/c5b 0 2024-02-26T10:47:57.096 INFO:tasks.workunit.client.0.smithi060.stdout:8/321: fsync d9/d15/d29/d2f/f31 0 2024-02-26T10:47:57.098 INFO:tasks.workunit.client.0.smithi060.stdout:8/322: getdents d9/d15/d29/d2f 0 2024-02-26T10:47:57.099 INFO:tasks.workunit.client.0.smithi060.stdout:8/323: chown d9/d15/f52 126761279 1 2024-02-26T10:47:57.106 INFO:tasks.workunit.client.0.smithi060.stdout:3/265: dwrite d5/de/df/d11/f1c [0,4194304] 0 2024-02-26T10:47:57.113 INFO:tasks.workunit.client.0.smithi060.stdout:3/266: unlink d5/l2e 0 2024-02-26T10:47:57.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:56 smithi060 ceph-mon[31539]: pgmap v484: 129 pgs: 129 active+clean; 549 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s rd, 80 MiB/s wr, 1.05k op/s 2024-02-26T10:47:57.113 INFO:tasks.workunit.client.0.smithi060.stdout:3/267: write d5/f25 [473141,128151] 0 2024-02-26T10:47:57.114 INFO:tasks.workunit.client.0.smithi060.stdout:8/324: creat d9/d5a/f5c x:0 0 0 2024-02-26T10:47:57.114 INFO:tasks.workunit.client.0.smithi060.stdout:9/374: rename d3/d1e/d5e/d1b/d5f/dc/d32/d3c/l42 to d3/d1e/l7b 0 2024-02-26T10:47:57.116 INFO:tasks.workunit.client.0.smithi060.stdout:3/268: mkdir d5/de/df/d11/d47 0 2024-02-26T10:47:57.121 INFO:tasks.workunit.client.0.smithi060.stdout:9/375: creat d3/d1e/d5e/d35/f7c x:0 0 0 2024-02-26T10:47:57.121 INFO:tasks.workunit.client.0.smithi060.stdout:1/447: creat d3/d35/d53/d68/d85/d8a/f9a x:0 0 0 2024-02-26T10:47:57.123 INFO:tasks.workunit.client.0.smithi060.stdout:9/376: mknod d3/c7d 0 2024-02-26T10:47:57.124 INFO:tasks.workunit.client.0.smithi060.stdout:9/377: fdatasync d3/d1e/d5e/d1b/d5f/dc/f3b 0 2024-02-26T10:47:57.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:56 smithi100 ceph-mon[36051]: pgmap v484: 129 pgs: 129 active+clean; 549 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 63 MiB/s rd, 80 MiB/s wr, 1.05k op/s 2024-02-26T10:47:57.126 INFO:tasks.workunit.client.0.smithi060.stdout:9/378: unlink d3/d1e/d5e/d1b/d5f/dc/ff 0 2024-02-26T10:47:57.136 INFO:tasks.workunit.client.0.smithi060.stdout:9/379: write d3/d1e/d5e/d1b/d5f/f1f [4831946,25303] 0 2024-02-26T10:47:57.151 INFO:tasks.workunit.client.0.smithi060.stdout:9/380: dread d3/d1e/d5e/d1b/d5f/d10/f4f [0,4194304] 0 2024-02-26T10:47:57.151 INFO:tasks.workunit.client.0.smithi060.stdout:9/381: chown d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f6d 15923 1 2024-02-26T10:47:57.161 INFO:tasks.workunit.client.0.smithi060.stdout:8/325: dwrite d9/d15/d29/d2f/f31 [0,4194304] 0 2024-02-26T10:47:57.167 INFO:tasks.workunit.client.0.smithi060.stdout:8/326: write d9/d15/d40/f46 [50688,88430] 0 2024-02-26T10:47:57.169 INFO:tasks.workunit.client.0.smithi060.stdout:8/327: mknod d9/c5d 0 2024-02-26T10:47:57.169 INFO:tasks.workunit.client.0.smithi060.stdout:8/328: fdatasync d9/d15/f1c 0 2024-02-26T10:47:57.169 INFO:tasks.workunit.client.0.smithi060.stdout:8/329: dread - d9/d5a/f5c zero size 2024-02-26T10:47:57.169 INFO:tasks.workunit.client.0.smithi060.stdout:8/330: chown d9/d19/l30 2393722 1 2024-02-26T10:47:57.171 INFO:tasks.workunit.client.0.smithi060.stdout:8/331: rename d9/d15/c1a to d9/d15/d40/c5e 0 2024-02-26T10:47:57.171 INFO:tasks.workunit.client.0.smithi060.stdout:8/332: truncate d9/f14 4085576 0 2024-02-26T10:47:57.171 INFO:tasks.workunit.client.0.smithi060.stdout:8/333: chown d9/d15/d40/f46 436 1 2024-02-26T10:47:57.171 INFO:tasks.workunit.client.0.smithi060.stdout:8/334: fdatasync d9/d15/f51 0 2024-02-26T10:47:57.173 INFO:tasks.workunit.client.0.smithi060.stdout:8/335: unlink d9/d15/d40/f53 0 2024-02-26T10:47:57.173 INFO:tasks.workunit.client.0.smithi060.stdout:3/269: dwrite d5/de/d16/d17/f30 [0,4194304] 0 2024-02-26T10:47:57.175 INFO:tasks.workunit.client.0.smithi060.stdout:8/336: write d9/f1e [889594,87339] 0 2024-02-26T10:47:57.175 INFO:tasks.workunit.client.0.smithi060.stdout:9/382: dread d3/d1e/d5e/d1b/d5f/d10/f21 [0,4194304] 0 2024-02-26T10:47:57.177 INFO:tasks.workunit.client.0.smithi060.stdout:3/270: creat d5/d45/f48 x:0 0 0 2024-02-26T10:47:57.177 INFO:tasks.workunit.client.0.smithi060.stdout:9/383: creat d3/d1e/d5e/d1b/d5f/d10/f7e x:0 0 0 2024-02-26T10:47:57.179 INFO:tasks.workunit.client.0.smithi060.stdout:8/337: mknod d9/d5a/d58/c5f 0 2024-02-26T10:47:57.179 INFO:tasks.workunit.client.0.smithi060.stdout:3/271: stat d5/de/df/d11/f36 0 2024-02-26T10:47:57.181 INFO:tasks.workunit.client.0.smithi060.stdout:8/338: rename d9/d15/f1b to d9/d15/f60 0 2024-02-26T10:47:57.181 INFO:tasks.workunit.client.0.smithi060.stdout:8/339: write d9/fc [1892418,98369] 0 2024-02-26T10:47:57.181 INFO:tasks.workunit.client.0.smithi060.stdout:3/272: creat d5/d45/f49 x:0 0 0 2024-02-26T10:47:57.183 INFO:tasks.workunit.client.0.smithi060.stdout:8/340: creat d9/d15/d40/f61 x:0 0 0 2024-02-26T10:47:57.184 INFO:tasks.workunit.client.0.smithi060.stdout:3/273: truncate d5/f24 322559 0 2024-02-26T10:47:57.184 INFO:tasks.workunit.client.0.smithi060.stdout:8/341: creat d9/d19/f62 x:0 0 0 2024-02-26T10:47:57.184 INFO:tasks.workunit.client.0.smithi060.stdout:8/342: stat d9/l38 0 2024-02-26T10:47:57.185 INFO:tasks.workunit.client.0.smithi060.stdout:8/343: mkdir d9/d15/d40/d63 0 2024-02-26T10:47:57.185 INFO:tasks.workunit.client.0.smithi060.stdout:8/344: chown d9/d15/d40/l48 114 1 2024-02-26T10:47:57.193 INFO:tasks.workunit.client.0.smithi060.stdout:9/384: rename d3/d1e/d5e/d1b/d5f/d10/f4f to d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f7f 0 2024-02-26T10:47:57.202 INFO:tasks.workunit.client.0.smithi060.stdout:9/385: link d3/d1e/d5e/d1b/d5f/f5b d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f80 0 2024-02-26T10:47:57.204 INFO:tasks.workunit.client.0.smithi060.stdout:9/386: mknod d3/d1e/c81 0 2024-02-26T10:47:57.205 INFO:tasks.workunit.client.0.smithi060.stdout:8/345: rename d9/d5a/f5c to d9/d15/d29/f64 0 2024-02-26T10:47:57.216 INFO:tasks.workunit.client.0.smithi060.stdout:3/274: dwrite d5/de/f3b [0,4194304] 0 2024-02-26T10:47:57.220 INFO:tasks.workunit.client.0.smithi060.stdout:2/325: sync 2024-02-26T10:47:57.220 INFO:tasks.workunit.client.0.smithi060.stdout:9/387: rename d3/d1e/d5e/d1b/d5f/d10/f2c to d3/d1e/f82 0 2024-02-26T10:47:57.235 INFO:tasks.workunit.client.0.smithi060.stdout:9/388: rename d3/d1e/d5e/d1b/l22 to d3/d1e/d5e/d1b/d5f/dc/d32/l83 0 2024-02-26T10:47:57.235 INFO:tasks.workunit.client.0.smithi060.stdout:9/389: chown d3/d1e/d5e/d1b/d5f/f2f 335454 1 2024-02-26T10:47:57.237 INFO:tasks.workunit.client.0.smithi060.stdout:9/390: mkdir d3/d1e/d5e/d35/d84 0 2024-02-26T10:47:57.237 INFO:tasks.workunit.client.0.smithi060.stdout:9/391: dread - d3/d1e/d5e/d1b/d5f/d10/f4e zero size 2024-02-26T10:47:57.237 INFO:tasks.workunit.client.0.smithi060.stdout:9/392: chown d3/d1e/d5e/f5c 598 1 2024-02-26T10:47:57.241 INFO:tasks.workunit.client.0.smithi060.stdout:2/326: dread dd/d43/d60/f1e [0,4194304] 0 2024-02-26T10:47:57.243 INFO:tasks.workunit.client.0.smithi060.stdout:2/327: getdents dd/d43/d60/d41/d37/d4c 0 2024-02-26T10:47:57.248 INFO:tasks.workunit.client.0.smithi060.stdout:9/393: rename d3/d1e/c81 to d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/d7a/c85 0 2024-02-26T10:47:57.248 INFO:tasks.workunit.client.0.smithi060.stdout:9/394: truncate d3/d1e/f58 907829 0 2024-02-26T10:47:57.249 INFO:tasks.workunit.client.0.smithi060.stdout:8/346: dwrite d9/d15/f27 [0,4194304] 0 2024-02-26T10:47:57.249 INFO:tasks.workunit.client.0.smithi060.stdout:8/347: readlink d9/d15/d29/d2f/l37 0 2024-02-26T10:47:57.250 INFO:tasks.workunit.client.0.smithi060.stdout:8/348: truncate d9/d19/f42 3860881 0 2024-02-26T10:47:57.250 INFO:tasks.workunit.client.0.smithi060.stdout:8/349: readlink d9/d15/d3e/l56 0 2024-02-26T10:47:57.252 INFO:tasks.workunit.client.0.smithi060.stdout:8/350: creat d9/d15/d29/f65 x:0 0 0 2024-02-26T10:47:57.252 INFO:tasks.workunit.client.0.smithi060.stdout:8/351: chown d9/f24 3483430 1 2024-02-26T10:47:57.254 INFO:tasks.workunit.client.0.smithi060.stdout:8/352: symlink d9/d5a/l66 0 2024-02-26T10:47:57.259 INFO:tasks.workunit.client.0.smithi060.stdout:9/395: link d3/d1e/d5e/d1b/d5f/dc/c69 d3/d1e/d5e/d1b/d23/c86 0 2024-02-26T10:47:57.269 INFO:tasks.workunit.client.0.smithi060.stdout:2/328: dread dd/d43/d60/f3f [4194304,4194304] 0 2024-02-26T10:47:57.270 INFO:tasks.workunit.client.0.smithi060.stdout:2/329: readlink dd/l32 0 2024-02-26T10:47:57.271 INFO:tasks.workunit.client.0.smithi060.stdout:2/330: read dd/d11/f44 [606419,17053] 0 2024-02-26T10:47:57.271 INFO:tasks.workunit.client.0.smithi060.stdout:2/331: truncate dd/f2b 1876452 0 2024-02-26T10:47:57.272 INFO:tasks.workunit.client.0.smithi060.stdout:2/332: creat dd/f68 x:0 0 0 2024-02-26T10:47:57.283 INFO:tasks.workunit.client.0.smithi060.stdout:8/353: write d9/f20 [617449,103676] 0 2024-02-26T10:47:57.283 INFO:tasks.workunit.client.0.smithi060.stdout:3/275: dwrite d5/de/df/f33 [0,4194304] 0 2024-02-26T10:47:57.290 INFO:tasks.workunit.client.0.smithi060.stdout:3/276: write d5/de/f14 [5138816,59480] 0 2024-02-26T10:47:57.291 INFO:tasks.workunit.client.0.smithi060.stdout:3/277: creat d5/f4a x:0 0 0 2024-02-26T10:47:57.292 INFO:tasks.workunit.client.0.smithi060.stdout:3/278: mknod d5/d37/c4b 0 2024-02-26T10:47:57.292 INFO:tasks.workunit.client.0.smithi060.stdout:3/279: read - d5/de/f46 zero size 2024-02-26T10:47:57.293 INFO:tasks.workunit.client.0.smithi060.stdout:9/396: write d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f4c [3840864,76997] 0 2024-02-26T10:47:57.294 INFO:tasks.workunit.client.0.smithi060.stdout:3/280: mkdir d5/de/df/d4c 0 2024-02-26T10:47:57.294 INFO:tasks.workunit.client.0.smithi060.stdout:3/281: chown d5/de/f46 4624253 1 2024-02-26T10:47:57.294 INFO:tasks.workunit.client.0.smithi060.stdout:9/397: unlink d3/d1e/d5e/l27 0 2024-02-26T10:47:57.297 INFO:tasks.workunit.client.0.smithi060.stdout:9/398: rename d3/d52 to d3/d1e/d5e/d35/d87 0 2024-02-26T10:47:57.297 INFO:tasks.workunit.client.0.smithi060.stdout:9/399: write d3/d1e/d5e/d1b/d5f/d10/f4e [737688,32186] 0 2024-02-26T10:47:57.299 INFO:tasks.workunit.client.0.smithi060.stdout:9/400: creat d3/d1e/d57/f88 x:0 0 0 2024-02-26T10:47:57.303 INFO:tasks.workunit.client.0.smithi060.stdout:9/401: getdents d3 0 2024-02-26T10:47:57.306 INFO:tasks.workunit.client.0.smithi060.stdout:9/402: symlink d3/d1e/d5e/d35/l89 0 2024-02-26T10:47:57.333 INFO:tasks.workunit.client.0.smithi060.stdout:8/354: dwrite d9/d15/f60 [0,4194304] 0 2024-02-26T10:47:57.333 INFO:tasks.workunit.client.0.smithi060.stdout:8/355: chown d9/d15/d29/d2f/f31 146649 1 2024-02-26T10:47:57.333 INFO:tasks.workunit.client.0.smithi060.stdout:8/356: chown d9/d15/d29/d2f/f3d 9 1 2024-02-26T10:47:57.338 INFO:tasks.workunit.client.0.smithi060.stdout:8/357: read d9/d5a/f49 [16749,52961] 0 2024-02-26T10:47:57.341 INFO:tasks.workunit.client.0.smithi060.stdout:3/282: dwrite d5/f4a [0,4194304] 0 2024-02-26T10:47:57.341 INFO:tasks.workunit.client.0.smithi060.stdout:3/283: fsync d5/de/df/d11/f36 0 2024-02-26T10:47:57.341 INFO:tasks.workunit.client.0.smithi060.stdout:3/284: write d5/de/df/d11/f20 [4578769,25845] 0 2024-02-26T10:47:57.343 INFO:tasks.workunit.client.0.smithi060.stdout:3/285: symlink d5/de/df/d4c/l4d 0 2024-02-26T10:47:57.343 INFO:tasks.workunit.client.0.smithi060.stdout:3/286: fdatasync d5/f25 0 2024-02-26T10:47:57.345 INFO:tasks.workunit.client.0.smithi060.stdout:0/335: sync 2024-02-26T10:47:57.345 INFO:tasks.workunit.client.0.smithi060.stdout:5/367: sync 2024-02-26T10:47:57.345 INFO:tasks.workunit.client.0.smithi060.stdout:4/364: sync 2024-02-26T10:47:57.345 INFO:tasks.workunit.client.0.smithi060.stdout:0/336: read - d0/f15 zero size 2024-02-26T10:47:57.345 INFO:tasks.workunit.client.0.smithi060.stdout:6/380: sync 2024-02-26T10:47:57.345 INFO:tasks.workunit.client.0.smithi060.stdout:7/473: sync 2024-02-26T10:47:57.346 INFO:tasks.workunit.client.0.smithi060.stdout:5/368: mkdir d0/de/d70/d84 0 2024-02-26T10:47:57.348 INFO:tasks.workunit.client.0.smithi060.stdout:0/337: link d0/ce d0/dd/d33/d6a/d6c/c70 0 2024-02-26T10:47:57.348 INFO:tasks.workunit.client.0.smithi060.stdout:6/381: creat d6/d7/d25/d1c/d39/f77 x:0 0 0 2024-02-26T10:47:57.351 INFO:tasks.workunit.client.0.smithi060.stdout:7/474: rename da/d15/d21/d2f/d37/f9c to da/d15/d88/d3b/f9d 0 2024-02-26T10:47:57.352 INFO:tasks.workunit.client.0.smithi060.stdout:7/475: write da/d15/d21/d4b/d67/f85 [416563,48438] 0 2024-02-26T10:47:57.353 INFO:tasks.workunit.client.0.smithi060.stdout:6/382: rename d6/d7/d25/d1c/d1f/d23/d2b/d63 to d6/d7/d25/d1c/d1f/d23/d2b/d67/d78 0 2024-02-26T10:47:57.355 INFO:tasks.workunit.client.0.smithi060.stdout:6/383: symlink d6/d7/d25/d1c/d1f/d23/d2b/d67/l79 0 2024-02-26T10:47:57.355 INFO:tasks.workunit.client.0.smithi060.stdout:9/403: dwrite d3/d1e/d5e/d1b/d5f/f5b [4194304,4194304] 0 2024-02-26T10:47:57.357 INFO:tasks.workunit.client.0.smithi060.stdout:9/404: stat d3/d1e/d5e/d1b/d5f/dc/c69 0 2024-02-26T10:47:57.362 INFO:tasks.workunit.client.0.smithi060.stdout:9/405: mkdir d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d44/d8a 0 2024-02-26T10:47:57.362 INFO:tasks.workunit.client.0.smithi060.stdout:9/406: write d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/f70 [130851,110626] 0 2024-02-26T10:47:57.362 INFO:tasks.workunit.client.0.smithi060.stdout:9/407: dread - d3/d1e/d5e/d1b/d5f/f54 zero size 2024-02-26T10:47:57.362 INFO:tasks.workunit.client.0.smithi060.stdout:9/408: truncate d3/d1e/d57/f79 648793 0 2024-02-26T10:47:57.362 INFO:tasks.workunit.client.0.smithi060.stdout:9/409: stat d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d44/f48 0 2024-02-26T10:47:57.362 INFO:tasks.workunit.client.0.smithi060.stdout:9/410: write d3/d1e/f58 [626944,25957] 0 2024-02-26T10:47:57.363 INFO:tasks.workunit.client.0.smithi060.stdout:9/411: chown d3/d1e/d5e/d35/l89 29911093 1 2024-02-26T10:47:57.363 INFO:tasks.workunit.client.0.smithi060.stdout:9/412: readlink d3/d1e/d5e/d1b/d5f/dc/d32/d3c/l4d 0 2024-02-26T10:47:57.363 INFO:tasks.workunit.client.0.smithi060.stdout:9/413: chown d3/d1e/d5e/d1b/d5f/d10/c3f 20998 1 2024-02-26T10:47:57.365 INFO:tasks.workunit.client.0.smithi060.stdout:9/414: creat d3/d1e/d5e/d1b/d5f/dc/f8b x:0 0 0 2024-02-26T10:47:57.365 INFO:tasks.workunit.client.0.smithi060.stdout:9/415: readlink d3/d1e/d5e/d1b/d5f/l4a 0 2024-02-26T10:47:57.368 INFO:tasks.workunit.client.0.smithi060.stdout:9/416: mkdir d3/d1e/d5e/d1b/d5f/d8c 0 2024-02-26T10:47:57.368 INFO:tasks.workunit.client.0.smithi060.stdout:9/417: chown d3/d1e/d5e/d1b/d5f/d10/f6c 794855 1 2024-02-26T10:47:57.368 INFO:tasks.workunit.client.0.smithi060.stdout:9/418: write d3/d1e/d5e/d35/f7c [414689,39346] 0 2024-02-26T10:47:57.368 INFO:tasks.workunit.client.0.smithi060.stdout:6/384: dread d6/d7/d25/f2d [0,4194304] 0 2024-02-26T10:47:57.368 INFO:tasks.workunit.client.0.smithi060.stdout:6/385: write d6/d7/d25/f59 [264258,30515] 0 2024-02-26T10:47:57.369 INFO:tasks.workunit.client.0.smithi060.stdout:8/358: dread d9/d19/f3a [0,4194304] 0 2024-02-26T10:47:57.371 INFO:tasks.workunit.client.0.smithi060.stdout:6/386: unlink d6/d7/d2e/l76 0 2024-02-26T10:47:57.371 INFO:tasks.workunit.client.0.smithi060.stdout:6/387: truncate d6/f71 1000496 0 2024-02-26T10:47:57.372 INFO:tasks.workunit.client.0.smithi060.stdout:7/476: dread da/d15/d21/f2c [0,4194304] 0 2024-02-26T10:47:57.372 INFO:tasks.workunit.client.0.smithi060.stdout:6/388: mkdir d6/d7/d25/d1c/d1f/d23/d7a 0 2024-02-26T10:47:57.373 INFO:tasks.workunit.client.0.smithi060.stdout:7/477: mknod da/d94/c9e 0 2024-02-26T10:47:57.375 INFO:tasks.workunit.client.0.smithi060.stdout:9/419: dread d3/d1e/d57/f79 [0,4194304] 0 2024-02-26T10:47:57.375 INFO:tasks.workunit.client.0.smithi060.stdout:9/420: write d3/d1e/d5e/d1b/d5f/f54 [815050,112034] 0 2024-02-26T10:47:57.381 INFO:tasks.workunit.client.0.smithi060.stdout:6/389: dread d6/d7/d25/d1c/d1f/f35 [0,4194304] 0 2024-02-26T10:47:57.381 INFO:tasks.workunit.client.0.smithi060.stdout:6/390: read - d6/d7/d25/d1c/f60 zero size 2024-02-26T10:47:57.399 INFO:tasks.workunit.client.0.smithi060.stdout:9/421: dread d3/d1e/d5e/d1b/d5f/f1f [0,4194304] 0 2024-02-26T10:47:57.399 INFO:tasks.workunit.client.0.smithi060.stdout:9/422: chown d3/d1e/f58 101573381 1 2024-02-26T10:47:57.415 INFO:tasks.workunit.client.0.smithi060.stdout:6/391: dwrite d6/d7/fd [0,4194304] 0 2024-02-26T10:47:57.416 INFO:tasks.workunit.client.0.smithi060.stdout:6/392: symlink d6/d7/d47/d69/l7b 0 2024-02-26T10:47:57.417 INFO:tasks.workunit.client.0.smithi060.stdout:6/393: dread - d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f74 zero size 2024-02-26T10:47:57.417 INFO:tasks.workunit.client.0.smithi060.stdout:7/478: rename da/d15/d21/d2f/l59 to da/d15/d21/d2f/d78/l9f 0 2024-02-26T10:47:57.418 INFO:tasks.workunit.client.0.smithi060.stdout:5/369: dwrite d0/d33/f3e [0,4194304] 0 2024-02-26T10:47:57.419 INFO:tasks.workunit.client.0.smithi060.stdout:5/370: write d0/de/d15/f44 [5072038,30687] 0 2024-02-26T10:47:57.419 INFO:tasks.workunit.client.0.smithi060.stdout:6/394: rename d6/f75 to d6/f7c 0 2024-02-26T10:47:57.419 INFO:tasks.workunit.client.0.smithi060.stdout:6/395: chown d6/d7/d25/d1c/d1f/d23 474 1 2024-02-26T10:47:57.419 INFO:tasks.workunit.client.0.smithi060.stdout:6/396: chown f4 447 1 2024-02-26T10:47:57.419 INFO:tasks.workunit.client.0.smithi060.stdout:5/371: write d0/d33/f5c [278500,87792] 0 2024-02-26T10:47:57.419 INFO:tasks.workunit.client.0.smithi060.stdout:6/397: write d6/d7/f70 [31255,119756] 0 2024-02-26T10:47:57.420 INFO:tasks.workunit.client.0.smithi060.stdout:7/479: mknod da/d15/d21/d41/d6c/ca0 0 2024-02-26T10:47:57.420 INFO:tasks.workunit.client.0.smithi060.stdout:7/480: chown da/d15/d21/d4b 2829922 1 2024-02-26T10:47:57.420 INFO:tasks.workunit.client.0.smithi060.stdout:5/372: creat d0/de/d20/f85 x:0 0 0 2024-02-26T10:47:57.421 INFO:tasks.workunit.client.0.smithi060.stdout:5/373: readlink d0/de/d15/l1b 0 2024-02-26T10:47:57.421 INFO:tasks.workunit.client.0.smithi060.stdout:5/374: chown d0/de/f22 3314 1 2024-02-26T10:47:57.422 INFO:tasks.workunit.client.0.smithi060.stdout:6/398: mknod d6/d7/d25/d1c/d1f/d23/d2b/c7d 0 2024-02-26T10:47:57.423 INFO:tasks.workunit.client.0.smithi060.stdout:7/481: truncate da/d15/d88/f5c 221475 0 2024-02-26T10:47:57.425 INFO:tasks.workunit.client.0.smithi060.stdout:6/399: mknod d6/d7/d25/c7e 0 2024-02-26T10:47:57.425 INFO:tasks.workunit.client.0.smithi060.stdout:0/338: dwrite d0/dd/d39/d40/f4d [0,4194304] 0 2024-02-26T10:47:57.428 INFO:tasks.workunit.client.0.smithi060.stdout:0/339: write d0/d42/f60 [248565,68243] 0 2024-02-26T10:47:57.428 INFO:tasks.workunit.client.0.smithi060.stdout:7/482: getdents da/d66 0 2024-02-26T10:47:57.431 INFO:tasks.workunit.client.0.smithi060.stdout:8/359: dread d9/d15/d29/d2f/f31 [0,4194304] 0 2024-02-26T10:47:57.438 INFO:tasks.workunit.client.0.smithi060.stdout:6/400: creat d6/d7/d25/d1c/d1f/d23/d28/f7f x:0 0 0 2024-02-26T10:47:57.438 INFO:tasks.workunit.client.0.smithi060.stdout:6/401: chown d6/d7/d25/d1c/d39/f44 26 1 2024-02-26T10:47:57.438 INFO:tasks.workunit.client.0.smithi060.stdout:7/483: dread da/d15/d21/f22 [0,4194304] 0 2024-02-26T10:47:57.440 INFO:tasks.workunit.client.0.smithi060.stdout:6/402: mkdir d6/d7/d25/d80 0 2024-02-26T10:47:57.442 INFO:tasks.workunit.client.0.smithi060.stdout:6/403: symlink d6/d7/d2e/d56/d73/l81 0 2024-02-26T10:47:57.445 INFO:tasks.workunit.client.0.smithi060.stdout:6/404: creat d6/d7/d47/d69/f82 x:0 0 0 2024-02-26T10:47:57.448 INFO:tasks.workunit.client.0.smithi060.stdout:6/405: mknod d6/d7/d25/d1c/d1f/c83 0 2024-02-26T10:47:57.449 INFO:tasks.workunit.client.0.smithi060.stdout:6/406: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84 0 2024-02-26T10:47:57.451 INFO:tasks.workunit.client.0.smithi060.stdout:6/407: link d6/d7/d25/f49 d6/d7/d25/d1c/d1f/d23/f85 0 2024-02-26T10:47:57.452 INFO:tasks.workunit.client.0.smithi060.stdout:9/423: rename d3/d1e/d5e/d1b/d5f/f62 to d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d44/f8d 0 2024-02-26T10:47:57.452 INFO:tasks.workunit.client.0.smithi060.stdout:9/424: stat d3/d1e/d5e 0 2024-02-26T10:47:57.453 INFO:tasks.workunit.client.0.smithi060.stdout:6/408: fsync d6/d7/d25/d1c/d1f/d23/d28/f4c 0 2024-02-26T10:47:57.454 INFO:tasks.workunit.client.0.smithi060.stdout:1/448: sync 2024-02-26T10:47:57.454 INFO:tasks.workunit.client.0.smithi060.stdout:2/333: sync 2024-02-26T10:47:57.454 INFO:tasks.workunit.client.0.smithi060.stdout:1/449: chown d3/db/d1e/l37 0 1 2024-02-26T10:47:57.455 INFO:tasks.workunit.client.0.smithi060.stdout:0/340: dread d0/d37/d44/d58/f5b [0,4194304] 0 2024-02-26T10:47:57.455 INFO:tasks.workunit.client.0.smithi060.stdout:0/341: stat d0/dd/d39/d40/f4d 0 2024-02-26T10:47:57.455 INFO:tasks.workunit.client.0.smithi060.stdout:5/375: dread d0/de/d15/f44 [0,4194304] 0 2024-02-26T10:47:57.455 INFO:tasks.workunit.client.0.smithi060.stdout:5/376: chown d0/de/d20/d32/d4c 0 1 2024-02-26T10:47:57.456 INFO:tasks.workunit.client.0.smithi060.stdout:7/484: dread da/d15/d21/d41/f97 [0,4194304] 0 2024-02-26T10:47:57.457 INFO:tasks.workunit.client.0.smithi060.stdout:1/450: truncate d3/d35/f65 2122831 0 2024-02-26T10:47:57.458 INFO:tasks.workunit.client.0.smithi060.stdout:7/485: creat da/d15/d21/d2f/d78/fa1 x:0 0 0 2024-02-26T10:47:57.458 INFO:tasks.workunit.client.0.smithi060.stdout:4/365: sync 2024-02-26T10:47:57.462 INFO:tasks.workunit.client.0.smithi060.stdout:3/287: sync 2024-02-26T10:47:57.465 INFO:tasks.workunit.client.0.smithi060.stdout:6/409: dread d6/f40 [0,4194304] 0 2024-02-26T10:47:57.465 INFO:tasks.workunit.client.0.smithi060.stdout:6/410: read - d6/d7/d47/f55 zero size 2024-02-26T10:47:57.465 INFO:tasks.workunit.client.0.smithi060.stdout:6/411: chown d6/d7/d47/c4f 1455 1 2024-02-26T10:47:57.465 INFO:tasks.workunit.client.0.smithi060.stdout:6/412: write d6/d7/d25/f2d [612382,57483] 0 2024-02-26T10:47:57.465 INFO:tasks.workunit.client.0.smithi060.stdout:7/486: link l3 da/d94/la2 0 2024-02-26T10:47:57.465 INFO:tasks.workunit.client.0.smithi060.stdout:3/288: link l1 d5/de/df/l4e 0 2024-02-26T10:47:57.466 INFO:tasks.workunit.client.0.smithi060.stdout:6/413: link d6/l5a d6/d7/d25/d80/l86 0 2024-02-26T10:47:57.469 INFO:tasks.workunit.client.0.smithi060.stdout:7/487: symlink da/d15/d88/d3b/la3 0 2024-02-26T10:47:57.469 INFO:tasks.workunit.client.0.smithi060.stdout:7/488: chown da/d15/d21/l6a 105 1 2024-02-26T10:47:57.469 INFO:tasks.workunit.client.0.smithi060.stdout:7/489: dread - da/d15/d21/d2f/f91 zero size 2024-02-26T10:47:57.473 INFO:tasks.workunit.client.0.smithi060.stdout:6/414: mknod d6/d7/d25/c87 0 2024-02-26T10:47:57.473 INFO:tasks.workunit.client.0.smithi060.stdout:6/415: truncate d6/d7/d25/d1c/d1f/d23/f4a 1021952 0 2024-02-26T10:47:57.476 INFO:tasks.workunit.client.0.smithi060.stdout:7/490: chown da/l11 11028226 1 2024-02-26T10:47:57.476 INFO:tasks.workunit.client.0.smithi060.stdout:7/491: stat da/d15/d21/d41/l7f 0 2024-02-26T10:47:57.476 INFO:tasks.workunit.client.0.smithi060.stdout:6/416: link d6/d7/d25/f4b d6/d7/d25/d1c/d1f/d23/d7a/f88 0 2024-02-26T10:47:57.479 INFO:tasks.workunit.client.0.smithi060.stdout:8/360: rename d9/d15/c2e to d9/d19/d50/c67 0 2024-02-26T10:47:57.479 INFO:tasks.workunit.client.0.smithi060.stdout:6/417: symlink d6/d7/d25/d1c/d1f/l89 0 2024-02-26T10:47:57.479 INFO:tasks.workunit.client.0.smithi060.stdout:6/418: chown d6/d7/d25/d1c/d39 27 1 2024-02-26T10:47:57.479 INFO:tasks.workunit.client.0.smithi060.stdout:8/361: chown d9/d15/d29/d2f/c33 22637928 1 2024-02-26T10:47:57.479 INFO:tasks.workunit.client.0.smithi060.stdout:8/362: write d9/d15/f52 [48096,50308] 0 2024-02-26T10:47:57.484 INFO:tasks.workunit.client.0.smithi060.stdout:0/342: rename d0/l6b to d0/dd/l71 0 2024-02-26T10:47:57.487 INFO:tasks.workunit.client.0.smithi060.stdout:6/419: link d6/d7/d25/d1c/d1f/d23/f85 d6/d7/d25/d1c/d1f/d23/d2b/d67/f8a 0 2024-02-26T10:47:57.487 INFO:tasks.workunit.client.0.smithi060.stdout:6/420: dread - d6/d7/d47/f72 zero size 2024-02-26T10:47:57.490 INFO:tasks.workunit.client.0.smithi060.stdout:6/421: fsync d6/d7/d25/d1c/d1f/d23/d28/f33 0 2024-02-26T10:47:57.503 INFO:tasks.workunit.client.0.smithi060.stdout:0/343: write d0/f3 [3811490,120198] 0 2024-02-26T10:47:57.503 INFO:tasks.workunit.client.0.smithi060.stdout:0/344: rename d0/dd to d0/dd/d33/d72 22 2024-02-26T10:47:57.503 INFO:tasks.workunit.client.0.smithi060.stdout:0/345: truncate d0/dd/d39/d40/f1f 5145823 0 2024-02-26T10:47:57.503 INFO:tasks.workunit.client.0.smithi060.stdout:1/451: dread d3/f15 [0,4194304] 0 2024-02-26T10:47:57.510 INFO:tasks.workunit.client.0.smithi060.stdout:9/425: dwrite d3/d1e/d5e/d1b/d5f/f38 [0,4194304] 0 2024-02-26T10:47:57.511 INFO:tasks.workunit.client.0.smithi060.stdout:9/426: dread - d3/d1e/d5e/d1b/d5f/dc/f8b zero size 2024-02-26T10:47:57.511 INFO:tasks.workunit.client.0.smithi060.stdout:5/377: write d0/d33/f63 [2009297,66649] 0 2024-02-26T10:47:57.511 INFO:tasks.workunit.client.0.smithi060.stdout:9/427: rmdir d3/d1e/d5e/d35/d87 39 2024-02-26T10:47:57.513 INFO:tasks.workunit.client.0.smithi060.stdout:5/378: mkdir d0/de/d15/d65/d86 0 2024-02-26T10:47:57.513 INFO:tasks.workunit.client.0.smithi060.stdout:5/379: truncate d0/de/d20/f85 567170 0 2024-02-26T10:47:57.517 INFO:tasks.workunit.client.0.smithi060.stdout:5/380: creat d0/de/d20/d82/f87 x:0 0 0 2024-02-26T10:47:57.517 INFO:tasks.workunit.client.0.smithi060.stdout:5/381: rename d0/de/d15/d65/l7a to d0/l88 0 2024-02-26T10:47:57.519 INFO:tasks.workunit.client.0.smithi060.stdout:4/366: dwrite d0/d7/d29/f32 [0,4194304] 0 2024-02-26T10:47:57.544 INFO:tasks.workunit.client.0.smithi060.stdout:4/367: dread d0/d27/d2f/f35 [0,4194304] 0 2024-02-26T10:47:57.544 INFO:tasks.workunit.client.0.smithi060.stdout:4/368: truncate d0/d7/d29/f5b 346535 0 2024-02-26T10:47:57.544 INFO:tasks.workunit.client.0.smithi060.stdout:9/428: dread d3/d1e/d5e/d1b/d5f/f5b [0,4194304] 0 2024-02-26T10:47:57.547 INFO:tasks.workunit.client.0.smithi060.stdout:8/363: dread d9/d19/f3a [0,4194304] 0 2024-02-26T10:47:57.547 INFO:tasks.workunit.client.0.smithi060.stdout:9/429: unlink d3/d1e/d5e/d1b/d5f/l4a 0 2024-02-26T10:47:57.547 INFO:tasks.workunit.client.0.smithi060.stdout:9/430: fdatasync d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f6d 0 2024-02-26T10:47:57.547 INFO:tasks.workunit.client.0.smithi060.stdout:4/369: symlink d0/d7/l68 0 2024-02-26T10:47:57.552 INFO:tasks.workunit.client.0.smithi060.stdout:9/431: mknod d3/d1e/c8e 0 2024-02-26T10:47:57.552 INFO:tasks.workunit.client.0.smithi060.stdout:9/432: write d3/d1e/d5e/d1b/d5f/d10/f7e [112734,4148] 0 2024-02-26T10:47:57.553 INFO:tasks.workunit.client.0.smithi060.stdout:3/289: write d5/de/f3b [1372510,44393] 0 2024-02-26T10:47:57.554 INFO:tasks.workunit.client.0.smithi060.stdout:2/334: dwrite f3 [4194304,4194304] 0 2024-02-26T10:47:57.556 INFO:tasks.workunit.client.0.smithi060.stdout:2/335: mkdir dd/d43/d60/d41/d37/d69 0 2024-02-26T10:47:57.559 INFO:tasks.workunit.client.0.smithi060.stdout:2/336: creat dd/f6a x:0 0 0 2024-02-26T10:47:57.563 INFO:tasks.workunit.client.0.smithi060.stdout:6/422: dwrite d6/d7/d25/d1c/d1f/d23/d7a/f88 [0,4194304] 0 2024-02-26T10:47:57.564 INFO:tasks.workunit.client.0.smithi060.stdout:9/433: rmdir d3/d1e/d5e/d1b 39 2024-02-26T10:47:57.565 INFO:tasks.workunit.client.0.smithi060.stdout:7/492: dwrite da/fe [0,4194304] 0 2024-02-26T10:47:57.565 INFO:tasks.workunit.client.0.smithi060.stdout:1/452: dread d3/d35/d76/f98 [0,4194304] 0 2024-02-26T10:47:57.565 INFO:tasks.workunit.client.0.smithi060.stdout:1/453: readlink d3/db/d1e/l37 0 2024-02-26T10:47:57.566 INFO:tasks.workunit.client.0.smithi060.stdout:8/364: dread f3 [0,4194304] 0 2024-02-26T10:47:57.566 INFO:tasks.workunit.client.0.smithi060.stdout:3/290: creat d5/de/df/d4c/f4f x:0 0 0 2024-02-26T10:47:57.566 INFO:tasks.workunit.client.0.smithi060.stdout:3/291: chown d5/d45/f49 5447242 1 2024-02-26T10:47:57.566 INFO:tasks.workunit.client.0.smithi060.stdout:0/346: dwrite d0/f3 [0,4194304] 0 2024-02-26T10:47:57.566 INFO:tasks.workunit.client.0.smithi060.stdout:0/347: write d0/dd/f2c [4019372,98563] 0 2024-02-26T10:47:57.567 INFO:tasks.workunit.client.0.smithi060.stdout:1/454: mkdir d3/db/d1e/d39/d9b 0 2024-02-26T10:47:57.567 INFO:tasks.workunit.client.0.smithi060.stdout:8/365: creat d9/d5a/f68 x:0 0 0 2024-02-26T10:47:57.569 INFO:tasks.workunit.client.0.smithi060.stdout:8/366: creat d9/d5a/f69 x:0 0 0 2024-02-26T10:47:57.569 INFO:tasks.workunit.client.0.smithi060.stdout:8/367: fdatasync d9/d15/f36 0 2024-02-26T10:47:57.569 INFO:tasks.workunit.client.0.smithi060.stdout:8/368: write d9/d5a/f69 [81559,109102] 0 2024-02-26T10:47:57.569 INFO:tasks.workunit.client.0.smithi060.stdout:8/369: read - d9/f24 zero size 2024-02-26T10:47:57.570 INFO:tasks.workunit.client.0.smithi060.stdout:7/493: creat da/d15/d88/d39/fa4 x:0 0 0 2024-02-26T10:47:57.570 INFO:tasks.workunit.client.0.smithi060.stdout:7/494: stat da/d15/d88/d3b/f54 0 2024-02-26T10:47:57.572 INFO:tasks.workunit.client.0.smithi060.stdout:3/292: symlink d5/de/df/d11/l50 0 2024-02-26T10:47:57.572 INFO:tasks.workunit.client.0.smithi060.stdout:3/293: dread - d5/d37/f41 zero size 2024-02-26T10:47:57.573 INFO:tasks.workunit.client.0.smithi060.stdout:3/294: readlink d5/de/df/d11/l13 0 2024-02-26T10:47:57.575 INFO:tasks.workunit.client.0.smithi060.stdout:9/434: getdents d3/d1e 0 2024-02-26T10:47:57.575 INFO:tasks.workunit.client.0.smithi060.stdout:9/435: write d3/d1e/d5e/d1b/f41 [110446,87220] 0 2024-02-26T10:47:57.579 INFO:tasks.workunit.client.0.smithi060.stdout:1/455: creat d3/d35/f9c x:0 0 0 2024-02-26T10:47:57.580 INFO:tasks.workunit.client.0.smithi060.stdout:9/436: creat d3/d1e/d5e/d1b/d5f/d74/f8f x:0 0 0 2024-02-26T10:47:57.580 INFO:tasks.workunit.client.0.smithi060.stdout:9/437: chown d3/d1e/d5e/d1b/d5f/d74 863 1 2024-02-26T10:47:57.581 INFO:tasks.workunit.client.0.smithi060.stdout:9/438: readlink d3/d1e/d5e/d1b/d5f/dc/d32/l83 0 2024-02-26T10:47:57.582 INFO:tasks.workunit.client.0.smithi060.stdout:1/456: mknod d3/db/d1e/d6d/d78/c9d 0 2024-02-26T10:47:57.582 INFO:tasks.workunit.client.0.smithi060.stdout:2/337: dread f5 [0,4194304] 0 2024-02-26T10:47:57.585 INFO:tasks.workunit.client.0.smithi060.stdout:1/457: creat d3/d35/d53/d68/d6b/f9e x:0 0 0 2024-02-26T10:47:57.585 INFO:tasks.workunit.client.0.smithi060.stdout:1/458: chown d3/d35/d53/d68/d90 629516593 1 2024-02-26T10:47:57.596 INFO:tasks.workunit.client.0.smithi060.stdout:8/370: rename d9/d15/d40 to d9/d19/d50/d6a 0 2024-02-26T10:47:57.596 INFO:tasks.workunit.client.0.smithi060.stdout:8/371: dread - d9/d19/d50/d6a/f61 zero size 2024-02-26T10:47:57.598 INFO:tasks.workunit.client.0.smithi060.stdout:5/382: dwrite d0/fd [0,4194304] 0 2024-02-26T10:47:57.601 INFO:tasks.workunit.client.0.smithi060.stdout:4/370: dread d0/da/d3b/f23 [0,4194304] 0 2024-02-26T10:47:57.603 INFO:tasks.workunit.client.0.smithi060.stdout:2/338: read dd/d43/d60/d41/f31 [924077,66278] 0 2024-02-26T10:47:57.603 INFO:tasks.workunit.client.0.smithi060.stdout:5/383: creat d0/de/d20/d32/f89 x:0 0 0 2024-02-26T10:47:57.609 INFO:tasks.workunit.client.0.smithi060.stdout:7/495: rename f7 to da/d15/d21/d4b/fa5 0 2024-02-26T10:47:57.609 INFO:tasks.workunit.client.0.smithi060.stdout:6/423: dread d6/f11 [0,4194304] 0 2024-02-26T10:47:57.622 INFO:tasks.workunit.client.0.smithi060.stdout:7/496: symlink da/d15/la6 0 2024-02-26T10:47:57.622 INFO:tasks.workunit.client.0.smithi060.stdout:7/497: fdatasync da/d15/d21/d32/f63 0 2024-02-26T10:47:57.622 INFO:tasks.workunit.client.0.smithi060.stdout:2/339: mknod dd/d43/d60/c6b 0 2024-02-26T10:47:57.622 INFO:tasks.workunit.client.0.smithi060.stdout:2/340: creat dd/d43/f6c x:0 0 0 2024-02-26T10:47:57.623 INFO:tasks.workunit.client.0.smithi060.stdout:2/341: creat dd/d43/d60/d41/f6d x:0 0 0 2024-02-26T10:47:57.632 INFO:tasks.workunit.client.0.smithi060.stdout:0/348: dwrite d0/f5f [0,4194304] 0 2024-02-26T10:47:57.633 INFO:tasks.workunit.client.0.smithi060.stdout:0/349: creat d0/d37/d44/d58/f73 x:0 0 0 2024-02-26T10:47:57.633 INFO:tasks.workunit.client.0.smithi060.stdout:4/371: read d0/d7/d29/f38 [1649645,97069] 0 2024-02-26T10:47:57.633 INFO:tasks.workunit.client.0.smithi060.stdout:4/372: write d0/da/d57/f67 [2932,31155] 0 2024-02-26T10:47:57.633 INFO:tasks.workunit.client.0.smithi060.stdout:0/350: write d0/dd/d39/d40/f17 [4967635,95802] 0 2024-02-26T10:47:57.633 INFO:tasks.workunit.client.0.smithi060.stdout:0/351: fsync d0/dd/f31 0 2024-02-26T10:47:57.637 INFO:tasks.workunit.client.0.smithi060.stdout:4/373: creat d0/da/d57/f69 x:0 0 0 2024-02-26T10:47:57.639 INFO:tasks.workunit.client.0.smithi060.stdout:2/342: truncate f3 6216913 0 2024-02-26T10:47:57.640 INFO:tasks.workunit.client.0.smithi060.stdout:2/343: mkdir dd/d43/d55/d6e 0 2024-02-26T10:47:57.643 INFO:tasks.workunit.client.0.smithi060.stdout:2/344: mkdir dd/d43/d60/d41/d42/d6f 0 2024-02-26T10:47:57.643 INFO:tasks.workunit.client.0.smithi060.stdout:2/345: fsync dd/d11/f62 0 2024-02-26T10:47:57.645 INFO:tasks.workunit.client.0.smithi060.stdout:2/346: write dd/d11/f24 [5027032,70100] 0 2024-02-26T10:47:57.645 INFO:tasks.workunit.client.0.smithi060.stdout:2/347: write dd/d11/f62 [609561,13264] 0 2024-02-26T10:47:57.645 INFO:tasks.workunit.client.0.smithi060.stdout:2/348: dread - dd/d43/d60/d41/f6d zero size 2024-02-26T10:47:57.645 INFO:tasks.workunit.client.0.smithi060.stdout:3/295: dwrite d5/fd [0,4194304] 0 2024-02-26T10:47:57.645 INFO:tasks.workunit.client.0.smithi060.stdout:3/296: fdatasync d5/d45/f48 0 2024-02-26T10:47:57.645 INFO:tasks.workunit.client.0.smithi060.stdout:3/297: unlink d5/de/df/l4e 0 2024-02-26T10:47:57.646 INFO:tasks.workunit.client.0.smithi060.stdout:3/298: creat d5/de/d16/d17/f51 x:0 0 0 2024-02-26T10:47:57.647 INFO:tasks.workunit.client.0.smithi060.stdout:3/299: creat d5/f52 x:0 0 0 2024-02-26T10:47:57.647 INFO:tasks.workunit.client.0.smithi060.stdout:3/300: dread - d5/d37/f41 zero size 2024-02-26T10:47:57.650 INFO:tasks.workunit.client.0.smithi060.stdout:8/372: dwrite d9/d15/d29/d2f/f55 [0,4194304] 0 2024-02-26T10:47:57.656 INFO:tasks.workunit.client.0.smithi060.stdout:9/439: dwrite d3/d1e/d5e/d1b/d5f/d10/f4e [0,4194304] 0 2024-02-26T10:47:57.660 INFO:tasks.workunit.client.0.smithi060.stdout:9/440: mknod d3/d1e/d5e/d1b/d5f/dc/d32/d3c/c90 0 2024-02-26T10:47:57.660 INFO:tasks.workunit.client.0.smithi060.stdout:9/441: mkdir d3/d1e/d5e/d1b/d5f/d8c/d91 0 2024-02-26T10:47:57.664 INFO:tasks.workunit.client.0.smithi060.stdout:9/442: symlink d3/d1e/d5e/d1b/l92 0 2024-02-26T10:47:57.665 INFO:tasks.workunit.client.0.smithi060.stdout:0/352: truncate d0/d42/f60 301413 0 2024-02-26T10:47:57.666 INFO:tasks.workunit.client.0.smithi060.stdout:1/459: dwrite d3/f5c [0,4194304] 0 2024-02-26T10:47:57.666 INFO:tasks.workunit.client.0.smithi060.stdout:1/460: fsync d3/d54/f7c 0 2024-02-26T10:47:57.667 INFO:tasks.workunit.client.0.smithi060.stdout:1/461: symlink d3/db/d1e/d39/l9f 0 2024-02-26T10:47:57.668 INFO:tasks.workunit.client.0.smithi060.stdout:0/353: mknod d0/dd/c74 0 2024-02-26T10:47:57.670 INFO:tasks.workunit.client.0.smithi060.stdout:1/462: getdents d3/db 0 2024-02-26T10:47:57.675 INFO:tasks.workunit.client.0.smithi060.stdout:1/463: creat d3/d35/fa0 x:0 0 0 2024-02-26T10:47:57.675 INFO:tasks.workunit.client.0.smithi060.stdout:1/464: dread - d3/db/d1e/d6d/d78/f86 zero size 2024-02-26T10:47:57.675 INFO:tasks.workunit.client.0.smithi060.stdout:1/465: truncate d3/db/d1e/d6d/d78/f99 1008172 0 2024-02-26T10:47:57.680 INFO:tasks.workunit.client.0.smithi060.stdout:1/466: link d3/db/d2e/d51/l6f d3/d54/la1 0 2024-02-26T10:47:57.680 INFO:tasks.workunit.client.0.smithi060.stdout:1/467: chown d3/d35/d53/d68/d85/c41 112706 1 2024-02-26T10:47:57.681 INFO:tasks.workunit.client.0.smithi060.stdout:1/468: readlink d3/db/d1e/d38/d6c/l7b 0 2024-02-26T10:47:57.684 INFO:tasks.workunit.client.0.smithi060.stdout:7/498: dwrite da/d15/d21/d41/d58/f70 [0,4194304] 0 2024-02-26T10:47:57.688 INFO:tasks.workunit.client.0.smithi060.stdout:1/469: mknod d3/db/d2e/d51/ca2 0 2024-02-26T10:47:57.690 INFO:tasks.workunit.client.0.smithi060.stdout:1/470: mkdir d3/db/d1e/d6d/da3 0 2024-02-26T10:47:57.701 INFO:tasks.workunit.client.0.smithi060.stdout:4/374: dwrite d0/f3c [0,4194304] 0 2024-02-26T10:47:57.704 INFO:tasks.workunit.client.0.smithi060.stdout:5/384: dwrite d0/d33/f43 [4194304,4194304] 0 2024-02-26T10:47:57.704 INFO:tasks.workunit.client.0.smithi060.stdout:2/349: dread f4 [0,4194304] 0 2024-02-26T10:47:57.704 INFO:tasks.workunit.client.0.smithi060.stdout:2/350: dread - dd/d11/f65 zero size 2024-02-26T10:47:57.705 INFO:tasks.workunit.client.0.smithi060.stdout:2/351: mknod dd/d43/d60/d41/d37/d69/c70 0 2024-02-26T10:47:57.707 INFO:tasks.workunit.client.0.smithi060.stdout:2/352: symlink dd/d43/d60/d41/d37/d38/d57/l71 0 2024-02-26T10:47:57.708 INFO:tasks.workunit.client.0.smithi060.stdout:0/354: read d0/dd/d39/d40/f34 [1469618,27140] 0 2024-02-26T10:47:57.708 INFO:tasks.workunit.client.0.smithi060.stdout:1/471: rename d3/db/l1a to d3/db/d1e/d6d/la4 0 2024-02-26T10:47:57.709 INFO:tasks.workunit.client.0.smithi060.stdout:2/353: rename dd/d43/d60/f3d to dd/d43/d60/d41/d37/d69/f72 0 2024-02-26T10:47:57.710 INFO:tasks.workunit.client.0.smithi060.stdout:8/373: dread d9/d15/d29/d2f/f31 [0,4194304] 0 2024-02-26T10:47:57.710 INFO:tasks.workunit.client.0.smithi060.stdout:0/355: link d0/dd/d39/d40/c3d d0/dd/d39/d63/c75 0 2024-02-26T10:47:57.711 INFO:tasks.workunit.client.0.smithi060.stdout:5/385: mknod d0/de/d20/d32/d4d/c8a 0 2024-02-26T10:47:57.711 INFO:tasks.workunit.client.0.smithi060.stdout:8/374: mknod d9/d19/d50/c6b 0 2024-02-26T10:47:57.712 INFO:tasks.workunit.client.0.smithi060.stdout:1/472: creat d3/d35/fa5 x:0 0 0 2024-02-26T10:47:57.713 INFO:tasks.workunit.client.0.smithi060.stdout:1/473: chown d3/db/d1e/l37 196221 1 2024-02-26T10:47:57.713 INFO:tasks.workunit.client.0.smithi060.stdout:5/386: link d0/de/d15/c27 d0/de/d15/d26/c8b 0 2024-02-26T10:47:57.714 INFO:tasks.workunit.client.0.smithi060.stdout:0/356: stat d0/dd/d39/d40/c3d 0 2024-02-26T10:47:57.714 INFO:tasks.workunit.client.0.smithi060.stdout:1/474: mkdir d3/db/d1e/d39/d9b/da6 0 2024-02-26T10:47:57.714 INFO:tasks.workunit.client.0.smithi060.stdout:1/475: truncate d3/d35/d53/d68/d6b/f9e 82534 0 2024-02-26T10:47:57.714 INFO:tasks.workunit.client.0.smithi060.stdout:1/476: chown d3/db/d1e/c3a 146 1 2024-02-26T10:47:57.715 INFO:tasks.workunit.client.0.smithi060.stdout:0/357: mknod d0/d37/c76 0 2024-02-26T10:47:57.716 INFO:tasks.workunit.client.0.smithi060.stdout:1/477: rmdir d3/d35/d76 39 2024-02-26T10:47:57.718 INFO:tasks.workunit.client.0.smithi060.stdout:0/358: truncate d0/d37/d44/d58/f5b 1491024 0 2024-02-26T10:47:57.720 INFO:tasks.workunit.client.0.smithi060.stdout:1/478: mknod d3/d35/d5e/ca7 0 2024-02-26T10:47:57.720 INFO:tasks.workunit.client.0.smithi060.stdout:1/479: chown d3/d35/d5e/f97 187851351 1 2024-02-26T10:47:57.720 INFO:tasks.workunit.client.0.smithi060.stdout:1/480: write d3/db/d1e/d38/f8e [325413,108199] 0 2024-02-26T10:47:57.720 INFO:tasks.workunit.client.0.smithi060.stdout:1/481: chown d3/d35/l44 2 1 2024-02-26T10:47:57.722 INFO:tasks.workunit.client.0.smithi060.stdout:2/354: dread dd/d11/f62 [0,4194304] 0 2024-02-26T10:47:57.722 INFO:tasks.workunit.client.0.smithi060.stdout:1/482: symlink d3/d35/d53/d68/d85/d8a/d91/la8 0 2024-02-26T10:47:57.722 INFO:tasks.workunit.client.0.smithi060.stdout:2/355: write dd/d43/d55/f5a [525376,101614] 0 2024-02-26T10:47:57.722 INFO:tasks.workunit.client.0.smithi060.stdout:2/356: readlink dd/l3a 0 2024-02-26T10:47:57.723 INFO:tasks.workunit.client.0.smithi060.stdout:0/359: rename d0/dd/d39/d40/f28 to d0/dd/f77 0 2024-02-26T10:47:57.724 INFO:tasks.workunit.client.0.smithi060.stdout:0/360: link d0/dd/f31 d0/f78 0 2024-02-26T10:47:57.726 INFO:tasks.workunit.client.0.smithi060.stdout:0/361: rename d0/dd/d39/f3f to d0/d37/d44/f79 0 2024-02-26T10:47:57.728 INFO:tasks.workunit.client.0.smithi060.stdout:0/362: mkdir d0/d42/d7a 0 2024-02-26T10:47:57.728 INFO:tasks.workunit.client.0.smithi060.stdout:4/375: link d0/da/d3b/d16/f39 d0/d7/f6a 0 2024-02-26T10:47:57.728 INFO:tasks.workunit.client.0.smithi060.stdout:4/376: write d0/d27/d2f/f55 [3677674,114411] 0 2024-02-26T10:47:57.732 INFO:tasks.workunit.client.0.smithi060.stdout:8/375: write d9/d15/d29/d2f/f31 [3241594,70841] 0 2024-02-26T10:47:57.733 INFO:tasks.workunit.client.0.smithi060.stdout:8/376: truncate d9/d15/d29/f65 844962 0 2024-02-26T10:47:57.743 INFO:tasks.workunit.client.0.smithi060.stdout:9/443: dwrite d3/d1e/d5e/d1b/f41 [0,4194304] 0 2024-02-26T10:47:57.749 INFO:tasks.workunit.client.0.smithi060.stdout:2/357: dread dd/d43/d60/d41/f36 [0,4194304] 0 2024-02-26T10:47:57.750 INFO:tasks.workunit.client.0.smithi060.stdout:2/358: symlink dd/d43/d55/l73 0 2024-02-26T10:47:57.751 INFO:tasks.workunit.client.0.smithi060.stdout:2/359: mkdir dd/d43/d55/d5e/d74 0 2024-02-26T10:47:57.751 INFO:tasks.workunit.client.0.smithi060.stdout:2/360: truncate dd/d43/f6c 498776 0 2024-02-26T10:47:57.757 INFO:tasks.workunit.client.0.smithi060.stdout:8/377: write d9/d15/f27 [427444,109197] 0 2024-02-26T10:47:57.758 INFO:tasks.workunit.client.0.smithi060.stdout:2/361: rmdir dd/d52 39 2024-02-26T10:47:57.758 INFO:tasks.workunit.client.0.smithi060.stdout:2/362: chown dd/d43/d60/d41/d37/d69/f72 0 1 2024-02-26T10:47:57.761 INFO:tasks.workunit.client.0.smithi060.stdout:9/444: unlink d3/d1e/d5e/d35/d87/l59 0 2024-02-26T10:47:57.762 INFO:tasks.workunit.client.0.smithi060.stdout:1/483: write d3/d35/d53/d57/f87 [2411651,126883] 0 2024-02-26T10:47:57.762 INFO:tasks.workunit.client.0.smithi060.stdout:1/484: symlink d3/db/d1e/d39/la9 0 2024-02-26T10:47:57.763 INFO:tasks.workunit.client.0.smithi060.stdout:9/445: rmdir d3/d1e/d5e/d1b 39 2024-02-26T10:47:57.763 INFO:tasks.workunit.client.0.smithi060.stdout:1/485: mkdir d3/db/d1e/d6d/da3/daa 0 2024-02-26T10:47:57.765 INFO:tasks.workunit.client.0.smithi060.stdout:9/446: rename d3/d1e/d5e/d35/d84 to d3/d1e/d93 0 2024-02-26T10:47:57.765 INFO:tasks.workunit.client.0.smithi060.stdout:1/486: symlink d3/d35/d76/lab 0 2024-02-26T10:47:57.767 INFO:tasks.workunit.client.0.smithi060.stdout:1/487: creat d3/db/d2e/d51/fac x:0 0 0 2024-02-26T10:47:57.767 INFO:tasks.workunit.client.0.smithi060.stdout:1/488: dread - d3/d35/d53/d68/d85/d8a/f9a zero size 2024-02-26T10:47:57.767 INFO:tasks.workunit.client.0.smithi060.stdout:1/489: dread - d3/db/d2e/d51/d63/f92 zero size 2024-02-26T10:47:57.769 INFO:tasks.workunit.client.0.smithi060.stdout:1/490: mkdir d3/d35/d53/d68/dad 0 2024-02-26T10:47:57.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/447: dread d3/d1e/d5e/d35/f7c [0,4194304] 0 2024-02-26T10:47:57.777 INFO:tasks.workunit.client.0.smithi060.stdout:9/448: mknod d3/d1e/d5e/d1b/d5f/dc/d32/c94 0 2024-02-26T10:47:57.778 INFO:tasks.workunit.client.0.smithi060.stdout:4/377: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:57.779 INFO:tasks.workunit.client.0.smithi060.stdout:4/378: mknod d0/c6b 0 2024-02-26T10:47:57.780 INFO:tasks.workunit.client.0.smithi060.stdout:9/449: rmdir d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47 39 2024-02-26T10:47:57.793 INFO:tasks.workunit.client.0.smithi060.stdout:1/491: rename d3/d35/d53/d68/d85/d8a/d91/c93 to d3/d54/cae 0 2024-02-26T10:47:57.793 INFO:tasks.workunit.client.0.smithi060.stdout:1/492: fdatasync d3/f34 0 2024-02-26T10:47:57.798 INFO:tasks.workunit.client.0.smithi060.stdout:1/493: creat d3/d35/d76/faf x:0 0 0 2024-02-26T10:47:57.802 INFO:tasks.workunit.client.0.smithi060.stdout:2/363: dwrite dd/d43/d60/d41/f31 [0,4194304] 0 2024-02-26T10:47:57.807 INFO:tasks.workunit.client.0.smithi060.stdout:4/379: rename d0/da/d3b/f2b to d0/da/d3b/d16/f6c 0 2024-02-26T10:47:57.807 INFO:tasks.workunit.client.0.smithi060.stdout:4/380: write d0/da/f14 [4407599,99343] 0 2024-02-26T10:47:57.808 INFO:tasks.workunit.client.0.smithi060.stdout:2/364: dread dd/d43/f6c [0,4194304] 0 2024-02-26T10:47:57.809 INFO:tasks.workunit.client.0.smithi060.stdout:4/381: mknod d0/da/d57/c6d 0 2024-02-26T10:47:57.817 INFO:tasks.workunit.client.0.smithi060.stdout:4/382: dread d0/d7/f40 [0,4194304] 0 2024-02-26T10:47:57.823 INFO:tasks.workunit.client.0.smithi060.stdout:4/383: link d0/da/d57/f67 d0/d27/d2f/f6e 0 2024-02-26T10:47:57.830 INFO:tasks.workunit.client.0.smithi060.stdout:2/365: write dd/d43/d60/f3f [6556190,109135] 0 2024-02-26T10:47:57.830 INFO:tasks.workunit.client.0.smithi060.stdout:8/378: dwrite d9/f1e [0,4194304] 0 2024-02-26T10:47:57.831 INFO:tasks.workunit.client.0.smithi060.stdout:4/384: dread d0/d7/d29/f5b [0,4194304] 0 2024-02-26T10:47:57.831 INFO:tasks.workunit.client.0.smithi060.stdout:4/385: write d0/d7/f40 [770756,61889] 0 2024-02-26T10:47:57.831 INFO:tasks.workunit.client.0.smithi060.stdout:4/386: stat d0/c22 0 2024-02-26T10:47:57.831 INFO:tasks.workunit.client.0.smithi060.stdout:2/366: symlink dd/d43/d60/d41/d37/l75 0 2024-02-26T10:47:57.833 INFO:tasks.workunit.client.0.smithi060.stdout:2/367: rmdir dd/d43/d55 39 2024-02-26T10:47:57.839 INFO:tasks.workunit.client.0.smithi060.stdout:2/368: mkdir dd/d43/d60/d41/d37/d4c/d76 0 2024-02-26T10:47:57.839 INFO:tasks.workunit.client.0.smithi060.stdout:2/369: write dd/d43/d60/d41/d37/d38/f40 [705924,50049] 0 2024-02-26T10:47:57.839 INFO:tasks.workunit.client.0.smithi060.stdout:8/379: rename d9/d19/d50/d6a/l48 to d9/d15/l6c 0 2024-02-26T10:47:57.839 INFO:tasks.workunit.client.0.smithi060.stdout:8/380: stat d9/d15/c26 0 2024-02-26T10:47:57.840 INFO:tasks.workunit.client.0.smithi060.stdout:8/381: creat d9/d15/f6d x:0 0 0 2024-02-26T10:47:57.841 INFO:tasks.workunit.client.0.smithi060.stdout:3/301: sync 2024-02-26T10:47:57.841 INFO:tasks.workunit.client.0.smithi060.stdout:6/424: sync 2024-02-26T10:47:57.841 INFO:tasks.workunit.client.0.smithi060.stdout:7/499: sync 2024-02-26T10:47:57.842 INFO:tasks.workunit.client.0.smithi060.stdout:8/382: creat d9/d5a/f6e x:0 0 0 2024-02-26T10:47:57.845 INFO:tasks.workunit.client.0.smithi060.stdout:6/425: unlink d6/d7/d25/d1c/d1f/d23/f85 0 2024-02-26T10:47:57.846 INFO:tasks.workunit.client.0.smithi060.stdout:6/426: creat d6/d7/d25/d1c/d1f/d23/d2b/d4d/f8b x:0 0 0 2024-02-26T10:47:57.849 INFO:tasks.workunit.client.0.smithi060.stdout:6/427: mkdir d6/d7/d2e/d56/d73/d8c 0 2024-02-26T10:47:57.857 INFO:tasks.workunit.client.0.smithi060.stdout:4/387: dread d0/da/d3b/f34 [0,4194304] 0 2024-02-26T10:47:57.867 INFO:tasks.workunit.client.0.smithi060.stdout:7/500: rename da/d15/d88/f5c to da/d15/d21/d2f/d78/fa7 0 2024-02-26T10:47:57.869 INFO:tasks.workunit.client.0.smithi060.stdout:7/501: mkdir da/d15/d21/d4b/d67/da8 0 2024-02-26T10:47:57.871 INFO:tasks.workunit.client.0.smithi060.stdout:7/502: symlink da/d15/d21/d41/d6c/la9 0 2024-02-26T10:47:57.871 INFO:tasks.workunit.client.0.smithi060.stdout:7/503: fdatasync da/d15/d21/d2f/f68 0 2024-02-26T10:47:57.887 INFO:tasks.workunit.client.0.smithi060.stdout:3/302: read d5/de/df/f21 [2217577,119789] 0 2024-02-26T10:47:57.890 INFO:tasks.workunit.client.0.smithi060.stdout:3/303: getdents d5 0 2024-02-26T10:47:57.890 INFO:tasks.workunit.client.0.smithi060.stdout:3/304: chown f3 0 1 2024-02-26T10:47:57.891 INFO:tasks.workunit.client.0.smithi060.stdout:3/305: rmdir d5/de/df/d4c 39 2024-02-26T10:47:57.894 INFO:tasks.workunit.client.0.smithi060.stdout:3/306: creat d5/d45/f53 x:0 0 0 2024-02-26T10:47:57.894 INFO:tasks.workunit.client.0.smithi060.stdout:3/307: fsync d5/de/f14 0 2024-02-26T10:47:57.900 INFO:tasks.workunit.client.0.smithi060.stdout:2/370: dwrite f0 [4194304,4194304] 0 2024-02-26T10:47:57.904 INFO:tasks.workunit.client.0.smithi060.stdout:2/371: symlink dd/d43/d60/d41/d37/d38/l77 0 2024-02-26T10:47:57.906 INFO:tasks.workunit.client.0.smithi060.stdout:6/428: dwrite d6/f11 [0,4194304] 0 2024-02-26T10:47:57.907 INFO:tasks.workunit.client.0.smithi060.stdout:6/429: mkdir d6/d8d 0 2024-02-26T10:47:57.908 INFO:tasks.workunit.client.0.smithi060.stdout:6/430: creat d6/f8e x:0 0 0 2024-02-26T10:47:57.908 INFO:tasks.workunit.client.0.smithi060.stdout:6/431: rename d6/d7/d25/d1c to d6/d7/d25/d1c/d1f/d23/d2b/d4d/d8f 22 2024-02-26T10:47:57.914 INFO:tasks.workunit.client.0.smithi060.stdout:8/383: dwrite d9/d15/d29/f65 [0,4194304] 0 2024-02-26T10:47:57.915 INFO:tasks.workunit.client.0.smithi060.stdout:8/384: read d9/d15/d29/d2f/f31 [3013801,45262] 0 2024-02-26T10:47:57.915 INFO:tasks.workunit.client.0.smithi060.stdout:8/385: write d9/f13 [2070967,94235] 0 2024-02-26T10:47:57.915 INFO:tasks.workunit.client.0.smithi060.stdout:8/386: fsync d9/d15/d29/f65 0 2024-02-26T10:47:57.915 INFO:tasks.workunit.client.0.smithi060.stdout:8/387: dread - d9/d19/f62 zero size 2024-02-26T10:47:57.916 INFO:tasks.workunit.client.0.smithi060.stdout:4/388: dwrite d0/d7/f37 [4194304,4194304] 0 2024-02-26T10:47:57.920 INFO:tasks.workunit.client.0.smithi060.stdout:4/389: symlink d0/d7/d29/d60/l6f 0 2024-02-26T10:47:57.923 INFO:tasks.workunit.client.0.smithi060.stdout:4/390: mknod d0/da/d3b/d1b/c70 0 2024-02-26T10:47:57.928 INFO:tasks.workunit.client.0.smithi060.stdout:3/308: read f3 [3839369,50078] 0 2024-02-26T10:47:57.932 INFO:tasks.workunit.client.0.smithi060.stdout:7/504: dwrite da/d15/d88/f5b [0,4194304] 0 2024-02-26T10:47:57.940 INFO:tasks.workunit.client.0.smithi060.stdout:6/432: dread d6/d7/d25/d1c/d1f/d23/d2b/f3b [0,4194304] 0 2024-02-26T10:47:57.940 INFO:tasks.workunit.client.0.smithi060.stdout:6/433: fsync d6/d7/d25/d1c/d1f/d23/d2b/d67/f8a 0 2024-02-26T10:47:57.941 INFO:tasks.workunit.client.0.smithi060.stdout:6/434: chown d6/d7/fd 1312476 1 2024-02-26T10:47:57.941 INFO:tasks.workunit.client.0.smithi060.stdout:6/435: write d6/d7/d25/d1c/d39/f77 [1031021,56552] 0 2024-02-26T10:47:57.942 INFO:tasks.workunit.client.0.smithi060.stdout:7/505: creat da/d94/faa x:0 0 0 2024-02-26T10:47:57.943 INFO:tasks.workunit.client.0.smithi060.stdout:6/436: write d6/d7/d25/d1c/d1f/d23/d28/f29 [545909,89558] 0 2024-02-26T10:47:57.945 INFO:tasks.workunit.client.0.smithi060.stdout:5/387: sync 2024-02-26T10:47:57.953 INFO:tasks.workunit.client.0.smithi060.stdout:1/494: sync 2024-02-26T10:47:57.953 INFO:tasks.workunit.client.0.smithi060.stdout:1/495: write d3/d35/d53/d68/d85/d8a/f9a [679238,4319] 0 2024-02-26T10:47:57.953 INFO:tasks.workunit.client.0.smithi060.stdout:0/363: sync 2024-02-26T10:47:57.953 INFO:tasks.workunit.client.0.smithi060.stdout:1/496: write d3/d35/d53/d68/d85/f94 [143462,130087] 0 2024-02-26T10:47:57.953 INFO:tasks.workunit.client.0.smithi060.stdout:9/450: sync 2024-02-26T10:47:57.953 INFO:tasks.workunit.client.0.smithi060.stdout:1/497: mknod d3/db/d2e/d51/d63/cb0 0 2024-02-26T10:47:57.954 INFO:tasks.workunit.client.0.smithi060.stdout:3/309: dread f3 [0,4194304] 0 2024-02-26T10:47:57.962 INFO:tasks.workunit.client.0.smithi060.stdout:3/310: truncate d5/fa 4024034 0 2024-02-26T10:47:57.962 INFO:tasks.workunit.client.0.smithi060.stdout:2/372: dwrite dd/d43/d60/f53 [0,4194304] 0 2024-02-26T10:47:57.962 INFO:tasks.workunit.client.0.smithi060.stdout:8/388: dread d9/d19/d50/d6a/f45 [0,4194304] 0 2024-02-26T10:47:57.962 INFO:tasks.workunit.client.0.smithi060.stdout:8/389: fsync d9/f23 0 2024-02-26T10:47:57.965 INFO:tasks.workunit.client.0.smithi060.stdout:8/390: truncate d9/d19/d50/d6a/f46 1196091 0 2024-02-26T10:47:57.966 INFO:tasks.workunit.client.0.smithi060.stdout:8/391: mkdir d9/d6f 0 2024-02-26T10:47:57.968 INFO:tasks.workunit.client.0.smithi060.stdout:8/392: mknod d9/c70 0 2024-02-26T10:47:57.969 INFO:tasks.workunit.client.0.smithi060.stdout:8/393: creat d9/d19/d50/d6a/f71 x:0 0 0 2024-02-26T10:47:57.974 INFO:tasks.workunit.client.0.smithi060.stdout:2/373: mknod dd/c78 0 2024-02-26T10:47:57.974 INFO:tasks.workunit.client.0.smithi060.stdout:2/374: stat dd/d43/d60/d41/d42/d6f 0 2024-02-26T10:47:57.981 INFO:tasks.workunit.client.0.smithi060.stdout:7/506: dread da/d15/d21/f22 [0,4194304] 0 2024-02-26T10:47:57.985 INFO:tasks.workunit.client.0.smithi060.stdout:9/451: write d3/d1e/d5e/d35/f3e [2234024,113127] 0 2024-02-26T10:47:57.985 INFO:tasks.workunit.client.0.smithi060.stdout:9/452: truncate d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d44/f8d 945539 0 2024-02-26T10:47:57.987 INFO:tasks.workunit.client.0.smithi060.stdout:9/453: mknod d3/d1e/c95 0 2024-02-26T10:47:57.988 INFO:tasks.workunit.client.0.smithi060.stdout:9/454: mkdir d3/d1e/d5e/d1b/d5f/d8c/d91/d96 0 2024-02-26T10:47:57.989 INFO:tasks.workunit.client.0.smithi060.stdout:6/437: dwrite d6/d7/d25/f4b [0,4194304] 0 2024-02-26T10:47:57.990 INFO:tasks.workunit.client.0.smithi060.stdout:9/455: creat d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/f97 x:0 0 0 2024-02-26T10:47:57.990 INFO:tasks.workunit.client.0.smithi060.stdout:6/438: rmdir d6/d7/d2e/d56/d73 39 2024-02-26T10:47:57.991 INFO:tasks.workunit.client.0.smithi060.stdout:9/456: mkdir d3/d1e/d5e/d1b/d5f/d10/d98 0 2024-02-26T10:47:57.992 INFO:tasks.workunit.client.0.smithi060.stdout:6/439: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f90 x:0 0 0 2024-02-26T10:47:57.992 INFO:tasks.workunit.client.0.smithi060.stdout:9/457: symlink d3/d1e/d5e/d1b/d5f/d10/l99 0 2024-02-26T10:47:57.994 INFO:tasks.workunit.client.0.smithi060.stdout:9/458: rmdir d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/d7a 39 2024-02-26T10:47:57.994 INFO:tasks.workunit.client.0.smithi060.stdout:9/459: mknod d3/d68/c9a 0 2024-02-26T10:47:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:9/460: write d3/d1e/d57/f79 [1242208,73109] 0 2024-02-26T10:47:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:5/388: write d0/de/d15/f44 [3036908,53171] 0 2024-02-26T10:47:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:5/389: dread - d0/de/d15/f47 zero size 2024-02-26T10:47:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:4/391: dwrite d0/ff [0,4194304] 0 2024-02-26T10:47:58.010 INFO:tasks.workunit.client.0.smithi060.stdout:2/375: creat dd/d43/d60/d41/d37/d38/d57/f79 x:0 0 0 2024-02-26T10:47:58.033 INFO:tasks.workunit.client.0.smithi060.stdout:8/394: rename d9/d19 to d9/d15/d29/d2f/d34/d72 0 2024-02-26T10:47:58.034 INFO:tasks.workunit.client.0.smithi060.stdout:6/440: write d6/f11 [1109429,8908] 0 2024-02-26T10:47:58.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/395: creat d9/d15/f73 x:0 0 0 2024-02-26T10:47:58.036 INFO:tasks.workunit.client.0.smithi060.stdout:0/364: dwrite d0/d42/f62 [0,4194304] 0 2024-02-26T10:47:58.036 INFO:tasks.workunit.client.0.smithi060.stdout:0/365: fsync d0/dd/d39/d40/f4d 0 2024-02-26T10:47:58.036 INFO:tasks.workunit.client.0.smithi060.stdout:0/366: fsync d0/dd/d39/f54 0 2024-02-26T10:47:58.036 INFO:tasks.workunit.client.0.smithi060.stdout:0/367: truncate d0/d37/f6d 355823 0 2024-02-26T10:47:58.037 INFO:tasks.workunit.client.0.smithi060.stdout:0/368: readlink d0/dd/l4f 0 2024-02-26T10:47:58.044 INFO:tasks.workunit.client.0.smithi060.stdout:7/507: rename da/d15/d21/d41/f73 to da/d15/d21/d32/fab 0 2024-02-26T10:47:58.044 INFO:tasks.workunit.client.0.smithi060.stdout:7/508: fdatasync da/d94/f98 0 2024-02-26T10:47:58.044 INFO:tasks.workunit.client.0.smithi060.stdout:7/509: chown da/d15/d21/d4b/d67/f85 21535845 1 2024-02-26T10:47:58.044 INFO:tasks.workunit.client.0.smithi060.stdout:6/441: creat d6/d7/d25/f91 x:0 0 0 2024-02-26T10:47:58.044 INFO:tasks.workunit.client.0.smithi060.stdout:6/442: chown d6/d7/d2e/d56/f61 1616 1 2024-02-26T10:47:58.044 INFO:tasks.workunit.client.0.smithi060.stdout:6/443: chown d6/fe 156 1 2024-02-26T10:47:58.044 INFO:tasks.workunit.client.0.smithi060.stdout:7/510: creat da/d15/d21/d2f/d78/fac x:0 0 0 2024-02-26T10:47:58.045 INFO:tasks.workunit.client.0.smithi060.stdout:6/444: symlink d6/d7/d25/l92 0 2024-02-26T10:47:58.045 INFO:tasks.workunit.client.0.smithi060.stdout:6/445: truncate d6/d7/d25/d1c/f60 406328 0 2024-02-26T10:47:58.045 INFO:tasks.workunit.client.0.smithi060.stdout:6/446: fdatasync d6/d7/d25/d1c/d1f/d23/d28/f7f 0 2024-02-26T10:47:58.046 INFO:tasks.workunit.client.0.smithi060.stdout:7/511: readlink da/d15/l1c 0 2024-02-26T10:47:58.047 INFO:tasks.workunit.client.0.smithi060.stdout:6/447: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93 0 2024-02-26T10:47:58.049 INFO:tasks.workunit.client.0.smithi060.stdout:3/311: write d5/de/d16/d17/f30 [894275,46945] 0 2024-02-26T10:47:58.049 INFO:tasks.workunit.client.0.smithi060.stdout:6/448: link d6/d7/d25/d1c/d1f/d23/d2b/d67/f8a d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/f94 0 2024-02-26T10:47:58.049 INFO:tasks.workunit.client.0.smithi060.stdout:1/498: dwrite d3/f2a [0,4194304] 0 2024-02-26T10:47:58.049 INFO:tasks.workunit.client.0.smithi060.stdout:1/499: stat d3/d35/d5e/f70 0 2024-02-26T10:47:58.051 INFO:tasks.workunit.client.0.smithi060.stdout:3/312: mknod d5/d45/c54 0 2024-02-26T10:47:58.057 INFO:tasks.workunit.client.0.smithi060.stdout:9/461: dwrite d3/d1e/d5e/d1b/f5a [0,4194304] 0 2024-02-26T10:47:58.057 INFO:tasks.workunit.client.0.smithi060.stdout:9/462: chown d3/d1e/d5e/d1b/d5f/dc/d32/l83 53068732 1 2024-02-26T10:47:58.057 INFO:tasks.workunit.client.0.smithi060.stdout:9/463: write d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f6d [68282,118267] 0 2024-02-26T10:47:58.064 INFO:tasks.workunit.client.0.smithi060.stdout:2/376: dwrite dd/d43/d60/f3f [0,4194304] 0 2024-02-26T10:47:58.064 INFO:tasks.workunit.client.0.smithi060.stdout:8/396: dread d9/d15/f36 [0,4194304] 0 2024-02-26T10:47:58.072 INFO:tasks.workunit.client.0.smithi060.stdout:9/464: truncate d3/d1e/d5e/d1b/d5f/d10/f37 1006988 0 2024-02-26T10:47:58.074 INFO:tasks.workunit.client.0.smithi060.stdout:5/390: dwrite d0/d33/f3e [0,4194304] 0 2024-02-26T10:47:58.078 INFO:tasks.workunit.client.0.smithi060.stdout:9/465: truncate d3/d1e/d5e/d1b/d23/f34 200041 0 2024-02-26T10:47:58.079 INFO:tasks.workunit.client.0.smithi060.stdout:8/397: rename f3 to d9/d15/d29/d2f/d34/d72/d50/d6a/f74 0 2024-02-26T10:47:58.081 INFO:tasks.workunit.client.0.smithi060.stdout:0/369: write d0/f3 [6137554,111296] 0 2024-02-26T10:47:58.083 INFO:tasks.workunit.client.0.smithi060.stdout:0/370: mknod d0/d42/d7a/c7b 0 2024-02-26T10:47:58.084 INFO:tasks.workunit.client.0.smithi060.stdout:0/371: creat d0/dd/d39/f7c x:0 0 0 2024-02-26T10:47:58.085 INFO:tasks.workunit.client.0.smithi060.stdout:0/372: getdents d0/d37/d44 0 2024-02-26T10:47:58.086 INFO:tasks.workunit.client.0.smithi060.stdout:4/392: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:58.086 INFO:tasks.workunit.client.0.smithi060.stdout:4/393: write d0/d7/d29/f45 [5065776,1125] 0 2024-02-26T10:47:58.086 INFO:tasks.workunit.client.0.smithi060.stdout:4/394: fdatasync d0/d7/f8 0 2024-02-26T10:47:58.089 INFO:tasks.workunit.client.0.smithi060.stdout:0/373: mknod d0/dd/d39/d40/c7d 0 2024-02-26T10:47:58.089 INFO:tasks.workunit.client.0.smithi060.stdout:4/395: creat d0/d27/d52/f71 x:0 0 0 2024-02-26T10:47:58.103 INFO:tasks.workunit.client.0.smithi060.stdout:2/377: rename f3 to dd/d52/f7a 0 2024-02-26T10:47:58.105 INFO:tasks.workunit.client.0.smithi060.stdout:2/378: unlink dd/d43/d60/d41/d37/d38/l77 0 2024-02-26T10:47:58.105 INFO:tasks.workunit.client.0.smithi060.stdout:2/379: chown dd/d43/d60/c6b 76584 1 2024-02-26T10:47:58.105 INFO:tasks.workunit.client.0.smithi060.stdout:2/380: dread - dd/d11/f65 zero size 2024-02-26T10:47:58.107 INFO:tasks.workunit.client.0.smithi060.stdout:2/381: symlink dd/d43/d60/d41/d37/d38/d57/l7b 0 2024-02-26T10:47:58.107 INFO:tasks.workunit.client.0.smithi060.stdout:2/382: write dd/f2b [1085462,34745] 0 2024-02-26T10:47:58.110 INFO:tasks.workunit.client.0.smithi060.stdout:8/398: dread d9/d15/d29/d2f/d34/d72/f4c [0,4194304] 0 2024-02-26T10:47:58.111 INFO:tasks.workunit.client.0.smithi060.stdout:8/399: creat d9/d15/d29/d2f/f75 x:0 0 0 2024-02-26T10:47:58.111 INFO:tasks.workunit.client.0.smithi060.stdout:5/391: dread d0/f14 [0,4194304] 0 2024-02-26T10:47:58.112 INFO:tasks.workunit.client.0.smithi060.stdout:8/400: link d9/d15/l6c d9/d15/l76 0 2024-02-26T10:47:58.121 INFO:tasks.workunit.client.0.smithi060.stdout:4/396: write d0/f44 [4032486,24553] 0 2024-02-26T10:47:58.126 INFO:tasks.workunit.client.0.smithi060.stdout:5/392: mkdir d0/d4b/d8c 0 2024-02-26T10:47:58.126 INFO:tasks.workunit.client.0.smithi060.stdout:5/393: fdatasync d0/d4b/f57 0 2024-02-26T10:47:58.127 INFO:tasks.workunit.client.0.smithi060.stdout:9/466: rename d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d44 to d3/d1e/d5e/d35/d87/d9b 0 2024-02-26T10:47:58.129 INFO:tasks.workunit.client.0.smithi060.stdout:3/313: dwrite d5/de/f3b [0,4194304] 0 2024-02-26T10:47:58.129 INFO:tasks.workunit.client.0.smithi060.stdout:3/314: chown d5/de/df/d11/l42 6772418 1 2024-02-26T10:47:58.129 INFO:tasks.workunit.client.0.smithi060.stdout:5/394: unlink d0/de/f22 0 2024-02-26T10:47:58.132 INFO:tasks.workunit.client.0.smithi060.stdout:4/397: dread d0/da/f4c [0,4194304] 0 2024-02-26T10:47:58.132 INFO:tasks.workunit.client.0.smithi060.stdout:4/398: fdatasync d0/da/d3b/d16/f19 0 2024-02-26T10:47:58.134 INFO:tasks.workunit.client.0.smithi060.stdout:4/399: creat d0/da/d3b/d16/f72 x:0 0 0 2024-02-26T10:47:58.134 INFO:tasks.workunit.client.0.smithi060.stdout:4/400: write d0/d27/d2f/f55 [2949406,48326] 0 2024-02-26T10:47:58.135 INFO:tasks.workunit.client.0.smithi060.stdout:4/401: read - d0/d27/d2f/f66 zero size 2024-02-26T10:47:58.135 INFO:tasks.workunit.client.0.smithi060.stdout:4/402: fsync d0/da/d3b/d16/f19 0 2024-02-26T10:47:58.137 INFO:tasks.workunit.client.0.smithi060.stdout:4/403: unlink d0/da/d3b/d16/f48 0 2024-02-26T10:47:58.139 INFO:tasks.workunit.client.0.smithi060.stdout:9/467: truncate d3/d1e/d57/f79 34458 0 2024-02-26T10:47:58.148 INFO:tasks.workunit.client.0.smithi060.stdout:4/404: link d0/da/d3b/c1f d0/d27/c73 0 2024-02-26T10:47:58.148 INFO:tasks.workunit.client.0.smithi060.stdout:4/405: write d0/f4b [943946,87685] 0 2024-02-26T10:47:58.149 INFO:tasks.workunit.client.0.smithi060.stdout:4/406: creat d0/f74 x:0 0 0 2024-02-26T10:47:58.149 INFO:tasks.workunit.client.0.smithi060.stdout:4/407: dread - d0/da/d3b/f59 zero size 2024-02-26T10:47:58.156 INFO:tasks.workunit.client.0.smithi060.stdout:4/408: link d0/da/d3b/d16/f72 d0/da/d57/f75 0 2024-02-26T10:47:58.157 INFO:tasks.workunit.client.0.smithi060.stdout:4/409: mkdir d0/d27/d76 0 2024-02-26T10:47:58.158 INFO:tasks.workunit.client.0.smithi060.stdout:4/410: creat d0/da/d57/f77 x:0 0 0 2024-02-26T10:47:58.160 INFO:tasks.workunit.client.0.smithi060.stdout:4/411: creat d0/d7/f78 x:0 0 0 2024-02-26T10:47:58.160 INFO:tasks.workunit.client.0.smithi060.stdout:4/412: chown d0/d7 2249791 1 2024-02-26T10:47:58.164 INFO:tasks.workunit.client.0.smithi060.stdout:6/449: dread d6/f9 [0,4194304] 0 2024-02-26T10:47:58.164 INFO:tasks.workunit.client.0.smithi060.stdout:7/512: dwrite da/d15/d21/d2f/d78/f96 [0,4194304] 0 2024-02-26T10:47:58.167 INFO:tasks.workunit.client.0.smithi060.stdout:4/413: dread d0/d27/d2f/f35 [0,4194304] 0 2024-02-26T10:47:58.174 INFO:tasks.workunit.client.0.smithi060.stdout:6/450: mknod d6/d7/d25/c95 0 2024-02-26T10:47:58.174 INFO:tasks.workunit.client.0.smithi060.stdout:8/401: dwrite d9/d15/d29/d2f/d34/d72/f62 [0,4194304] 0 2024-02-26T10:47:58.174 INFO:tasks.workunit.client.0.smithi060.stdout:1/500: dread d3/d35/d53/d68/d85/d8a/f9a [0,4194304] 0 2024-02-26T10:47:58.177 INFO:tasks.workunit.client.0.smithi060.stdout:1/501: symlink d3/db/d1e/d38/d6c/lb1 0 2024-02-26T10:47:58.177 INFO:tasks.workunit.client.0.smithi060.stdout:1/502: mknod d3/db/d1e/d38/cb2 0 2024-02-26T10:47:58.190 INFO:tasks.workunit.client.0.smithi060.stdout:1/503: dread d3/d35/d53/d68/d6b/f9e [0,4194304] 0 2024-02-26T10:47:58.192 INFO:tasks.workunit.client.0.smithi060.stdout:3/315: dwrite d5/d37/f38 [0,4194304] 0 2024-02-26T10:47:58.192 INFO:tasks.workunit.client.0.smithi060.stdout:3/316: fdatasync d5/de/df/f21 0 2024-02-26T10:47:58.192 INFO:tasks.workunit.client.0.smithi060.stdout:3/317: fdatasync d5/de/d16/f2d 0 2024-02-26T10:47:58.192 INFO:tasks.workunit.client.0.smithi060.stdout:3/318: dread - d5/d45/f53 zero size 2024-02-26T10:47:58.196 INFO:tasks.workunit.client.0.smithi060.stdout:3/319: creat d5/de/df/d11/d47/f55 x:0 0 0 2024-02-26T10:47:58.203 INFO:tasks.workunit.client.0.smithi060.stdout:5/395: dread d0/de/d15/d26/f45 [0,4194304] 0 2024-02-26T10:47:58.203 INFO:tasks.workunit.client.0.smithi060.stdout:5/396: dread - d0/de/f46 zero size 2024-02-26T10:47:58.208 INFO:tasks.workunit.client.0.smithi060.stdout:0/374: dwrite d0/f78 [0,4194304] 0 2024-02-26T10:47:58.211 INFO:tasks.workunit.client.0.smithi060.stdout:8/402: rename d9/d15/d29/c2b to d9/d5a/c77 0 2024-02-26T10:47:58.214 INFO:tasks.workunit.client.0.smithi060.stdout:7/513: dwrite da/d15/d21/f2c [0,4194304] 0 2024-02-26T10:47:58.214 INFO:tasks.workunit.client.0.smithi060.stdout:3/320: truncate d5/d37/f38 2204725 0 2024-02-26T10:47:58.216 INFO:tasks.workunit.client.0.smithi060.stdout:7/514: unlink da/d15/d21/d4b/d67/f7a 0 2024-02-26T10:47:58.217 INFO:tasks.workunit.client.0.smithi060.stdout:3/321: creat d5/d37/f56 x:0 0 0 2024-02-26T10:47:58.217 INFO:tasks.workunit.client.0.smithi060.stdout:7/515: readlink da/l36 0 2024-02-26T10:47:58.220 INFO:tasks.workunit.client.0.smithi060.stdout:7/516: rename da/d15/d88/d39 to da/d66/dad 0 2024-02-26T10:47:58.220 INFO:tasks.workunit.client.0.smithi060.stdout:7/517: chown da/d15/d21/d2f/d78/fac 25 1 2024-02-26T10:47:58.221 INFO:tasks.workunit.client.0.smithi060.stdout:1/504: write d3/db/d1e/f3b [86618,13194] 0 2024-02-26T10:47:58.221 INFO:tasks.workunit.client.0.smithi060.stdout:1/505: dread - d3/db/d2e/d51/d63/f92 zero size 2024-02-26T10:47:58.222 INFO:tasks.workunit.client.0.smithi060.stdout:7/518: rename da/d15/d88/d3b/c89 to da/d15/d88/d3b/cae 0 2024-02-26T10:47:58.223 INFO:tasks.workunit.client.0.smithi060.stdout:1/506: truncate d3/f9 1065315 0 2024-02-26T10:47:58.224 INFO:tasks.workunit.client.0.smithi060.stdout:7/519: creat da/d15/d21/d32/faf x:0 0 0 2024-02-26T10:47:58.224 INFO:tasks.workunit.client.0.smithi060.stdout:7/520: write da/d15/d21/f79 [1084981,98238] 0 2024-02-26T10:47:58.228 INFO:tasks.workunit.client.0.smithi060.stdout:7/521: mknod da/d15/d21/d2f/cb0 0 2024-02-26T10:47:58.230 INFO:tasks.workunit.client.0.smithi060.stdout:1/507: unlink d3/db/d1e/c47 0 2024-02-26T10:47:58.230 INFO:tasks.workunit.client.0.smithi060.stdout:7/522: unlink da/d15/d21/d4b/fa5 0 2024-02-26T10:47:58.231 INFO:tasks.workunit.client.0.smithi060.stdout:1/508: mknod d3/db/d1e/d39/d9b/da6/cb3 0 2024-02-26T10:47:58.231 INFO:tasks.workunit.client.0.smithi060.stdout:7/523: readlink da/d15/l42 0 2024-02-26T10:47:58.233 INFO:tasks.workunit.client.0.smithi060.stdout:1/509: creat d3/d35/d53/d68/d85/d8a/d91/fb4 x:0 0 0 2024-02-26T10:47:58.233 INFO:tasks.workunit.client.0.smithi060.stdout:6/451: dwrite d6/d7/d47/d69/f82 [0,4194304] 0 2024-02-26T10:47:58.234 INFO:tasks.workunit.client.0.smithi060.stdout:7/524: rename da/d15/d21/d41/d58/c6b to da/d94/cb1 0 2024-02-26T10:47:58.237 INFO:tasks.workunit.client.0.smithi060.stdout:6/452: link d6/d7/d47/f72 d6/d6e/f96 0 2024-02-26T10:47:58.239 INFO:tasks.workunit.client.0.smithi060.stdout:6/453: truncate f4 2464811 0 2024-02-26T10:47:58.239 INFO:tasks.workunit.client.0.smithi060.stdout:6/454: stat d6/d7/l22 0 2024-02-26T10:47:58.241 INFO:tasks.workunit.client.0.smithi060.stdout:6/455: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/d97 0 2024-02-26T10:47:58.243 INFO:tasks.workunit.client.0.smithi060.stdout:0/375: dread d0/f32 [0,4194304] 0 2024-02-26T10:47:58.244 INFO:tasks.workunit.client.0.smithi060.stdout:6/456: mkdir d6/d7/d25/d1c/d39/d98 0 2024-02-26T10:47:58.246 INFO:tasks.workunit.client.0.smithi060.stdout:0/376: rename d0/d42/f62 to d0/dd/d39/f7e 0 2024-02-26T10:47:58.246 INFO:tasks.workunit.client.0.smithi060.stdout:6/457: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f99 x:0 0 0 2024-02-26T10:47:58.246 INFO:tasks.workunit.client.0.smithi060.stdout:6/458: dread - d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f99 zero size 2024-02-26T10:47:58.247 INFO:tasks.workunit.client.0.smithi060.stdout:6/459: dread - d6/d6e/f96 zero size 2024-02-26T10:47:58.247 INFO:tasks.workunit.client.0.smithi060.stdout:0/377: mkdir d0/d42/d7a/d7f 0 2024-02-26T10:47:58.248 INFO:tasks.workunit.client.0.smithi060.stdout:6/460: mknod d6/d7/d25/d1c/d1f/d23/d7a/c9a 0 2024-02-26T10:47:58.254 INFO:tasks.workunit.client.0.smithi060.stdout:7/525: dread da/d15/d21/f28 [0,4194304] 0 2024-02-26T10:47:58.262 INFO:tasks.workunit.client.0.smithi060.stdout:6/461: dread d6/d7/d25/d1c/d1f/d23/d28/f4c [0,4194304] 0 2024-02-26T10:47:58.280 INFO:tasks.workunit.client.0.smithi060.stdout:1/510: dwrite d3/d35/f65 [0,4194304] 0 2024-02-26T10:47:58.282 INFO:tasks.workunit.client.0.smithi060.stdout:1/511: rename d3/d54/la1 to d3/lb5 0 2024-02-26T10:47:58.293 INFO:tasks.workunit.client.0.smithi060.stdout:5/397: dwrite d0/d4b/f51 [0,4194304] 0 2024-02-26T10:47:58.298 INFO:tasks.workunit.client.0.smithi060.stdout:5/398: rename d0/de/d15/d26/l61 to d0/de/d20/l8d 0 2024-02-26T10:47:58.302 INFO:tasks.workunit.client.0.smithi060.stdout:5/399: rename d0/l4a to d0/de/d70/d84/l8e 0 2024-02-26T10:47:58.304 INFO:tasks.workunit.client.0.smithi060.stdout:5/400: mknod d0/d4b/d8c/c8f 0 2024-02-26T10:47:58.304 INFO:tasks.workunit.client.0.smithi060.stdout:5/401: truncate d0/d4b/f57 877538 0 2024-02-26T10:47:58.307 INFO:tasks.workunit.client.0.smithi060.stdout:5/402: rename d0/de/d15/f7f to d0/f90 0 2024-02-26T10:47:58.310 INFO:tasks.workunit.client.0.smithi060.stdout:5/403: symlink d0/de/d20/d82/l91 0 2024-02-26T10:47:58.310 INFO:tasks.workunit.client.0.smithi060.stdout:5/404: write d0/de/d20/d82/f87 [403636,32941] 0 2024-02-26T10:47:58.317 INFO:tasks.workunit.client.0.smithi060.stdout:5/405: rename d0/d4b/d8c to d0/de/d70/d84/d92 0 2024-02-26T10:47:58.317 INFO:tasks.workunit.client.0.smithi060.stdout:5/406: chown d0/de/f5d 30 1 2024-02-26T10:47:58.318 INFO:tasks.workunit.client.0.smithi060.stdout:5/407: chown d0/de/d15/d26/l67 303213 1 2024-02-26T10:47:58.321 INFO:tasks.workunit.client.0.smithi060.stdout:5/408: symlink d0/de/d15/d65/l93 0 2024-02-26T10:47:58.321 INFO:tasks.workunit.client.0.smithi060.stdout:3/322: dwrite d5/fd [4194304,4194304] 0 2024-02-26T10:47:58.321 INFO:tasks.workunit.client.0.smithi060.stdout:3/323: write d5/d37/f41 [578327,52035] 0 2024-02-26T10:47:58.323 INFO:tasks.workunit.client.0.smithi060.stdout:0/378: dwrite d0/f15 [0,4194304] 0 2024-02-26T10:47:58.323 INFO:tasks.workunit.client.0.smithi060.stdout:0/379: write d0/f55 [915122,33958] 0 2024-02-26T10:47:58.323 INFO:tasks.workunit.client.0.smithi060.stdout:0/380: fdatasync d0/dd/f2d 0 2024-02-26T10:47:58.323 INFO:tasks.workunit.client.0.smithi060.stdout:3/324: mkdir d5/de/df/d57 0 2024-02-26T10:47:58.323 INFO:tasks.workunit.client.0.smithi060.stdout:3/325: chown l1 78 1 2024-02-26T10:47:58.324 INFO:tasks.workunit.client.0.smithi060.stdout:8/403: dwrite d9/d15/f36 [0,4194304] 0 2024-02-26T10:47:58.324 INFO:tasks.workunit.client.0.smithi060.stdout:7/526: dwrite f6 [0,4194304] 0 2024-02-26T10:47:58.324 INFO:tasks.workunit.client.0.smithi060.stdout:7/527: stat da 0 2024-02-26T10:47:58.324 INFO:tasks.workunit.client.0.smithi060.stdout:8/404: write d9/d5a/f69 [1105462,51318] 0 2024-02-26T10:47:58.327 INFO:tasks.workunit.client.0.smithi060.stdout:8/405: creat d9/d5a/d58/f78 x:0 0 0 2024-02-26T10:47:58.328 INFO:tasks.workunit.client.0.smithi060.stdout:8/406: mknod d9/d6f/c79 0 2024-02-26T10:47:58.330 INFO:tasks.workunit.client.0.smithi060.stdout:6/462: dwrite d6/fe [0,4194304] 0 2024-02-26T10:47:58.331 INFO:tasks.workunit.client.0.smithi060.stdout:8/407: rmdir d9/d15/d29 39 2024-02-26T10:47:58.332 INFO:tasks.workunit.client.0.smithi060.stdout:7/528: creat da/d15/d21/fb2 x:0 0 0 2024-02-26T10:47:58.334 INFO:tasks.workunit.client.0.smithi060.stdout:6/463: creat d6/d7/d47/f9b x:0 0 0 2024-02-26T10:47:58.335 INFO:tasks.workunit.client.0.smithi060.stdout:7/529: getdents da/d94 0 2024-02-26T10:47:58.336 INFO:tasks.workunit.client.0.smithi060.stdout:6/464: creat d6/d7/d25/d1c/d1f/d23/f9c x:0 0 0 2024-02-26T10:47:58.337 INFO:tasks.workunit.client.0.smithi060.stdout:6/465: write d6/f11 [4289186,77789] 0 2024-02-26T10:47:58.340 INFO:tasks.workunit.client.0.smithi060.stdout:7/530: rename da/d15/d88/d3b/la3 to da/d66/dad/lb3 0 2024-02-26T10:47:58.341 INFO:tasks.workunit.client.0.smithi060.stdout:8/408: creat d9/d5a/d58/f7a x:0 0 0 2024-02-26T10:47:58.346 INFO:tasks.workunit.client.0.smithi060.stdout:3/326: read d5/de/df/d11/f20 [2460375,130362] 0 2024-02-26T10:47:58.346 INFO:tasks.workunit.client.0.smithi060.stdout:4/414: sync 2024-02-26T10:47:58.346 INFO:tasks.workunit.client.0.smithi060.stdout:6/466: rmdir d6/d7/d25/d1c/d1f/d23 39 2024-02-26T10:47:58.346 INFO:tasks.workunit.client.0.smithi060.stdout:6/467: fsync d6/d7/d25/d1c/d39/f44 0 2024-02-26T10:47:58.346 INFO:tasks.workunit.client.0.smithi060.stdout:6/468: truncate d6/f8e 582453 0 2024-02-26T10:47:58.346 INFO:tasks.workunit.client.0.smithi060.stdout:6/469: fdatasync d6/d7/d25/f4b 0 2024-02-26T10:47:58.347 INFO:tasks.workunit.client.0.smithi060.stdout:7/531: truncate da/fd 7246604 0 2024-02-26T10:47:58.347 INFO:tasks.workunit.client.0.smithi060.stdout:0/381: dread d0/dd/d39/d40/f17 [0,4194304] 0 2024-02-26T10:47:58.347 INFO:tasks.workunit.client.0.smithi060.stdout:2/383: sync 2024-02-26T10:47:58.347 INFO:tasks.workunit.client.0.smithi060.stdout:9/468: sync 2024-02-26T10:47:58.348 INFO:tasks.workunit.client.0.smithi060.stdout:9/469: getdents d3/d1e/d5e/d1b/d5f/d10/d98 0 2024-02-26T10:47:58.348 INFO:tasks.workunit.client.0.smithi060.stdout:0/382: link d0/c47 d0/dd/d33/c80 0 2024-02-26T10:47:58.350 INFO:tasks.workunit.client.0.smithi060.stdout:0/383: symlink d0/d37/l81 0 2024-02-26T10:47:58.350 INFO:tasks.workunit.client.0.smithi060.stdout:0/384: write d0/dd/d39/f7e [5126782,114416] 0 2024-02-26T10:47:58.350 INFO:tasks.workunit.client.0.smithi060.stdout:2/384: link dd/d43/d60/d41/d37/d38/l4b dd/d11/l7c 0 2024-02-26T10:47:58.352 INFO:tasks.workunit.client.0.smithi060.stdout:2/385: mknod dd/d43/d60/d41/d37/d4c/d76/c7d 0 2024-02-26T10:47:58.352 INFO:tasks.workunit.client.0.smithi060.stdout:0/385: link d0/dd/d39/d40/f1f d0/f82 0 2024-02-26T10:47:58.352 INFO:tasks.workunit.client.0.smithi060.stdout:2/386: chown dd/d11/c16 0 1 2024-02-26T10:47:58.353 INFO:tasks.workunit.client.0.smithi060.stdout:0/386: mkdir d0/dd/d39/d40/d83 0 2024-02-26T10:47:58.354 INFO:tasks.workunit.client.0.smithi060.stdout:2/387: creat dd/d43/d60/d41/d37/d69/f7e x:0 0 0 2024-02-26T10:47:58.355 INFO:tasks.workunit.client.0.smithi060.stdout:2/388: mknod dd/d43/d60/d41/c7f 0 2024-02-26T10:47:58.355 INFO:tasks.workunit.client.0.smithi060.stdout:2/389: write dd/d11/f44 [1006783,112624] 0 2024-02-26T10:47:58.357 INFO:tasks.workunit.client.0.smithi060.stdout:2/390: creat dd/d43/d60/d41/d37/f80 x:0 0 0 2024-02-26T10:47:58.358 INFO:tasks.workunit.client.0.smithi060.stdout:0/387: symlink d0/dd/d53/l84 0 2024-02-26T10:47:58.358 INFO:tasks.workunit.client.0.smithi060.stdout:0/388: fdatasync d0/d37/f6d 0 2024-02-26T10:47:58.358 INFO:tasks.workunit.client.0.smithi060.stdout:0/389: stat d0/dd/d39/d40/f17 0 2024-02-26T10:47:58.359 INFO:tasks.workunit.client.0.smithi060.stdout:0/390: truncate d0/f15 5057650 0 2024-02-26T10:47:58.359 INFO:tasks.workunit.client.0.smithi060.stdout:0/391: fsync d0/f15 0 2024-02-26T10:47:58.364 INFO:tasks.workunit.client.0.smithi060.stdout:0/392: symlink d0/d37/d44/d58/l85 0 2024-02-26T10:47:58.377 INFO:tasks.workunit.client.0.smithi060.stdout:6/470: link d6/d7/d2e/c57 d6/d7/d25/d1c/d39/c9d 0 2024-02-26T10:47:58.401 INFO:tasks.workunit.client.0.smithi060.stdout:3/327: dwrite d5/f27 [4194304,4194304] 0 2024-02-26T10:47:58.404 INFO:tasks.workunit.client.0.smithi060.stdout:3/328: rename d5/d45/f53 to d5/de/f58 0 2024-02-26T10:47:58.408 INFO:tasks.workunit.client.0.smithi060.stdout:3/329: truncate d5/de/f3b 2720472 0 2024-02-26T10:47:58.411 INFO:tasks.workunit.client.0.smithi060.stdout:3/330: chown d5/de/df/l3f 3757081 1 2024-02-26T10:47:58.411 INFO:tasks.workunit.client.0.smithi060.stdout:3/331: readlink d5/de/df/d11/l13 0 2024-02-26T10:47:58.414 INFO:tasks.workunit.client.0.smithi060.stdout:3/332: link d5/de/df/d11/c22 d5/de/d16/d17/c59 0 2024-02-26T10:47:58.417 INFO:tasks.workunit.client.0.smithi060.stdout:4/415: dwrite d0/ff [8388608,4194304] 0 2024-02-26T10:47:58.418 INFO:tasks.workunit.client.0.smithi060.stdout:3/333: creat d5/de/df/d11/d47/f5a x:0 0 0 2024-02-26T10:47:58.420 INFO:tasks.workunit.client.0.smithi060.stdout:9/470: dwrite d3/d1e/d5e/d1b/d23/f34 [0,4194304] 0 2024-02-26T10:47:58.420 INFO:tasks.workunit.client.0.smithi060.stdout:9/471: dread - d3/d1e/d57/f88 zero size 2024-02-26T10:47:58.423 INFO:tasks.workunit.client.0.smithi060.stdout:5/409: dwrite d0/fd [0,4194304] 0 2024-02-26T10:47:58.426 INFO:tasks.workunit.client.0.smithi060.stdout:9/472: mkdir d3/d1e/d57/d9c 0 2024-02-26T10:47:58.426 INFO:tasks.workunit.client.0.smithi060.stdout:5/410: rename d0/de/d15/d26/l67 to d0/de/d15/l94 0 2024-02-26T10:47:58.426 INFO:tasks.workunit.client.0.smithi060.stdout:3/334: rmdir d5/de/d16/d17 39 2024-02-26T10:47:58.433 INFO:tasks.workunit.client.0.smithi060.stdout:3/335: creat d5/de/df/f5b x:0 0 0 2024-02-26T10:47:58.434 INFO:tasks.workunit.client.0.smithi060.stdout:6/471: dwrite d6/f1d [0,4194304] 0 2024-02-26T10:47:58.438 INFO:tasks.workunit.client.0.smithi060.stdout:9/473: chown d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/c53 98900 1 2024-02-26T10:47:58.438 INFO:tasks.workunit.client.0.smithi060.stdout:6/472: rename d6/d7/d25/d1c/d1f/f24 to d6/d7/d25/d1c/f9e 0 2024-02-26T10:47:58.439 INFO:tasks.workunit.client.0.smithi060.stdout:6/473: write d6/d7/d47/f72 [808842,43060] 0 2024-02-26T10:47:58.442 INFO:tasks.workunit.client.0.smithi060.stdout:8/409: dwrite d9/d5a/d58/f7a [0,4194304] 0 2024-02-26T10:47:58.442 INFO:tasks.workunit.client.0.smithi060.stdout:8/410: chown d9/c5d 381621776 1 2024-02-26T10:47:58.442 INFO:tasks.workunit.client.0.smithi060.stdout:8/411: write d9/d5a/f6e [787860,16312] 0 2024-02-26T10:47:58.447 INFO:tasks.workunit.client.0.smithi060.stdout:4/416: link d0/da/d3b/c11 d0/d7/d29/c79 0 2024-02-26T10:47:58.448 INFO:tasks.workunit.client.0.smithi060.stdout:7/532: dwrite da/d15/f46 [0,4194304] 0 2024-02-26T10:47:58.449 INFO:tasks.workunit.client.0.smithi060.stdout:4/417: symlink d0/d27/d76/l7a 0 2024-02-26T10:47:58.450 INFO:tasks.workunit.client.0.smithi060.stdout:7/533: getdents da/d15/d21/d2f/d37 0 2024-02-26T10:47:58.452 INFO:tasks.workunit.client.0.smithi060.stdout:7/534: rename da/c87 to da/d15/d88/cb4 0 2024-02-26T10:47:58.469 INFO:tasks.workunit.client.0.smithi060.stdout:5/411: dread d0/de/d20/d82/f87 [0,4194304] 0 2024-02-26T10:47:58.469 INFO:tasks.workunit.client.0.smithi060.stdout:5/412: readlink d0/d4b/l73 0 2024-02-26T10:47:58.470 INFO:tasks.workunit.client.0.smithi060.stdout:6/474: write d6/d7/d25/d1c/d1f/d23/d2b/f3b [664169,55429] 0 2024-02-26T10:47:58.473 INFO:tasks.workunit.client.0.smithi060.stdout:3/336: dread d5/de/df/d11/f1c [0,4194304] 0 2024-02-26T10:47:58.480 INFO:tasks.workunit.client.0.smithi060.stdout:4/418: dread d0/f5 [0,4194304] 0 2024-02-26T10:47:58.481 INFO:tasks.workunit.client.0.smithi060.stdout:9/474: dread d3/d1e/d5e/d1b/d5f/f5b [4194304,4194304] 0 2024-02-26T10:47:58.484 INFO:tasks.workunit.client.0.smithi060.stdout:6/475: rename d6/d7/d25/d1c/d1f/d23/l5d to d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/l9f 0 2024-02-26T10:47:58.484 INFO:tasks.workunit.client.0.smithi060.stdout:6/476: fdatasync d6/d7/ff 0 2024-02-26T10:47:58.484 INFO:tasks.workunit.client.0.smithi060.stdout:6/477: fsync d6/d7/d25/d1c/d1f/d23/d28/f33 0 2024-02-26T10:47:58.485 INFO:tasks.workunit.client.0.smithi060.stdout:5/413: dread d0/de/d20/f31 [0,4194304] 0 2024-02-26T10:47:58.485 INFO:tasks.workunit.client.0.smithi060.stdout:6/478: mkdir d6/d7/d25/d1c/d1f/d23/d7a/da0 0 2024-02-26T10:47:58.485 INFO:tasks.workunit.client.0.smithi060.stdout:6/479: chown d6/l5a 6559 1 2024-02-26T10:47:58.486 INFO:tasks.workunit.client.0.smithi060.stdout:6/480: creat d6/d6e/d6f/fa1 x:0 0 0 2024-02-26T10:47:58.487 INFO:tasks.workunit.client.0.smithi060.stdout:5/414: getdents d0/d4b 0 2024-02-26T10:47:58.488 INFO:tasks.workunit.client.0.smithi060.stdout:6/481: symlink d6/d7/d25/d1c/d1f/d23/d2b/la2 0 2024-02-26T10:47:58.488 INFO:tasks.workunit.client.0.smithi060.stdout:0/393: dwrite d0/dd/f4e [0,4194304] 0 2024-02-26T10:47:58.488 INFO:tasks.workunit.client.0.smithi060.stdout:0/394: chown d0/c57 36759385 1 2024-02-26T10:47:58.489 INFO:tasks.workunit.client.0.smithi060.stdout:5/415: rename d0/de/d15/l1b to d0/d4b/d7b/l95 0 2024-02-26T10:47:58.490 INFO:tasks.workunit.client.0.smithi060.stdout:6/482: creat d6/d7/d25/d1c/d1f/d23/d28/fa3 x:0 0 0 2024-02-26T10:47:58.490 INFO:tasks.workunit.client.0.smithi060.stdout:6/483: chown d6/d7/d25/d1c/d39/c45 55224 1 2024-02-26T10:47:58.491 INFO:tasks.workunit.client.0.smithi060.stdout:0/395: link d0/dd/d33/c80 d0/d42/c86 0 2024-02-26T10:47:58.491 INFO:tasks.workunit.client.0.smithi060.stdout:5/416: symlink d0/l96 0 2024-02-26T10:47:58.493 INFO:tasks.workunit.client.0.smithi060.stdout:0/396: mkdir d0/dd/d53/d87 0 2024-02-26T10:47:58.496 INFO:tasks.workunit.client.0.smithi060.stdout:0/397: creat d0/dd/d33/f88 x:0 0 0 2024-02-26T10:47:58.497 INFO:tasks.workunit.client.0.smithi060.stdout:5/417: mkdir d0/de/d20/d32/d4d/d97 0 2024-02-26T10:47:58.497 INFO:tasks.workunit.client.0.smithi060.stdout:5/418: getdents d0/de/d15/d65/d86 0 2024-02-26T10:47:58.497 INFO:tasks.workunit.client.0.smithi060.stdout:5/419: readlink d0/de/d20/l2a 0 2024-02-26T10:47:58.515 INFO:tasks.workunit.client.0.smithi060.stdout:7/535: dwrite da/d94/f98 [0,4194304] 0 2024-02-26T10:47:58.516 INFO:tasks.workunit.client.0.smithi060.stdout:4/419: dread d0/d7/f25 [0,4194304] 0 2024-02-26T10:47:58.518 INFO:tasks.workunit.client.0.smithi060.stdout:4/420: creat d0/f7b x:0 0 0 2024-02-26T10:47:58.518 INFO:tasks.workunit.client.0.smithi060.stdout:4/421: read d0/d27/d2f/f6e [7779,109161] 0 2024-02-26T10:47:58.518 INFO:tasks.workunit.client.0.smithi060.stdout:4/422: chown d0/da/d57/f65 26 1 2024-02-26T10:47:58.519 INFO:tasks.workunit.client.0.smithi060.stdout:4/423: creat d0/da/f7c x:0 0 0 2024-02-26T10:47:58.524 INFO:tasks.workunit.client.0.smithi060.stdout:9/475: dread d3/d1e/d5e/d1b/f56 [0,4194304] 0 2024-02-26T10:47:58.524 INFO:tasks.workunit.client.0.smithi060.stdout:8/412: dwrite d9/d15/d29/d2f/f75 [0,4194304] 0 2024-02-26T10:47:58.528 INFO:tasks.workunit.client.0.smithi060.stdout:6/484: rename d6/d7/d25/d1c/d1f/d23/f4a to d6/d7/d2e/d56/d73/fa4 0 2024-02-26T10:47:58.529 INFO:tasks.workunit.client.0.smithi060.stdout:9/476: creat d3/d1e/d5e/d1b/f9d x:0 0 0 2024-02-26T10:47:58.529 INFO:tasks.workunit.client.0.smithi060.stdout:9/477: chown d3/l1c 7 1 2024-02-26T10:47:58.529 INFO:tasks.workunit.client.0.smithi060.stdout:9/478: read - d3/d1e/d5e/d1b/d5f/dc/d32/d3c/f63 zero size 2024-02-26T10:47:58.529 INFO:tasks.workunit.client.0.smithi060.stdout:9/479: stat d3/d1e/d5e/d1b/d5f/dc/d32/c94 0 2024-02-26T10:47:58.529 INFO:tasks.workunit.client.0.smithi060.stdout:9/480: fsync d3/d1e/d5e/d35/f3e 0 2024-02-26T10:47:58.529 INFO:tasks.workunit.client.0.smithi060.stdout:9/481: write d3/d1e/f58 [1122776,39148] 0 2024-02-26T10:47:58.533 INFO:tasks.workunit.client.0.smithi060.stdout:2/391: sync 2024-02-26T10:47:58.533 INFO:tasks.workunit.client.0.smithi060.stdout:1/512: sync 2024-02-26T10:47:58.533 INFO:tasks.workunit.client.0.smithi060.stdout:2/392: chown lb 9738 1 2024-02-26T10:47:58.540 INFO:tasks.workunit.client.0.smithi060.stdout:0/398: dread d0/d37/d44/f79 [0,4194304] 0 2024-02-26T10:47:58.542 INFO:tasks.workunit.client.0.smithi060.stdout:3/337: dwrite d5/d45/f49 [0,4194304] 0 2024-02-26T10:47:58.545 INFO:tasks.workunit.client.0.smithi060.stdout:7/536: rename da/d66/f90 to da/d94/fb5 0 2024-02-26T10:47:58.545 INFO:tasks.workunit.client.0.smithi060.stdout:7/537: fdatasync da/fd 0 2024-02-26T10:47:58.547 INFO:tasks.workunit.client.0.smithi060.stdout:5/420: dread d0/de/d15/d26/f38 [0,4194304] 0 2024-02-26T10:47:58.547 INFO:tasks.workunit.client.0.smithi060.stdout:1/513: rmdir d3/db/d2e 39 2024-02-26T10:47:58.549 INFO:tasks.workunit.client.0.smithi060.stdout:4/424: dread d0/da/f14 [0,4194304] 0 2024-02-26T10:47:58.550 INFO:tasks.workunit.client.0.smithi060.stdout:5/421: symlink d0/de/d15/d26/l98 0 2024-02-26T10:47:58.551 INFO:tasks.workunit.client.0.smithi060.stdout:7/538: creat da/d15/d21/d41/d58/fb6 x:0 0 0 2024-02-26T10:47:58.553 INFO:tasks.workunit.client.0.smithi060.stdout:8/413: dread d9/d5a/d58/f7a [0,4194304] 0 2024-02-26T10:47:58.554 INFO:tasks.workunit.client.0.smithi060.stdout:8/414: link d9/d15/d29/d2f/d34/d72/f62 d9/d15/d29/d2f/d34/d72/d50/f7b 0 2024-02-26T10:47:58.555 INFO:tasks.workunit.client.0.smithi060.stdout:5/422: symlink d0/de/d20/d32/d4d/d97/l99 0 2024-02-26T10:47:58.555 INFO:tasks.workunit.client.0.smithi060.stdout:7/539: write da/d15/f53 [226881,75150] 0 2024-02-26T10:47:58.555 INFO:tasks.workunit.client.0.smithi060.stdout:7/540: chown da/d66/dad/lb3 38675 1 2024-02-26T10:47:58.556 INFO:tasks.workunit.client.0.smithi060.stdout:3/338: write d5/de/d16/d17/f1e [640316,43906] 0 2024-02-26T10:47:58.563 INFO:tasks.workunit.client.0.smithi060.stdout:8/415: unlink d9/d15/f1c 0 2024-02-26T10:47:58.563 INFO:tasks.workunit.client.0.smithi060.stdout:3/339: mknod d5/de/df/d4c/c5c 0 2024-02-26T10:47:58.564 INFO:tasks.workunit.client.0.smithi060.stdout:9/482: truncate d3/d1e/d5e/d1b/f56 3069726 0 2024-02-26T10:47:58.568 INFO:tasks.workunit.client.0.smithi060.stdout:0/399: write d0/d37/d44/f79 [4070085,117115] 0 2024-02-26T10:47:58.570 INFO:tasks.workunit.client.0.smithi060.stdout:0/400: mkdir d0/dd/d39/d40/d83/d89 0 2024-02-26T10:47:58.570 INFO:tasks.workunit.client.0.smithi060.stdout:3/340: mknod d5/de/df/c5d 0 2024-02-26T10:47:58.571 INFO:tasks.workunit.client.0.smithi060.stdout:0/401: truncate d0/dd/d39/d40/f26 600780 0 2024-02-26T10:47:58.571 INFO:tasks.workunit.client.0.smithi060.stdout:3/341: write d5/fa [5035931,29421] 0 2024-02-26T10:47:58.571 INFO:tasks.workunit.client.0.smithi060.stdout:1/514: write d3/db/d1e/d39/f80 [2725790,127463] 0 2024-02-26T10:47:58.574 INFO:tasks.workunit.client.0.smithi060.stdout:1/515: creat d3/d35/d53/d68/dad/fb6 x:0 0 0 2024-02-26T10:47:58.581 INFO:tasks.workunit.client.0.smithi060.stdout:1/516: creat d3/db/d1e/fb7 x:0 0 0 2024-02-26T10:47:58.582 INFO:tasks.workunit.client.0.smithi060.stdout:7/541: write da/d15/d88/f51 [827413,34971] 0 2024-02-26T10:47:58.593 INFO:tasks.workunit.client.0.smithi060.stdout:0/402: read d0/d37/d44/d58/f5b [374714,30126] 0 2024-02-26T10:47:58.593 INFO:tasks.workunit.client.0.smithi060.stdout:7/542: rename da/d15/l4c to da/d15/d88/lb7 0 2024-02-26T10:47:58.593 INFO:tasks.workunit.client.0.smithi060.stdout:7/543: chown da/d66/dad/lb3 89861047 1 2024-02-26T10:47:58.602 INFO:tasks.workunit.client.0.smithi060.stdout:4/425: dwrite d0/d27/d52/f62 [0,4194304] 0 2024-02-26T10:47:58.608 INFO:tasks.workunit.client.0.smithi060.stdout:6/485: dwrite d6/d7/d2e/d56/d73/fa4 [0,4194304] 0 2024-02-26T10:47:58.608 INFO:tasks.workunit.client.0.smithi060.stdout:0/403: dread d0/dd/d39/d40/f26 [0,4194304] 0 2024-02-26T10:47:58.608 INFO:tasks.workunit.client.0.smithi060.stdout:0/404: dread - d0/dd/d53/f64 zero size 2024-02-26T10:47:58.608 INFO:tasks.workunit.client.0.smithi060.stdout:3/342: write d5/de/df/d11/f1c [3195272,88571] 0 2024-02-26T10:47:58.608 INFO:tasks.workunit.client.0.smithi060.stdout:3/343: chown d5/de/d16/d17/f30 261136 1 2024-02-26T10:47:58.616 INFO:tasks.workunit.client.0.smithi060.stdout:6/486: creat d6/d7/d25/d1c/d39/fa5 x:0 0 0 2024-02-26T10:47:58.620 INFO:tasks.workunit.client.0.smithi060.stdout:3/344: truncate d5/de/df/d11/f1c 3183727 0 2024-02-26T10:47:58.620 INFO:tasks.workunit.client.0.smithi060.stdout:7/544: write da/d15/d88/f5b [2181013,94427] 0 2024-02-26T10:47:58.621 INFO:tasks.workunit.client.0.smithi060.stdout:3/345: mkdir d5/de/df/d57/d5e 0 2024-02-26T10:47:58.621 INFO:tasks.workunit.client.0.smithi060.stdout:7/545: mknod da/d15/d21/d41/d6c/cb8 0 2024-02-26T10:47:58.622 INFO:tasks.workunit.client.0.smithi060.stdout:7/546: readlink da/d15/d21/d2f/d37/l44 0 2024-02-26T10:47:58.627 INFO:tasks.workunit.client.0.smithi060.stdout:3/346: symlink d5/de/df/d11/l5f 0 2024-02-26T10:47:58.627 INFO:tasks.workunit.client.0.smithi060.stdout:3/347: write d5/de/df/d11/f1c [89792,123811] 0 2024-02-26T10:47:58.631 INFO:tasks.workunit.client.0.smithi060.stdout:1/517: dread d3/f12 [0,4194304] 0 2024-02-26T10:47:58.631 INFO:tasks.workunit.client.0.smithi060.stdout:1/518: dread - d3/db/d1e/d38/d6c/f88 zero size 2024-02-26T10:47:58.632 INFO:tasks.workunit.client.0.smithi060.stdout:1/519: getdents d3/d35/d53/d68 0 2024-02-26T10:47:58.636 INFO:tasks.workunit.client.0.smithi060.stdout:1/520: write d3/db/d2e/d51/fac [652144,55563] 0 2024-02-26T10:47:58.636 INFO:tasks.workunit.client.0.smithi060.stdout:8/416: dwrite f8 [0,4194304] 0 2024-02-26T10:47:58.641 INFO:tasks.workunit.client.0.smithi060.stdout:4/426: dread d0/d27/f4d [0,4194304] 0 2024-02-26T10:47:58.653 INFO:tasks.workunit.client.0.smithi060.stdout:9/483: dwrite d3/f6b [0,4194304] 0 2024-02-26T10:47:58.653 INFO:tasks.workunit.client.0.smithi060.stdout:9/484: read d3/d1e/d5e/d1b/d23/f36 [2181125,49063] 0 2024-02-26T10:47:58.657 INFO:tasks.workunit.client.0.smithi060.stdout:0/405: dread d0/f3 [0,4194304] 0 2024-02-26T10:47:58.657 INFO:tasks.workunit.client.0.smithi060.stdout:8/417: link c1 d9/d15/d29/d2f/d34/d72/d50/c7c 0 2024-02-26T10:47:58.659 INFO:tasks.workunit.client.0.smithi060.stdout:0/406: symlink d0/d37/d44/l8a 0 2024-02-26T10:47:58.659 INFO:tasks.workunit.client.0.smithi060.stdout:0/407: stat d0/d37/l49 0 2024-02-26T10:47:58.659 INFO:tasks.workunit.client.0.smithi060.stdout:0/408: fdatasync d0/dd/d53/f64 0 2024-02-26T10:47:58.659 INFO:tasks.workunit.client.0.smithi060.stdout:8/418: getdents d9/d15/d29/d2f/d34/d72/d50 0 2024-02-26T10:47:58.660 INFO:tasks.workunit.client.0.smithi060.stdout:0/409: mknod d0/dd/d53/d87/c8b 0 2024-02-26T10:47:58.661 INFO:tasks.workunit.client.0.smithi060.stdout:8/419: creat d9/d15/d29/f7d x:0 0 0 2024-02-26T10:47:58.662 INFO:tasks.workunit.client.0.smithi060.stdout:7/547: dread da/d15/d21/d32/f47 [0,4194304] 0 2024-02-26T10:47:58.662 INFO:tasks.workunit.client.0.smithi060.stdout:7/548: fsync da/d15/d21/d2f/d78/fa7 0 2024-02-26T10:47:58.663 INFO:tasks.workunit.client.0.smithi060.stdout:8/420: mkdir d9/d15/d29/d2f/d34/d72/d7e 0 2024-02-26T10:47:58.663 INFO:tasks.workunit.client.0.smithi060.stdout:8/421: readlink d9/d15/d29/d2f/l37 0 2024-02-26T10:47:58.664 INFO:tasks.workunit.client.0.smithi060.stdout:7/549: mkdir da/d15/d21/d4b/db9 0 2024-02-26T10:47:58.665 INFO:tasks.workunit.client.0.smithi060.stdout:8/422: write d9/d15/d29/d2f/d34/d72/d50/d6a/f61 [350077,47096] 0 2024-02-26T10:47:58.667 INFO:tasks.workunit.client.0.smithi060.stdout:6/487: dwrite d6/d7/fd [0,4194304] 0 2024-02-26T10:47:58.672 INFO:tasks.workunit.client.0.smithi060.stdout:6/488: symlink d6/d7/d25/d80/la6 0 2024-02-26T10:47:58.681 INFO:tasks.workunit.client.0.smithi060.stdout:9/485: read d3/d1e/f82 [1730450,47635] 0 2024-02-26T10:47:58.682 INFO:tasks.workunit.client.0.smithi060.stdout:6/489: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/da7 0 2024-02-26T10:47:58.686 INFO:tasks.workunit.client.0.smithi060.stdout:9/486: mkdir d3/d1e/d5e/d9e 0 2024-02-26T10:47:58.686 INFO:tasks.workunit.client.0.smithi060.stdout:9/487: chown d3/f6b 840 1 2024-02-26T10:47:58.689 INFO:tasks.workunit.client.0.smithi060.stdout:9/488: getdents d3/d1e/d5e/d35 0 2024-02-26T10:47:58.690 INFO:tasks.workunit.client.0.smithi060.stdout:9/489: symlink d3/d1e/d5e/d1b/d5f/dc/d32/l9f 0 2024-02-26T10:47:58.691 INFO:tasks.workunit.client.0.smithi060.stdout:1/521: dwrite d3/db/d1e/fb7 [0,4194304] 0 2024-02-26T10:47:58.691 INFO:tasks.workunit.client.0.smithi060.stdout:1/522: write d3/db/d1e/d6d/d78/f86 [565334,82867] 0 2024-02-26T10:47:58.691 INFO:tasks.workunit.client.0.smithi060.stdout:9/490: creat d3/d1e/d5e/d1b/d5f/d8c/d91/d96/fa0 x:0 0 0 2024-02-26T10:47:58.693 INFO:tasks.workunit.client.0.smithi060.stdout:9/491: mknod d3/d1e/d5e/ca1 0 2024-02-26T10:47:58.693 INFO:tasks.workunit.client.0.smithi060.stdout:9/492: write d3/d1e/d5e/d1b/f5a [4926750,66775] 0 2024-02-26T10:47:58.693 INFO:tasks.workunit.client.0.smithi060.stdout:9/493: chown d3/d1e/d5e/d1b/d23 201 1 2024-02-26T10:47:58.695 INFO:tasks.workunit.client.0.smithi060.stdout:3/348: dwrite d5/de/df/d11/f34 [0,4194304] 0 2024-02-26T10:47:58.697 INFO:tasks.workunit.client.0.smithi060.stdout:9/494: mkdir d3/d1e/d93/da2 0 2024-02-26T10:47:58.697 INFO:tasks.workunit.client.0.smithi060.stdout:9/495: read - d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/f97 zero size 2024-02-26T10:47:58.699 INFO:tasks.workunit.client.0.smithi060.stdout:9/496: creat d3/d1e/d5e/d1b/d5f/d10/d98/fa3 x:0 0 0 2024-02-26T10:47:58.701 INFO:tasks.workunit.client.0.smithi060.stdout:8/423: rename d9/d15/f6d to d9/d5a/f7f 0 2024-02-26T10:47:58.730 INFO:tasks.workunit.client.0.smithi060.stdout:4/427: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:47:58.735 INFO:tasks.workunit.client.0.smithi060.stdout:4/428: creat d0/d27/d2f/f7d x:0 0 0 2024-02-26T10:47:58.738 INFO:tasks.workunit.client.0.smithi060.stdout:4/429: symlink d0/da/d3b/d1b/l7e 0 2024-02-26T10:47:58.739 INFO:tasks.workunit.client.0.smithi060.stdout:7/550: dread da/d15/d21/d41/d58/f70 [0,4194304] 0 2024-02-26T10:47:58.739 INFO:tasks.workunit.client.0.smithi060.stdout:7/551: stat da/d66/dad/c64 0 2024-02-26T10:47:58.739 INFO:tasks.workunit.client.0.smithi060.stdout:7/552: dread - da/d5a/f82 zero size 2024-02-26T10:47:58.740 INFO:tasks.workunit.client.0.smithi060.stdout:4/430: mkdir d0/d27/d7f 0 2024-02-26T10:47:58.740 INFO:tasks.workunit.client.0.smithi060.stdout:4/431: fsync d0/d7/d29/f32 0 2024-02-26T10:47:58.740 INFO:tasks.workunit.client.0.smithi060.stdout:4/432: write d0/d27/d52/f71 [517709,80262] 0 2024-02-26T10:47:58.741 INFO:tasks.workunit.client.0.smithi060.stdout:7/553: mknod da/d15/d88/cba 0 2024-02-26T10:47:58.742 INFO:tasks.workunit.client.0.smithi060.stdout:7/554: mknod da/d15/d21/cbb 0 2024-02-26T10:47:58.747 INFO:tasks.workunit.client.0.smithi060.stdout:3/349: dwrite d5/f52 [0,4194304] 0 2024-02-26T10:47:58.751 INFO:tasks.workunit.client.0.smithi060.stdout:1/523: dwrite d3/db/d1e/f3b [0,4194304] 0 2024-02-26T10:47:58.752 INFO:tasks.workunit.client.0.smithi060.stdout:9/497: write d3/d1e/d5e/d1b/f5a [2945596,4991] 0 2024-02-26T10:47:58.752 INFO:tasks.workunit.client.0.smithi060.stdout:7/555: link da/d15/f2e da/fbc 0 2024-02-26T10:47:58.755 INFO:tasks.workunit.client.0.smithi060.stdout:7/556: mkdir da/d66/dbd 0 2024-02-26T10:47:58.764 INFO:tasks.workunit.client.0.smithi060.stdout:4/433: write d0/d7/f25 [3105955,109681] 0 2024-02-26T10:47:58.767 INFO:tasks.workunit.client.0.smithi060.stdout:0/410: dwrite d0/dd/d39/f61 [0,4194304] 0 2024-02-26T10:47:58.767 INFO:tasks.workunit.client.0.smithi060.stdout:9/498: rename d3/d1e/d5e/d1b/d5f/f5b to d3/d1e/d5e/d1b/d5f/d10/d98/fa4 0 2024-02-26T10:47:58.767 INFO:tasks.workunit.client.0.smithi060.stdout:0/411: chown d0/dd/d39/d40/f19 5 1 2024-02-26T10:47:58.767 INFO:tasks.workunit.client.0.smithi060.stdout:9/499: write d3/d1e/d57/f88 [349966,448] 0 2024-02-26T10:47:58.767 INFO:tasks.workunit.client.0.smithi060.stdout:6/490: dread d6/d7/d25/d1c/d39/f77 [0,4194304] 0 2024-02-26T10:47:58.767 INFO:tasks.workunit.client.0.smithi060.stdout:9/500: fdatasync d3/d1e/d5e/d1b/d23/f34 0 2024-02-26T10:47:58.767 INFO:tasks.workunit.client.0.smithi060.stdout:4/434: stat d0/da/d3b/d16/f51 0 2024-02-26T10:47:58.779 INFO:tasks.workunit.client.0.smithi060.stdout:0/412: rename d0/dd/d39/d40/f1c to d0/dd/d33/f8c 0 2024-02-26T10:47:58.779 INFO:tasks.workunit.client.0.smithi060.stdout:6/491: truncate d6/d7/d25/d1c/d39/f77 263542 0 2024-02-26T10:47:58.779 INFO:tasks.workunit.client.0.smithi060.stdout:0/413: fdatasync d0/d37/d44/f66 0 2024-02-26T10:47:58.779 INFO:tasks.workunit.client.0.smithi060.stdout:9/501: creat d3/d1e/d57/d9c/fa5 x:0 0 0 2024-02-26T10:47:58.779 INFO:tasks.workunit.client.0.smithi060.stdout:9/502: chown d3/d1e/d5e/d1b/d5f/f1f 19 1 2024-02-26T10:47:58.779 INFO:tasks.workunit.client.0.smithi060.stdout:7/557: dread da/d15/d21/d2f/d37/f38 [0,4194304] 0 2024-02-26T10:47:58.779 INFO:tasks.workunit.client.0.smithi060.stdout:4/435: creat d0/da/f80 x:0 0 0 2024-02-26T10:47:58.779 INFO:tasks.workunit.client.0.smithi060.stdout:4/436: truncate d0/d27/d2f/f7d 767973 0 2024-02-26T10:47:58.780 INFO:tasks.workunit.client.0.smithi060.stdout:4/437: dread - d0/da/d3b/d16/f3a zero size 2024-02-26T10:47:58.783 INFO:tasks.workunit.client.0.smithi060.stdout:9/503: rename d3/d1e/d5e/d1b/c1d to d3/d1e/ca6 0 2024-02-26T10:47:58.784 INFO:tasks.workunit.client.0.smithi060.stdout:9/504: write d3/d1e/d5e/d1b/d5f/f2f [5980392,49849] 0 2024-02-26T10:47:58.784 INFO:tasks.workunit.client.0.smithi060.stdout:5/423: sync 2024-02-26T10:47:58.784 INFO:tasks.workunit.client.0.smithi060.stdout:2/393: sync 2024-02-26T10:47:58.785 INFO:tasks.workunit.client.0.smithi060.stdout:5/424: chown d0/de/d15/f44 40 1 2024-02-26T10:47:58.785 INFO:tasks.workunit.client.0.smithi060.stdout:9/505: getdents d3/d1e/d5e/d35/d87 0 2024-02-26T10:47:58.786 INFO:tasks.workunit.client.0.smithi060.stdout:6/492: dread d6/d7/d25/f59 [0,4194304] 0 2024-02-26T10:47:58.787 INFO:tasks.workunit.client.0.smithi060.stdout:2/394: rename dd/d11/l7c to dd/d43/d60/d41/d37/l81 0 2024-02-26T10:47:58.788 INFO:tasks.workunit.client.0.smithi060.stdout:9/506: mknod d3/d1e/d5e/d1b/d5f/d10/ca7 0 2024-02-26T10:47:58.790 INFO:tasks.workunit.client.0.smithi060.stdout:6/493: creat d6/d7/d25/d1c/d39/fa8 x:0 0 0 2024-02-26T10:47:58.791 INFO:tasks.workunit.client.0.smithi060.stdout:2/395: creat dd/d43/d60/d41/d37/d4c/f82 x:0 0 0 2024-02-26T10:47:58.791 INFO:tasks.workunit.client.0.smithi060.stdout:6/494: symlink d6/d7/d25/d80/la9 0 2024-02-26T10:47:58.791 INFO:tasks.workunit.client.0.smithi060.stdout:2/396: stat dd/d43/d60/d41/d37/d4c/f82 0 2024-02-26T10:47:58.793 INFO:tasks.workunit.client.0.smithi060.stdout:2/397: symlink dd/d52/l83 0 2024-02-26T10:47:58.793 INFO:tasks.workunit.client.0.smithi060.stdout:0/414: dread d0/dd/f2b [0,4194304] 0 2024-02-26T10:47:58.793 INFO:tasks.workunit.client.0.smithi060.stdout:9/507: symlink d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/la8 0 2024-02-26T10:47:58.794 INFO:tasks.workunit.client.0.smithi060.stdout:6/495: unlink d6/d7/d25/d1c/d1f/d23/d2b/c7d 0 2024-02-26T10:47:58.794 INFO:tasks.workunit.client.0.smithi060.stdout:6/496: truncate d6/d7/d47/f9b 1012476 0 2024-02-26T10:47:58.794 INFO:tasks.workunit.client.0.smithi060.stdout:9/508: creat d3/d1e/d5e/d1b/d5f/d74/fa9 x:0 0 0 2024-02-26T10:47:58.794 INFO:tasks.workunit.client.0.smithi060.stdout:9/509: write d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/f70 [188386,101953] 0 2024-02-26T10:47:58.794 INFO:tasks.workunit.client.0.smithi060.stdout:9/510: stat d3/d1e/d5e/d1b/d23/f36 0 2024-02-26T10:47:58.794 INFO:tasks.workunit.client.0.smithi060.stdout:9/511: chown d3/d1e/d5e/ca1 973545538 1 2024-02-26T10:47:58.795 INFO:tasks.workunit.client.0.smithi060.stdout:0/415: link d0/dd/f2b d0/dd/d39/d40/f8d 0 2024-02-26T10:47:58.796 INFO:tasks.workunit.client.0.smithi060.stdout:2/398: creat dd/d43/d60/d41/d37/d4c/d76/f84 x:0 0 0 2024-02-26T10:47:58.796 INFO:tasks.workunit.client.0.smithi060.stdout:6/497: symlink d6/d7/d25/d1c/d6b/laa 0 2024-02-26T10:47:58.797 INFO:tasks.workunit.client.0.smithi060.stdout:6/498: write d6/d7/d25/d1c/d1f/d23/f3a [919542,97790] 0 2024-02-26T10:47:58.797 INFO:tasks.workunit.client.0.smithi060.stdout:9/512: symlink d3/d1e/d5e/d9e/laa 0 2024-02-26T10:47:58.799 INFO:tasks.workunit.client.0.smithi060.stdout:9/513: creat d3/d1e/d93/fab x:0 0 0 2024-02-26T10:47:58.799 INFO:tasks.workunit.client.0.smithi060.stdout:9/514: chown d3/d68 10193 1 2024-02-26T10:47:58.799 INFO:tasks.workunit.client.0.smithi060.stdout:9/515: fdatasync d3/d1e/d5e/d1b/d5f/d10/d98/fa3 0 2024-02-26T10:47:58.800 INFO:tasks.workunit.client.0.smithi060.stdout:6/499: read d6/f9 [319494,100201] 0 2024-02-26T10:47:58.800 INFO:tasks.workunit.client.0.smithi060.stdout:6/500: dread - d6/d7/d25/d1c/f6d zero size 2024-02-26T10:47:58.800 INFO:tasks.workunit.client.0.smithi060.stdout:6/501: chown d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93 1086267 1 2024-02-26T10:47:58.800 INFO:tasks.workunit.client.0.smithi060.stdout:9/516: truncate d3/d1e/d5e/d1b/d5f/d10/f21 3357494 0 2024-02-26T10:47:58.801 INFO:tasks.workunit.client.0.smithi060.stdout:1/524: dwrite d3/d35/d53/d68/f8c [0,4194304] 0 2024-02-26T10:47:58.801 INFO:tasks.workunit.client.0.smithi060.stdout:1/525: write d3/d35/d53/d68/d6b/f9e [822592,46768] 0 2024-02-26T10:47:58.801 INFO:tasks.workunit.client.0.smithi060.stdout:6/502: read d6/f8e [461728,71978] 0 2024-02-26T10:47:58.807 INFO:tasks.workunit.client.0.smithi060.stdout:6/503: rmdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/d97 0 2024-02-26T10:47:58.808 INFO:tasks.workunit.client.0.smithi060.stdout:0/416: dread d0/d37/d44/d58/f5b [0,4194304] 0 2024-02-26T10:47:58.813 INFO:tasks.workunit.client.0.smithi060.stdout:6/504: creat d6/d7/d2e/fab x:0 0 0 2024-02-26T10:47:58.813 INFO:tasks.workunit.client.0.smithi060.stdout:6/505: chown d6/d7/d2e/d56/d73/fa4 20944 1 2024-02-26T10:47:58.822 INFO:tasks.workunit.client.0.smithi060.stdout:5/425: rename d0/de/d15/d26/l98 to d0/de/d20/d32/d4d/d97/l9a 0 2024-02-26T10:47:58.828 INFO:tasks.workunit.client.0.smithi060.stdout:4/438: dwrite d0/da/f14 [0,4194304] 0 2024-02-26T10:47:58.829 INFO:tasks.workunit.client.0.smithi060.stdout:9/517: dread d3/f6b [0,4194304] 0 2024-02-26T10:47:58.829 INFO:tasks.workunit.client.0.smithi060.stdout:4/439: symlink d0/d7/l81 0 2024-02-26T10:47:58.830 INFO:tasks.workunit.client.0.smithi060.stdout:9/518: symlink d3/d1e/d5e/d1b/d5f/dc/d32/d3c/lac 0 2024-02-26T10:47:58.832 INFO:tasks.workunit.client.0.smithi060.stdout:9/519: creat d3/d1e/d5e/d1b/d5f/d10/fad x:0 0 0 2024-02-26T10:47:58.832 INFO:tasks.workunit.client.0.smithi060.stdout:9/520: chown d3/d1e/d93/da2 80340570 1 2024-02-26T10:47:58.832 INFO:tasks.workunit.client.0.smithi060.stdout:9/521: dread - d3/d1e/d5e/d1b/d5f/d10/f6c zero size 2024-02-26T10:47:58.835 INFO:tasks.workunit.client.0.smithi060.stdout:9/522: creat d3/d68/fae x:0 0 0 2024-02-26T10:47:58.835 INFO:tasks.workunit.client.0.smithi060.stdout:9/523: stat d3/d1e/d5e/d35 0 2024-02-26T10:47:58.842 INFO:tasks.workunit.client.0.smithi060.stdout:9/524: mknod d3/d1e/d5e/d1b/caf 0 2024-02-26T10:47:58.842 INFO:tasks.workunit.client.0.smithi060.stdout:7/558: dwrite da/d15/f74 [0,4194304] 0 2024-02-26T10:47:58.850 INFO:tasks.workunit.client.0.smithi060.stdout:5/426: rename d0/de/d20/d32/d4d/d60 to d0/de/d15/d65/d86/d9b 0 2024-02-26T10:47:58.850 INFO:tasks.workunit.client.0.smithi060.stdout:7/559: dread da/d15/f2e [0,4194304] 0 2024-02-26T10:47:58.851 INFO:tasks.workunit.client.0.smithi060.stdout:7/560: readlink da/l36 0 2024-02-26T10:47:58.852 INFO:tasks.workunit.client.0.smithi060.stdout:7/561: creat da/d15/d21/d32/fbe x:0 0 0 2024-02-26T10:47:58.852 INFO:tasks.workunit.client.0.smithi060.stdout:7/562: dread - da/d15/d21/d2f/f91 zero size 2024-02-26T10:47:58.852 INFO:tasks.workunit.client.0.smithi060.stdout:7/563: chown da/d15/d21/d41/d58/f70 72902 1 2024-02-26T10:47:58.853 INFO:tasks.workunit.client.0.smithi060.stdout:5/427: mkdir d0/de/d15/d26/d9c 0 2024-02-26T10:47:58.853 INFO:tasks.workunit.client.0.smithi060.stdout:5/428: chown d0/de/d20/d32/d4d/d97 4072 1 2024-02-26T10:47:58.854 INFO:tasks.workunit.client.0.smithi060.stdout:5/429: creat d0/de/d15/d65/f9d x:0 0 0 2024-02-26T10:47:58.854 INFO:tasks.workunit.client.0.smithi060.stdout:6/506: write d6/d7/d25/d1c/d1f/d23/d7a/f88 [1705051,105104] 0 2024-02-26T10:47:58.855 INFO:tasks.workunit.client.0.smithi060.stdout:5/430: mknod d0/d4b/c9e 0 2024-02-26T10:47:58.858 INFO:tasks.workunit.client.0.smithi060.stdout:1/526: dwrite d3/d35/d53/d57/f71 [0,4194304] 0 2024-02-26T10:47:58.859 INFO:tasks.workunit.client.0.smithi060.stdout:1/527: fdatasync d3/d35/d53/d57/f81 0 2024-02-26T10:47:58.861 INFO:tasks.workunit.client.0.smithi060.stdout:5/431: rmdir d0/de 39 2024-02-26T10:47:58.861 INFO:tasks.workunit.client.0.smithi060.stdout:5/432: write d0/fd [5153660,4821] 0 2024-02-26T10:47:58.861 INFO:tasks.workunit.client.0.smithi060.stdout:5/433: chown d0/d4b/f51 32477 1 2024-02-26T10:47:58.864 INFO:tasks.workunit.client.0.smithi060.stdout:5/434: creat d0/de/f9f x:0 0 0 2024-02-26T10:47:58.864 INFO:tasks.workunit.client.0.smithi060.stdout:5/435: fdatasync d0/f1d 0 2024-02-26T10:47:58.869 INFO:tasks.workunit.client.0.smithi060.stdout:6/507: dread d6/d6e/f96 [0,4194304] 0 2024-02-26T10:47:58.871 INFO:tasks.workunit.client.0.smithi060.stdout:2/399: write f4 [3086725,3090] 0 2024-02-26T10:47:58.872 INFO:tasks.workunit.client.0.smithi060.stdout:6/508: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac 0 2024-02-26T10:47:58.875 INFO:tasks.workunit.client.0.smithi060.stdout:7/564: dread f6 [0,4194304] 0 2024-02-26T10:47:58.875 INFO:tasks.workunit.client.0.smithi060.stdout:7/565: truncate da/d15/d88/d3b/f9d 186585 0 2024-02-26T10:47:58.879 INFO:tasks.workunit.client.0.smithi060.stdout:2/400: link dd/f1d dd/d43/d60/f85 0 2024-02-26T10:47:58.880 INFO:tasks.workunit.client.0.smithi060.stdout:4/440: rename d0/d27/l42 to d0/da/d3b/d16/l82 0 2024-02-26T10:47:58.880 INFO:tasks.workunit.client.0.smithi060.stdout:4/441: readlink d0/l2c 0 2024-02-26T10:47:58.883 INFO:tasks.workunit.client.0.smithi060.stdout:4/442: link d0/d7/f8 d0/d27/f83 0 2024-02-26T10:47:58.887 INFO:tasks.workunit.client.0.smithi060.stdout:4/443: creat d0/da/d3b/d16/f84 x:0 0 0 2024-02-26T10:47:58.887 INFO:tasks.workunit.client.0.smithi060.stdout:7/566: link da/d15/d21/d2f/f48 da/d15/d21/d2f/fbf 0 2024-02-26T10:47:58.888 INFO:tasks.workunit.client.0.smithi060.stdout:0/417: dwrite d0/f78 [0,4194304] 0 2024-02-26T10:47:58.888 INFO:tasks.workunit.client.0.smithi060.stdout:0/418: chown d0/cc 23619765 1 2024-02-26T10:47:58.911 INFO:tasks.workunit.client.0.smithi060.stdout:8/424: sync 2024-02-26T10:47:58.911 INFO:tasks.workunit.client.0.smithi060.stdout:3/350: sync 2024-02-26T10:47:58.912 INFO:tasks.workunit.client.0.smithi060.stdout:1/528: rename d3/db/d1e/f27 to d3/d35/d53/d68/dad/fb8 0 2024-02-26T10:47:58.927 INFO:tasks.workunit.client.0.smithi060.stdout:9/525: dwrite d3/d1e/d5e/f17 [0,4194304] 0 2024-02-26T10:47:58.932 INFO:tasks.workunit.client.0.smithi060.stdout:1/529: rename d3/d35/fa0 to d3/db/d1e/d38/d6c/fb9 0 2024-02-26T10:47:58.932 INFO:tasks.workunit.client.0.smithi060.stdout:6/509: dwrite d6/d7/d47/f55 [0,4194304] 0 2024-02-26T10:47:58.933 INFO:tasks.workunit.client.0.smithi060.stdout:2/401: dread dd/d52/f7a [0,4194304] 0 2024-02-26T10:47:58.935 INFO:tasks.workunit.client.0.smithi060.stdout:6/510: symlink d6/d7/d2e/d56/lad 0 2024-02-26T10:47:58.938 INFO:tasks.workunit.client.0.smithi060.stdout:6/511: write d6/d7/d25/d1c/d1f/d23/d2b/f3b [3223311,92945] 0 2024-02-26T10:47:58.938 INFO:tasks.workunit.client.0.smithi060.stdout:6/512: dread - d6/f7c zero size 2024-02-26T10:47:58.938 INFO:tasks.workunit.client.0.smithi060.stdout:6/513: truncate d6/d7/d2e/d56/f61 309620 0 2024-02-26T10:47:58.938 INFO:tasks.workunit.client.0.smithi060.stdout:2/402: mkdir dd/d43/d55/d5e/d86 0 2024-02-26T10:47:58.938 INFO:tasks.workunit.client.0.smithi060.stdout:6/514: truncate d6/d7/d25/d1c/f6d 927158 0 2024-02-26T10:47:58.945 INFO:tasks.workunit.client.0.smithi060.stdout:9/526: link d3/d1e/d5e/d1b/d5f/d10/f11 d3/d1e/d5e/d35/d87/d9b/d8a/fb0 0 2024-02-26T10:47:58.946 INFO:tasks.workunit.client.0.smithi060.stdout:4/444: dwrite d0/d27/d2f/f6e [0,4194304] 0 2024-02-26T10:47:58.952 INFO:tasks.workunit.client.0.smithi060.stdout:1/530: rename d3/db/d1e/d38/l4e to d3/d35/d53/d68/d6b/lba 0 2024-02-26T10:47:58.953 INFO:tasks.workunit.client.0.smithi060.stdout:1/531: write d3/d35/d53/f74 [808310,44618] 0 2024-02-26T10:47:58.957 INFO:tasks.workunit.client.0.smithi060.stdout:8/425: dwrite d9/f20 [0,4194304] 0 2024-02-26T10:47:58.957 INFO:tasks.workunit.client.0.smithi060.stdout:8/426: read - d9/d5a/d58/f78 zero size 2024-02-26T10:47:58.958 INFO:tasks.workunit.client.0.smithi060.stdout:9/527: mkdir d3/d1e/d5e/d1b/d23/db1 0 2024-02-26T10:47:58.958 INFO:tasks.workunit.client.0.smithi060.stdout:8/427: mknod d9/d5a/d58/c80 0 2024-02-26T10:47:58.959 INFO:tasks.workunit.client.0.smithi060.stdout:9/528: write d3/d1e/d5e/d1b/d5f/d10/f37 [1224317,28113] 0 2024-02-26T10:47:58.959 INFO:tasks.workunit.client.0.smithi060.stdout:9/529: chown d3/d1e/d5e/d1b/d5f/d10/l99 1431117467 1 2024-02-26T10:47:58.960 INFO:tasks.workunit.client.0.smithi060.stdout:0/419: dwrite d0/dd/d39/d40/f34 [4194304,4194304] 0 2024-02-26T10:47:58.963 INFO:tasks.workunit.client.0.smithi060.stdout:1/532: mkdir d3/db/d2e/d51/dbb 0 2024-02-26T10:47:58.964 INFO:tasks.workunit.client.0.smithi060.stdout:9/530: rename d3/d1e/d5e/d1b/c75 to d3/d1e/d5e/d35/d87/d9b/d8a/cb2 0 2024-02-26T10:47:58.964 INFO:tasks.workunit.client.0.smithi060.stdout:7/567: dwrite da/d66/dad/fa4 [0,4194304] 0 2024-02-26T10:47:58.964 INFO:tasks.workunit.client.0.smithi060.stdout:1/533: getdents d3/db/d1e/d39/d9b 0 2024-02-26T10:47:58.965 INFO:tasks.workunit.client.0.smithi060.stdout:7/568: mknod da/d15/d21/d41/d6c/cc0 0 2024-02-26T10:47:58.971 INFO:tasks.workunit.client.0.smithi060.stdout:8/428: dread d9/d15/d29/d2f/d34/d72/d50/d6a/f45 [0,4194304] 0 2024-02-26T10:47:58.976 INFO:tasks.workunit.client.0.smithi060.stdout:8/429: unlink d9/d15/d29/d2f/f55 0 2024-02-26T10:47:58.983 INFO:tasks.workunit.client.0.smithi060.stdout:1/534: rename d3/f12 to d3/d35/d53/d68/d90/fbc 0 2024-02-26T10:47:58.986 INFO:tasks.workunit.client.0.smithi060.stdout:1/535: write d3/d35/d53/d57/f87 [4690725,59838] 0 2024-02-26T10:47:58.986 INFO:tasks.workunit.client.0.smithi060.stdout:8/430: dread d9/d15/d29/d2f/d34/d72/d50/d6a/f61 [0,4194304] 0 2024-02-26T10:47:58.987 INFO:tasks.workunit.client.0.smithi060.stdout:8/431: rename d9/d15/d29/d2f/f3d to d9/d15/f81 0 2024-02-26T10:47:58.996 INFO:tasks.workunit.client.0.smithi060.stdout:8/432: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f82 x:0 0 0 2024-02-26T10:47:58.996 INFO:tasks.workunit.client.0.smithi060.stdout:8/433: dread - d9/d5a/f68 zero size 2024-02-26T10:47:58.996 INFO:tasks.workunit.client.0.smithi060.stdout:9/531: dread d3/d1e/d5e/d1b/d5f/f2e [0,4194304] 0 2024-02-26T10:47:58.997 INFO:tasks.workunit.client.0.smithi060.stdout:9/532: creat d3/d1e/d93/da2/fb3 x:0 0 0 2024-02-26T10:47:58.999 INFO:tasks.workunit.client.0.smithi060.stdout:9/533: creat d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/fb4 x:0 0 0 2024-02-26T10:47:58.999 INFO:tasks.workunit.client.0.smithi060.stdout:8/434: mknod d9/d15/d29/d2f/d34/d72/d50/d6a/c83 0 2024-02-26T10:47:59.001 INFO:tasks.workunit.client.0.smithi060.stdout:9/534: rename d3/d1e/d5e/d1b/d5f/d74/fa9 to d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/d7a/fb5 0 2024-02-26T10:47:59.003 INFO:tasks.workunit.client.0.smithi060.stdout:6/515: dwrite d6/d7/d25/f5c [0,4194304] 0 2024-02-26T10:47:59.003 INFO:tasks.workunit.client.0.smithi060.stdout:6/516: readlink d6/l36 0 2024-02-26T10:47:59.003 INFO:tasks.workunit.client.0.smithi060.stdout:6/517: dread - d6/d7/d25/d1c/d39/fa5 zero size 2024-02-26T10:47:59.003 INFO:tasks.workunit.client.0.smithi060.stdout:6/518: stat d6/d7/d25/d1c/d1f/d23/d2b/c41 0 2024-02-26T10:47:59.008 INFO:tasks.workunit.client.0.smithi060.stdout:1/536: write d3/db/d2e/f30 [1877469,115478] 0 2024-02-26T10:47:59.009 INFO:tasks.workunit.client.0.smithi060.stdout:1/537: creat d3/db/d1e/d6d/da3/fbd x:0 0 0 2024-02-26T10:47:59.028 INFO:tasks.workunit.client.0.smithi060.stdout:4/445: dwrite d0/da/d57/f67 [0,4194304] 0 2024-02-26T10:47:59.037 INFO:tasks.workunit.client.0.smithi060.stdout:8/435: dwrite d9/d15/f81 [0,4194304] 0 2024-02-26T10:47:59.042 INFO:tasks.workunit.client.0.smithi060.stdout:7/569: dwrite da/d15/d21/d4b/d67/f77 [0,4194304] 0 2024-02-26T10:47:59.042 INFO:tasks.workunit.client.0.smithi060.stdout:1/538: dread d3/d35/d76/f98 [0,4194304] 0 2024-02-26T10:47:59.045 INFO:tasks.workunit.client.0.smithi060.stdout:7/570: mknod da/d15/d21/d41/d6c/cc1 0 2024-02-26T10:47:59.045 INFO:tasks.workunit.client.0.smithi060.stdout:7/571: chown da/d15/d21/d41/d6c/la9 80729 1 2024-02-26T10:47:59.052 INFO:tasks.workunit.client.0.smithi060.stdout:7/572: unlink da/d15/d21/d29/l2a 0 2024-02-26T10:47:59.052 INFO:tasks.workunit.client.0.smithi060.stdout:7/573: fdatasync da/d15/d21/d2f/f68 0 2024-02-26T10:47:59.055 INFO:tasks.workunit.client.0.smithi060.stdout:6/519: dwrite d6/d7/d25/f4b [0,4194304] 0 2024-02-26T10:47:59.058 INFO:tasks.workunit.client.0.smithi060.stdout:8/436: rename d9/d15/d29/d2f/d34/d72/d50/d6a/c83 to d9/d5a/d58/c84 0 2024-02-26T10:47:59.060 INFO:tasks.workunit.client.0.smithi060.stdout:7/574: dread da/d15/d21/f28 [0,4194304] 0 2024-02-26T10:47:59.062 INFO:tasks.workunit.client.0.smithi060.stdout:7/575: symlink da/d15/d21/d4b/d67/lc2 0 2024-02-26T10:47:59.063 INFO:tasks.workunit.client.0.smithi060.stdout:7/576: rmdir da/d15/d21/d41 39 2024-02-26T10:47:59.064 INFO:tasks.workunit.client.0.smithi060.stdout:7/577: truncate da/d15/f53 1813633 0 2024-02-26T10:47:59.065 INFO:tasks.workunit.client.0.smithi060.stdout:7/578: creat da/d15/d21/d29/fc3 x:0 0 0 2024-02-26T10:47:59.065 INFO:tasks.workunit.client.0.smithi060.stdout:7/579: fsync da/d66/dad/f56 0 2024-02-26T10:47:59.067 INFO:tasks.workunit.client.0.smithi060.stdout:7/580: link da/d15/d21/d32/faf da/d66/dad/d8f/fc4 0 2024-02-26T10:47:59.069 INFO:tasks.workunit.client.0.smithi060.stdout:7/581: symlink da/d66/dad/lc5 0 2024-02-26T10:47:59.069 INFO:tasks.workunit.client.0.smithi060.stdout:7/582: chown da/d15/d21/d41/l7b 37617176 1 2024-02-26T10:47:59.071 INFO:tasks.workunit.client.0.smithi060.stdout:7/583: unlink da/d15/d21/d32/f43 0 2024-02-26T10:47:59.074 INFO:tasks.workunit.client.0.smithi060.stdout:7/584: rename da/l55 to da/d94/lc6 0 2024-02-26T10:47:59.074 INFO:tasks.workunit.client.0.smithi060.stdout:7/585: dread - da/d15/d21/d32/faf zero size 2024-02-26T10:47:59.074 INFO:tasks.workunit.client.0.smithi060.stdout:7/586: dread - da/d5a/f81 zero size 2024-02-26T10:47:59.075 INFO:tasks.workunit.client.0.smithi060.stdout:7/587: mkdir da/d15/d21/d41/dc7 0 2024-02-26T10:47:59.077 INFO:tasks.workunit.client.0.smithi060.stdout:7/588: getdents da/d66/dad 0 2024-02-26T10:47:59.077 INFO:tasks.workunit.client.0.smithi060.stdout:7/589: chown da/d15/d21/f2c 74948688 1 2024-02-26T10:47:59.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:47:58 smithi113 ceph-mon[36126]: pgmap v485: 129 pgs: 129 active+clean; 608 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 89 MiB/s wr, 424 op/s 2024-02-26T10:47:59.083 INFO:tasks.workunit.client.0.smithi060.stdout:5/436: sync 2024-02-26T10:47:59.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/437: read - d0/f81 zero size 2024-02-26T10:47:59.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/438: truncate d0/f81 315983 0 2024-02-26T10:47:59.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/439: truncate d0/de/d15/d65/f6c 146696 0 2024-02-26T10:47:59.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/440: fdatasync d0/de/d15/d65/f6c 0 2024-02-26T10:47:59.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/441: stat d0/f17 0 2024-02-26T10:47:59.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/442: truncate d0/de/d15/f5a 919907 0 2024-02-26T10:47:59.084 INFO:tasks.workunit.client.0.smithi060.stdout:8/437: dread d9/f13 [0,4194304] 0 2024-02-26T10:47:59.085 INFO:tasks.workunit.client.0.smithi060.stdout:5/443: creat d0/de/d15/d65/fa0 x:0 0 0 2024-02-26T10:47:59.088 INFO:tasks.workunit.client.0.smithi060.stdout:5/444: rename d0/de/d15/d26/f41 to d0/d33/fa1 0 2024-02-26T10:47:59.089 INFO:tasks.workunit.client.0.smithi060.stdout:5/445: mknod d0/de/d70/d84/ca2 0 2024-02-26T10:47:59.092 INFO:tasks.workunit.client.0.smithi060.stdout:5/446: mkdir d0/de/d20/d32/d4d/da3 0 2024-02-26T10:47:59.102 INFO:tasks.workunit.client.0.smithi060.stdout:5/447: rmdir d0/de/d20/d32/d4d 39 2024-02-26T10:47:59.103 INFO:tasks.workunit.client.0.smithi060.stdout:5/448: mkdir d0/de/da4 0 2024-02-26T10:47:59.104 INFO:tasks.workunit.client.0.smithi060.stdout:8/438: rename d9/d5a/d58/c5f to d9/d6f/c85 0 2024-02-26T10:47:59.107 INFO:tasks.workunit.client.0.smithi060.stdout:8/439: rename c1 to d9/d6f/c86 0 2024-02-26T10:47:59.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:47:58 smithi060 ceph-mon[31539]: pgmap v485: 129 pgs: 129 active+clean; 608 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 89 MiB/s wr, 424 op/s 2024-02-26T10:47:59.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:47:58 smithi100 ceph-mon[36051]: pgmap v485: 129 pgs: 129 active+clean; 608 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 89 MiB/s wr, 424 op/s 2024-02-26T10:47:59.126 INFO:tasks.workunit.client.0.smithi060.stdout:7/590: dwrite da/d15/d21/d2f/f68 [0,4194304] 0 2024-02-26T10:47:59.150 INFO:tasks.workunit.client.0.smithi060.stdout:5/449: dwrite d0/de/d15/f2f [0,4194304] 0 2024-02-26T10:47:59.151 INFO:tasks.workunit.client.0.smithi060.stdout:7/591: write da/d15/f19 [3327633,65300] 0 2024-02-26T10:47:59.153 INFO:tasks.workunit.client.0.smithi060.stdout:8/440: dwrite d9/d5a/d58/f78 [0,4194304] 0 2024-02-26T10:47:59.157 INFO:tasks.workunit.client.0.smithi060.stdout:5/450: mkdir d0/de/d20/d32/d4d/da3/da5 0 2024-02-26T10:47:59.158 INFO:tasks.workunit.client.0.smithi060.stdout:8/441: rename d9/l21 to d9/d15/d29/d2f/l87 0 2024-02-26T10:47:59.158 INFO:tasks.workunit.client.0.smithi060.stdout:8/442: readlink d9/d15/d29/d2f/d34/d72/l30 0 2024-02-26T10:47:59.160 INFO:tasks.workunit.client.0.smithi060.stdout:2/403: sync 2024-02-26T10:47:59.160 INFO:tasks.workunit.client.0.smithi060.stdout:0/420: sync 2024-02-26T10:47:59.160 INFO:tasks.workunit.client.0.smithi060.stdout:3/351: sync 2024-02-26T10:47:59.160 INFO:tasks.workunit.client.0.smithi060.stdout:8/443: truncate d9/d15/d29/d2f/d34/d72/d50/d6a/f74 1687439 0 2024-02-26T10:47:59.161 INFO:tasks.workunit.client.0.smithi060.stdout:2/404: creat dd/d43/d60/d41/d37/d38/d57/f87 x:0 0 0 2024-02-26T10:47:59.161 INFO:tasks.workunit.client.0.smithi060.stdout:2/405: write dd/d43/d60/d41/d37/d4c/d76/f84 [13840,49696] 0 2024-02-26T10:47:59.162 INFO:tasks.workunit.client.0.smithi060.stdout:3/352: rename d5/de/d16/d17 to d5/de/df/d57/d60 0 2024-02-26T10:47:59.163 INFO:tasks.workunit.client.0.smithi060.stdout:2/406: creat dd/d11/f88 x:0 0 0 2024-02-26T10:47:59.164 INFO:tasks.workunit.client.0.smithi060.stdout:3/353: mkdir d5/d45/d61 0 2024-02-26T10:47:59.165 INFO:tasks.workunit.client.0.smithi060.stdout:3/354: stat l1 0 2024-02-26T10:47:59.165 INFO:tasks.workunit.client.0.smithi060.stdout:3/355: chown d5/f24 1821 1 2024-02-26T10:47:59.167 INFO:tasks.workunit.client.0.smithi060.stdout:3/356: link d5/de/df/d57/d60/f30 d5/de/df/d11/f62 0 2024-02-26T10:47:59.168 INFO:tasks.workunit.client.0.smithi060.stdout:5/451: rmdir d0/de/d15/d26/d9c 0 2024-02-26T10:47:59.168 INFO:tasks.workunit.client.0.smithi060.stdout:3/357: mkdir d5/de/df/d11/d63 0 2024-02-26T10:47:59.168 INFO:tasks.workunit.client.0.smithi060.stdout:3/358: fdatasync d5/f24 0 2024-02-26T10:47:59.168 INFO:tasks.workunit.client.0.smithi060.stdout:3/359: truncate d5/d45/f49 4406674 0 2024-02-26T10:47:59.175 INFO:tasks.workunit.client.0.smithi060.stdout:8/444: rmdir d9/d15/d29/d2f/d34/d72 39 2024-02-26T10:47:59.187 INFO:tasks.workunit.client.0.smithi060.stdout:2/407: write dd/d43/d60/d41/f1a [349589,18379] 0 2024-02-26T10:47:59.192 INFO:tasks.workunit.client.0.smithi060.stdout:7/592: dwrite da/d66/dad/fa4 [0,4194304] 0 2024-02-26T10:47:59.192 INFO:tasks.workunit.client.0.smithi060.stdout:7/593: read da/f60 [3607598,27587] 0 2024-02-26T10:47:59.194 INFO:tasks.workunit.client.0.smithi060.stdout:7/594: mkdir da/d15/d21/d4b/dc8 0 2024-02-26T10:47:59.197 INFO:tasks.workunit.client.0.smithi060.stdout:5/452: write d0/de/d15/d65/d86/d9b/f72 [432989,68110] 0 2024-02-26T10:47:59.197 INFO:tasks.workunit.client.0.smithi060.stdout:5/453: truncate d0/de/d15/d65/fa0 6251 0 2024-02-26T10:47:59.198 INFO:tasks.workunit.client.0.smithi060.stdout:5/454: unlink d0/d33/c79 0 2024-02-26T10:47:59.199 INFO:tasks.workunit.client.0.smithi060.stdout:5/455: write d0/de/d15/d26/f78 [800695,109095] 0 2024-02-26T10:47:59.200 INFO:tasks.workunit.client.0.smithi060.stdout:5/456: fsync d0/f90 0 2024-02-26T10:47:59.201 INFO:tasks.workunit.client.0.smithi060.stdout:5/457: symlink d0/de/d20/d32/d4d/da3/da5/la6 0 2024-02-26T10:47:59.201 INFO:tasks.workunit.client.0.smithi060.stdout:5/458: chown d0/d33/l77 1 1 2024-02-26T10:47:59.201 INFO:tasks.workunit.client.0.smithi060.stdout:5/459: dread - d0/de/d20/f56 zero size 2024-02-26T10:47:59.202 INFO:tasks.workunit.client.0.smithi060.stdout:5/460: stat d0/d33/c49 0 2024-02-26T10:47:59.202 INFO:tasks.workunit.client.0.smithi060.stdout:5/461: fdatasync d0/de/d15/d26/f38 0 2024-02-26T10:47:59.202 INFO:tasks.workunit.client.0.smithi060.stdout:5/462: write d0/de/d20/f85 [473572,32193] 0 2024-02-26T10:47:59.203 INFO:tasks.workunit.client.0.smithi060.stdout:5/463: symlink d0/de/da4/la7 0 2024-02-26T10:47:59.203 INFO:tasks.workunit.client.0.smithi060.stdout:5/464: chown d0/de/f29 517631 1 2024-02-26T10:47:59.203 INFO:tasks.workunit.client.0.smithi060.stdout:5/465: stat d0/d4b/d7b 0 2024-02-26T10:47:59.203 INFO:tasks.workunit.client.0.smithi060.stdout:5/466: dread - d0/de/f6a zero size 2024-02-26T10:47:59.204 INFO:tasks.workunit.client.0.smithi060.stdout:5/467: creat d0/de/d20/fa8 x:0 0 0 2024-02-26T10:47:59.206 INFO:tasks.workunit.client.0.smithi060.stdout:8/445: write d9/f20 [2659925,33482] 0 2024-02-26T10:47:59.208 INFO:tasks.workunit.client.0.smithi060.stdout:8/446: creat d9/d15/d3e/f88 x:0 0 0 2024-02-26T10:47:59.209 INFO:tasks.workunit.client.0.smithi060.stdout:8/447: mkdir d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89 0 2024-02-26T10:47:59.210 INFO:tasks.workunit.client.0.smithi060.stdout:8/448: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f8a x:0 0 0 2024-02-26T10:47:59.210 INFO:tasks.workunit.client.0.smithi060.stdout:4/446: sync 2024-02-26T10:47:59.211 INFO:tasks.workunit.client.0.smithi060.stdout:9/535: sync 2024-02-26T10:47:59.212 INFO:tasks.workunit.client.0.smithi060.stdout:9/536: creat d3/d1e/d5e/d1b/d5f/fb6 x:0 0 0 2024-02-26T10:47:59.228 INFO:tasks.workunit.client.0.smithi060.stdout:8/449: link d9/d15/d29/d2f/d34/d72/d50/c54 d9/d15/d29/d2f/d34/d72/d50/d6a/c8b 0 2024-02-26T10:47:59.244 INFO:tasks.workunit.client.0.smithi060.stdout:2/408: dwrite dd/d43/d60/d41/d37/d4c/d76/f84 [0,4194304] 0 2024-02-26T10:47:59.244 INFO:tasks.workunit.client.0.smithi060.stdout:2/409: write f4 [4054222,120871] 0 2024-02-26T10:47:59.247 INFO:tasks.workunit.client.0.smithi060.stdout:5/468: dwrite d0/de/d15/d65/d86/d9b/f75 [0,4194304] 0 2024-02-26T10:47:59.250 INFO:tasks.workunit.client.0.smithi060.stdout:5/469: truncate d0/fd 3221265 0 2024-02-26T10:47:59.250 INFO:tasks.workunit.client.0.smithi060.stdout:5/470: stat d0/de/d20/d32/d4c/f74 0 2024-02-26T10:47:59.252 INFO:tasks.workunit.client.0.smithi060.stdout:4/447: dwrite d0/d7/d29/f64 [0,4194304] 0 2024-02-26T10:47:59.256 INFO:tasks.workunit.client.0.smithi060.stdout:4/448: symlink d0/l85 0 2024-02-26T10:47:59.256 INFO:tasks.workunit.client.0.smithi060.stdout:4/449: write d0/da/d3b/d16/f51 [2670858,2152] 0 2024-02-26T10:47:59.259 INFO:tasks.workunit.client.0.smithi060.stdout:4/450: unlink d0/l85 0 2024-02-26T10:47:59.259 INFO:tasks.workunit.client.0.smithi060.stdout:4/451: write d0/d27/f49 [88111,64465] 0 2024-02-26T10:47:59.260 INFO:tasks.workunit.client.0.smithi060.stdout:5/471: unlink d0/de/cf 0 2024-02-26T10:47:59.262 INFO:tasks.workunit.client.0.smithi060.stdout:4/452: creat d0/da/d3b/d1b/f86 x:0 0 0 2024-02-26T10:47:59.270 INFO:tasks.workunit.client.0.smithi060.stdout:7/595: rename da/d5a/f6f to da/d15/d88/d3b/fc9 0 2024-02-26T10:47:59.281 INFO:tasks.workunit.client.0.smithi060.stdout:7/596: creat da/d66/dbd/fca x:0 0 0 2024-02-26T10:47:59.281 INFO:tasks.workunit.client.0.smithi060.stdout:7/597: chown da/d15/f19 935985642 1 2024-02-26T10:47:59.284 INFO:tasks.workunit.client.0.smithi060.stdout:8/450: rename d9/d5a/d58/f78 to d9/d15/d29/d2f/d34/f8c 0 2024-02-26T10:47:59.286 INFO:tasks.workunit.client.0.smithi060.stdout:8/451: symlink d9/d15/d29/d2f/d34/l8d 0 2024-02-26T10:47:59.286 INFO:tasks.workunit.client.0.smithi060.stdout:5/472: rename d0/de/d15/c54 to d0/de/ca9 0 2024-02-26T10:47:59.287 INFO:tasks.workunit.client.0.smithi060.stdout:5/473: stat d0/de/d20/f85 0 2024-02-26T10:47:59.287 INFO:tasks.workunit.client.0.smithi060.stdout:5/474: fdatasync d0/de/f12 0 2024-02-26T10:47:59.288 INFO:tasks.workunit.client.0.smithi060.stdout:5/475: rename d0/d33/c49 to d0/de/d20/caa 0 2024-02-26T10:47:59.290 INFO:tasks.workunit.client.0.smithi060.stdout:8/452: fsync d9/f20 0 2024-02-26T10:47:59.290 INFO:tasks.workunit.client.0.smithi060.stdout:8/453: fdatasync d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f8a 0 2024-02-26T10:47:59.291 INFO:tasks.workunit.client.0.smithi060.stdout:5/476: mkdir d0/de/d20/d32/dab 0 2024-02-26T10:47:59.291 INFO:tasks.workunit.client.0.smithi060.stdout:5/477: dread - d0/de/d20/f56 zero size 2024-02-26T10:47:59.291 INFO:tasks.workunit.client.0.smithi060.stdout:5/478: fsync d0/de/d15/d65/d86/d9b/f75 0 2024-02-26T10:47:59.291 INFO:tasks.workunit.client.0.smithi060.stdout:5/479: fdatasync d0/de/f6a 0 2024-02-26T10:47:59.293 INFO:tasks.workunit.client.0.smithi060.stdout:8/454: creat d9/d15/d29/d2f/d34/d72/d50/d6a/f8e x:0 0 0 2024-02-26T10:47:59.293 INFO:tasks.workunit.client.0.smithi060.stdout:8/455: getdents d9/d15/d29/d2f/d34/d72/d50 0 2024-02-26T10:47:59.294 INFO:tasks.workunit.client.0.smithi060.stdout:5/480: symlink d0/lac 0 2024-02-26T10:47:59.299 INFO:tasks.workunit.client.0.smithi060.stdout:9/537: dwrite d3/d1e/d5e/d1b/d5f/d10/d98/fa4 [4194304,4194304] 0 2024-02-26T10:47:59.299 INFO:tasks.workunit.client.0.smithi060.stdout:9/538: write d3/d1e/d5e/d1b/d5f/f2f [2235373,83026] 0 2024-02-26T10:47:59.304 INFO:tasks.workunit.client.0.smithi060.stdout:9/539: stat d3/d1e/d5e/d1b/d5f/dc/d32/d3c/f65 0 2024-02-26T10:47:59.304 INFO:tasks.workunit.client.0.smithi060.stdout:9/540: chown d3/d1e/d5e/d1b/d5f/dc/d32/d3a 15221204 1 2024-02-26T10:47:59.304 INFO:tasks.workunit.client.0.smithi060.stdout:9/541: stat d3/d1e/d57/l67 0 2024-02-26T10:47:59.304 INFO:tasks.workunit.client.0.smithi060.stdout:9/542: chown d3/d1e/d5e/d1b/d5f/d8c/d91 915 1 2024-02-26T10:47:59.305 INFO:tasks.workunit.client.0.smithi060.stdout:9/543: mkdir d3/d1e/d93/da2/db7 0 2024-02-26T10:47:59.307 INFO:tasks.workunit.client.0.smithi060.stdout:7/598: link da/d15/d21/d2f/d78/l9f da/d15/d21/d4b/db9/lcb 0 2024-02-26T10:47:59.316 INFO:tasks.workunit.client.0.smithi060.stdout:2/410: dwrite dd/d43/d60/d41/d37/d4c/d76/f84 [0,4194304] 0 2024-02-26T10:47:59.319 INFO:tasks.workunit.client.0.smithi060.stdout:2/411: unlink dd/d52/l83 0 2024-02-26T10:47:59.319 INFO:tasks.workunit.client.0.smithi060.stdout:2/412: write dd/d43/d60/d41/f6d [386567,117008] 0 2024-02-26T10:47:59.320 INFO:tasks.workunit.client.0.smithi060.stdout:7/599: dread da/d15/d21/d2f/d78/fa7 [0,4194304] 0 2024-02-26T10:47:59.320 INFO:tasks.workunit.client.0.smithi060.stdout:7/600: write da/d15/d21/d29/fc3 [573166,102773] 0 2024-02-26T10:47:59.322 INFO:tasks.workunit.client.0.smithi060.stdout:2/413: creat dd/d43/d60/d41/d37/d38/f89 x:0 0 0 2024-02-26T10:47:59.323 INFO:tasks.workunit.client.0.smithi060.stdout:2/414: readlink dd/d52/l63 0 2024-02-26T10:47:59.325 INFO:tasks.workunit.client.0.smithi060.stdout:2/415: mknod dd/d43/d60/d41/d42/d6f/c8a 0 2024-02-26T10:47:59.325 INFO:tasks.workunit.client.0.smithi060.stdout:2/416: fdatasync dd/d11/f88 0 2024-02-26T10:47:59.325 INFO:tasks.workunit.client.0.smithi060.stdout:9/544: write d3/d1e/d5e/d35/d87/d9b/d8a/fb0 [255363,24617] 0 2024-02-26T10:47:59.326 INFO:tasks.workunit.client.0.smithi060.stdout:2/417: write dd/d11/f88 [488916,22845] 0 2024-02-26T10:47:59.328 INFO:tasks.workunit.client.0.smithi060.stdout:2/418: mknod dd/d43/d55/d6e/c8b 0 2024-02-26T10:47:59.328 INFO:tasks.workunit.client.0.smithi060.stdout:4/453: dwrite d0/da/d3b/d16/f72 [0,4194304] 0 2024-02-26T10:47:59.333 INFO:tasks.workunit.client.0.smithi060.stdout:0/421: sync 2024-02-26T10:47:59.333 INFO:tasks.workunit.client.0.smithi060.stdout:1/539: sync 2024-02-26T10:47:59.333 INFO:tasks.workunit.client.0.smithi060.stdout:6/520: sync 2024-02-26T10:47:59.333 INFO:tasks.workunit.client.0.smithi060.stdout:3/360: sync 2024-02-26T10:47:59.335 INFO:tasks.workunit.client.0.smithi060.stdout:1/540: unlink d3/lb5 0 2024-02-26T10:47:59.335 INFO:tasks.workunit.client.0.smithi060.stdout:1/541: chown d3/db/c16 984 1 2024-02-26T10:47:59.336 INFO:tasks.workunit.client.0.smithi060.stdout:1/542: mkdir d3/d35/d53/d68/d85/d8a/dbe 0 2024-02-26T10:47:59.344 INFO:tasks.workunit.client.0.smithi060.stdout:8/456: dwrite d9/d15/f27 [0,4194304] 0 2024-02-26T10:47:59.344 INFO:tasks.workunit.client.0.smithi060.stdout:8/457: chown d9/d15/d29/d2f/d34/d72/d50/c54 19671549 1 2024-02-26T10:47:59.345 INFO:tasks.workunit.client.0.smithi060.stdout:6/521: rmdir d6/d6e 39 2024-02-26T10:47:59.345 INFO:tasks.workunit.client.0.smithi060.stdout:6/522: write d6/f9 [443624,25847] 0 2024-02-26T10:47:59.345 INFO:tasks.workunit.client.0.smithi060.stdout:3/361: getdents d5/de/df/d57 0 2024-02-26T10:47:59.346 INFO:tasks.workunit.client.0.smithi060.stdout:8/458: creat d9/d5a/d58/f8f x:0 0 0 2024-02-26T10:47:59.347 INFO:tasks.workunit.client.0.smithi060.stdout:8/459: mkdir d9/d15/d29/d2f/d34/d72/d50/d90 0 2024-02-26T10:47:59.348 INFO:tasks.workunit.client.0.smithi060.stdout:3/362: creat d5/de/df/d11/d63/f64 x:0 0 0 2024-02-26T10:47:59.348 INFO:tasks.workunit.client.0.smithi060.stdout:8/460: fdatasync d9/fc 0 2024-02-26T10:47:59.356 INFO:tasks.workunit.client.0.smithi060.stdout:4/454: write d0/d27/f83 [2439226,40584] 0 2024-02-26T10:47:59.360 INFO:tasks.workunit.client.0.smithi060.stdout:4/455: getdents d0/da/d3b/d16 0 2024-02-26T10:47:59.361 INFO:tasks.workunit.client.0.smithi060.stdout:1/543: rename d3/d35/d76/lab to d3/db/d1e/d38/lbf 0 2024-02-26T10:47:59.373 INFO:tasks.workunit.client.0.smithi060.stdout:9/545: dwrite d3/d1e/f82 [0,4194304] 0 2024-02-26T10:47:59.375 INFO:tasks.workunit.client.0.smithi060.stdout:9/546: getdents d3/d1e/d5e/d1b/d23 0 2024-02-26T10:47:59.378 INFO:tasks.workunit.client.0.smithi060.stdout:9/547: unlink d3/d1e/d5e/d35/d87/d9b/f48 0 2024-02-26T10:47:59.381 INFO:tasks.workunit.client.0.smithi060.stdout:9/548: truncate d3/d1e/d5e/d1b/d5f/f38 3223905 0 2024-02-26T10:47:59.383 INFO:tasks.workunit.client.0.smithi060.stdout:1/544: dread d3/db/d1e/f3b [0,4194304] 0 2024-02-26T10:47:59.384 INFO:tasks.workunit.client.0.smithi060.stdout:9/549: symlink d3/d1e/d5e/d35/d87/d9b/lb8 0 2024-02-26T10:47:59.385 INFO:tasks.workunit.client.0.smithi060.stdout:8/461: dread d9/f1e [4194304,4194304] 0 2024-02-26T10:47:59.385 INFO:tasks.workunit.client.0.smithi060.stdout:9/550: rmdir d3/d1e 39 2024-02-26T10:47:59.387 INFO:tasks.workunit.client.0.smithi060.stdout:8/462: creat d9/f91 x:0 0 0 2024-02-26T10:47:59.387 INFO:tasks.workunit.client.0.smithi060.stdout:8/463: write d9/d5a/d58/f7a [4406897,2356] 0 2024-02-26T10:47:59.387 INFO:tasks.workunit.client.0.smithi060.stdout:8/464: truncate d9/d15/d29/d2f/f39 386753 0 2024-02-26T10:47:59.388 INFO:tasks.workunit.client.0.smithi060.stdout:2/419: dwrite dd/f68 [0,4194304] 0 2024-02-26T10:47:59.390 INFO:tasks.workunit.client.0.smithi060.stdout:9/551: unlink d3/d1e/d5e/d1b/d5f/d10/fad 0 2024-02-26T10:47:59.391 INFO:tasks.workunit.client.0.smithi060.stdout:9/552: mkdir d3/d1e/d5e/d1b/d5f/dc/d32/d3c/db9 0 2024-02-26T10:47:59.397 INFO:tasks.workunit.client.0.smithi060.stdout:3/363: dwrite d5/f6 [4194304,4194304] 0 2024-02-26T10:47:59.401 INFO:tasks.workunit.client.0.smithi060.stdout:9/553: write d3/d1e/d5e/d1b/d5f/dc/d32/d3a/f80 [4137059,45847] 0 2024-02-26T10:47:59.409 INFO:tasks.workunit.client.0.smithi060.stdout:1/545: write d3/f15 [4088387,124382] 0 2024-02-26T10:47:59.412 INFO:tasks.workunit.client.0.smithi060.stdout:1/546: mkdir d3/d54/dc0 0 2024-02-26T10:47:59.416 INFO:tasks.workunit.client.0.smithi060.stdout:2/420: dread dd/d43/d60/f1e [0,4194304] 0 2024-02-26T10:47:59.418 INFO:tasks.workunit.client.0.smithi060.stdout:2/421: mknod dd/d52/c8c 0 2024-02-26T10:47:59.418 INFO:tasks.workunit.client.0.smithi060.stdout:2/422: read - dd/d43/d60/d41/d37/d69/f7e zero size 2024-02-26T10:47:59.419 INFO:tasks.workunit.client.0.smithi060.stdout:2/423: creat dd/f8d x:0 0 0 2024-02-26T10:47:59.419 INFO:tasks.workunit.client.0.smithi060.stdout:2/424: truncate dd/d11/f88 843055 0 2024-02-26T10:47:59.419 INFO:tasks.workunit.client.0.smithi060.stdout:2/425: write dd/d43/d60/d41/d37/f80 [436030,61239] 0 2024-02-26T10:47:59.421 INFO:tasks.workunit.client.0.smithi060.stdout:2/426: creat dd/d43/d55/f8e x:0 0 0 2024-02-26T10:47:59.421 INFO:tasks.workunit.client.0.smithi060.stdout:2/427: write dd/d11/f88 [1516206,69855] 0 2024-02-26T10:47:59.426 INFO:tasks.workunit.client.0.smithi060.stdout:2/428: rmdir dd/d43/d60/d41 39 2024-02-26T10:47:59.427 INFO:tasks.workunit.client.0.smithi060.stdout:1/547: link d3/db/c21 d3/d35/d53/d68/d85/d8a/cc1 0 2024-02-26T10:47:59.427 INFO:tasks.workunit.client.0.smithi060.stdout:2/429: mkdir dd/d43/d60/d41/d42/d6f/d8f 0 2024-02-26T10:47:59.428 INFO:tasks.workunit.client.0.smithi060.stdout:2/430: rmdir dd/d43/d60/d41/d37/d4c 39 2024-02-26T10:47:59.428 INFO:tasks.workunit.client.0.smithi060.stdout:1/548: mkdir d3/d35/dc2 0 2024-02-26T10:47:59.429 INFO:tasks.workunit.client.0.smithi060.stdout:1/549: creat d3/db/d1e/d6d/fc3 x:0 0 0 2024-02-26T10:47:59.430 INFO:tasks.workunit.client.0.smithi060.stdout:1/550: mkdir d3/db/d1e/d6d/d78/dc4 0 2024-02-26T10:47:59.431 INFO:tasks.workunit.client.0.smithi060.stdout:2/431: mknod dd/c90 0 2024-02-26T10:47:59.433 INFO:tasks.workunit.client.0.smithi060.stdout:2/432: mkdir dd/d11/d91 0 2024-02-26T10:47:59.438 INFO:tasks.workunit.client.0.smithi060.stdout:9/554: dread d3/d1e/d5e/d1b/d5f/d10/f11 [0,4194304] 0 2024-02-26T10:47:59.441 INFO:tasks.workunit.client.0.smithi060.stdout:8/465: dwrite d9/d15/d3e/f88 [0,4194304] 0 2024-02-26T10:47:59.442 INFO:tasks.workunit.client.0.smithi060.stdout:9/555: mknod d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/cba 0 2024-02-26T10:47:59.445 INFO:tasks.workunit.client.0.smithi060.stdout:3/364: dwrite d5/de/df/d11/d47/f55 [0,4194304] 0 2024-02-26T10:47:59.449 INFO:tasks.workunit.client.0.smithi060.stdout:9/556: dread d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/f70 [0,4194304] 0 2024-02-26T10:47:59.456 INFO:tasks.workunit.client.0.smithi060.stdout:8/466: mknod d9/d15/d29/d2f/d34/d72/c92 0 2024-02-26T10:47:59.458 INFO:tasks.workunit.client.0.smithi060.stdout:8/467: rmdir d9/d15/d29/d2f/d34/d72/d50/d90 0 2024-02-26T10:47:59.462 INFO:tasks.workunit.client.0.smithi060.stdout:8/468: rmdir d9/d15/d29/d2f/d34/d72/d50/d6a/d63 39 2024-02-26T10:47:59.462 INFO:tasks.workunit.client.0.smithi060.stdout:9/557: truncate d3/d1e/d5e/d1b/d5f/d10/f4e 979457 0 2024-02-26T10:47:59.462 INFO:tasks.workunit.client.0.smithi060.stdout:1/551: write d3/d35/d53/d57/f83 [4545817,73572] 0 2024-02-26T10:47:59.463 INFO:tasks.workunit.client.0.smithi060.stdout:9/558: mkdir d3/d1e/dbb 0 2024-02-26T10:47:59.464 INFO:tasks.workunit.client.0.smithi060.stdout:9/559: creat d3/d1e/d5e/d9e/fbc x:0 0 0 2024-02-26T10:47:59.464 INFO:tasks.workunit.client.0.smithi060.stdout:9/560: truncate d3/d1e/d5e/d1b/d5f/fb6 733132 0 2024-02-26T10:47:59.470 INFO:tasks.workunit.client.0.smithi060.stdout:9/561: creat d3/d1e/d5e/d1b/d5f/dc/d32/d3c/fbd x:0 0 0 2024-02-26T10:47:59.479 INFO:tasks.workunit.client.0.smithi060.stdout:3/365: read d5/de/df/d11/d47/f55 [2208737,107131] 0 2024-02-26T10:47:59.482 INFO:tasks.workunit.client.0.smithi060.stdout:5/481: sync 2024-02-26T10:47:59.482 INFO:tasks.workunit.client.0.smithi060.stdout:5/482: fsync d0/f14 0 2024-02-26T10:47:59.489 INFO:tasks.workunit.client.0.smithi060.stdout:1/552: rename d3/db/d2e/d51/ca2 to d3/d35/d53/d68/d85/d8a/dbe/cc5 0 2024-02-26T10:47:59.489 INFO:tasks.workunit.client.0.smithi060.stdout:8/469: read d9/d5a/d58/f7a [879854,36256] 0 2024-02-26T10:47:59.489 INFO:tasks.workunit.client.0.smithi060.stdout:8/470: chown d9/d15/d29/d2f/d34/d72/c92 0 1 2024-02-26T10:47:59.489 INFO:tasks.workunit.client.0.smithi060.stdout:1/553: mknod d3/d54/cc6 0 2024-02-26T10:47:59.491 INFO:tasks.workunit.client.0.smithi060.stdout:1/554: creat d3/db/d2e/d51/fc7 x:0 0 0 2024-02-26T10:47:59.492 INFO:tasks.workunit.client.0.smithi060.stdout:8/471: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f93 x:0 0 0 2024-02-26T10:47:59.494 INFO:tasks.workunit.client.0.smithi060.stdout:8/472: mkdir d9/d15/d29/d2f/d34/d72/d94 0 2024-02-26T10:47:59.494 INFO:tasks.workunit.client.0.smithi060.stdout:8/473: fsync d9/d5a/f69 0 2024-02-26T10:47:59.494 INFO:tasks.workunit.client.0.smithi060.stdout:8/474: chown d9/d15/d3e/l41 13 1 2024-02-26T10:47:59.494 INFO:tasks.workunit.client.0.smithi060.stdout:8/475: chown d9/d15/l6c 17392146 1 2024-02-26T10:47:59.494 INFO:tasks.workunit.client.0.smithi060.stdout:2/433: dwrite dd/d11/f25 [0,4194304] 0 2024-02-26T10:47:59.495 INFO:tasks.workunit.client.0.smithi060.stdout:8/476: fsync d9/f14 0 2024-02-26T10:47:59.495 INFO:tasks.workunit.client.0.smithi060.stdout:8/477: truncate f8 54290 0 2024-02-26T10:47:59.495 INFO:tasks.workunit.client.0.smithi060.stdout:8/478: fdatasync d9/d15/d29/d2f/d34/d72/d50/d6a/f46 0 2024-02-26T10:47:59.504 INFO:tasks.workunit.client.0.smithi060.stdout:9/562: dread d3/f16 [0,4194304] 0 2024-02-26T10:47:59.512 INFO:tasks.workunit.client.0.smithi060.stdout:1/555: rename d3/d35/d53/d68/d85/d8a to d3/db/dc8 0 2024-02-26T10:47:59.515 INFO:tasks.workunit.client.0.smithi060.stdout:3/366: dread d5/de/f3b [0,4194304] 0 2024-02-26T10:47:59.540 INFO:tasks.workunit.client.0.smithi060.stdout:2/434: dwrite dd/d43/d60/d41/d37/d38/d57/f87 [0,4194304] 0 2024-02-26T10:47:59.565 INFO:tasks.workunit.client.0.smithi060.stdout:9/563: dwrite d3/d1e/d5e/d1b/d5f/dc/d32/d3c/d47/d7a/fb5 [0,4194304] 0 2024-02-26T10:47:59.565 INFO:tasks.workunit.client.0.smithi060.stdout:9/564: readlink d3/d1e/d5e/d1b/d5f/dc/d32/d3c/lac 0 2024-02-26T10:47:59.565 INFO:tasks.workunit.client.0.smithi060.stdout:3/367: dread d5/de/df/d11/f34 [0,4194304] 0 2024-02-26T10:47:59.565 INFO:tasks.workunit.client.0.smithi060.stdout:9/565: readlink d3/d1e/d5e/d1b/d5f/dc/d32/l78 0 2024-02-26T10:47:59.565 INFO:tasks.workunit.client.0.smithi060.stdout:8/479: dwrite d9/d15/f36 [0,4194304] 0 2024-02-26T10:47:59.565 INFO:tasks.workunit.client.0.smithi060.stdout:2/435: dread dd/d11/f26 [0,4194304] 0 2024-02-26T10:47:59.566 INFO:tasks.workunit.client.0.smithi060.stdout:8/480: mkdir d9/d15/d29/d95 0 2024-02-26T10:47:59.568 INFO:tasks.workunit.client.0.smithi060.stdout:8/481: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/f96 x:0 0 0 2024-02-26T10:47:59.572 INFO:tasks.workunit.client.0.smithi060.stdout:5/483: dwrite d0/f3a [0,4194304] 0 2024-02-26T10:47:59.572 INFO:tasks.workunit.client.0.smithi060.stdout:5/484: write d0/d33/fa1 [1171120,10169] 0 2024-02-26T10:47:59.575 INFO:tasks.workunit.client.0.smithi060.stdout:5/485: rmdir d0/de/d20/d32/dab 0 2024-02-26T10:47:59.575 INFO:tasks.workunit.client.0.smithi060.stdout:5/486: write d0/d33/f63 [4500429,12799] 0 2024-02-26T10:47:59.576 INFO:tasks.workunit.client.0.smithi060.stdout:5/487: read d0/de/d15/f3b [5369,4460] 0 2024-02-26T10:47:59.576 INFO:tasks.workunit.client.0.smithi060.stdout:3/368: write d5/d37/f38 [936836,80361] 0 2024-02-26T10:47:59.580 INFO:tasks.workunit.client.0.smithi060.stdout:3/369: symlink d5/l65 0 2024-02-26T10:47:59.587 INFO:tasks.workunit.client.0.smithi060.stdout:2/436: read dd/d11/f4e [3090042,50237] 0 2024-02-26T10:47:59.596 INFO:tasks.workunit.client.0.smithi060.stdout:3/370: dread f3 [0,4194304] 0 2024-02-26T10:47:59.596 INFO:tasks.workunit.client.0.smithi060.stdout:3/371: write d5/f24 [750388,47140] 0 2024-02-26T10:47:59.597 INFO:tasks.workunit.client.0.smithi060.stdout:9/566: rename d3/d1e/d5e/d1b/d5f to d3/d1e/d5e/d35/d87/d9b/d8a/dbe 0 2024-02-26T10:47:59.597 INFO:tasks.workunit.client.0.smithi060.stdout:5/488: dread d0/de/d20/f24 [0,4194304] 0 2024-02-26T10:47:59.598 INFO:tasks.workunit.client.0.smithi060.stdout:7/601: sync 2024-02-26T10:47:59.598 INFO:tasks.workunit.client.0.smithi060.stdout:3/372: symlink d5/de/df/d57/d60/l66 0 2024-02-26T10:47:59.601 INFO:tasks.workunit.client.0.smithi060.stdout:9/567: link d3/d1e/c50 d3/d1e/d57/d9c/cbf 0 2024-02-26T10:47:59.601 INFO:tasks.workunit.client.0.smithi060.stdout:8/482: write d9/f20 [7163748,116345] 0 2024-02-26T10:47:59.601 INFO:tasks.workunit.client.0.smithi060.stdout:3/373: rmdir d5/de/df/d57/d5e 0 2024-02-26T10:47:59.602 INFO:tasks.workunit.client.0.smithi060.stdout:3/374: chown d5/l65 24604145 1 2024-02-26T10:47:59.602 INFO:tasks.workunit.client.0.smithi060.stdout:1/556: dwrite d3/f1b [0,4194304] 0 2024-02-26T10:47:59.602 INFO:tasks.workunit.client.0.smithi060.stdout:5/489: creat d0/de/d15/d65/d86/d9b/fad x:0 0 0 2024-02-26T10:47:59.602 INFO:tasks.workunit.client.0.smithi060.stdout:5/490: readlink d0/de/d15/d65/l93 0 2024-02-26T10:47:59.603 INFO:tasks.workunit.client.0.smithi060.stdout:9/568: symlink d3/d1e/d57/lc0 0 2024-02-26T10:47:59.603 INFO:tasks.workunit.client.0.smithi060.stdout:9/569: truncate d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f6c 902005 0 2024-02-26T10:47:59.604 INFO:tasks.workunit.client.0.smithi060.stdout:1/557: mkdir d3/d35/d53/d68/d6b/dc9 0 2024-02-26T10:47:59.605 INFO:tasks.workunit.client.0.smithi060.stdout:5/491: symlink d0/de/d15/d65/lae 0 2024-02-26T10:47:59.611 INFO:tasks.workunit.client.0.smithi060.stdout:3/375: mknod d5/de/c67 0 2024-02-26T10:47:59.612 INFO:tasks.workunit.client.0.smithi060.stdout:7/602: stat da/d15/d21/d29 0 2024-02-26T10:47:59.612 INFO:tasks.workunit.client.0.smithi060.stdout:3/376: write d5/d45/f49 [5197312,83850] 0 2024-02-26T10:47:59.612 INFO:tasks.workunit.client.0.smithi060.stdout:2/437: dread dd/d11/f88 [0,4194304] 0 2024-02-26T10:47:59.614 INFO:tasks.workunit.client.0.smithi060.stdout:7/603: creat da/d15/fcc x:0 0 0 2024-02-26T10:47:59.614 INFO:tasks.workunit.client.0.smithi060.stdout:7/604: chown da/d15/d88/d3b/fc9 71 1 2024-02-26T10:47:59.615 INFO:tasks.workunit.client.0.smithi060.stdout:4/456: sync 2024-02-26T10:47:59.615 INFO:tasks.workunit.client.0.smithi060.stdout:6/523: sync 2024-02-26T10:47:59.615 INFO:tasks.workunit.client.0.smithi060.stdout:0/422: sync 2024-02-26T10:47:59.615 INFO:tasks.workunit.client.0.smithi060.stdout:0/423: chown d0/l14 213107 1 2024-02-26T10:47:59.615 INFO:tasks.workunit.client.0.smithi060.stdout:0/424: readlink d0/dd/d39/d40/l20 0 2024-02-26T10:47:59.615 INFO:tasks.workunit.client.0.smithi060.stdout:6/524: fsync d6/d7/d25/f49 0 2024-02-26T10:47:59.615 INFO:tasks.workunit.client.0.smithi060.stdout:2/438: creat dd/d43/d60/d41/d37/d4c/f92 x:0 0 0 2024-02-26T10:47:59.615 INFO:tasks.workunit.client.0.smithi060.stdout:2/439: chown dd/d43/d60/d41/d37/l75 7759 1 2024-02-26T10:47:59.616 INFO:tasks.workunit.client.0.smithi060.stdout:3/377: rename d5/de/df/d11/d47 to d5/de/df/d4c/d68 0 2024-02-26T10:47:59.616 INFO:tasks.workunit.client.0.smithi060.stdout:2/440: symlink dd/d43/d60/d41/l93 0 2024-02-26T10:47:59.616 INFO:tasks.workunit.client.0.smithi060.stdout:2/441: write dd/d43/d60/f3f [7823097,116297] 0 2024-02-26T10:47:59.617 INFO:tasks.workunit.client.0.smithi060.stdout:4/457: creat d0/d7/d29/f87 x:0 0 0 2024-02-26T10:47:59.617 INFO:tasks.workunit.client.0.smithi060.stdout:4/458: fdatasync d0/da/d3b/d1b/f86 0 2024-02-26T10:47:59.617 INFO:tasks.workunit.client.0.smithi060.stdout:4/459: write d0/d27/f5e [282061,61044] 0 2024-02-26T10:47:59.618 INFO:tasks.workunit.client.0.smithi060.stdout:0/425: symlink d0/dd/d53/l8e 0 2024-02-26T10:47:59.618 INFO:tasks.workunit.client.0.smithi060.stdout:0/426: fsync d0/dd/d39/f7e 0 2024-02-26T10:47:59.619 INFO:tasks.workunit.client.0.smithi060.stdout:6/525: creat d6/d7/d25/d80/fae x:0 0 0 2024-02-26T10:47:59.620 INFO:tasks.workunit.client.0.smithi060.stdout:2/442: unlink dd/d11/f24 0 2024-02-26T10:47:59.621 INFO:tasks.workunit.client.0.smithi060.stdout:2/443: creat dd/f94 x:0 0 0 2024-02-26T10:47:59.622 INFO:tasks.workunit.client.0.smithi060.stdout:0/427: rmdir d0/d37/d44/d58 39 2024-02-26T10:47:59.623 INFO:tasks.workunit.client.0.smithi060.stdout:2/444: creat dd/d43/d60/d41/f95 x:0 0 0 2024-02-26T10:47:59.623 INFO:tasks.workunit.client.0.smithi060.stdout:6/526: creat d6/d7/d25/d1c/d1f/d23/faf x:0 0 0 2024-02-26T10:47:59.624 INFO:tasks.workunit.client.0.smithi060.stdout:6/527: write d6/d7/d25/d1c/d1f/d23/d28/f7f [419425,115238] 0 2024-02-26T10:47:59.626 INFO:tasks.workunit.client.0.smithi060.stdout:0/428: creat d0/dd/d39/d40/d83/f8f x:0 0 0 2024-02-26T10:47:59.628 INFO:tasks.workunit.client.0.smithi060.stdout:0/429: mknod d0/dd/d33/c90 0 2024-02-26T10:47:59.647 INFO:tasks.workunit.client.0.smithi060.stdout:4/460: rename d0/da/d3b/d16/l82 to d0/d27/d52/l88 0 2024-02-26T10:47:59.647 INFO:tasks.workunit.client.0.smithi060.stdout:0/430: dread d0/dd/d39/d40/f8d [0,4194304] 0 2024-02-26T10:47:59.647 INFO:tasks.workunit.client.0.smithi060.stdout:1/558: dwrite d3/d35/d53/d57/f83 [0,4194304] 0 2024-02-26T10:47:59.648 INFO:tasks.workunit.client.0.smithi060.stdout:0/431: symlink d0/dd/d53/d87/l91 0 2024-02-26T10:47:59.652 INFO:tasks.workunit.client.0.smithi060.stdout:8/483: dread d9/d15/d29/d2f/d34/d72/f62 [0,4194304] 0 2024-02-26T10:47:59.655 INFO:tasks.workunit.client.0.smithi060.stdout:4/461: getdents d0/da/d3b/d1b 0 2024-02-26T10:47:59.656 INFO:tasks.workunit.client.0.smithi060.stdout:8/484: symlink d9/d15/d3e/l97 0 2024-02-26T10:47:59.656 INFO:tasks.workunit.client.0.smithi060.stdout:8/485: chown d9/d6f 67308 1 2024-02-26T10:47:59.657 INFO:tasks.workunit.client.0.smithi060.stdout:1/559: getdents d3/d35/d53/d68/dad 0 2024-02-26T10:47:59.657 INFO:tasks.workunit.client.0.smithi060.stdout:1/560: truncate d3/db/f56 1616651 0 2024-02-26T10:47:59.657 INFO:tasks.workunit.client.0.smithi060.stdout:1/561: read d3/f69 [200235,106942] 0 2024-02-26T10:47:59.658 INFO:tasks.workunit.client.0.smithi060.stdout:8/486: creat d9/d15/d29/d2f/f98 x:0 0 0 2024-02-26T10:47:59.660 INFO:tasks.workunit.client.0.smithi060.stdout:8/487: mknod d9/d15/c99 0 2024-02-26T10:47:59.665 INFO:tasks.workunit.client.0.smithi060.stdout:1/562: mknod d3/db/d1e/d38/d6c/cca 0 2024-02-26T10:47:59.667 INFO:tasks.workunit.client.0.smithi060.stdout:1/563: unlink d3/db/d2e/d51/l6f 0 2024-02-26T10:47:59.673 INFO:tasks.workunit.client.0.smithi060.stdout:8/488: link d9/d15/d29/l4e d9/d5a/d58/l9a 0 2024-02-26T10:47:59.679 INFO:tasks.workunit.client.0.smithi060.stdout:7/605: dwrite da/d15/d21/d4b/d67/f77 [4194304,4194304] 0 2024-02-26T10:47:59.681 INFO:tasks.workunit.client.0.smithi060.stdout:3/378: dwrite d5/de/df/d57/d60/f3a [0,4194304] 0 2024-02-26T10:47:59.682 INFO:tasks.workunit.client.0.smithi060.stdout:3/379: fdatasync d5/d45/f48 0 2024-02-26T10:47:59.682 INFO:tasks.workunit.client.0.smithi060.stdout:3/380: fsync d5/f25 0 2024-02-26T10:47:59.684 INFO:tasks.workunit.client.0.smithi060.stdout:7/606: rename da/fd to da/d66/fcd 0 2024-02-26T10:47:59.686 INFO:tasks.workunit.client.0.smithi060.stdout:3/381: rmdir d5/de 39 2024-02-26T10:47:59.686 INFO:tasks.workunit.client.0.smithi060.stdout:3/382: dread - d5/de/df/d57/d60/f51 zero size 2024-02-26T10:47:59.686 INFO:tasks.workunit.client.0.smithi060.stdout:3/383: chown d5/de/df/d11/l12 840364674 1 2024-02-26T10:47:59.687 INFO:tasks.workunit.client.0.smithi060.stdout:4/462: dread d0/d27/d2f/f6e [0,4194304] 0 2024-02-26T10:47:59.688 INFO:tasks.workunit.client.0.smithi060.stdout:7/607: truncate da/f60 1653073 0 2024-02-26T10:47:59.689 INFO:tasks.workunit.client.0.smithi060.stdout:3/384: truncate d5/de/f14 6772114 0 2024-02-26T10:47:59.689 INFO:tasks.workunit.client.0.smithi060.stdout:3/385: fsync d5/de/df/d11/f20 0 2024-02-26T10:47:59.695 INFO:tasks.workunit.client.0.smithi060.stdout:7/608: unlink da/d15/d21/d32/f47 0 2024-02-26T10:47:59.696 INFO:tasks.workunit.client.0.smithi060.stdout:1/564: read d3/db/d2e/f33 [485716,47696] 0 2024-02-26T10:47:59.705 INFO:tasks.workunit.client.0.smithi060.stdout:2/445: dwrite dd/d43/f6c [0,4194304] 0 2024-02-26T10:47:59.705 INFO:tasks.workunit.client.0.smithi060.stdout:0/432: dwrite d0/dd/f4e [0,4194304] 0 2024-02-26T10:47:59.706 INFO:tasks.workunit.client.0.smithi060.stdout:0/433: chown d0/dd/f2d 320223 1 2024-02-26T10:47:59.706 INFO:tasks.workunit.client.0.smithi060.stdout:3/386: link d5/l65 d5/d45/l69 0 2024-02-26T10:47:59.707 INFO:tasks.workunit.client.0.smithi060.stdout:8/489: write d9/d15/f60 [2419241,130229] 0 2024-02-26T10:47:59.707 INFO:tasks.workunit.client.0.smithi060.stdout:8/490: write d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f82 [506113,126639] 0 2024-02-26T10:47:59.708 INFO:tasks.workunit.client.0.smithi060.stdout:6/528: dwrite d6/f7c [0,4194304] 0 2024-02-26T10:47:59.709 INFO:tasks.workunit.client.0.smithi060.stdout:0/434: mknod d0/c92 0 2024-02-26T10:47:59.709 INFO:tasks.workunit.client.0.smithi060.stdout:0/435: chown d0/l22 25 1 2024-02-26T10:47:59.709 INFO:tasks.workunit.client.0.smithi060.stdout:0/436: fsync d0/dd/d39/d40/f8d 0 2024-02-26T10:47:59.709 INFO:tasks.workunit.client.0.smithi060.stdout:2/446: unlink dd/d11/f88 0 2024-02-26T10:47:59.710 INFO:tasks.workunit.client.0.smithi060.stdout:2/447: mkdir dd/d43/d96 0 2024-02-26T10:47:59.710 INFO:tasks.workunit.client.0.smithi060.stdout:2/448: stat dd/c90 0 2024-02-26T10:47:59.710 INFO:tasks.workunit.client.0.smithi060.stdout:3/387: mknod d5/d37/c6a 0 2024-02-26T10:47:59.711 INFO:tasks.workunit.client.0.smithi060.stdout:7/609: rename da/d15/d21/d32 to da/d15/d88/dce 0 2024-02-26T10:47:59.711 INFO:tasks.workunit.client.0.smithi060.stdout:2/449: mkdir dd/d43/d55/d5e/d86/d97 0 2024-02-26T10:47:59.712 INFO:tasks.workunit.client.0.smithi060.stdout:1/565: rmdir d3/db 39 2024-02-26T10:47:59.712 INFO:tasks.workunit.client.0.smithi060.stdout:1/566: write d3/d35/d53/d68/d85/f94 [1283437,41467] 0 2024-02-26T10:47:59.714 INFO:tasks.workunit.client.0.smithi060.stdout:8/491: getdents d9/d15/d3e 0 2024-02-26T10:47:59.717 INFO:tasks.workunit.client.0.smithi060.stdout:0/437: mkdir d0/dd/d39/d93 0 2024-02-26T10:47:59.717 INFO:tasks.workunit.client.0.smithi060.stdout:0/438: write d0/dd/d39/f7c [514372,33946] 0 2024-02-26T10:47:59.719 INFO:tasks.workunit.client.0.smithi060.stdout:1/567: truncate d3/f2b 304171 0 2024-02-26T10:47:59.720 INFO:tasks.workunit.client.0.smithi060.stdout:2/450: symlink dd/d11/l98 0 2024-02-26T10:47:59.721 INFO:tasks.workunit.client.0.smithi060.stdout:2/451: fdatasync dd/d43/d60/d41/d37/d38/d57/f87 0 2024-02-26T10:47:59.721 INFO:tasks.workunit.client.0.smithi060.stdout:0/439: mkdir d0/dd/d53/d87/d94 0 2024-02-26T10:47:59.721 INFO:tasks.workunit.client.0.smithi060.stdout:8/492: dread d9/d5a/f49 [0,4194304] 0 2024-02-26T10:47:59.722 INFO:tasks.workunit.client.0.smithi060.stdout:1/568: creat d3/db/d1e/d38/d6c/fcb x:0 0 0 2024-02-26T10:47:59.722 INFO:tasks.workunit.client.0.smithi060.stdout:1/569: chown d3/db/d1e/d38/cb2 1230 1 2024-02-26T10:47:59.722 INFO:tasks.workunit.client.0.smithi060.stdout:7/610: mknod da/d15/d21/d2f/ccf 0 2024-02-26T10:47:59.722 INFO:tasks.workunit.client.0.smithi060.stdout:7/611: readlink da/l11 0 2024-02-26T10:47:59.722 INFO:tasks.workunit.client.0.smithi060.stdout:2/452: symlink dd/d43/l99 0 2024-02-26T10:47:59.722 INFO:tasks.workunit.client.0.smithi060.stdout:2/453: getdents dd/d11/d91 0 2024-02-26T10:47:59.723 INFO:tasks.workunit.client.0.smithi060.stdout:0/440: mknod d0/dd/d39/d93/c95 0 2024-02-26T10:47:59.725 INFO:tasks.workunit.client.0.smithi060.stdout:0/441: symlink d0/dd/d53/d87/l96 0 2024-02-26T10:47:59.725 INFO:tasks.workunit.client.0.smithi060.stdout:0/442: chown d0/dd/f2d 1579013823 1 2024-02-26T10:47:59.725 INFO:tasks.workunit.client.0.smithi060.stdout:0/443: write d0/dd/f2b [1528074,32756] 0 2024-02-26T10:47:59.725 INFO:tasks.workunit.client.0.smithi060.stdout:0/444: chown d0/f15 4434 1 2024-02-26T10:47:59.727 INFO:tasks.workunit.client.0.smithi060.stdout:0/445: getdents d0/dd/d33/d6a 0 2024-02-26T10:47:59.729 INFO:tasks.workunit.client.0.smithi060.stdout:0/446: creat d0/dd/d33/d6a/d6c/f97 x:0 0 0 2024-02-26T10:47:59.734 INFO:tasks.workunit.client.0.smithi060.stdout:2/454: dread dd/d43/d60/d41/d37/d38/f40 [0,4194304] 0 2024-02-26T10:47:59.736 INFO:tasks.workunit.client.0.smithi060.stdout:4/463: dwrite d0/da/d57/f58 [0,4194304] 0 2024-02-26T10:47:59.736 INFO:tasks.workunit.client.0.smithi060.stdout:4/464: chown d0/fb 112917 1 2024-02-26T10:47:59.737 INFO:tasks.workunit.client.0.smithi060.stdout:3/388: dread d5/fd [0,4194304] 0 2024-02-26T10:47:59.739 INFO:tasks.workunit.client.0.smithi060.stdout:0/447: dread d0/d37/f6d [0,4194304] 0 2024-02-26T10:47:59.739 INFO:tasks.workunit.client.0.smithi060.stdout:4/465: truncate d0/d27/f5e 7673 0 2024-02-26T10:47:59.751 INFO:tasks.workunit.client.0.smithi060.stdout:1/570: rename d3/d54/l7d to d3/d35/d53/d68/d6b/dc9/lcc 0 2024-02-26T10:47:59.758 INFO:tasks.workunit.client.0.smithi060.stdout:7/612: dread da/d15/d21/d2f/f68 [0,4194304] 0 2024-02-26T10:47:59.758 INFO:tasks.workunit.client.0.smithi060.stdout:7/613: dread - da/d15/d88/d3b/f8c zero size 2024-02-26T10:47:59.760 INFO:tasks.workunit.client.0.smithi060.stdout:1/571: rename d3/db/d1e/d39/d9b/da6 to d3/db/d1e/d6d/da3/daa/dcd 0 2024-02-26T10:47:59.760 INFO:tasks.workunit.client.0.smithi060.stdout:3/389: dread d5/de/df/d11/f1c [0,4194304] 0 2024-02-26T10:47:59.767 INFO:tasks.workunit.client.0.smithi060.stdout:4/466: link d0/f5 d0/da/d3b/f89 0 2024-02-26T10:47:59.777 INFO:tasks.workunit.client.0.smithi060.stdout:4/467: creat d0/d27/d76/f8a x:0 0 0 2024-02-26T10:47:59.777 INFO:tasks.workunit.client.0.smithi060.stdout:4/468: getdents d0/da/d3b/d1b 0 2024-02-26T10:47:59.782 INFO:tasks.workunit.client.0.smithi060.stdout:4/469: write d0/d27/d52/f71 [169949,72518] 0 2024-02-26T10:47:59.784 INFO:tasks.workunit.client.0.smithi060.stdout:4/470: creat d0/da/f8b x:0 0 0 2024-02-26T10:47:59.785 INFO:tasks.workunit.client.0.smithi060.stdout:4/471: truncate d0/da/d57/f58 706274 0 2024-02-26T10:47:59.786 INFO:tasks.workunit.client.0.smithi060.stdout:2/455: dread dd/d43/d60/d41/f1a [0,4194304] 0 2024-02-26T10:47:59.786 INFO:tasks.workunit.client.0.smithi060.stdout:8/493: dwrite d9/f1e [4194304,4194304] 0 2024-02-26T10:47:59.787 INFO:tasks.workunit.client.0.smithi060.stdout:2/456: mkdir dd/d43/d60/d41/d37/d9a 0 2024-02-26T10:47:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:2/457: symlink dd/d43/d60/l9b 0 2024-02-26T10:47:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:2/458: chown dd/d11/l98 1862604740 1 2024-02-26T10:47:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:2/459: write dd/d43/d60/d41/f95 [601352,120514] 0 2024-02-26T10:47:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:4/472: unlink d0/d27/d2f/f4e 0 2024-02-26T10:47:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:4/473: symlink d0/d27/d76/l8c 0 2024-02-26T10:47:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:4/474: mknod d0/d7/c8d 0 2024-02-26T10:47:59.805 INFO:tasks.workunit.client.0.smithi060.stdout:8/494: dread d9/d15/d29/d2f/d34/d72/d50/d6a/f45 [0,4194304] 0 2024-02-26T10:47:59.805 INFO:tasks.workunit.client.0.smithi060.stdout:7/614: dwrite da/fe [0,4194304] 0 2024-02-26T10:47:59.805 INFO:tasks.workunit.client.0.smithi060.stdout:8/495: fdatasync d9/d15/f52 0 2024-02-26T10:47:59.807 INFO:tasks.workunit.client.0.smithi060.stdout:7/615: creat da/d15/d88/d3b/fd0 x:0 0 0 2024-02-26T10:47:59.807 INFO:tasks.workunit.client.0.smithi060.stdout:7/616: fdatasync da/d15/d88/d3b/fd0 0 2024-02-26T10:47:59.808 INFO:tasks.workunit.client.0.smithi060.stdout:8/496: truncate d9/d15/d29/d2f/f4d 3496713 0 2024-02-26T10:47:59.808 INFO:tasks.workunit.client.0.smithi060.stdout:7/617: write da/d66/dad/f56 [18874,115960] 0 2024-02-26T10:47:59.808 INFO:tasks.workunit.client.0.smithi060.stdout:8/497: write d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f8a [850196,88991] 0 2024-02-26T10:47:59.810 INFO:tasks.workunit.client.0.smithi060.stdout:7/618: symlink da/d66/dbd/ld1 0 2024-02-26T10:47:59.810 INFO:tasks.workunit.client.0.smithi060.stdout:8/498: truncate d9/f24 444076 0 2024-02-26T10:47:59.810 INFO:tasks.workunit.client.0.smithi060.stdout:8/499: readlink d9/d5a/d58/l9a 0 2024-02-26T10:47:59.811 INFO:tasks.workunit.client.0.smithi060.stdout:7/619: creat da/d15/d21/fd2 x:0 0 0 2024-02-26T10:47:59.816 INFO:tasks.workunit.client.0.smithi060.stdout:7/620: stat da/d15/d21/d2f/d78 0 2024-02-26T10:47:59.816 INFO:tasks.workunit.client.0.smithi060.stdout:8/500: dread d9/d15/d29/d2f/d34/d72/d50/d6a/f45 [0,4194304] 0 2024-02-26T10:47:59.818 INFO:tasks.workunit.client.0.smithi060.stdout:8/501: creat d9/d15/d3e/f9b x:0 0 0 2024-02-26T10:47:59.819 INFO:tasks.workunit.client.0.smithi060.stdout:2/460: write dd/d43/f6c [3025210,52614] 0 2024-02-26T10:47:59.828 INFO:tasks.workunit.client.0.smithi060.stdout:1/572: write d3/d35/d53/d57/f71 [2786424,12340] 0 2024-02-26T10:47:59.828 INFO:tasks.workunit.client.0.smithi060.stdout:0/448: dwrite d0/dd/d39/f7e [4194304,4194304] 0 2024-02-26T10:47:59.828 INFO:tasks.workunit.client.0.smithi060.stdout:3/390: dwrite d5/d45/f49 [0,4194304] 0 2024-02-26T10:47:59.829 INFO:tasks.workunit.client.0.smithi060.stdout:3/391: truncate d5/de/df/d11/d63/f64 455773 0 2024-02-26T10:47:59.835 INFO:tasks.workunit.client.0.smithi060.stdout:1/573: mkdir d3/db/dc8/dce 0 2024-02-26T10:47:59.835 INFO:tasks.workunit.client.0.smithi060.stdout:1/574: write d3/db/d2e/f95 [928975,33586] 0 2024-02-26T10:47:59.835 INFO:tasks.workunit.client.0.smithi060.stdout:1/575: readlink d3/db/d1e/l31 0 2024-02-26T10:47:59.836 INFO:tasks.workunit.client.0.smithi060.stdout:1/576: truncate d3/d35/d53/d57/f73 405679 0 2024-02-26T10:47:59.836 INFO:tasks.workunit.client.0.smithi060.stdout:1/577: fdatasync d3/f23 0 2024-02-26T10:47:59.837 INFO:tasks.workunit.client.0.smithi060.stdout:7/621: rename da/d15/d21/d4b to da/d15/d21/d2f/d78/dd3 0 2024-02-26T10:47:59.837 INFO:tasks.workunit.client.0.smithi060.stdout:7/622: truncate da/d15/d88/d3b/fc9 366652 0 2024-02-26T10:47:59.837 INFO:tasks.workunit.client.0.smithi060.stdout:9/570: sync 2024-02-26T10:47:59.838 INFO:tasks.workunit.client.0.smithi060.stdout:5/492: sync 2024-02-26T10:47:59.838 INFO:tasks.workunit.client.0.smithi060.stdout:6/529: sync 2024-02-26T10:47:59.839 INFO:tasks.workunit.client.0.smithi060.stdout:9/571: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f6c [897258,110803] 0 2024-02-26T10:47:59.839 INFO:tasks.workunit.client.0.smithi060.stdout:4/475: dread d0/d7/f8 [4194304,4194304] 0 2024-02-26T10:47:59.839 INFO:tasks.workunit.client.0.smithi060.stdout:3/392: rename d5/de/df/d4c/d68/f5a to d5/de/df/d4c/f6b 0 2024-02-26T10:47:59.840 INFO:tasks.workunit.client.0.smithi060.stdout:1/578: symlink d3/db/d1e/d6d/lcf 0 2024-02-26T10:47:59.840 INFO:tasks.workunit.client.0.smithi060.stdout:1/579: stat d3/db/d2e 0 2024-02-26T10:47:59.840 INFO:tasks.workunit.client.0.smithi060.stdout:1/580: fsync d3/db/f59 0 2024-02-26T10:47:59.840 INFO:tasks.workunit.client.0.smithi060.stdout:1/581: write d3/d35/d53/d57/f71 [1670374,43620] 0 2024-02-26T10:47:59.841 INFO:tasks.workunit.client.0.smithi060.stdout:7/623: unlink da/d15/d21/d41/l7b 0 2024-02-26T10:47:59.842 INFO:tasks.workunit.client.0.smithi060.stdout:2/461: dread dd/d43/d55/f66 [0,4194304] 0 2024-02-26T10:47:59.842 INFO:tasks.workunit.client.0.smithi060.stdout:2/462: chown dd/d43/d60/d41/c7f 223 1 2024-02-26T10:47:59.842 INFO:tasks.workunit.client.0.smithi060.stdout:5/493: unlink d0/de/d15/l76 0 2024-02-26T10:47:59.843 INFO:tasks.workunit.client.0.smithi060.stdout:2/463: mkdir dd/d43/d55/d5e/d86/d97/d9c 0 2024-02-26T10:47:59.843 INFO:tasks.workunit.client.0.smithi060.stdout:9/572: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f7f [411702,96404] 0 2024-02-26T10:47:59.844 INFO:tasks.workunit.client.0.smithi060.stdout:5/494: write d0/de/d15/d26/f48 [69864,12620] 0 2024-02-26T10:47:59.847 INFO:tasks.workunit.client.0.smithi060.stdout:9/573: unlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/l99 0 2024-02-26T10:47:59.847 INFO:tasks.workunit.client.0.smithi060.stdout:9/574: dread - d3/d1e/d93/da2/fb3 zero size 2024-02-26T10:47:59.848 INFO:tasks.workunit.client.0.smithi060.stdout:3/393: link d5/de/df/d4c/l4d d5/de/l6c 0 2024-02-26T10:47:59.852 INFO:tasks.workunit.client.0.smithi060.stdout:3/394: rename d5/f25 to d5/de/d16/f6d 0 2024-02-26T10:47:59.852 INFO:tasks.workunit.client.0.smithi060.stdout:3/395: write d5/d37/f38 [77983,94873] 0 2024-02-26T10:47:59.854 INFO:tasks.workunit.client.0.smithi060.stdout:3/396: creat d5/de/df/d11/f6e x:0 0 0 2024-02-26T10:47:59.873 INFO:tasks.workunit.client.0.smithi060.stdout:8/502: dwrite d9/d15/f52 [0,4194304] 0 2024-02-26T10:47:59.874 INFO:tasks.workunit.client.0.smithi060.stdout:9/575: dread d3/d1e/d57/f79 [0,4194304] 0 2024-02-26T10:47:59.875 INFO:tasks.workunit.client.0.smithi060.stdout:8/503: write d9/f23 [4480463,5808] 0 2024-02-26T10:47:59.875 INFO:tasks.workunit.client.0.smithi060.stdout:9/576: chown d3/d1e/d5e/ca1 6098 1 2024-02-26T10:47:59.875 INFO:tasks.workunit.client.0.smithi060.stdout:9/577: readlink d3/d1e/l7b 0 2024-02-26T10:47:59.909 INFO:tasks.workunit.client.0.smithi060.stdout:4/476: dread d0/da/d3b/f1a [0,4194304] 0 2024-02-26T10:47:59.911 INFO:tasks.workunit.client.0.smithi060.stdout:4/477: mkdir d0/d7/d29/d8e 0 2024-02-26T10:47:59.911 INFO:tasks.workunit.client.0.smithi060.stdout:4/478: write d0/f5 [5163634,92001] 0 2024-02-26T10:47:59.917 INFO:tasks.workunit.client.0.smithi060.stdout:4/479: symlink d0/d27/d7f/l8f 0 2024-02-26T10:47:59.917 INFO:tasks.workunit.client.0.smithi060.stdout:5/495: dwrite d0/f90 [0,4194304] 0 2024-02-26T10:47:59.917 INFO:tasks.workunit.client.0.smithi060.stdout:5/496: write d0/de/d15/d26/f78 [1851570,25083] 0 2024-02-26T10:47:59.921 INFO:tasks.workunit.client.0.smithi060.stdout:6/530: dwrite d6/d7/f70 [0,4194304] 0 2024-02-26T10:47:59.921 INFO:tasks.workunit.client.0.smithi060.stdout:6/531: write d6/d7/d25/d1c/d1f/d23/d2b/d4d/f58 [1391802,100479] 0 2024-02-26T10:47:59.924 INFO:tasks.workunit.client.0.smithi060.stdout:7/624: dwrite da/f18 [0,4194304] 0 2024-02-26T10:47:59.927 INFO:tasks.workunit.client.0.smithi060.stdout:7/625: creat da/d15/d88/fd4 x:0 0 0 2024-02-26T10:47:59.928 INFO:tasks.workunit.client.0.smithi060.stdout:5/497: dread d0/d33/f5c [0,4194304] 0 2024-02-26T10:47:59.929 INFO:tasks.workunit.client.0.smithi060.stdout:2/464: dwrite dd/d43/d60/d41/f95 [0,4194304] 0 2024-02-26T10:47:59.929 INFO:tasks.workunit.client.0.smithi060.stdout:7/626: creat da/d15/d21/d41/fd5 x:0 0 0 2024-02-26T10:47:59.931 INFO:tasks.workunit.client.0.smithi060.stdout:7/627: creat da/d94/fd6 x:0 0 0 2024-02-26T10:47:59.936 INFO:tasks.workunit.client.0.smithi060.stdout:3/397: dwrite d5/d37/f38 [0,4194304] 0 2024-02-26T10:47:59.936 INFO:tasks.workunit.client.0.smithi060.stdout:3/398: write d5/de/df/d11/f6e [168008,98644] 0 2024-02-26T10:47:59.937 INFO:tasks.workunit.client.0.smithi060.stdout:5/498: symlink d0/d4b/d7b/laf 0 2024-02-26T10:47:59.937 INFO:tasks.workunit.client.0.smithi060.stdout:5/499: fdatasync d0/d4b/f51 0 2024-02-26T10:47:59.937 INFO:tasks.workunit.client.0.smithi060.stdout:5/500: rename d0/de/f9f to d0/de/d20/d32/d4c/fb0 0 2024-02-26T10:47:59.937 INFO:tasks.workunit.client.0.smithi060.stdout:5/501: chown d0/de/d20/d32/d4c/f74 41253074 1 2024-02-26T10:47:59.937 INFO:tasks.workunit.client.0.smithi060.stdout:3/399: creat d5/de/df/d4c/f6f x:0 0 0 2024-02-26T10:47:59.937 INFO:tasks.workunit.client.0.smithi060.stdout:5/502: creat d0/de/d15/d65/fb1 x:0 0 0 2024-02-26T10:47:59.938 INFO:tasks.workunit.client.0.smithi060.stdout:3/400: creat d5/de/df/d4c/f70 x:0 0 0 2024-02-26T10:47:59.945 INFO:tasks.workunit.client.0.smithi060.stdout:7/628: dread da/d15/f2e [0,4194304] 0 2024-02-26T10:47:59.950 INFO:tasks.workunit.client.0.smithi060.stdout:2/465: link lb dd/d43/d55/d5e/d74/l9d 0 2024-02-26T10:47:59.950 INFO:tasks.workunit.client.0.smithi060.stdout:9/578: dwrite d3/d1e/f58 [0,4194304] 0 2024-02-26T10:47:59.956 INFO:tasks.workunit.client.0.smithi060.stdout:7/629: dread da/d15/d21/f22 [0,4194304] 0 2024-02-26T10:47:59.963 INFO:tasks.workunit.client.0.smithi060.stdout:2/466: creat dd/d43/f9e x:0 0 0 2024-02-26T10:47:59.967 INFO:tasks.workunit.client.0.smithi060.stdout:3/401: dread d5/de/df/d4c/d68/f55 [0,4194304] 0 2024-02-26T10:47:59.967 INFO:tasks.workunit.client.0.smithi060.stdout:9/579: read d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f80 [7549952,54170] 0 2024-02-26T10:47:59.968 INFO:tasks.workunit.client.0.smithi060.stdout:3/402: truncate d5/de/df/d57/d60/f1e 756295 0 2024-02-26T10:47:59.968 INFO:tasks.workunit.client.0.smithi060.stdout:7/630: dread da/d94/fb5 [0,4194304] 0 2024-02-26T10:47:59.969 INFO:tasks.workunit.client.0.smithi060.stdout:3/403: symlink d5/de/df/d4c/l71 0 2024-02-26T10:47:59.969 INFO:tasks.workunit.client.0.smithi060.stdout:3/404: stat d5/de/df/d11/d63/f64 0 2024-02-26T10:47:59.970 INFO:tasks.workunit.client.0.smithi060.stdout:7/631: mkdir da/d15/d21/d2f/d78/dd3/db9/dd7 0 2024-02-26T10:47:59.972 INFO:tasks.workunit.client.0.smithi060.stdout:3/405: link d5/l19 d5/d37/l72 0 2024-02-26T10:47:59.973 INFO:tasks.workunit.client.0.smithi060.stdout:3/406: truncate d5/de/df/d11/f62 3919327 0 2024-02-26T10:47:59.976 INFO:tasks.workunit.client.0.smithi060.stdout:3/407: mknod d5/de/c73 0 2024-02-26T10:47:59.979 INFO:tasks.workunit.client.0.smithi060.stdout:2/467: dread dd/d43/d55/f66 [0,4194304] 0 2024-02-26T10:47:59.979 INFO:tasks.workunit.client.0.smithi060.stdout:2/468: fsync dd/d43/d60/d41/d37/d69/f72 0 2024-02-26T10:47:59.980 INFO:tasks.workunit.client.0.smithi060.stdout:2/469: stat dd/l2d 0 2024-02-26T10:47:59.987 INFO:tasks.workunit.client.0.smithi060.stdout:9/580: rename d3/d1e/d5e/d35/d87/d9b/d8a/dbe/f2e to d3/d1e/d5e/d35/d87/fc1 0 2024-02-26T10:47:59.987 INFO:tasks.workunit.client.0.smithi060.stdout:5/503: dwrite d0/de/d20/d32/f89 [0,4194304] 0 2024-02-26T10:47:59.987 INFO:tasks.workunit.client.0.smithi060.stdout:5/504: readlink d0/de/da4/la7 0 2024-02-26T10:47:59.987 INFO:tasks.workunit.client.0.smithi060.stdout:5/505: truncate d0/de/f46 467295 0 2024-02-26T10:47:59.988 INFO:tasks.workunit.client.0.smithi060.stdout:9/581: mkdir d3/d1e/dbb/dc2 0 2024-02-26T10:47:59.990 INFO:tasks.workunit.client.0.smithi060.stdout:2/470: dread dd/d43/d55/f66 [0,4194304] 0 2024-02-26T10:47:59.990 INFO:tasks.workunit.client.0.smithi060.stdout:2/471: fdatasync dd/d43/d60/f3f 0 2024-02-26T10:47:59.992 INFO:tasks.workunit.client.0.smithi060.stdout:2/472: rename dd/l2d to dd/d43/d60/d41/d37/l9f 0 2024-02-26T10:47:59.993 INFO:tasks.workunit.client.0.smithi060.stdout:4/480: dwrite d0/d27/f30 [0,4194304] 0 2024-02-26T10:47:59.994 INFO:tasks.workunit.client.0.smithi060.stdout:4/481: dread - d0/f74 zero size 2024-02-26T10:47:59.994 INFO:tasks.workunit.client.0.smithi060.stdout:4/482: fdatasync d0/d7/d29/f38 0 2024-02-26T10:47:59.995 INFO:tasks.workunit.client.0.smithi060.stdout:5/506: dread d0/d33/f5c [0,4194304] 0 2024-02-26T10:48:00.003 INFO:tasks.workunit.client.0.smithi060.stdout:5/507: mknod d0/de/da4/cb2 0 2024-02-26T10:48:00.016 INFO:tasks.workunit.client.0.smithi060.stdout:2/473: dread dd/d43/d60/f53 [0,4194304] 0 2024-02-26T10:48:00.016 INFO:tasks.workunit.client.0.smithi060.stdout:2/474: chown dd/l32 3248161 1 2024-02-26T10:48:00.019 INFO:tasks.workunit.client.0.smithi060.stdout:6/532: dwrite d6/d7/f70 [4194304,4194304] 0 2024-02-26T10:48:00.020 INFO:tasks.workunit.client.0.smithi060.stdout:9/582: read d3/d1e/d5e/d35/d87/fc1 [1604240,79723] 0 2024-02-26T10:48:00.021 INFO:tasks.workunit.client.0.smithi060.stdout:6/533: creat d6/d7/d25/fb0 x:0 0 0 2024-02-26T10:48:00.022 INFO:tasks.workunit.client.0.smithi060.stdout:9/583: creat d3/d1e/dbb/dc2/fc3 x:0 0 0 2024-02-26T10:48:00.025 INFO:tasks.workunit.client.0.smithi060.stdout:9/584: symlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/lc4 0 2024-02-26T10:48:00.031 INFO:tasks.workunit.client.0.smithi060.stdout:9/585: symlink d3/d1e/d5e/d9e/lc5 0 2024-02-26T10:48:00.034 INFO:tasks.workunit.client.0.smithi060.stdout:7/632: dwrite da/d15/d21/d2f/f91 [0,4194304] 0 2024-02-26T10:48:00.040 INFO:tasks.workunit.client.0.smithi060.stdout:4/483: dwrite d0/d27/f4d [0,4194304] 0 2024-02-26T10:48:00.041 INFO:tasks.workunit.client.0.smithi060.stdout:7/633: rmdir da/d15/d88 39 2024-02-26T10:48:00.041 INFO:tasks.workunit.client.0.smithi060.stdout:7/634: truncate da/d15/d21/d2f/d78/fac 882502 0 2024-02-26T10:48:00.042 INFO:tasks.workunit.client.0.smithi060.stdout:7/635: stat da/d5a/f81 0 2024-02-26T10:48:00.042 INFO:tasks.workunit.client.0.smithi060.stdout:7/636: fsync da/d15/d21/d2f/d78/fa1 0 2024-02-26T10:48:00.042 INFO:tasks.workunit.client.0.smithi060.stdout:4/484: truncate d0/d27/d2f/f35 227185 0 2024-02-26T10:48:00.044 INFO:tasks.workunit.client.0.smithi060.stdout:4/485: getdents d0/da/d3b/d16 0 2024-02-26T10:48:00.047 INFO:tasks.workunit.client.0.smithi060.stdout:4/486: mknod d0/da/d57/c90 0 2024-02-26T10:48:00.047 INFO:tasks.workunit.client.0.smithi060.stdout:4/487: write d0/d27/f83 [4433228,55949] 0 2024-02-26T10:48:00.048 INFO:tasks.workunit.client.0.smithi060.stdout:2/475: rename f0 to dd/d43/d60/d41/d37/fa0 0 2024-02-26T10:48:00.048 INFO:tasks.workunit.client.0.smithi060.stdout:4/488: creat d0/d27/d76/f91 x:0 0 0 2024-02-26T10:48:00.049 INFO:tasks.workunit.client.0.smithi060.stdout:1/582: sync 2024-02-26T10:48:00.049 INFO:tasks.workunit.client.0.smithi060.stdout:3/408: dwrite d5/de/df/d57/d60/f51 [0,4194304] 0 2024-02-26T10:48:00.053 INFO:tasks.workunit.client.0.smithi060.stdout:0/449: sync 2024-02-26T10:48:00.053 INFO:tasks.workunit.client.0.smithi060.stdout:6/534: write d6/d7/d47/d69/f82 [1363840,4673] 0 2024-02-26T10:48:00.053 INFO:tasks.workunit.client.0.smithi060.stdout:0/450: stat d0/dd/d39/d63/f67 0 2024-02-26T10:48:00.053 INFO:tasks.workunit.client.0.smithi060.stdout:4/489: rename d0/da/d3b/d16/f72 to d0/da/d3b/d16/f92 0 2024-02-26T10:48:00.053 INFO:tasks.workunit.client.0.smithi060.stdout:8/504: sync 2024-02-26T10:48:00.055 INFO:tasks.workunit.client.0.smithi060.stdout:8/505: mkdir d9/d5a/d9c 0 2024-02-26T10:48:00.055 INFO:tasks.workunit.client.0.smithi060.stdout:8/506: truncate d9/d15/d29/d2f/d34/d72/d50/d6a/f71 203780 0 2024-02-26T10:48:00.056 INFO:tasks.workunit.client.0.smithi060.stdout:0/451: creat d0/dd/d39/d40/f98 x:0 0 0 2024-02-26T10:48:00.060 INFO:tasks.workunit.client.0.smithi060.stdout:0/452: link d0/dd/d39/d40/l20 d0/dd/d53/l99 0 2024-02-26T10:48:00.062 INFO:tasks.workunit.client.0.smithi060.stdout:9/586: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/f54 [0,4194304] 0 2024-02-26T10:48:00.063 INFO:tasks.workunit.client.0.smithi060.stdout:0/453: creat d0/dd/d33/d6a/d6c/f9a x:0 0 0 2024-02-26T10:48:00.065 INFO:tasks.workunit.client.0.smithi060.stdout:0/454: mknod d0/dd/d53/d87/c9b 0 2024-02-26T10:48:00.066 INFO:tasks.workunit.client.0.smithi060.stdout:0/455: chown d0/dd/l27 43762546 1 2024-02-26T10:48:00.071 INFO:tasks.workunit.client.0.smithi060.stdout:5/508: dwrite d0/de/d15/f18 [0,4194304] 0 2024-02-26T10:48:00.080 INFO:tasks.workunit.client.0.smithi060.stdout:1/583: dread d3/db/d2e/f4d [0,4194304] 0 2024-02-26T10:48:00.080 INFO:tasks.workunit.client.0.smithi060.stdout:2/476: write dd/f68 [859211,43737] 0 2024-02-26T10:48:00.081 INFO:tasks.workunit.client.0.smithi060.stdout:5/509: getdents d0/de 0 2024-02-26T10:48:00.081 INFO:tasks.workunit.client.0.smithi060.stdout:7/637: rename da/d5a to da/d66/dad/dd8 0 2024-02-26T10:48:00.081 INFO:tasks.workunit.client.0.smithi060.stdout:7/638: write da/d66/fcd [790229,63859] 0 2024-02-26T10:48:00.084 INFO:tasks.workunit.client.0.smithi060.stdout:3/409: write d5/f27 [6258501,111786] 0 2024-02-26T10:48:00.085 INFO:tasks.workunit.client.0.smithi060.stdout:3/410: fsync d5/de/df/d57/d60/f3a 0 2024-02-26T10:48:00.093 INFO:tasks.workunit.client.0.smithi060.stdout:2/477: symlink dd/d43/d55/d5e/d86/la1 0 2024-02-26T10:48:00.093 INFO:tasks.workunit.client.0.smithi060.stdout:2/478: chown dd/d43/d60/f3f 249213 1 2024-02-26T10:48:00.093 INFO:tasks.workunit.client.0.smithi060.stdout:8/507: link d9/d15/d29/f64 d9/d5a/d9c/f9d 0 2024-02-26T10:48:00.093 INFO:tasks.workunit.client.0.smithi060.stdout:5/510: creat d0/de/da4/fb3 x:0 0 0 2024-02-26T10:48:00.093 INFO:tasks.workunit.client.0.smithi060.stdout:3/411: creat d5/de/df/d57/f74 x:0 0 0 2024-02-26T10:48:00.093 INFO:tasks.workunit.client.0.smithi060.stdout:5/511: fsync d0/f3a 0 2024-02-26T10:48:00.095 INFO:tasks.workunit.client.0.smithi060.stdout:3/412: symlink d5/de/d16/l75 0 2024-02-26T10:48:00.095 INFO:tasks.workunit.client.0.smithi060.stdout:5/512: creat d0/de/d20/d32/d4d/fb4 x:0 0 0 2024-02-26T10:48:00.095 INFO:tasks.workunit.client.0.smithi060.stdout:5/513: chown d0/de/d20 378118 1 2024-02-26T10:48:00.096 INFO:tasks.workunit.client.0.smithi060.stdout:6/535: dread d6/f18 [0,4194304] 0 2024-02-26T10:48:00.096 INFO:tasks.workunit.client.0.smithi060.stdout:3/413: mkdir d5/d76 0 2024-02-26T10:48:00.100 INFO:tasks.workunit.client.0.smithi060.stdout:1/584: link d3/db/d1e/d6d/d78/f99 d3/d35/d53/d68/d85/fd0 0 2024-02-26T10:48:00.103 INFO:tasks.workunit.client.0.smithi060.stdout:9/587: dread d3/d1e/d5e/d1b/f41 [0,4194304] 0 2024-02-26T10:48:00.108 INFO:tasks.workunit.client.0.smithi060.stdout:1/585: rmdir d3/d35/d53 39 2024-02-26T10:48:00.109 INFO:tasks.workunit.client.0.smithi060.stdout:8/508: link d9/d15/d3e/f88 d9/d15/d29/d2f/d34/d72/d94/f9e 0 2024-02-26T10:48:00.109 INFO:tasks.workunit.client.0.smithi060.stdout:0/456: write d0/dd/d39/d40/f1f [435234,101279] 0 2024-02-26T10:48:00.113 INFO:tasks.workunit.client.0.smithi060.stdout:6/536: dread d6/d7/d25/d1c/d1f/d23/f3a [0,4194304] 0 2024-02-26T10:48:00.113 INFO:tasks.workunit.client.0.smithi060.stdout:8/509: rename d9/d15/d3e/l56 to d9/d15/d29/d2f/d34/d72/d50/d6a/d63/l9f 0 2024-02-26T10:48:00.113 INFO:tasks.workunit.client.0.smithi060.stdout:1/586: mkdir d3/db/d1e/d6d/dd1 0 2024-02-26T10:48:00.114 INFO:tasks.workunit.client.0.smithi060.stdout:8/510: readlink d9/d15/d29/d2f/d34/d72/d50/d6a/l4f 0 2024-02-26T10:48:00.114 INFO:tasks.workunit.client.0.smithi060.stdout:8/511: stat d9/d15/d29/d2f/l32 0 2024-02-26T10:48:00.115 INFO:tasks.workunit.client.0.smithi060.stdout:8/512: truncate d9/d5a/f69 1135084 0 2024-02-26T10:48:00.115 INFO:tasks.workunit.client.0.smithi060.stdout:8/513: chown d9/d15/f36 308356 1 2024-02-26T10:48:00.115 INFO:tasks.workunit.client.0.smithi060.stdout:8/514: stat d9/d5a/d58/c80 0 2024-02-26T10:48:00.116 INFO:tasks.workunit.client.0.smithi060.stdout:8/515: creat d9/d5a/d58/fa0 x:0 0 0 2024-02-26T10:48:00.118 INFO:tasks.workunit.client.0.smithi060.stdout:8/516: mknod d9/d15/ca1 0 2024-02-26T10:48:00.118 INFO:tasks.workunit.client.0.smithi060.stdout:8/517: chown d9/d6f/c86 0 1 2024-02-26T10:48:00.136 INFO:tasks.workunit.client.0.smithi060.stdout:4/490: dwrite d0/da/d3b/f1a [0,4194304] 0 2024-02-26T10:48:00.139 INFO:tasks.workunit.client.0.smithi060.stdout:4/491: link d0/d27/d76/f8a d0/d27/d76/f93 0 2024-02-26T10:48:00.140 INFO:tasks.workunit.client.0.smithi060.stdout:4/492: truncate d0/da/d3b/d16/f18 5283055 0 2024-02-26T10:48:00.145 INFO:tasks.workunit.client.0.smithi060.stdout:7/639: dread da/d15/d21/d2f/d37/f38 [0,4194304] 0 2024-02-26T10:48:00.150 INFO:tasks.workunit.client.0.smithi060.stdout:7/640: mknod da/d15/cd9 0 2024-02-26T10:48:00.150 INFO:tasks.workunit.client.0.smithi060.stdout:7/641: write da/f18 [5421990,31153] 0 2024-02-26T10:48:00.150 INFO:tasks.workunit.client.0.smithi060.stdout:7/642: fsync da/d15/f74 0 2024-02-26T10:48:00.151 INFO:tasks.workunit.client.0.smithi060.stdout:2/479: dwrite f4 [4194304,4194304] 0 2024-02-26T10:48:00.173 INFO:tasks.workunit.client.0.smithi060.stdout:3/414: dwrite d5/de/df/d57/d60/f51 [0,4194304] 0 2024-02-26T10:48:00.182 INFO:tasks.workunit.client.0.smithi060.stdout:7/643: rename da/d15/d88/f5b to da/d15/d21/d2f/d78/fda 0 2024-02-26T10:48:00.188 INFO:tasks.workunit.client.0.smithi060.stdout:7/644: creat da/d66/dad/dd8/fdb x:0 0 0 2024-02-26T10:48:00.189 INFO:tasks.workunit.client.0.smithi060.stdout:9/588: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/d47/f70 [0,4194304] 0 2024-02-26T10:48:00.194 INFO:tasks.workunit.client.0.smithi060.stdout:2/480: rename dd/d43/d55/f66 to dd/d43/d60/d41/d37/d4c/fa2 0 2024-02-26T10:48:00.195 INFO:tasks.workunit.client.0.smithi060.stdout:8/518: dwrite d9/d15/f52 [0,4194304] 0 2024-02-26T10:48:00.195 INFO:tasks.workunit.client.0.smithi060.stdout:8/519: write d9/f14 [4413536,77329] 0 2024-02-26T10:48:00.196 INFO:tasks.workunit.client.0.smithi060.stdout:8/520: chown d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/f96 25 1 2024-02-26T10:48:00.196 INFO:tasks.workunit.client.0.smithi060.stdout:2/481: creat dd/d43/d60/d41/d37/d4c/d76/fa3 x:0 0 0 2024-02-26T10:48:00.196 INFO:tasks.workunit.client.0.smithi060.stdout:8/521: mknod d9/d15/d29/d2f/d34/d72/d94/ca2 0 2024-02-26T10:48:00.197 INFO:tasks.workunit.client.0.smithi060.stdout:8/522: creat d9/d15/d29/d2f/fa3 x:0 0 0 2024-02-26T10:48:00.198 INFO:tasks.workunit.client.0.smithi060.stdout:6/537: dread d6/f7c [0,4194304] 0 2024-02-26T10:48:00.199 INFO:tasks.workunit.client.0.smithi060.stdout:6/538: creat d6/d7/d47/d69/fb1 x:0 0 0 2024-02-26T10:48:00.200 INFO:tasks.workunit.client.0.smithi060.stdout:8/523: mkdir d9/da4 0 2024-02-26T10:48:00.205 INFO:tasks.workunit.client.0.smithi060.stdout:5/514: dwrite d0/d4b/f51 [0,4194304] 0 2024-02-26T10:48:00.206 INFO:tasks.workunit.client.0.smithi060.stdout:0/457: dread d0/dd/f2d [0,4194304] 0 2024-02-26T10:48:00.210 INFO:tasks.workunit.client.0.smithi060.stdout:8/524: write d9/d5a/f69 [719449,2273] 0 2024-02-26T10:48:00.210 INFO:tasks.workunit.client.0.smithi060.stdout:8/525: write d9/d15/d29/d2f/fa3 [539799,7531] 0 2024-02-26T10:48:00.210 INFO:tasks.workunit.client.0.smithi060.stdout:8/526: stat d9/d15/f27 0 2024-02-26T10:48:00.210 INFO:tasks.workunit.client.0.smithi060.stdout:0/458: getdents d0/d42/d7a 0 2024-02-26T10:48:00.214 INFO:tasks.workunit.client.0.smithi060.stdout:3/415: write d5/f4a [2269426,7057] 0 2024-02-26T10:48:00.214 INFO:tasks.workunit.client.0.smithi060.stdout:8/527: rename d9/d15/f51 to d9/d15/d29/fa5 0 2024-02-26T10:48:00.214 INFO:tasks.workunit.client.0.smithi060.stdout:4/493: dwrite d0/d7/f6a [0,4194304] 0 2024-02-26T10:48:00.215 INFO:tasks.workunit.client.0.smithi060.stdout:8/528: getdents d9/d15/d29/d2f/d34/d72/d50 0 2024-02-26T10:48:00.215 INFO:tasks.workunit.client.0.smithi060.stdout:8/529: readlink d9/d15/d29/d2f/d34/d72/l22 0 2024-02-26T10:48:00.215 INFO:tasks.workunit.client.0.smithi060.stdout:9/589: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f11 [0,4194304] 0 2024-02-26T10:48:00.216 INFO:tasks.workunit.client.0.smithi060.stdout:3/416: rename d5/de/df/d4c/l4d to d5/l77 0 2024-02-26T10:48:00.217 INFO:tasks.workunit.client.0.smithi060.stdout:4/494: mknod d0/d27/d7f/c94 0 2024-02-26T10:48:00.217 INFO:tasks.workunit.client.0.smithi060.stdout:5/515: symlink d0/de/d15/d26/lb5 0 2024-02-26T10:48:00.219 INFO:tasks.workunit.client.0.smithi060.stdout:8/530: rename d9/d15/c28 to d9/d15/ca6 0 2024-02-26T10:48:00.219 INFO:tasks.workunit.client.0.smithi060.stdout:8/531: fdatasync d9/d15/d29/d2f/d34/d72/d50/d6a/f46 0 2024-02-26T10:48:00.219 INFO:tasks.workunit.client.0.smithi060.stdout:8/532: stat d9/d15/d29/d2f/d34/d72/d50/c67 0 2024-02-26T10:48:00.220 INFO:tasks.workunit.client.0.smithi060.stdout:4/495: creat d0/da/d3b/f95 x:0 0 0 2024-02-26T10:48:00.221 INFO:tasks.workunit.client.0.smithi060.stdout:6/539: write d6/d7/d25/d1c/d1f/d23/d7a/f88 [1497311,128910] 0 2024-02-26T10:48:00.221 INFO:tasks.workunit.client.0.smithi060.stdout:8/533: truncate d9/d15/d29/f64 46904 0 2024-02-26T10:48:00.222 INFO:tasks.workunit.client.0.smithi060.stdout:8/534: chown d9/d15/d29/d2f/d34 37346 1 2024-02-26T10:48:00.223 INFO:tasks.workunit.client.0.smithi060.stdout:6/540: getdents d6/d7/d25/d80 0 2024-02-26T10:48:00.223 INFO:tasks.workunit.client.0.smithi060.stdout:6/541: getdents d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/da7 0 2024-02-26T10:48:00.223 INFO:tasks.workunit.client.0.smithi060.stdout:6/542: truncate d6/f18 5067596 0 2024-02-26T10:48:00.223 INFO:tasks.workunit.client.0.smithi060.stdout:6/543: truncate d6/d7/d25/d1c/f6d 1755002 0 2024-02-26T10:48:00.223 INFO:tasks.workunit.client.0.smithi060.stdout:6/544: truncate d6/f71 1570120 0 2024-02-26T10:48:00.225 INFO:tasks.workunit.client.0.smithi060.stdout:6/545: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/fb2 x:0 0 0 2024-02-26T10:48:00.227 INFO:tasks.workunit.client.0.smithi060.stdout:6/546: link d6/d7/f53 d6/d8d/fb3 0 2024-02-26T10:48:00.233 INFO:tasks.workunit.client.0.smithi060.stdout:1/587: dwrite d3/f1b [0,4194304] 0 2024-02-26T10:48:00.233 INFO:tasks.workunit.client.0.smithi060.stdout:1/588: chown d3/d35/d53/d68/f8c 171 1 2024-02-26T10:48:00.233 INFO:tasks.workunit.client.0.smithi060.stdout:1/589: fdatasync d3/f69 0 2024-02-26T10:48:00.234 INFO:tasks.workunit.client.0.smithi060.stdout:1/590: creat d3/d35/d53/fd2 x:0 0 0 2024-02-26T10:48:00.238 INFO:tasks.workunit.client.0.smithi060.stdout:1/591: creat d3/db/dc8/dbe/fd3 x:0 0 0 2024-02-26T10:48:00.240 INFO:tasks.workunit.client.0.smithi060.stdout:1/592: fdatasync d3/d35/d5e/f66 0 2024-02-26T10:48:00.240 INFO:tasks.workunit.client.0.smithi060.stdout:1/593: write d3/d54/f7c [6535793,128994] 0 2024-02-26T10:48:00.240 INFO:tasks.workunit.client.0.smithi060.stdout:1/594: truncate d3/db/d1e/d38/d6c/fb9 733743 0 2024-02-26T10:48:00.240 INFO:tasks.workunit.client.0.smithi060.stdout:1/595: chown d3/db/d1e/d38/d6c/fcb 131613 1 2024-02-26T10:48:00.243 INFO:tasks.workunit.client.0.smithi060.stdout:9/590: rename d3/d1e/d5e/d1b/d23/c86 to d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/cc6 0 2024-02-26T10:48:00.244 INFO:tasks.workunit.client.0.smithi060.stdout:9/591: symlink d3/d1e/d93/da2/lc7 0 2024-02-26T10:48:00.245 INFO:tasks.workunit.client.0.smithi060.stdout:4/496: rename d0/d7/f5a to d0/da/d57/f96 0 2024-02-26T10:48:00.246 INFO:tasks.workunit.client.0.smithi060.stdout:9/592: unlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/lc4 0 2024-02-26T10:48:00.248 INFO:tasks.workunit.client.0.smithi060.stdout:9/593: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8 0 2024-02-26T10:48:00.256 INFO:tasks.workunit.client.0.smithi060.stdout:0/459: dwrite d0/d42/f60 [0,4194304] 0 2024-02-26T10:48:00.259 INFO:tasks.workunit.client.0.smithi060.stdout:9/594: mkdir d3/d1e/d5e/dc9 0 2024-02-26T10:48:00.263 INFO:tasks.workunit.client.0.smithi060.stdout:0/460: link d0/dd/d53/d87/c9b d0/c9c 0 2024-02-26T10:48:00.264 INFO:tasks.workunit.client.0.smithi060.stdout:0/461: write d0/dd/d39/d40/f26 [977466,108533] 0 2024-02-26T10:48:00.264 INFO:tasks.workunit.client.0.smithi060.stdout:9/595: symlink d3/d1e/d5e/d35/d87/lca 0 2024-02-26T10:48:00.267 INFO:tasks.workunit.client.0.smithi060.stdout:4/497: read d0/d7/d29/f32 [715033,36884] 0 2024-02-26T10:48:00.268 INFO:tasks.workunit.client.0.smithi060.stdout:3/417: dwrite d5/de/df/d4c/f4f [0,4194304] 0 2024-02-26T10:48:00.268 INFO:tasks.workunit.client.0.smithi060.stdout:4/498: truncate d0/f7b 134362 0 2024-02-26T10:48:00.268 INFO:tasks.workunit.client.0.smithi060.stdout:0/462: link d0/dd/f2c d0/dd/d39/d40/d83/d89/f9d 0 2024-02-26T10:48:00.268 INFO:tasks.workunit.client.0.smithi060.stdout:0/463: write d0/dd/d39/f7e [9333934,5016] 0 2024-02-26T10:48:00.269 INFO:tasks.workunit.client.0.smithi060.stdout:3/418: creat d5/d76/f78 x:0 0 0 2024-02-26T10:48:00.271 INFO:tasks.workunit.client.0.smithi060.stdout:4/499: getdents d0 0 2024-02-26T10:48:00.274 INFO:tasks.workunit.client.0.smithi060.stdout:3/419: link d5/de/df/l3f d5/de/d16/l79 0 2024-02-26T10:48:00.274 INFO:tasks.workunit.client.0.smithi060.stdout:3/420: write d5/de/df/d11/f36 [759844,1921] 0 2024-02-26T10:48:00.275 INFO:tasks.workunit.client.0.smithi060.stdout:4/500: mknod d0/d7/d29/d60/c97 0 2024-02-26T10:48:00.277 INFO:tasks.workunit.client.0.smithi060.stdout:3/421: mkdir d5/de/df/d57/d7a 0 2024-02-26T10:48:00.278 INFO:tasks.workunit.client.0.smithi060.stdout:7/645: dwrite da/d94/faa [0,4194304] 0 2024-02-26T10:48:00.278 INFO:tasks.workunit.client.0.smithi060.stdout:7/646: dread - da/d15/d21/d2f/d37/f8e zero size 2024-02-26T10:48:00.278 INFO:tasks.workunit.client.0.smithi060.stdout:7/647: chown da/d15/d88/d3b/f54 96736900 1 2024-02-26T10:48:00.283 INFO:tasks.workunit.client.0.smithi060.stdout:8/535: rename d9/d6f/c79 to d9/d5a/d9c/ca7 0 2024-02-26T10:48:00.283 INFO:tasks.workunit.client.0.smithi060.stdout:8/536: chown d9/d5a/d9c 621 1 2024-02-26T10:48:00.283 INFO:tasks.workunit.client.0.smithi060.stdout:8/537: dread - d9/d5a/d58/fa0 zero size 2024-02-26T10:48:00.283 INFO:tasks.workunit.client.0.smithi060.stdout:8/538: stat d9/d5a/f7f 0 2024-02-26T10:48:00.287 INFO:tasks.workunit.client.0.smithi060.stdout:2/482: read dd/d43/d60/f53 [1576487,103562] 0 2024-02-26T10:48:00.287 INFO:tasks.workunit.client.0.smithi060.stdout:2/483: truncate dd/f6a 96728 0 2024-02-26T10:48:00.287 INFO:tasks.workunit.client.0.smithi060.stdout:4/501: rename d0/f44 to d0/f98 0 2024-02-26T10:48:00.287 INFO:tasks.workunit.client.0.smithi060.stdout:2/484: dread - dd/d43/d60/d41/d37/d4c/f92 zero size 2024-02-26T10:48:00.287 INFO:tasks.workunit.client.0.smithi060.stdout:2/485: chown dd/d43/d60/d41/d37/d38/d57/l7b 0 1 2024-02-26T10:48:00.287 INFO:tasks.workunit.client.0.smithi060.stdout:2/486: stat dd/d43/d60/d41/d37/d4c/d76 0 2024-02-26T10:48:00.288 INFO:tasks.workunit.client.0.smithi060.stdout:2/487: truncate dd/f2b 240049 0 2024-02-26T10:48:00.288 INFO:tasks.workunit.client.0.smithi060.stdout:2/488: stat dd/d43/d60/d41/d37/d4c 0 2024-02-26T10:48:00.289 INFO:tasks.workunit.client.0.smithi060.stdout:2/489: fdatasync dd/d11/f62 0 2024-02-26T10:48:00.292 INFO:tasks.workunit.client.0.smithi060.stdout:7/648: mkdir da/d15/d88/d3b/ddc 0 2024-02-26T10:48:00.292 INFO:tasks.workunit.client.0.smithi060.stdout:8/539: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/fa8 x:0 0 0 2024-02-26T10:48:00.292 INFO:tasks.workunit.client.0.smithi060.stdout:8/540: fsync d9/d5a/d9c/f9d 0 2024-02-26T10:48:00.293 INFO:tasks.workunit.client.0.smithi060.stdout:6/547: rename d6/d7/d47/d69/l7b to d6/d7/d47/lb4 0 2024-02-26T10:48:00.293 INFO:tasks.workunit.client.0.smithi060.stdout:4/502: creat d0/d7/d29/d8e/f99 x:0 0 0 2024-02-26T10:48:00.295 INFO:tasks.workunit.client.0.smithi060.stdout:4/503: truncate d0/da/d57/f75 2839450 0 2024-02-26T10:48:00.296 INFO:tasks.workunit.client.0.smithi060.stdout:3/422: symlink d5/d45/l7b 0 2024-02-26T10:48:00.297 INFO:tasks.workunit.client.0.smithi060.stdout:1/596: dwrite d3/d35/d76/f98 [4194304,4194304] 0 2024-02-26T10:48:00.298 INFO:tasks.workunit.client.0.smithi060.stdout:7/649: mknod da/d15/d88/d3b/ddc/cdd 0 2024-02-26T10:48:00.300 INFO:tasks.workunit.client.0.smithi060.stdout:8/541: mkdir d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9 0 2024-02-26T10:48:00.300 INFO:tasks.workunit.client.0.smithi060.stdout:2/490: link dd/l32 dd/d11/la4 0 2024-02-26T10:48:00.300 INFO:tasks.workunit.client.0.smithi060.stdout:2/491: truncate dd/d43/d60/d41/d37/d69/f7e 157704 0 2024-02-26T10:48:00.301 INFO:tasks.workunit.client.0.smithi060.stdout:3/423: write d5/de/d16/f6d [119109,43076] 0 2024-02-26T10:48:00.302 INFO:tasks.workunit.client.0.smithi060.stdout:6/548: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac/db5 0 2024-02-26T10:48:00.302 INFO:tasks.workunit.client.0.smithi060.stdout:5/516: dwrite d0/de/d15/d65/f7c [4194304,4194304] 0 2024-02-26T10:48:00.302 INFO:tasks.workunit.client.0.smithi060.stdout:2/492: mkdir dd/d43/d55/d5e/d74/da5 0 2024-02-26T10:48:00.306 INFO:tasks.workunit.client.0.smithi060.stdout:7/650: creat da/d15/d88/d3b/fde x:0 0 0 2024-02-26T10:48:00.309 INFO:tasks.workunit.client.0.smithi060.stdout:5/517: truncate d0/de/d15/f2f 902618 0 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:4/504: creat d0/da/f9a x:0 0 0 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:3/424: creat d5/de/df/d11/d63/f7c x:0 0 0 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:5/518: mknod d0/de/d15/cb6 0 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:5/519: fsync d0/de/d20/f56 0 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:3/425: mkdir d5/d76/d7d 0 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:6/549: creat d6/d7/d2e/fb6 x:0 0 0 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:3/426: link d5/de/d16/c1a d5/d76/d7d/c7e 0 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:3/427: chown d5/d37/f56 0 1 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:6/550: creat d6/d7/d25/d1c/d1f/fb7 x:0 0 0 2024-02-26T10:48:00.323 INFO:tasks.workunit.client.0.smithi060.stdout:3/428: creat d5/de/df/d11/d63/f7f x:0 0 0 2024-02-26T10:48:00.324 INFO:tasks.workunit.client.0.smithi060.stdout:6/551: symlink d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/lb8 0 2024-02-26T10:48:00.325 INFO:tasks.workunit.client.0.smithi060.stdout:6/552: dread - d6/d6e/d6f/fa1 zero size 2024-02-26T10:48:00.326 INFO:tasks.workunit.client.0.smithi060.stdout:3/429: mkdir d5/d45/d61/d80 0 2024-02-26T10:48:00.326 INFO:tasks.workunit.client.0.smithi060.stdout:6/553: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d4d/db9 0 2024-02-26T10:48:00.332 INFO:tasks.workunit.client.0.smithi060.stdout:6/554: rename d6/d7/d25/c87 to d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac/cba 0 2024-02-26T10:48:00.337 INFO:tasks.workunit.client.0.smithi060.stdout:6/555: truncate d6/d7/d25/d1c/f2c 676110 0 2024-02-26T10:48:00.338 INFO:tasks.workunit.client.0.smithi060.stdout:1/597: dread d3/f2a [0,4194304] 0 2024-02-26T10:48:00.338 INFO:tasks.workunit.client.0.smithi060.stdout:1/598: write d3/d35/d5e/f70 [952806,90087] 0 2024-02-26T10:48:00.346 INFO:tasks.workunit.client.0.smithi060.stdout:1/599: getdents d3/db/dc8/d91 0 2024-02-26T10:48:00.346 INFO:tasks.workunit.client.0.smithi060.stdout:1/600: write d3/d35/fa5 [630199,98066] 0 2024-02-26T10:48:00.346 INFO:tasks.workunit.client.0.smithi060.stdout:1/601: stat d3/f2a 0 2024-02-26T10:48:00.347 INFO:tasks.workunit.client.0.smithi060.stdout:1/602: mknod d3/d35/dc2/cd4 0 2024-02-26T10:48:00.347 INFO:tasks.workunit.client.0.smithi060.stdout:1/603: readlink d3/db/dc8/d91/la8 0 2024-02-26T10:48:00.352 INFO:tasks.workunit.client.0.smithi060.stdout:4/505: dread d0/d27/d2f/f6e [0,4194304] 0 2024-02-26T10:48:00.355 INFO:tasks.workunit.client.0.smithi060.stdout:4/506: symlink d0/da/l9b 0 2024-02-26T10:48:00.355 INFO:tasks.workunit.client.0.smithi060.stdout:4/507: fdatasync d0/d7/d29/f87 0 2024-02-26T10:48:00.357 INFO:tasks.workunit.client.0.smithi060.stdout:5/520: read d0/f90 [6161208,68540] 0 2024-02-26T10:48:00.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/508: creat d0/da/d3b/d1b/f9c x:0 0 0 2024-02-26T10:48:00.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/509: write d0/da/d3b/f34 [4248647,54800] 0 2024-02-26T10:48:00.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/510: readlink d0/d7/d29/d60/l6f 0 2024-02-26T10:48:00.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/511: write d0/f98 [3154749,19639] 0 2024-02-26T10:48:00.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/512: write d0/d7/d29/f64 [5216919,58975] 0 2024-02-26T10:48:00.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/513: symlink d0/d27/d7f/l9d 0 2024-02-26T10:48:00.368 INFO:tasks.workunit.client.0.smithi060.stdout:9/596: dwrite d3/d1e/d5e/f2a [0,4194304] 0 2024-02-26T10:48:00.372 INFO:tasks.workunit.client.0.smithi060.stdout:0/464: dwrite d0/f32 [0,4194304] 0 2024-02-26T10:48:00.378 INFO:tasks.workunit.client.0.smithi060.stdout:6/556: fsync d6/d7/d25/d1c/d1f/d23/d2b/d4d/f58 0 2024-02-26T10:48:00.378 INFO:tasks.workunit.client.0.smithi060.stdout:6/557: write d6/d7/d25/d1c/d1f/d23/d28/fa3 [339591,25007] 0 2024-02-26T10:48:00.388 INFO:tasks.workunit.client.0.smithi060.stdout:7/651: dread da/f30 [4194304,4194304] 0 2024-02-26T10:48:00.388 INFO:tasks.workunit.client.0.smithi060.stdout:7/652: stat da/d15/l1c 0 2024-02-26T10:48:00.396 INFO:tasks.workunit.client.0.smithi060.stdout:0/465: read d0/dd/d33/f8c [3050886,13245] 0 2024-02-26T10:48:00.396 INFO:tasks.workunit.client.0.smithi060.stdout:0/466: read - d0/dd/d33/d6a/d6c/f9a zero size 2024-02-26T10:48:00.396 INFO:tasks.workunit.client.0.smithi060.stdout:0/467: write d0/dd/d53/f64 [852395,71371] 0 2024-02-26T10:48:00.399 INFO:tasks.workunit.client.0.smithi060.stdout:8/542: dwrite d9/d15/f52 [0,4194304] 0 2024-02-26T10:48:00.402 INFO:tasks.workunit.client.0.smithi060.stdout:5/521: read d0/de/f12 [1579049,88579] 0 2024-02-26T10:48:00.406 INFO:tasks.workunit.client.0.smithi060.stdout:4/514: read d0/f3c [838896,124089] 0 2024-02-26T10:48:00.406 INFO:tasks.workunit.client.0.smithi060.stdout:4/515: chown d0/d27/d52/l88 1 1 2024-02-26T10:48:00.409 INFO:tasks.workunit.client.0.smithi060.stdout:3/430: dwrite d5/f52 [4194304,4194304] 0 2024-02-26T10:48:00.411 INFO:tasks.workunit.client.0.smithi060.stdout:1/604: dwrite d3/d35/f65 [0,4194304] 0 2024-02-26T10:48:00.412 INFO:tasks.workunit.client.0.smithi060.stdout:8/543: mknod d9/d15/d29/d2f/caa 0 2024-02-26T10:48:00.414 INFO:tasks.workunit.client.0.smithi060.stdout:4/516: creat d0/d27/d2f/f9e x:0 0 0 2024-02-26T10:48:00.418 INFO:tasks.workunit.client.0.smithi060.stdout:1/605: unlink d3/d35/f9c 0 2024-02-26T10:48:00.418 INFO:tasks.workunit.client.0.smithi060.stdout:1/606: stat d3/db/d1e/d38/cb2 0 2024-02-26T10:48:00.418 INFO:tasks.workunit.client.0.smithi060.stdout:4/517: stat d0/da/d57/f75 0 2024-02-26T10:48:00.418 INFO:tasks.workunit.client.0.smithi060.stdout:4/518: fsync d0/f74 0 2024-02-26T10:48:00.419 INFO:tasks.workunit.client.0.smithi060.stdout:4/519: fdatasync d0/da/d3b/d16/f24 0 2024-02-26T10:48:00.419 INFO:tasks.workunit.client.0.smithi060.stdout:4/520: fsync d0/d27/f49 0 2024-02-26T10:48:00.419 INFO:tasks.workunit.client.0.smithi060.stdout:4/521: chown d0/l36 1268 1 2024-02-26T10:48:00.422 INFO:tasks.workunit.client.0.smithi060.stdout:4/522: unlink d0/d27/d2f/f9e 0 2024-02-26T10:48:00.424 INFO:tasks.workunit.client.0.smithi060.stdout:4/523: mkdir d0/d27/d76/d9f 0 2024-02-26T10:48:00.425 INFO:tasks.workunit.client.0.smithi060.stdout:4/524: creat d0/da/fa0 x:0 0 0 2024-02-26T10:48:00.426 INFO:tasks.workunit.client.0.smithi060.stdout:0/468: read d0/dd/f2d [1629799,73779] 0 2024-02-26T10:48:00.426 INFO:tasks.workunit.client.0.smithi060.stdout:0/469: fdatasync d0/f3 0 2024-02-26T10:48:00.426 INFO:tasks.workunit.client.0.smithi060.stdout:0/470: chown d0/dd/l4f 20752098 1 2024-02-26T10:48:00.431 INFO:tasks.workunit.client.0.smithi060.stdout:0/471: truncate d0/f55 1370447 0 2024-02-26T10:48:00.432 INFO:tasks.workunit.client.0.smithi060.stdout:9/597: dread d3/d1e/d5e/d1b/f5a [0,4194304] 0 2024-02-26T10:48:00.439 INFO:tasks.workunit.client.0.smithi060.stdout:0/472: mknod d0/dd/d33/d6a/c9e 0 2024-02-26T10:48:00.439 INFO:tasks.workunit.client.0.smithi060.stdout:0/473: chown d0/d37/d44 1 1 2024-02-26T10:48:00.439 INFO:tasks.workunit.client.0.smithi060.stdout:0/474: symlink d0/dd/d53/l9f 0 2024-02-26T10:48:00.440 INFO:tasks.workunit.client.0.smithi060.stdout:3/431: dread d5/d45/f49 [0,4194304] 0 2024-02-26T10:48:00.461 INFO:tasks.workunit.client.0.smithi060.stdout:5/522: dwrite d0/f1d [0,4194304] 0 2024-02-26T10:48:00.461 INFO:tasks.workunit.client.0.smithi060.stdout:5/523: chown d0/d33/f3e 45 1 2024-02-26T10:48:00.462 INFO:tasks.workunit.client.0.smithi060.stdout:5/524: truncate d0/d33/f5f 373388 0 2024-02-26T10:48:00.462 INFO:tasks.workunit.client.0.smithi060.stdout:5/525: creat d0/de/d20/d32/d4c/fb7 x:0 0 0 2024-02-26T10:48:00.462 INFO:tasks.workunit.client.0.smithi060.stdout:8/544: rename d9/d15/d29/fa5 to d9/d5a/d58/fab 0 2024-02-26T10:48:00.462 INFO:tasks.workunit.client.0.smithi060.stdout:5/526: symlink d0/de/d15/d65/lb8 0 2024-02-26T10:48:00.463 INFO:tasks.workunit.client.0.smithi060.stdout:5/527: dread - d0/de/d20/f56 zero size 2024-02-26T10:48:00.463 INFO:tasks.workunit.client.0.smithi060.stdout:8/545: rename d9/d15/d29/d2f/f4d to d9/d15/d3e/fac 0 2024-02-26T10:48:00.464 INFO:tasks.workunit.client.0.smithi060.stdout:5/528: mkdir d0/de/d15/db9 0 2024-02-26T10:48:00.465 INFO:tasks.workunit.client.0.smithi060.stdout:5/529: rmdir d0/d4b/d7b 39 2024-02-26T10:48:00.473 INFO:tasks.workunit.client.0.smithi060.stdout:7/653: dwrite da/d66/dad/dd8/f81 [0,4194304] 0 2024-02-26T10:48:00.475 INFO:tasks.workunit.client.0.smithi060.stdout:7/654: creat da/d15/d21/d2f/d37/fdf x:0 0 0 2024-02-26T10:48:00.476 INFO:tasks.workunit.client.0.smithi060.stdout:7/655: mkdir da/d15/d21/d2f/d78/dd3/db9/dd7/de0 0 2024-02-26T10:48:00.476 INFO:tasks.workunit.client.0.smithi060.stdout:7/656: getdents da/d15/d21/d2f/d78/dd3/d67/da8 0 2024-02-26T10:48:00.477 INFO:tasks.workunit.client.0.smithi060.stdout:1/607: dwrite d3/db/d1e/d6d/d78/f86 [0,4194304] 0 2024-02-26T10:48:00.480 INFO:tasks.workunit.client.0.smithi060.stdout:7/657: rename da/d15/d21/f79 to da/d66/dbd/fe1 0 2024-02-26T10:48:00.480 INFO:tasks.workunit.client.0.smithi060.stdout:7/658: fdatasync da/d15/f46 0 2024-02-26T10:48:00.481 INFO:tasks.workunit.client.0.smithi060.stdout:7/659: symlink da/d15/d88/d3b/le2 0 2024-02-26T10:48:00.481 INFO:tasks.workunit.client.0.smithi060.stdout:7/660: write da/f30 [5058915,107982] 0 2024-02-26T10:48:00.490 INFO:tasks.workunit.client.0.smithi060.stdout:7/661: dread da/d15/d88/f51 [0,4194304] 0 2024-02-26T10:48:00.491 INFO:tasks.workunit.client.0.smithi060.stdout:9/598: write d3/d1e/d5e/d1b/f56 [2196960,11890] 0 2024-02-26T10:48:00.492 INFO:tasks.workunit.client.0.smithi060.stdout:6/558: dwrite d6/d7/d25/d1c/d1f/d23/d2b/d67/f8a [0,4194304] 0 2024-02-26T10:48:00.497 INFO:tasks.workunit.client.0.smithi060.stdout:9/599: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb 0 2024-02-26T10:48:00.498 INFO:tasks.workunit.client.0.smithi060.stdout:4/525: dread d0/da/d3b/d16/f51 [0,4194304] 0 2024-02-26T10:48:00.499 INFO:tasks.workunit.client.0.smithi060.stdout:7/662: mkdir da/d15/d21/d2f/d78/dd3/d67/da8/de3 0 2024-02-26T10:48:00.510 INFO:tasks.workunit.client.0.smithi060.stdout:9/600: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/fcc x:0 0 0 2024-02-26T10:48:00.513 INFO:tasks.workunit.client.0.smithi060.stdout:7/663: creat da/d15/d21/d29/fe4 x:0 0 0 2024-02-26T10:48:00.513 INFO:tasks.workunit.client.0.smithi060.stdout:7/664: chown da/d94/lc6 475485111 1 2024-02-26T10:48:00.513 INFO:tasks.workunit.client.0.smithi060.stdout:9/601: unlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/d47/cba 0 2024-02-26T10:48:00.513 INFO:tasks.workunit.client.0.smithi060.stdout:4/526: creat d0/d7/d29/d8e/fa1 x:0 0 0 2024-02-26T10:48:00.516 INFO:tasks.workunit.client.0.smithi060.stdout:7/665: mknod da/d15/d88/ce5 0 2024-02-26T10:48:00.516 INFO:tasks.workunit.client.0.smithi060.stdout:9/602: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8/dcd 0 2024-02-26T10:48:00.516 INFO:tasks.workunit.client.0.smithi060.stdout:4/527: mknod d0/d27/d76/d9f/ca2 0 2024-02-26T10:48:00.516 INFO:tasks.workunit.client.0.smithi060.stdout:7/666: link da/d15/d21/d2f/d78/dd3/f93 da/d15/d21/d2f/d78/dd3/db9/dd7/de0/fe6 0 2024-02-26T10:48:00.516 INFO:tasks.workunit.client.0.smithi060.stdout:4/528: mkdir d0/d27/d76/da3 0 2024-02-26T10:48:00.517 INFO:tasks.workunit.client.0.smithi060.stdout:7/667: creat da/d66/dad/dd8/fe7 x:0 0 0 2024-02-26T10:48:00.519 INFO:tasks.workunit.client.0.smithi060.stdout:7/668: mknod da/d15/d21/d2f/d78/dd3/d67/da8/de3/ce8 0 2024-02-26T10:48:00.526 INFO:tasks.workunit.client.0.smithi060.stdout:7/669: dread da/d94/fb5 [0,4194304] 0 2024-02-26T10:48:00.542 INFO:tasks.workunit.client.0.smithi060.stdout:9/603: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/d47/d7a/fb5 [2249528,49709] 0 2024-02-26T10:48:00.542 INFO:tasks.workunit.client.0.smithi060.stdout:9/604: creat d3/d1e/d93/fce x:0 0 0 2024-02-26T10:48:00.545 INFO:tasks.workunit.client.0.smithi060.stdout:0/475: dwrite d0/f5f [4194304,4194304] 0 2024-02-26T10:48:00.545 INFO:tasks.workunit.client.0.smithi060.stdout:0/476: readlink d0/dd/d33/l6f 0 2024-02-26T10:48:00.546 INFO:tasks.workunit.client.0.smithi060.stdout:9/605: creat d3/d1e/d93/da2/db7/fcf x:0 0 0 2024-02-26T10:48:00.547 INFO:tasks.workunit.client.0.smithi060.stdout:9/606: chown d3/d1e/d5e/d35/f7c 14 1 2024-02-26T10:48:00.547 INFO:tasks.workunit.client.0.smithi060.stdout:9/607: fsync d3/d1e/d5e/d1b/d23/f36 0 2024-02-26T10:48:00.548 INFO:tasks.workunit.client.0.smithi060.stdout:9/608: creat d3/d1e/d57/fd0 x:0 0 0 2024-02-26T10:48:00.556 INFO:tasks.workunit.client.0.smithi060.stdout:1/608: dwrite d3/f15 [0,4194304] 0 2024-02-26T10:48:00.559 INFO:tasks.workunit.client.0.smithi060.stdout:7/670: write da/d66/dad/fa4 [3148012,115063] 0 2024-02-26T10:48:00.559 INFO:tasks.workunit.client.0.smithi060.stdout:1/609: truncate d3/db/d1e/d6d/d78/f99 256256 0 2024-02-26T10:48:00.563 INFO:tasks.workunit.client.0.smithi060.stdout:5/530: dwrite d0/fd [0,4194304] 0 2024-02-26T10:48:00.565 INFO:tasks.workunit.client.0.smithi060.stdout:7/671: symlink da/d15/d21/d2f/d78/dd3/dc8/le9 0 2024-02-26T10:48:00.571 INFO:tasks.workunit.client.0.smithi060.stdout:6/559: dwrite d6/d7/d25/d1c/d1f/d23/d28/f34 [0,4194304] 0 2024-02-26T10:48:00.574 INFO:tasks.workunit.client.0.smithi060.stdout:4/529: dwrite d0/f74 [0,4194304] 0 2024-02-26T10:48:00.578 INFO:tasks.workunit.client.0.smithi060.stdout:4/530: write d0/d27/d2f/f7d [399421,40470] 0 2024-02-26T10:48:00.583 INFO:tasks.workunit.client.0.smithi060.stdout:1/610: creat d3/db/d2e/fd5 x:0 0 0 2024-02-26T10:48:00.583 INFO:tasks.workunit.client.0.smithi060.stdout:1/611: readlink d3/d35/l75 0 2024-02-26T10:48:00.584 INFO:tasks.workunit.client.0.smithi060.stdout:0/477: dwrite d0/dd/d39/d40/f8d [0,4194304] 0 2024-02-26T10:48:00.585 INFO:tasks.workunit.client.0.smithi060.stdout:9/609: dread d3/d1e/f82 [0,4194304] 0 2024-02-26T10:48:00.585 INFO:tasks.workunit.client.0.smithi060.stdout:9/610: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f63 15212131 1 2024-02-26T10:48:00.587 INFO:tasks.workunit.client.0.smithi060.stdout:9/611: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/d47/d7a/fd1 x:0 0 0 2024-02-26T10:48:00.589 INFO:tasks.workunit.client.0.smithi060.stdout:9/612: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/dd2 0 2024-02-26T10:48:00.591 INFO:tasks.workunit.client.0.smithi060.stdout:0/478: unlink d0/c4b 0 2024-02-26T10:48:00.593 INFO:tasks.workunit.client.0.smithi060.stdout:0/479: rename d0/dd/f2c to d0/dd/d33/d6a/d6c/fa0 0 2024-02-26T10:48:00.593 INFO:tasks.workunit.client.0.smithi060.stdout:0/480: chown d0/dd/c46 531834401 1 2024-02-26T10:48:00.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T10:48:00.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T10:48:00.599 INFO:tasks.workunit.client.0.smithi060.stdout:7/672: write da/d15/f46 [2611346,92239] 0 2024-02-26T10:48:00.599 INFO:tasks.workunit.client.0.smithi060.stdout:6/560: dread d6/d7/d25/f5c [0,4194304] 0 2024-02-26T10:48:00.605 INFO:tasks.workunit.client.0.smithi060.stdout:0/481: symlink d0/dd/d39/d40/d83/d89/la1 0 2024-02-26T10:48:00.607 INFO:tasks.workunit.client.0.smithi060.stdout:2/493: sync 2024-02-26T10:48:00.607 INFO:tasks.workunit.client.0.smithi060.stdout:3/432: sync 2024-02-26T10:48:00.607 INFO:tasks.workunit.client.0.smithi060.stdout:2/494: write dd/d43/d60/f53 [4038729,60874] 0 2024-02-26T10:48:00.607 INFO:tasks.workunit.client.0.smithi060.stdout:6/561: truncate d6/fe 2704433 0 2024-02-26T10:48:00.611 INFO:tasks.workunit.client.0.smithi060.stdout:2/495: mkdir dd/d43/d60/d41/d37/d4c/d76/da6 0 2024-02-26T10:48:00.611 INFO:tasks.workunit.client.0.smithi060.stdout:2/496: chown dd/d11/c16 62078 1 2024-02-26T10:48:00.611 INFO:tasks.workunit.client.0.smithi060.stdout:7/673: mkdir da/d15/d88/dce/dea 0 2024-02-26T10:48:00.611 INFO:tasks.workunit.client.0.smithi060.stdout:0/482: mkdir d0/dd/d53/da2 0 2024-02-26T10:48:00.613 INFO:tasks.workunit.client.0.smithi060.stdout:7/674: creat da/d15/d21/d2f/d37/feb x:0 0 0 2024-02-26T10:48:00.613 INFO:tasks.workunit.client.0.smithi060.stdout:1/612: dread d3/db/d1e/fb7 [0,4194304] 0 2024-02-26T10:48:00.616 INFO:tasks.workunit.client.0.smithi060.stdout:0/483: mkdir d0/d37/d44/da3 0 2024-02-26T10:48:00.616 INFO:tasks.workunit.client.0.smithi060.stdout:7/675: mkdir da/d15/d21/d2f/d78/dd3/db9/dd7/dec 0 2024-02-26T10:48:00.616 INFO:tasks.workunit.client.0.smithi060.stdout:0/484: write d0/dd/d39/d40/d83/f8f [313365,46712] 0 2024-02-26T10:48:00.616 INFO:tasks.workunit.client.0.smithi060.stdout:0/485: chown d0/dd/d39/d40/f34 3 1 2024-02-26T10:48:00.616 INFO:tasks.workunit.client.0.smithi060.stdout:7/676: readlink da/d15/l3a 0 2024-02-26T10:48:00.616 INFO:tasks.workunit.client.0.smithi060.stdout:7/677: chown da/d66/dad/dd8/l62 50 1 2024-02-26T10:48:00.616 INFO:tasks.workunit.client.0.smithi060.stdout:7/678: write da/d66/dad/d8f/fc4 [872076,28918] 0 2024-02-26T10:48:00.616 INFO:tasks.workunit.client.0.smithi060.stdout:1/613: unlink d3/db/d1e/f3b 0 2024-02-26T10:48:00.617 INFO:tasks.workunit.client.0.smithi060.stdout:8/546: sync 2024-02-26T10:48:00.617 INFO:tasks.workunit.client.0.smithi060.stdout:0/486: getdents d0/dd 0 2024-02-26T10:48:00.617 INFO:tasks.workunit.client.0.smithi060.stdout:0/487: chown d0/dd/d33/f88 3236703 1 2024-02-26T10:48:00.618 INFO:tasks.workunit.client.0.smithi060.stdout:4/531: dread d0/da/d3b/d16/f6c [0,4194304] 0 2024-02-26T10:48:00.618 INFO:tasks.workunit.client.0.smithi060.stdout:7/679: mknod da/d15/d88/d3b/ddc/ced 0 2024-02-26T10:48:00.618 INFO:tasks.workunit.client.0.smithi060.stdout:7/680: readlink da/d15/d21/d41/d6c/la9 0 2024-02-26T10:48:00.618 INFO:tasks.workunit.client.0.smithi060.stdout:8/547: mkdir d9/d15/d29/dad 0 2024-02-26T10:48:00.622 INFO:tasks.workunit.client.0.smithi060.stdout:7/681: unlink da/d15/d88/d3b/fc9 0 2024-02-26T10:48:00.622 INFO:tasks.workunit.client.0.smithi060.stdout:0/488: mkdir d0/dd/da4 0 2024-02-26T10:48:00.622 INFO:tasks.workunit.client.0.smithi060.stdout:8/548: mknod d9/d15/d3e/cae 0 2024-02-26T10:48:00.627 INFO:tasks.workunit.client.0.smithi060.stdout:5/531: write d0/fd [1148932,39850] 0 2024-02-26T10:48:00.627 INFO:tasks.workunit.client.0.smithi060.stdout:5/532: fsync d0/d33/f62 0 2024-02-26T10:48:00.628 INFO:tasks.workunit.client.0.smithi060.stdout:7/682: truncate da/f24 3890316 0 2024-02-26T10:48:00.632 INFO:tasks.workunit.client.0.smithi060.stdout:7/683: link da/d15/d21/d2f/d37/l45 da/d15/d21/lee 0 2024-02-26T10:48:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:4/532: dread d0/d27/f49 [0,4194304] 0 2024-02-26T10:48:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:4/533: truncate d0/d27/f83 7589204 0 2024-02-26T10:48:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:3/433: dread d5/de/df/d57/d60/f3a [0,4194304] 0 2024-02-26T10:48:00.644 INFO:tasks.workunit.client.0.smithi060.stdout:3/434: creat d5/de/df/f81 x:0 0 0 2024-02-26T10:48:00.650 INFO:tasks.workunit.client.0.smithi060.stdout:9/613: dwrite d3/d1e/d93/fab [0,4194304] 0 2024-02-26T10:48:00.652 INFO:tasks.workunit.client.0.smithi060.stdout:9/614: mknod d3/d1e/d93/da2/db7/cd3 0 2024-02-26T10:48:00.656 INFO:tasks.workunit.client.0.smithi060.stdout:9/615: unlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/f3d 0 2024-02-26T10:48:00.664 INFO:tasks.workunit.client.0.smithi060.stdout:9/616: rmdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a 39 2024-02-26T10:48:00.664 INFO:tasks.workunit.client.0.smithi060.stdout:9/617: truncate d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f7e 709126 0 2024-02-26T10:48:00.664 INFO:tasks.workunit.client.0.smithi060.stdout:9/618: dread - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/d47/d7a/fd1 zero size 2024-02-26T10:48:00.678 INFO:tasks.workunit.client.0.smithi060.stdout:8/549: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f93 [0,4194304] 0 2024-02-26T10:48:00.678 INFO:tasks.workunit.client.0.smithi060.stdout:9/619: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f80 [2479154,95535] 0 2024-02-26T10:48:00.679 INFO:tasks.workunit.client.0.smithi060.stdout:8/550: creat d9/d5a/faf x:0 0 0 2024-02-26T10:48:00.680 INFO:tasks.workunit.client.0.smithi060.stdout:9/620: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/d98/dd4 0 2024-02-26T10:48:00.681 INFO:tasks.workunit.client.0.smithi060.stdout:9/621: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/fcc [538986,62555] 0 2024-02-26T10:48:00.697 INFO:tasks.workunit.client.0.smithi060.stdout:3/435: write d5/de/df/d11/f20 [4238395,93747] 0 2024-02-26T10:48:00.697 INFO:tasks.workunit.client.0.smithi060.stdout:0/489: dwrite d0/dd/d39/d40/f34 [0,4194304] 0 2024-02-26T10:48:00.697 INFO:tasks.workunit.client.0.smithi060.stdout:0/490: write d0/dd/d39/f61 [4473936,33588] 0 2024-02-26T10:48:00.697 INFO:tasks.workunit.client.0.smithi060.stdout:0/491: write d0/dd/f5d [1666439,35743] 0 2024-02-26T10:48:00.698 INFO:tasks.workunit.client.0.smithi060.stdout:9/622: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/f38 [2179140,7391] 0 2024-02-26T10:48:00.698 INFO:tasks.workunit.client.0.smithi060.stdout:9/623: stat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb 0 2024-02-26T10:48:00.698 INFO:tasks.workunit.client.0.smithi060.stdout:0/492: mknod d0/dd/d39/d40/d83/ca5 0 2024-02-26T10:48:00.700 INFO:tasks.workunit.client.0.smithi060.stdout:0/493: rename d0/dd/d39/d40/d83/d89 to d0/dd/d53/d87/d94/da6 0 2024-02-26T10:48:00.702 INFO:tasks.workunit.client.0.smithi060.stdout:0/494: rmdir d0/d37/d44/da3 0 2024-02-26T10:48:00.703 INFO:tasks.workunit.client.0.smithi060.stdout:9/624: rmdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a 39 2024-02-26T10:48:00.710 INFO:tasks.workunit.client.0.smithi060.stdout:2/497: dwrite dd/f2b [0,4194304] 0 2024-02-26T10:48:00.712 INFO:tasks.workunit.client.0.smithi060.stdout:4/534: dwrite d0/d27/d76/f8a [0,4194304] 0 2024-02-26T10:48:00.713 INFO:tasks.workunit.client.0.smithi060.stdout:2/498: rename dd/d43/d60/f85 to dd/d43/d55/d5e/d74/da5/fa7 0 2024-02-26T10:48:00.715 INFO:tasks.workunit.client.0.smithi060.stdout:2/499: link dd/d43/d60/d41/d37/d38/d57/c67 dd/d43/d55/ca8 0 2024-02-26T10:48:00.715 INFO:tasks.workunit.client.0.smithi060.stdout:2/500: truncate dd/f6a 449509 0 2024-02-26T10:48:00.719 INFO:tasks.workunit.client.0.smithi060.stdout:2/501: creat dd/d43/d60/d41/d42/d6f/fa9 x:0 0 0 2024-02-26T10:48:00.721 INFO:tasks.workunit.client.0.smithi060.stdout:2/502: mknod dd/d43/d60/d41/d37/d4c/d76/da6/caa 0 2024-02-26T10:48:00.723 INFO:tasks.workunit.client.0.smithi060.stdout:2/503: link dd/d43/d60/d41/d42/l4f dd/d43/d60/d41/d37/d4c/lab 0 2024-02-26T10:48:00.733 INFO:tasks.workunit.client.0.smithi060.stdout:4/535: rename d0/d27/d76/d9f/ca2 to d0/da/d3b/ca4 0 2024-02-26T10:48:00.735 INFO:tasks.workunit.client.0.smithi060.stdout:3/436: dread d5/d37/f38 [0,4194304] 0 2024-02-26T10:48:00.735 INFO:tasks.workunit.client.0.smithi060.stdout:3/437: write d5/de/df/d4c/d68/f55 [1112636,101451] 0 2024-02-26T10:48:00.737 INFO:tasks.workunit.client.0.smithi060.stdout:3/438: symlink d5/d76/l82 0 2024-02-26T10:48:00.737 INFO:tasks.workunit.client.0.smithi060.stdout:3/439: chown d5/de/c3e 6 1 2024-02-26T10:48:00.738 INFO:tasks.workunit.client.0.smithi060.stdout:0/495: dread d0/dd/d39/d40/f17 [4194304,4194304] 0 2024-02-26T10:48:00.738 INFO:tasks.workunit.client.0.smithi060.stdout:1/614: dwrite d3/d35/d53/d57/f73 [0,4194304] 0 2024-02-26T10:48:00.738 INFO:tasks.workunit.client.0.smithi060.stdout:3/440: mknod d5/de/df/d57/c83 0 2024-02-26T10:48:00.740 INFO:tasks.workunit.client.0.smithi060.stdout:1/615: mkdir d3/db/d1e/d6d/da3/daa/dd6 0 2024-02-26T10:48:00.748 INFO:tasks.workunit.client.0.smithi060.stdout:6/562: read d6/d7/d2e/d56/d73/fa4 [1153549,52642] 0 2024-02-26T10:48:00.749 INFO:tasks.workunit.client.0.smithi060.stdout:6/563: chown d6/d7/d25/d1c/d1f/d23/d28 7624 1 2024-02-26T10:48:00.750 INFO:tasks.workunit.client.0.smithi060.stdout:2/504: write dd/d43/d60/f1e [842002,11547] 0 2024-02-26T10:48:00.753 INFO:tasks.workunit.client.0.smithi060.stdout:3/441: getdents d5/de/df/d57 0 2024-02-26T10:48:00.753 INFO:tasks.workunit.client.0.smithi060.stdout:3/442: read - d5/d37/f56 zero size 2024-02-26T10:48:00.753 INFO:tasks.workunit.client.0.smithi060.stdout:4/536: rename d0/da/d3b/d1b/l7e to d0/da/d57/la5 0 2024-02-26T10:48:00.753 INFO:tasks.workunit.client.0.smithi060.stdout:4/537: chown d0/d27/f4d 89974 1 2024-02-26T10:48:00.754 INFO:tasks.workunit.client.0.smithi060.stdout:4/538: mkdir d0/d7/d29/d60/da6 0 2024-02-26T10:48:00.757 INFO:tasks.workunit.client.0.smithi060.stdout:9/625: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/f2f [0,4194304] 0 2024-02-26T10:48:00.757 INFO:tasks.workunit.client.0.smithi060.stdout:9/626: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/d47/d7a/fb5 42845 1 2024-02-26T10:48:00.759 INFO:tasks.workunit.client.0.smithi060.stdout:9/627: creat d3/d1e/d5e/d1b/d23/fd5 x:0 0 0 2024-02-26T10:48:00.759 INFO:tasks.workunit.client.0.smithi060.stdout:9/628: stat d3/d1e/d5e/d1b/d23/f36 0 2024-02-26T10:48:00.760 INFO:tasks.workunit.client.0.smithi060.stdout:6/564: dread d6/d7/d25/d1c/d1f/f37 [0,4194304] 0 2024-02-26T10:48:00.763 INFO:tasks.workunit.client.0.smithi060.stdout:9/629: symlink d3/d1e/d5e/ld6 0 2024-02-26T10:48:00.763 INFO:tasks.workunit.client.0.smithi060.stdout:9/630: chown d3/d1e/d5e/d1b/f41 23 1 2024-02-26T10:48:00.769 INFO:tasks.workunit.client.0.smithi060.stdout:6/565: dread d6/d7/d25/d1c/d1f/d23/d28/f29 [0,4194304] 0 2024-02-26T10:48:00.772 INFO:tasks.workunit.client.0.smithi060.stdout:9/631: getdents d3/d1e/d5e/d35/d87/d9b/d8a/dbe 0 2024-02-26T10:48:00.773 INFO:tasks.workunit.client.0.smithi060.stdout:1/616: rename d3/db/c21 to d3/d35/d53/cd7 0 2024-02-26T10:48:00.773 INFO:tasks.workunit.client.0.smithi060.stdout:1/617: dread - d3/db/d1e/d38/f79 zero size 2024-02-26T10:48:00.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/632: creat d3/d1e/d5e/d35/fd7 x:0 0 0 2024-02-26T10:48:00.776 INFO:tasks.workunit.client.0.smithi060.stdout:2/505: rename dd/d43/d60/d41/d37/d38/d57/f87 to dd/d43/d55/d6e/fac 0 2024-02-26T10:48:00.776 INFO:tasks.workunit.client.0.smithi060.stdout:1/618: mknod d3/d35/dc2/cd8 0 2024-02-26T10:48:00.777 INFO:tasks.workunit.client.0.smithi060.stdout:9/633: mknod d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/dd2/cd8 0 2024-02-26T10:48:00.778 INFO:tasks.workunit.client.0.smithi060.stdout:2/506: creat dd/d43/d55/d5e/fad x:0 0 0 2024-02-26T10:48:00.786 INFO:tasks.workunit.client.0.smithi060.stdout:2/507: mknod dd/d43/d60/d41/d37/d69/cae 0 2024-02-26T10:48:00.788 INFO:tasks.workunit.client.0.smithi060.stdout:9/634: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fd9 x:0 0 0 2024-02-26T10:48:00.788 INFO:tasks.workunit.client.0.smithi060.stdout:9/635: fdatasync d3/d1e/d5e/d9e/fbc 0 2024-02-26T10:48:00.794 INFO:tasks.workunit.client.0.smithi060.stdout:2/508: getdents dd/d43/d55 0 2024-02-26T10:48:00.794 INFO:tasks.workunit.client.0.smithi060.stdout:2/509: symlink dd/d43/d96/laf 0 2024-02-26T10:48:00.794 INFO:tasks.workunit.client.0.smithi060.stdout:2/510: write dd/d43/d60/d41/d37/d4c/d76/fa3 [545832,11275] 0 2024-02-26T10:48:00.800 INFO:tasks.workunit.client.0.smithi060.stdout:2/511: link dd/d43/d55/d5e/d74/da5/fa7 dd/d43/fb0 0 2024-02-26T10:48:00.815 INFO:tasks.workunit.client.0.smithi060.stdout:4/539: dwrite d0/da/d3b/f89 [4194304,4194304] 0 2024-02-26T10:48:00.824 INFO:tasks.workunit.client.0.smithi060.stdout:4/540: getdents d0/da/d3b 0 2024-02-26T10:48:00.824 INFO:tasks.workunit.client.0.smithi060.stdout:6/566: dwrite d6/d7/d25/fb0 [0,4194304] 0 2024-02-26T10:48:00.825 INFO:tasks.workunit.client.0.smithi060.stdout:6/567: write d6/d7/d25/f2d [1708184,82021] 0 2024-02-26T10:48:00.825 INFO:tasks.workunit.client.0.smithi060.stdout:0/496: dread d0/dd/f31 [0,4194304] 0 2024-02-26T10:48:00.826 INFO:tasks.workunit.client.0.smithi060.stdout:5/533: dwrite d0/de/d15/f47 [0,4194304] 0 2024-02-26T10:48:00.833 INFO:tasks.workunit.client.0.smithi060.stdout:7/684: dwrite da/d15/d21/d2f/f91 [0,4194304] 0 2024-02-26T10:48:00.833 INFO:tasks.workunit.client.0.smithi060.stdout:8/551: dread d9/f1e [0,4194304] 0 2024-02-26T10:48:00.835 INFO:tasks.workunit.client.0.smithi060.stdout:1/619: dwrite d3/db/d1e/d38/d6c/fcb [0,4194304] 0 2024-02-26T10:48:00.836 INFO:tasks.workunit.client.0.smithi060.stdout:6/568: creat d6/d7/d25/d1c/d1f/fbb x:0 0 0 2024-02-26T10:48:00.836 INFO:tasks.workunit.client.0.smithi060.stdout:2/512: dread dd/d43/d60/d41/d37/f80 [0,4194304] 0 2024-02-26T10:48:00.837 INFO:tasks.workunit.client.0.smithi060.stdout:2/513: readlink dd/d43/d60/d41/d37/d38/l61 0 2024-02-26T10:48:00.837 INFO:tasks.workunit.client.0.smithi060.stdout:3/443: dwrite d5/de/df/f81 [0,4194304] 0 2024-02-26T10:48:00.837 INFO:tasks.workunit.client.0.smithi060.stdout:9/636: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/f1f [0,4194304] 0 2024-02-26T10:48:00.840 INFO:tasks.workunit.client.0.smithi060.stdout:9/637: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8/dcd/fda x:0 0 0 2024-02-26T10:48:00.841 INFO:tasks.workunit.client.0.smithi060.stdout:5/534: symlink d0/de/d20/d32/d4d/da3/da5/lba 0 2024-02-26T10:48:00.841 INFO:tasks.workunit.client.0.smithi060.stdout:7/685: creat da/d15/d21/d41/dc7/fef x:0 0 0 2024-02-26T10:48:00.843 INFO:tasks.workunit.client.0.smithi060.stdout:0/497: symlink d0/dd/d53/d87/d94/da6/la7 0 2024-02-26T10:48:00.844 INFO:tasks.workunit.client.0.smithi060.stdout:6/569: mkdir d6/d7/d25/d1c/d1f/d23/d7a/dbc 0 2024-02-26T10:48:00.845 INFO:tasks.workunit.client.0.smithi060.stdout:5/535: mknod d0/de/d70/cbb 0 2024-02-26T10:48:00.845 INFO:tasks.workunit.client.0.smithi060.stdout:5/536: fsync d0/de/d20/fa8 0 2024-02-26T10:48:00.845 INFO:tasks.workunit.client.0.smithi060.stdout:0/498: link d0/dd/d33/f88 d0/dd/fa8 0 2024-02-26T10:48:00.846 INFO:tasks.workunit.client.0.smithi060.stdout:0/499: write d0/d37/f6d [1088443,108137] 0 2024-02-26T10:48:00.861 INFO:tasks.workunit.client.0.smithi060.stdout:4/541: link d0/d27/d52/f62 d0/da/d3b/d1b/fa7 0 2024-02-26T10:48:00.861 INFO:tasks.workunit.client.0.smithi060.stdout:4/542: chown d0/d7/d29/c79 243 1 2024-02-26T10:48:00.862 INFO:tasks.workunit.client.0.smithi060.stdout:4/543: fsync d0/da/d57/f58 0 2024-02-26T10:48:00.864 INFO:tasks.workunit.client.0.smithi060.stdout:0/500: link d0/dd/d39/d40/f1f d0/dd/d53/d87/fa9 0 2024-02-26T10:48:00.864 INFO:tasks.workunit.client.0.smithi060.stdout:0/501: stat d0/d42 0 2024-02-26T10:48:00.867 INFO:tasks.workunit.client.0.smithi060.stdout:1/620: rename d3/db/dc8/d91 to d3/db/d1e/d6d/dd9 0 2024-02-26T10:48:00.867 INFO:tasks.workunit.client.0.smithi060.stdout:1/621: chown d3/db/d1e/d39/c5d 106024 1 2024-02-26T10:48:00.867 INFO:tasks.workunit.client.0.smithi060.stdout:4/544: rmdir d0/d7/d29/d60/da6 0 2024-02-26T10:48:00.867 INFO:tasks.workunit.client.0.smithi060.stdout:4/545: stat d0/da/d57/f65 0 2024-02-26T10:48:00.868 INFO:tasks.workunit.client.0.smithi060.stdout:0/502: creat d0/dd/d53/da2/faa x:0 0 0 2024-02-26T10:48:00.868 INFO:tasks.workunit.client.0.smithi060.stdout:0/503: creat d0/dd/d33/d6a/d6c/fab x:0 0 0 2024-02-26T10:48:00.870 INFO:tasks.workunit.client.0.smithi060.stdout:0/504: symlink d0/d42/d7a/d7f/lac 0 2024-02-26T10:48:00.870 INFO:tasks.workunit.client.0.smithi060.stdout:0/505: chown d0/f55 119 1 2024-02-26T10:48:00.872 INFO:tasks.workunit.client.0.smithi060.stdout:0/506: rename d0/dd/da4 to d0/dd/d39/d40/dad 0 2024-02-26T10:48:00.873 INFO:tasks.workunit.client.0.smithi060.stdout:8/552: write d9/d15/f36 [256337,43045] 0 2024-02-26T10:48:00.874 INFO:tasks.workunit.client.0.smithi060.stdout:0/507: rename d0/c92 to d0/dd/cae 0 2024-02-26T10:48:00.875 INFO:tasks.workunit.client.0.smithi060.stdout:8/553: creat d9/d15/d29/d2f/d34/d72/d50/fb0 x:0 0 0 2024-02-26T10:48:00.875 INFO:tasks.workunit.client.0.smithi060.stdout:4/546: symlink d0/da/d3b/d1b/la8 0 2024-02-26T10:48:00.875 INFO:tasks.workunit.client.0.smithi060.stdout:4/547: readlink d0/l2c 0 2024-02-26T10:48:00.877 INFO:tasks.workunit.client.0.smithi060.stdout:0/508: truncate d0/f3 2192586 0 2024-02-26T10:48:00.904 INFO:tasks.workunit.client.0.smithi060.stdout:1/622: link d3/d35/d5e/c7f d3/db/d1e/d38/cda 0 2024-02-26T10:48:00.928 INFO:tasks.workunit.client.0.smithi060.stdout:4/548: dread d0/da/d57/f96 [0,4194304] 0 2024-02-26T10:48:00.928 INFO:tasks.workunit.client.0.smithi060.stdout:4/549: write d0/da/f80 [360308,99908] 0 2024-02-26T10:48:00.942 INFO:tasks.workunit.client.0.smithi060.stdout:9/638: dwrite d3/d1e/d5e/d35/f3e [0,4194304] 0 2024-02-26T10:48:00.943 INFO:tasks.workunit.client.0.smithi060.stdout:7/686: dread da/d15/d88/dce/f5f [0,4194304] 0 2024-02-26T10:48:00.943 INFO:tasks.workunit.client.0.smithi060.stdout:5/537: dwrite d0/de/d20/d82/f87 [0,4194304] 0 2024-02-26T10:48:00.946 INFO:tasks.workunit.client.0.smithi060.stdout:5/538: mknod d0/de/d70/d84/cbc 0 2024-02-26T10:48:00.947 INFO:tasks.workunit.client.0.smithi060.stdout:5/539: chown d0/d4b/d7b/l95 37355390 1 2024-02-26T10:48:00.950 INFO:tasks.workunit.client.0.smithi060.stdout:3/444: dwrite d5/f6 [4194304,4194304] 0 2024-02-26T10:48:00.951 INFO:tasks.workunit.client.0.smithi060.stdout:0/509: dwrite d0/dd/d33/f4a [0,4194304] 0 2024-02-26T10:48:00.961 INFO:tasks.workunit.client.0.smithi060.stdout:3/445: creat d5/de/f84 x:0 0 0 2024-02-26T10:48:00.962 INFO:tasks.workunit.client.0.smithi060.stdout:0/510: getdents d0/dd/d39/d93 0 2024-02-26T10:48:00.962 INFO:tasks.workunit.client.0.smithi060.stdout:0/511: fsync d0/f55 0 2024-02-26T10:48:00.963 INFO:tasks.workunit.client.0.smithi060.stdout:9/639: rename d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/c3f to d3/d1e/d5e/d1b/cdb 0 2024-02-26T10:48:00.965 INFO:tasks.workunit.client.0.smithi060.stdout:5/540: rename d0/de/f6a to d0/de/da4/fbd 0 2024-02-26T10:48:00.965 INFO:tasks.workunit.client.0.smithi060.stdout:9/640: rmdir d3/d1e/d93/da2/db7 39 2024-02-26T10:48:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.984+0000 7fa541400640 1 -- 172.21.15.60:0/850650218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53c100160 msgr2=0x7fa53c100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.984+0000 7fa541400640 1 --2- 172.21.15.60:0/850650218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53c100160 0x7fa53c100580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa534009ca0 tx=0x7fa53402c100 comp rx=0 tx=0).stop 2024-02-26T10:48:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.984+0000 7fa541400640 1 -- 172.21.15.60:0/850650218 shutdown_connections 2024-02-26T10:48:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.984+0000 7fa541400640 1 --2- 172.21.15.60:0/850650218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53c101d10 0x7fa53c1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.984+0000 7fa541400640 1 --2- 172.21.15.60:0/850650218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53c101350 0x7fa53c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:00.978 INFO:tasks.workunit.client.0.smithi060.stdout:9/641: dread d3/d1e/d5e/d35/d87/d9b/f8d [0,4194304] 0 2024-02-26T10:48:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.985+0000 7fa541400640 1 --2- 172.21.15.60:0/850650218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53c100160 0x7fa53c100580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.985+0000 7fa541400640 1 -- 172.21.15.60:0/850650218 >> 172.21.15.60:0/850650218 conn(0x7fa53c0fb6f0 msgr2=0x7fa53c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.985+0000 7fa541400640 1 -- 172.21.15.60:0/850650218 shutdown_connections 2024-02-26T10:48:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.985+0000 7fa541400640 1 -- 172.21.15.60:0/850650218 wait complete. 2024-02-26T10:48:00.980 INFO:tasks.workunit.client.0.smithi060.stdout:7/687: write da/d15/d88/dce/fab [2155552,84297] 0 2024-02-26T10:48:00.980 INFO:tasks.workunit.client.0.smithi060.stdout:7/688: write f6 [1291527,95698] 0 2024-02-26T10:48:00.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.986+0000 7fa541400640 1 Processor -- start 2024-02-26T10:48:00.982 INFO:tasks.workunit.client.0.smithi060.stdout:9/642: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fdc x:0 0 0 2024-02-26T10:48:00.982 INFO:tasks.workunit.client.0.smithi060.stdout:7/689: creat da/d15/d88/dce/dea/ff0 x:0 0 0 2024-02-26T10:48:00.982 INFO:tasks.workunit.client.0.smithi060.stdout:7/690: getdents da/d15/d21/d2f/d78/dd3/d67 0 2024-02-26T10:48:00.982 INFO:tasks.workunit.client.0.smithi060.stdout:7/691: chown da/d15/c9b 57475 1 2024-02-26T10:48:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.987+0000 7fa541400640 1 -- start start 2024-02-26T10:48:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.988+0000 7fa541400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53c100160 0x7fa53c068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.988+0000 7fa541400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53c101350 0x7fa53c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.988+0000 7fa541400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53c101d10 0x7fa53c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.988+0000 7fa541400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c10bdf0 con 0x7fa53c100160 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.988+0000 7fa541400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c10bc70 con 0x7fa53c101d10 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.988+0000 7fa541400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c10bf70 con 0x7fa53c101350 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.988+0000 7fa53a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53c100160 0x7fa53c068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.988+0000 7fa53a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53c100160 0x7fa53c068f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46164/0 (socket says 172.21.15.60:46164) 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.988+0000 7fa53a800640 1 -- 172.21.15.60:0/3250288060 learned_addr learned my addr 172.21.15.60:0/3250288060 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.989+0000 7fa53b200640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53c101d10 0x7fa53c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.989+0000 7fa539e00640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53c101350 0x7fa53c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.989+0000 7fa53b200640 1 -- 172.21.15.60:0/3250288060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53c101350 msgr2=0x7fa53c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.989+0000 7fa53b200640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53c101350 0x7fa53c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.989+0000 7fa53b200640 1 -- 172.21.15.60:0/3250288060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53c100160 msgr2=0x7fa53c068f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.989+0000 7fa53b200640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53c100160 0x7fa53c068f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.989+0000 7fa53b200640 1 -- 172.21.15.60:0/3250288060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c1a13a0 con 0x7fa53c101d10 2024-02-26T10:48:00.985 INFO:tasks.workunit.client.0.smithi060.stdout:7/692: truncate da/d15/d21/f72 3953742 0 2024-02-26T10:48:00.985 INFO:tasks.workunit.client.0.smithi060.stdout:7/693: truncate da/d15/d88/d3b/f9d 759087 0 2024-02-26T10:48:00.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.991+0000 7fa53b200640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53c101d10 0x7fa53c1a0bb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e970 tx=0x7fa52c00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:00.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.992+0000 7fa533400640 1 -- 172.21.15.60:0/3250288060 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00ccc0 con 0x7fa53c101d10 2024-02-26T10:48:00.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.992+0000 7fa533400640 1 -- 172.21.15.60:0/3250288060 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa52c009e40 con 0x7fa53c101d10 2024-02-26T10:48:00.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.992+0000 7fa541400640 1 -- 172.21.15.60:0/3250288060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c1a1630 con 0x7fa53c101d10 2024-02-26T10:48:00.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.992+0000 7fa533400640 1 -- 172.21.15.60:0/3250288060 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c004a00 con 0x7fa53c101d10 2024-02-26T10:48:00.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.994+0000 7fa541400640 1 -- 172.21.15.60:0/3250288060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c1a1b20 con 0x7fa53c101d10 2024-02-26T10:48:00.991 INFO:tasks.workunit.client.0.smithi060.stdout:6/570: dwrite d6/d7/ff [0,4194304] 0 2024-02-26T10:48:00.994 INFO:tasks.workunit.client.0.smithi060.stdout:1/623: dwrite d3/db/d1e/d39/f58 [0,4194304] 0 2024-02-26T10:48:00.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:00.999+0000 7fa54442b740 1 Processor -- start 2024-02-26T10:48:00.995 INFO:tasks.workunit.client.0.smithi060.stdout:0/512: write d0/dd/d39/d40/f4d [2090947,5104] 0 2024-02-26T10:48:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.001+0000 7fa54442b740 1 -- start start 2024-02-26T10:48:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.001+0000 7fa533400640 1 -- 172.21.15.60:0/3250288060 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa52c004ba0 con 0x7fa53c101d10 2024-02-26T10:48:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.002+0000 7fa54442b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74686750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.002+0000 7fa54442b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c7468b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.002+0000 7fa54442b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 0x560c7468e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.002+0000 7fa54442b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c745ab060 con 0x560c7468be90 2024-02-26T10:48:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.002+0000 7fa54442b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c745e2d00 con 0x560c74686d20 2024-02-26T10:48:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.002+0000 7fa54442b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7458ba00 con 0x560c74686330 2024-02-26T10:48:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.002+0000 7fa539e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74686750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:00.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.003+0000 7fa539e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74686750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:32946/0 (socket says 172.21.15.60:32946) 2024-02-26T10:48:00.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.003+0000 7fa539e00640 1 -- 172.21.15.60:0/2884266512 learned_addr learned my addr 172.21.15.60:0/2884266512 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:00.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.003+0000 7fa53a800640 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c7468b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:00.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.003+0000 7fa539e00640 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 0x560c7468e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:00.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.003+0000 7fa533400640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5200a1e30 0x7fa5200a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:00.998 INFO:tasks.workunit.client.0.smithi060.stdout:0/513: rename d0/f15 to d0/dd/faf 0 2024-02-26T10:48:00.998 INFO:tasks.workunit.client.0.smithi060.stdout:0/514: fsync d0/dd/d53/f64 0 2024-02-26T10:48:00.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.004+0000 7fa539e00640 1 -- 172.21.15.60:0/2884266512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 msgr2=0x560c7468b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:00.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.004+0000 7fa539e00640 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c7468b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:00.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.004+0000 7fa539e00640 1 -- 172.21.15.60:0/2884266512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 msgr2=0x560c7468e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:00.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.004+0000 7fa539e00640 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 0x560c7468e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:00.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.004+0000 7fa539e00640 1 -- 172.21.15.60:0/2884266512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c743c17d0 con 0x560c74686330 2024-02-26T10:48:00.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.004+0000 7fa53a800640 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c7468b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:00.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.004+0000 7fa533400640 1 -- 172.21.15.60:0/3250288060 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa52c0cd530 con 0x7fa53c101d10 2024-02-26T10:48:00.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.005+0000 7fa53b200640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5200a1e30 0x7fa5200a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:00.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.005+0000 7fa539e00640 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74686750 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa524013980 tx=0x7fa52400a820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2fde51849d59fe server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:00.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.005+0000 7fa52b400640 1 -- 172.21.15.60:0/2884266512 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524011cb0 con 0x560c74686330 2024-02-26T10:48:01.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.006+0000 7fa52b400640 1 -- 172.21.15.60:0/2884266512 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa524011e50 con 0x560c74686330 2024-02-26T10:48:01.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.007+0000 7fa54442b740 1 -- 172.21.15.60:0/2884266512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 msgr2=0x560c74686750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.007+0000 7fa54442b740 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74686750 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa524013980 tx=0x7fa52400a820 comp rx=0 tx=0).stop 2024-02-26T10:48:01.001 INFO:tasks.workunit.client.0.smithi060.stdout:0/515: rename d0/d37/d44/d58/f73 to d0/dd/fb0 0 2024-02-26T10:48:01.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.007+0000 7fa53b200640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5200a1e30 0x7fa5200a42f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0a22a0 tx=0x7fa52c0053b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:01.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.008+0000 7fa54442b740 1 -- 172.21.15.60:0/2884266512 shutdown_connections 2024-02-26T10:48:01.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.008+0000 7fa54442b740 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c7468b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.008+0000 7fa54442b740 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 0x560c7468e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.008+0000 7fa54442b740 1 --2- 172.21.15.60:0/2884266512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74686750 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.008+0000 7fa54442b740 1 -- 172.21.15.60:0/2884266512 >> 172.21.15.60:0/2884266512 conn(0x560c74679090 msgr2=0x560c74679490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.008+0000 7fa54442b740 1 -- 172.21.15.60:0/2884266512 shutdown_connections 2024-02-26T10:48:01.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.008+0000 7fa54442b740 1 -- 172.21.15.60:0/2884266512 wait complete. 2024-02-26T10:48:01.002 INFO:tasks.workunit.client.0.smithi060.stdout:3/446: dwrite d5/de/f84 [0,4194304] 0 2024-02-26T10:48:01.003 INFO:tasks.workunit.client.0.smithi060.stdout:0/516: creat d0/d42/d7a/d7f/fb1 x:0 0 0 2024-02-26T10:48:01.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.009+0000 7fa54442b740 1 -- start start 2024-02-26T10:48:01.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.009+0000 7fa54442b740 1 Processor -- start 2024-02-26T10:48:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.010+0000 7fa54442b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74690730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.010+0000 7fa54442b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74690c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.010+0000 7fa54442b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 0x560c74697c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.005 INFO:tasks.workunit.client.0.smithi060.stdout:3/447: mknod d5/de/df/d57/d7a/c85 0 2024-02-26T10:48:01.005 INFO:tasks.workunit.client.0.smithi060.stdout:0/517: mknod d0/dd/d39/d40/d83/cb2 0 2024-02-26T10:48:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.010+0000 7fa54442b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c74581090 con 0x560c7468be90 2024-02-26T10:48:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.010+0000 7fa54442b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c745deda0 con 0x560c74686d20 2024-02-26T10:48:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.010+0000 7fa54442b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7468ee90 con 0x560c74686330 2024-02-26T10:48:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.010+0000 7fa54442b740 10 client.0 _ll_register_callbacks cb 0x560c7459e830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.010+0000 7fa539e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74690730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.011+0000 7fa539e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74690730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:32956/0 (socket says 172.21.15.60:32956) 2024-02-26T10:48:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.011+0000 7fa539e00640 1 -- 172.21.15.60:0/3783919527 learned_addr learned my addr 172.21.15.60:0/3783919527 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.011+0000 7fa539e00640 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 0x560c74697c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.012+0000 7fa53a800640 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74690c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.007 INFO:tasks.workunit.client.0.smithi060.stdout:0/518: mknod d0/dd/d53/da2/cb3 0 2024-02-26T10:48:01.007 INFO:tasks.workunit.client.0.smithi060.stdout:0/519: truncate d0/d37/d44/f66 288559 0 2024-02-26T10:48:01.007 INFO:tasks.workunit.client.0.smithi060.stdout:0/520: fsync d0/d37/f6d 0 2024-02-26T10:48:01.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.013+0000 7fa53a800640 1 -- 172.21.15.60:0/3783919527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 msgr2=0x560c74690730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:01.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.013+0000 7fa53a800640 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74690730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.013+0000 7fa53a800640 1 -- 172.21.15.60:0/3783919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 msgr2=0x560c74697c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:01.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.013+0000 7fa53a800640 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 0x560c74697c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.013+0000 7fa53a800640 1 -- 172.21.15.60:0/3783919527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c742f73f0 con 0x560c74686d20 2024-02-26T10:48:01.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.013+0000 7fa53a800640 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74690c70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa534000f80 tx=0x7fa534032c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c13ce0d802b3953 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:01.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.013+0000 7fa51fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:01.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.014+0000 7fa51fe00640 1 -- 172.21.15.60:0/3783919527 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa534044070 con 0x560c74686d20 2024-02-26T10:48:01.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.014+0000 7fa51fe00640 1 -- 172.21.15.60:0/3783919527 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa534003970 con 0x560c74686d20 2024-02-26T10:48:01.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.014+0000 7fa54442b740 1 -- 172.21.15.60:0/3783919527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa524011e50 con 0x560c74686d20 2024-02-26T10:48:01.009 INFO:tasks.workunit.client.0.smithi060.stdout:0/521: symlink d0/dd/d39/d40/lb4 0 2024-02-26T10:48:01.009 INFO:tasks.workunit.client.0.smithi060.stdout:0/522: fsync d0/dd/f31 0 2024-02-26T10:48:01.009 INFO:tasks.workunit.client.0.smithi060.stdout:0/523: chown d0/dd/f5d 565755101 1 2024-02-26T10:48:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.015+0000 7fa51fe00640 1 -- 172.21.15.60:0/3783919527 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa534039040 con 0x560c74686d20 2024-02-26T10:48:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.015+0000 7fa54442b740 10 client.26051 fetch_fsmap learned FSMap version 29 2024-02-26T10:48:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.015+0000 7fa54442b740 1 -- 172.21.15.60:0/3783919527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c742b70f0 con 0x560c74686d20 2024-02-26T10:48:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.016+0000 7fa51fe00640 1 -- 172.21.15.60:0/3783919527 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa524011e50 con 0x560c74686d20 2024-02-26T10:48:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.016+0000 7fa51fe00640 1 -- 172.21.15.60:0/3783919527 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa534040460 con 0x560c74686d20 2024-02-26T10:48:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.016+0000 7fa54442b740 10 client.26051 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:48:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.016+0000 7fa54442b740 10 client.26051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.016+0000 7fa54442b740 20 client.26051 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:01.011 INFO:tasks.workunit.client.0.smithi060.stdout:0/524: truncate d0/dd/f2d 3452150 0 2024-02-26T10:48:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.016+0000 7fa54442b740 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560c746ad250 0x560c746af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.016+0000 7fa54442b740 4 client.26051 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:48:01.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.016+0000 7fa54442b740 1 -- 172.21.15.60:0/3783919527 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c745f0240 con 0x560c746ad250 2024-02-26T10:48:01.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.016+0000 7fa539e00640 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560c746ad250 0x560c746af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.012 INFO:tasks.workunit.client.0.smithi060.stdout:0/525: write d0/d37/d44/f79 [933602,24765] 0 2024-02-26T10:48:01.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.017+0000 7fa539e00640 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560c746ad250 0x560c746af670 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a177c47176274e7a server_cookie=de8d3ca31adea689 in_seq=0 out_seq=0 2024-02-26T10:48:01.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.017+0000 7fa51fe00640 10 client.26051 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:01.013 INFO:tasks.workunit.client.0.smithi060.stdout:1/624: link d3/db/d1e/d38/d6c/cca d3/d35/d53/cdb 0 2024-02-26T10:48:01.013 INFO:tasks.workunit.client.0.smithi060.stdout:0/526: rmdir d0/dd/d53/d87 39 2024-02-26T10:48:01.013 INFO:tasks.workunit.client.0.smithi060.stdout:0/527: write d0/f78 [5112742,64538] 0 2024-02-26T10:48:01.013 INFO:tasks.workunit.client.0.smithi060.stdout:0/528: readlink d0/dd/d53/d87/l91 0 2024-02-26T10:48:01.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa51fe00640 1 -- 172.21.15.60:0/3783919527 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c745f0240 con 0x560c746ad250 2024-02-26T10:48:01.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa51fe00640 10 client.26051 handle_command_reply: tid=0 2024-02-26T10:48:01.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 client.26051 shutdown 2024-02-26T10:48:01.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 -- 172.21.15.60:0/3783919527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 msgr2=0x560c74690c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74690c70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa534000f80 tx=0x7fa534032c90 comp rx=0 tx=0).stop 2024-02-26T10:48:01.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 -- 172.21.15.60:0/3783919527 shutdown_connections 2024-02-26T10:48:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560c746ad250 0x560c746af670 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74690c70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c7468be90 0x560c74697c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa51fe00640 0 client.26051 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 --2- 172.21.15.60:0/3783919527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686330 0x560c74690730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 -- 172.21.15.60:0/3783919527 >> 172.21.15.60:0/3783919527 conn(0x560c74679090 msgr2=0x560c74678330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 -- 172.21.15.60:0/3783919527 shutdown_connections 2024-02-26T10:48:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.019+0000 7fa54442b740 1 -- 172.21.15.60:0/3783919527 wait complete. 2024-02-26T10:48:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.020+0000 7fa54442b740 20 client.26051 trim_cache size 0 max 16384 2024-02-26T10:48:01.015 INFO:tasks.workunit.client.0.smithi060.stdout:0/529: symlink d0/dd/d53/d87/d94/lb5 0 2024-02-26T10:48:01.019 INFO:tasks.workunit.client.0.smithi060.stdout:1/625: link d3/db/d1e/d38/d6c/f88 d3/d35/d53/d68/fdc 0 2024-02-26T10:48:01.019 INFO:tasks.workunit.client.0.smithi060.stdout:0/530: truncate d0/dd/f31 3585929 0 2024-02-26T10:48:01.019 INFO:tasks.workunit.client.0.smithi060.stdout:0/531: fsync d0/dd/d33/d6a/d6c/f9a 0 2024-02-26T10:48:01.020 INFO:tasks.workunit.client.0.smithi060.stdout:1/626: symlink d3/db/ldd 0 2024-02-26T10:48:01.022 INFO:tasks.workunit.client.0.smithi060.stdout:0/532: getdents d0/dd/d39/d63 0 2024-02-26T10:48:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.039+0000 7fa54442b740 1 Processor -- start 2024-02-26T10:48:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.039+0000 7fa54442b740 1 -- start start 2024-02-26T10:48:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.039+0000 7fa54442b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 0x560c74688270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.039+0000 7fa54442b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686d20 0x560c74697c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.039+0000 7fa54442b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c7468be90 0x560c746887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.039+0000 7fa54442b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c072f80 con 0x560c74686330 2024-02-26T10:48:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.039+0000 7fa54442b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c070ac0 con 0x560c7468be90 2024-02-26T10:48:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.039+0000 7fa54442b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c06eb60 con 0x560c74686d20 2024-02-26T10:48:01.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa53a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686d20 0x560c74697c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa53a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686d20 0x560c74697c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:32958/0 (socket says 172.21.15.60:32958) 2024-02-26T10:48:01.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa53a800640 1 -- 172.21.15.60:0/2870093519 learned_addr learned my addr 172.21.15.60:0/2870093519 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:01.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa539e00640 1 --2- 172.21.15.60:0/2870093519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 0x560c74688270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.035 INFO:tasks.workunit.client.0.smithi060.stdout:4/550: dwrite d0/d7/f6a [0,4194304] 0 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa53a800640 1 -- 172.21.15.60:0/2870093519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c7468be90 msgr2=0x560c746887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa53a800640 1 --2- 172.21.15.60:0/2870093519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c7468be90 0x560c746887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa53a800640 1 -- 172.21.15.60:0/2870093519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 msgr2=0x560c74688270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa53a800640 1 --2- 172.21.15.60:0/2870093519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 0x560c74688270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa53a800640 1 -- 172.21.15.60:0/2870093519 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c7468f010 con 0x560c74686d20 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.040+0000 7fa53a800640 1 --2- 172.21.15.60:0/2870093519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686d20 0x560c74697c00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa534009bf0 tx=0x7fa534009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68a6437154940540 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.041+0000 7fa52aa00640 1 -- 172.21.15.60:0/2870093519 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa534058070 con 0x560c74686d20 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.041+0000 7fa52aa00640 1 -- 172.21.15.60:0/2870093519 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa534005bd0 con 0x560c74686d20 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.041+0000 7fa52aa00640 1 -- 172.21.15.60:0/2870093519 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa534044040 con 0x560c74686d20 2024-02-26T10:48:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 -- 172.21.15.60:0/2870093519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686d20 msgr2=0x560c74697c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 --2- 172.21.15.60:0/2870093519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686d20 0x560c74697c00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa534009bf0 tx=0x7fa534009540 comp rx=0 tx=0).stop 2024-02-26T10:48:01.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 -- 172.21.15.60:0/2870093519 shutdown_connections 2024-02-26T10:48:01.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 --2- 172.21.15.60:0/2870093519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c7468be90 0x560c746887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 --2- 172.21.15.60:0/2870093519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c74686d20 0x560c74697c00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 --2- 172.21.15.60:0/2870093519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 0x560c74688270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 -- 172.21.15.60:0/2870093519 >> 172.21.15.60:0/2870093519 conn(0x560c74679090 msgr2=0x560c7468f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:01.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 -- 172.21.15.60:0/2870093519 shutdown_connections 2024-02-26T10:48:01.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 -- 172.21.15.60:0/2870093519 wait complete. 2024-02-26T10:48:01.037 INFO:tasks.workunit.client.0.smithi060.stdout:4/551: rename d0/da/d57/c6d to d0/da/d57/ca9 0 2024-02-26T10:48:01.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.042+0000 7fa54442b740 1 -- start start 2024-02-26T10:48:01.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.043+0000 7fa54442b740 1 Processor -- start 2024-02-26T10:48:01.038 INFO:tasks.workunit.client.0.smithi060.stdout:4/552: truncate d0/da/d3b/d16/f6c 1636332 0 2024-02-26T10:48:01.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.043+0000 7fa54442b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 0x560c74697930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.043+0000 7fa54442b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74695f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.043+0000 7fa54442b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c7468be90 0x560c74696480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.043+0000 7fa54442b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c06f160 con 0x560c74686330 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.043+0000 7fa54442b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c070460 con 0x560c74686d20 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.043+0000 7fa54442b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c06f7a0 con 0x560c7468be90 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.043+0000 7fa54442b740 10 client.0 _ll_register_callbacks cb 0x7fa534009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.044+0000 7fa539e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 0x560c74697930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.044+0000 7fa53a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74695f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.044+0000 7fa539e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 0x560c74697930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46202/0 (socket says 172.21.15.60:46202) 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.044+0000 7fa539e00640 1 -- 172.21.15.60:0/1361288812 learned_addr learned my addr 172.21.15.60:0/1361288812 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.044+0000 7fa539e00640 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c7468be90 0x560c74696480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa53a800640 1 -- 172.21.15.60:0/1361288812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c7468be90 msgr2=0x560c74696480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa53a800640 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c7468be90 0x560c74696480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa53a800640 1 -- 172.21.15.60:0/1361288812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 msgr2=0x560c74697930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa53a800640 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 0x560c74697930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa53a800640 1 -- 172.21.15.60:0/1361288812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c74696af0 con 0x560c74686d20 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa539e00640 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c7468be90 0x560c74696480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa53a800640 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74695f40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa534000f80 tx=0x7fa53402c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f406a40df18734 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa51fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa51fe00640 1 -- 172.21.15.60:0/1361288812 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa534058070 con 0x560c74686d20 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa54442b740 1 -- 172.21.15.60:0/1361288812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa534005bd0 con 0x560c74686d20 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.045+0000 7fa51fe00640 1 -- 172.21.15.60:0/1361288812 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa5340329f0 con 0x560c74686d20 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.046+0000 7fa51fe00640 1 -- 172.21.15.60:0/1361288812 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa534044070 con 0x560c74686d20 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.046+0000 7fa54442b740 10 client.26054 fetch_fsmap learned FSMap version 29 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.046+0000 7fa54442b740 1 -- 172.21.15.60:0/1361288812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c747045c0 con 0x560c74686d20 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.046+0000 7fa51fe00640 1 -- 172.21.15.60:0/1361288812 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa534005bd0 con 0x560c74686d20 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.047+0000 7fa51fe00640 1 -- 172.21.15.60:0/1361288812 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa53405b020 con 0x560c74686d20 2024-02-26T10:48:01.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.047+0000 7fa54442b740 10 client.26054 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:48:01.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.047+0000 7fa54442b740 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:01.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.047+0000 7fa54442b740 20 client.26054 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:01.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.047+0000 7fa54442b740 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560c74704ee0 0x560c74707300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.047+0000 7fa54442b740 4 client.26054 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T10:48:01.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.047+0000 7fa54442b740 1 -- 172.21.15.60:0/1361288812 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c745ea590 con 0x560c74704ee0 2024-02-26T10:48:01.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.047+0000 7fa539e00640 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560c74704ee0 0x560c74707300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.049+0000 7fa51fe00640 10 client.26054 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:01.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.049+0000 7fa539e00640 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560c74704ee0 0x560c74707300 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf5f5f5dbf8b9b7 server_cookie=bf1668c23418a0ec in_seq=0 out_seq=0 2024-02-26T10:48:01.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.049+0000 7fa51fe00640 1 -- 172.21.15.60:0/1361288812 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560c745ea590 con 0x560c74704ee0 2024-02-26T10:48:01.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.049+0000 7fa51fe00640 10 client.26054 handle_command_reply: tid=0 2024-02-26T10:48:01.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.049+0000 7fa54442b740 1 client.26054 shutdown 2024-02-26T10:48:01.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 -- 172.21.15.60:0/1361288812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 msgr2=0x560c74695f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74695f40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa534000f80 tx=0x7fa53402c100 comp rx=0 tx=0).stop 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 -- 172.21.15.60:0/1361288812 shutdown_connections 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560c74704ee0 0x560c74707300 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c74686d20 0x560c74695f40 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa534000f80 tx=0x7fa53402c100 comp rx=0 tx=0).stop 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c7468be90 0x560c74696480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 --2- 172.21.15.60:0/1361288812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c74686330 0x560c74697930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 -- 172.21.15.60:0/1361288812 >> 172.21.15.60:0/1361288812 conn(0x560c74679090 msgr2=0x560c74688270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa51fe00640 0 client.26054 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 -- 172.21.15.60:0/1361288812 shutdown_connections 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 1 -- 172.21.15.60:0/1361288812 wait complete. 2024-02-26T10:48:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.050+0000 7fa54442b740 20 client.26054 trim_cache size 0 max 16384 2024-02-26T10:48:01.046 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:48:01.046 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-02-26T10:48:01.046 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-02-26T10:48:01.046 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:48:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 -- 172.21.15.60:0/3250288060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5200a1e30 msgr2=0x7fa5200a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5200a1e30 0x7fa5200a42f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0a22a0 tx=0x7fa52c0053b0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 -- 172.21.15.60:0/3250288060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53c101d10 msgr2=0x7fa53c1a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53c101d10 0x7fa53c1a0bb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e970 tx=0x7fa52c00ed70 comp rx=0 tx=0).stop 2024-02-26T10:48:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 -- 172.21.15.60:0/3250288060 shutdown_connections 2024-02-26T10:48:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa5200a1e30 0x7fa5200a42f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53c101d10 0x7fa53c1a0bb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53c101350 0x7fa53c19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 --2- 172.21.15.60:0/3250288060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53c100160 0x7fa53c068f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 -- 172.21.15.60:0/3250288060 >> 172.21.15.60:0/3250288060 conn(0x7fa53c0fb6f0 msgr2=0x7fa53c104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:01.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 -- 172.21.15.60:0/3250288060 shutdown_connections 2024-02-26T10:48:01.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.051+0000 7fa541400640 1 -- 172.21.15.60:0/3250288060 wait complete. 2024-02-26T10:48:01.054 INFO:tasks.workunit.client.0.smithi060.stdout:7/694: dwrite da/d66/dad/f56 [0,4194304] 0 2024-02-26T10:48:01.054 INFO:tasks.workunit.client.0.smithi060.stdout:4/553: dread d0/da/d3b/f34 [4194304,4194304] 0 2024-02-26T10:48:01.057 INFO:tasks.workunit.client.0.smithi060.stdout:4/554: creat d0/d27/d76/faa x:0 0 0 2024-02-26T10:48:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:00 smithi060 ceph-mon[31539]: pgmap v486: 129 pgs: 129 active+clean; 831 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s rd, 124 MiB/s wr, 650 op/s 2024-02-26T10:48:01.064 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T10:48:01.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:48:01.072 INFO:tasks.workunit.client.0.smithi060.stdout:6/571: dwrite d6/d7/d25/d1c/d39/f44 [0,4194304] 0 2024-02-26T10:48:01.072 INFO:tasks.workunit.client.0.smithi060.stdout:6/572: chown d6/d6e/f96 27 1 2024-02-26T10:48:01.072 INFO:tasks.workunit.client.0.smithi060.stdout:6/573: fsync d6/d7/d25/d1c/d1f/d23/d28/f34 0 2024-02-26T10:48:01.072 INFO:tasks.workunit.client.0.smithi060.stdout:6/574: chown d6/d7/d25/d1c/d39 6124 1 2024-02-26T10:48:01.072 INFO:tasks.workunit.client.0.smithi060.stdout:6/575: chown d6/d6e/d6f 1535291 1 2024-02-26T10:48:01.075 INFO:tasks.workunit.client.0.smithi060.stdout:1/627: dwrite d3/db/d1e/d38/f6e [0,4194304] 0 2024-02-26T10:48:01.075 INFO:tasks.workunit.client.0.smithi060.stdout:4/555: rename d0/f4b to d0/da/d3b/d1b/fab 0 2024-02-26T10:48:01.075 INFO:tasks.workunit.client.0.smithi060.stdout:4/556: dread - d0/d27/d76/f91 zero size 2024-02-26T10:48:01.076 INFO:tasks.workunit.client.0.smithi060.stdout:1/628: creat d3/db/d1e/d6d/da3/daa/fde x:0 0 0 2024-02-26T10:48:01.079 INFO:tasks.workunit.client.0.smithi060.stdout:4/557: creat d0/d7/d29/d8e/fac x:0 0 0 2024-02-26T10:48:01.079 INFO:tasks.workunit.client.0.smithi060.stdout:4/558: chown d0/da/d3b/d16 116107060 1 2024-02-26T10:48:01.080 INFO:tasks.workunit.client.0.smithi060.stdout:4/559: unlink d0/d27/f49 0 2024-02-26T10:48:01.080 INFO:tasks.workunit.client.0.smithi060.stdout:1/629: creat d3/db/d1e/d6d/da3/daa/dd6/fdf x:0 0 0 2024-02-26T10:48:01.080 INFO:tasks.workunit.client.0.smithi060.stdout:1/630: fsync d3/f23 0 2024-02-26T10:48:01.080 INFO:tasks.workunit.client.0.smithi060.stdout:4/560: truncate d0/d27/d76/faa 480319 0 2024-02-26T10:48:01.080 INFO:tasks.workunit.client.0.smithi060.stdout:4/561: fsync d0/d27/d2f/f31 0 2024-02-26T10:48:01.080 INFO:tasks.workunit.client.0.smithi060.stdout:4/562: chown d0/da/f53 2 1 2024-02-26T10:48:01.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:00 smithi113 ceph-mon[36126]: pgmap v486: 129 pgs: 129 active+clean; 831 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s rd, 124 MiB/s wr, 650 op/s 2024-02-26T10:48:01.081 INFO:tasks.workunit.client.0.smithi060.stdout:6/576: dread d6/d6e/f96 [0,4194304] 0 2024-02-26T10:48:01.083 INFO:tasks.workunit.client.0.smithi060.stdout:6/577: symlink d6/d7/d2e/lbd 0 2024-02-26T10:48:01.088 INFO:tasks.workunit.client.0.smithi060.stdout:1/631: link d3/db/d1e/d6d/da3/daa/dcd/cb3 d3/db/dc8/dbe/ce0 0 2024-02-26T10:48:01.099 INFO:tasks.workunit.client.0.smithi060.stdout:7/695: dwrite da/d15/d88/f51 [0,4194304] 0 2024-02-26T10:48:01.100 INFO:tasks.workunit.client.0.smithi060.stdout:1/632: rename d3/db/d1e/c42 to d3/db/d1e/d6d/dd9/ce1 0 2024-02-26T10:48:01.103 INFO:tasks.workunit.client.0.smithi060.stdout:4/563: dread d0/da/d57/f67 [0,4194304] 0 2024-02-26T10:48:01.111 INFO:tasks.workunit.client.0.smithi060.stdout:6/578: write d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/f94 [1662599,59956] 0 2024-02-26T10:48:01.111 INFO:tasks.workunit.client.0.smithi060.stdout:7/696: stat da/d15/d21/d2f 0 2024-02-26T10:48:01.112 INFO:tasks.workunit.client.0.smithi060.stdout:7/697: chown da/d66/dbd/fe1 1623 1 2024-02-26T10:48:01.113 INFO:tasks.workunit.client.0.smithi060.stdout:4/564: mknod d0/da/cad 0 2024-02-26T10:48:01.113 INFO:tasks.workunit.client.0.smithi060.stdout:4/565: stat d0/da/d3b/d16/c46 0 2024-02-26T10:48:01.113 INFO:tasks.workunit.client.0.smithi060.stdout:4/566: write d0/da/d3b/f5f [476009,127764] 0 2024-02-26T10:48:01.113 INFO:tasks.workunit.client.0.smithi060.stdout:4/567: fdatasync d0/da/d3b/f59 0 2024-02-26T10:48:01.114 INFO:tasks.workunit.client.0.smithi060.stdout:4/568: mknod d0/da/d3b/d1b/cae 0 2024-02-26T10:48:01.115 INFO:tasks.workunit.client.0.smithi060.stdout:6/579: link d6/d8d/fb3 d6/d8d/fbe 0 2024-02-26T10:48:01.121 INFO:tasks.workunit.client.0.smithi060.stdout:6/580: symlink d6/d7/d25/d1c/d1f/lbf 0 2024-02-26T10:48:01.121 INFO:tasks.workunit.client.0.smithi060.stdout:6/581: write d6/d7/d25/d1c/d1f/d23/f50 [190561,119740] 0 2024-02-26T10:48:01.121 INFO:tasks.workunit.client.0.smithi060.stdout:6/582: fdatasync d6/d7/d25/d1c/d1f/d23/f3a 0 2024-02-26T10:48:01.122 INFO:tasks.workunit.client.0.smithi060.stdout:6/583: fdatasync d6/fe 0 2024-02-26T10:48:01.123 INFO:tasks.workunit.client.0.smithi060.stdout:6/584: unlink d6/d7/f2a 0 2024-02-26T10:48:01.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:00 smithi100 ceph-mon[36051]: pgmap v486: 129 pgs: 129 active+clean; 831 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s rd, 124 MiB/s wr, 650 op/s 2024-02-26T10:48:01.124 INFO:tasks.workunit.client.0.smithi060.stdout:6/585: getdents d6/d7/d25/d1c/d1f/d23/d7a 0 2024-02-26T10:48:01.129 INFO:tasks.workunit.client.0.smithi060.stdout:1/633: write d3/db/d1e/d6d/d78/f86 [1282369,50772] 0 2024-02-26T10:48:01.136 INFO:tasks.workunit.client.0.smithi060.stdout:6/586: dread d6/d7/d25/d1c/d1f/d23/d28/f29 [0,4194304] 0 2024-02-26T10:48:01.138 INFO:tasks.workunit.client.0.smithi060.stdout:6/587: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/dc0 0 2024-02-26T10:48:01.139 INFO:tasks.workunit.client.0.smithi060.stdout:1/634: rmdir d3/d35/d53/d68/d85 39 2024-02-26T10:48:01.139 INFO:tasks.workunit.client.0.smithi060.stdout:1/635: readlink d3/db/d1e/l37 0 2024-02-26T10:48:01.139 INFO:tasks.workunit.client.0.smithi060.stdout:1/636: stat d3/d35/d53/f74 0 2024-02-26T10:48:01.140 INFO:tasks.workunit.client.0.smithi060.stdout:6/588: creat d6/d7/d25/d1c/d1f/d23/d7a/fc1 x:0 0 0 2024-02-26T10:48:01.157 INFO:tasks.workunit.client.0.smithi060.stdout:4/569: dwrite d0/da/f80 [0,4194304] 0 2024-02-26T10:48:01.166 INFO:tasks.workunit.client.0.smithi060.stdout:2/514: sync 2024-02-26T10:48:01.168 INFO:tasks.workunit.client.0.smithi060.stdout:6/589: write d6/d7/fd [2531451,66864] 0 2024-02-26T10:48:01.169 INFO:tasks.workunit.client.0.smithi060.stdout:6/590: chown d6/d7/d25/d1c/d1f/d23/d2b/d4d/f58 408695 1 2024-02-26T10:48:01.169 INFO:tasks.workunit.client.0.smithi060.stdout:8/554: sync 2024-02-26T10:48:01.170 INFO:tasks.workunit.client.0.smithi060.stdout:1/637: write d3/db/d1e/d38/f6e [1173140,8518] 0 2024-02-26T10:48:01.170 INFO:tasks.workunit.client.0.smithi060.stdout:8/555: mknod d9/da4/cb1 0 2024-02-26T10:48:01.171 INFO:tasks.workunit.client.0.smithi060.stdout:8/556: truncate d9/d5a/f6e 915817 0 2024-02-26T10:48:01.171 INFO:tasks.workunit.client.0.smithi060.stdout:8/557: truncate d9/d15/d3e/f9b 301508 0 2024-02-26T10:48:01.171 INFO:tasks.workunit.client.0.smithi060.stdout:1/638: write d3/d35/d53/d57/f71 [216198,109124] 0 2024-02-26T10:48:01.176 INFO:tasks.workunit.client.0.smithi060.stdout:8/558: rmdir d9/d15/d29/dad 0 2024-02-26T10:48:01.176 INFO:tasks.workunit.client.0.smithi060.stdout:8/559: readlink d9/d15/l6c 0 2024-02-26T10:48:01.176 INFO:tasks.workunit.client.0.smithi060.stdout:7/698: dwrite da/d15/d21/d2f/d78/fda [0,4194304] 0 2024-02-26T10:48:01.176 INFO:tasks.workunit.client.0.smithi060.stdout:7/699: fsync da/d15/d21/d41/f97 0 2024-02-26T10:48:01.176 INFO:tasks.workunit.client.0.smithi060.stdout:7/700: fsync da/d15/d21/fb2 0 2024-02-26T10:48:01.186 INFO:tasks.workunit.client.0.smithi060.stdout:2/515: dread dd/d43/d55/d5e/d74/da5/fa7 [4194304,4194304] 0 2024-02-26T10:48:01.189 INFO:tasks.workunit.client.0.smithi060.stdout:8/560: dread d9/d15/d29/d2f/d34/d72/d50/d6a/f46 [0,4194304] 0 2024-02-26T10:48:01.189 INFO:tasks.workunit.client.0.smithi060.stdout:2/516: rmdir dd/d11/d91 0 2024-02-26T10:48:01.189 INFO:tasks.workunit.client.0.smithi060.stdout:2/517: readlink dd/d11/l98 0 2024-02-26T10:48:01.190 INFO:tasks.workunit.client.0.smithi060.stdout:2/518: fsync dd/d43/d60/d41/d37/d4c/d76/fa3 0 2024-02-26T10:48:01.192 INFO:tasks.workunit.client.0.smithi060.stdout:7/701: rename da/d15/d21/d2f/d78/dd3/db9 to da/df1 0 2024-02-26T10:48:01.192 INFO:tasks.workunit.client.0.smithi060.stdout:7/702: chown da/d15/f2e 718922 1 2024-02-26T10:48:01.192 INFO:tasks.workunit.client.0.smithi060.stdout:7/703: readlink da/d15/d21/d2f/d78/l9f 0 2024-02-26T10:48:01.192 INFO:tasks.workunit.client.0.smithi060.stdout:8/561: write d9/d15/d29/d2f/d34/d72/d50/d6a/f46 [413089,127451] 0 2024-02-26T10:48:01.196 INFO:tasks.workunit.client.0.smithi060.stdout:4/570: dread d0/da/d57/f75 [0,4194304] 0 2024-02-26T10:48:01.196 INFO:tasks.workunit.client.0.smithi060.stdout:4/571: chown d0/f98 8707 1 2024-02-26T10:48:01.196 INFO:tasks.workunit.client.0.smithi060.stdout:4/572: readlink d0/d7/l68 0 2024-02-26T10:48:01.197 INFO:tasks.workunit.client.0.smithi060.stdout:4/573: mknod d0/d7/d29/caf 0 2024-02-26T10:48:01.199 INFO:tasks.workunit.client.0.smithi060.stdout:4/574: truncate d0/d27/d2f/f31 4856111 0 2024-02-26T10:48:01.199 INFO:tasks.workunit.client.0.smithi060.stdout:4/575: dread - d0/da/d3b/f59 zero size 2024-02-26T10:48:01.200 INFO:tasks.workunit.client.0.smithi060.stdout:4/576: stat d0/da/d3b/d1b/fa7 0 2024-02-26T10:48:01.200 INFO:tasks.workunit.client.0.smithi060.stdout:8/562: dread d9/f24 [0,4194304] 0 2024-02-26T10:48:01.215 INFO:tasks.workunit.client.0.smithi060.stdout:4/577: getdents d0/d27/d2f 0 2024-02-26T10:48:01.215 INFO:tasks.workunit.client.0.smithi060.stdout:7/704: write da/d15/d21/d2f/f68 [3812289,24857] 0 2024-02-26T10:48:01.216 INFO:tasks.workunit.client.0.smithi060.stdout:4/578: mkdir d0/d27/d7f/db0 0 2024-02-26T10:48:01.217 INFO:tasks.workunit.client.0.smithi060.stdout:4/579: rename d0/f3c to d0/da/fb1 0 2024-02-26T10:48:01.218 INFO:tasks.workunit.client.0.smithi060.stdout:4/580: read d0/d27/d76/faa [263788,45098] 0 2024-02-26T10:48:01.244 INFO:tasks.workunit.client.0.smithi060.stdout:4/581: dread d0/f74 [0,4194304] 0 2024-02-26T10:48:01.247 INFO:tasks.workunit.client.0.smithi060.stdout:1/639: dwrite d3/d35/d53/d68/f8c [0,4194304] 0 2024-02-26T10:48:01.248 INFO:tasks.workunit.client.0.smithi060.stdout:4/582: rename d0/da/d57/f67 to d0/da/d57/fb2 0 2024-02-26T10:48:01.249 INFO:tasks.workunit.client.0.smithi060.stdout:4/583: write d0/d7/d29/d8e/f99 [60304,18779] 0 2024-02-26T10:48:01.250 INFO:tasks.workunit.client.0.smithi060.stdout:4/584: mkdir d0/d7/d29/d8e/db3 0 2024-02-26T10:48:01.253 INFO:tasks.workunit.client.0.smithi060.stdout:4/585: creat d0/d7/d29/d8e/fb4 x:0 0 0 2024-02-26T10:48:01.254 INFO:tasks.workunit.client.0.smithi060.stdout:4/586: mkdir d0/d27/d7f/db0/db5 0 2024-02-26T10:48:01.255 INFO:tasks.workunit.client.0.smithi060.stdout:4/587: creat d0/d7/d29/d60/fb6 x:0 0 0 2024-02-26T10:48:01.255 INFO:tasks.workunit.client.0.smithi060.stdout:4/588: stat d0/d27/c5d 0 2024-02-26T10:48:01.256 INFO:tasks.workunit.client.0.smithi060.stdout:4/589: write d0/d27/d52/f71 [211871,127424] 0 2024-02-26T10:48:01.273 INFO:tasks.workunit.client.0.smithi060.stdout:2/519: dwrite dd/d43/f6c [0,4194304] 0 2024-02-26T10:48:01.275 INFO:tasks.workunit.client.0.smithi060.stdout:1/640: dread d3/d35/d53/d68/d90/fbc [0,4194304] 0 2024-02-26T10:48:01.275 INFO:tasks.workunit.client.0.smithi060.stdout:7/705: dwrite da/d15/d21/d2f/f68 [0,4194304] 0 2024-02-26T10:48:01.275 INFO:tasks.workunit.client.0.smithi060.stdout:2/520: creat dd/d43/d60/d41/d42/d6f/fb1 x:0 0 0 2024-02-26T10:48:01.276 INFO:tasks.workunit.client.0.smithi060.stdout:7/706: mknod da/d66/cf2 0 2024-02-26T10:48:01.276 INFO:tasks.workunit.client.0.smithi060.stdout:1/641: creat d3/db/d1e/d38/fe2 x:0 0 0 2024-02-26T10:48:01.276 INFO:tasks.workunit.client.0.smithi060.stdout:1/642: readlink d3/d35/d53/d68/d6b/lba 0 2024-02-26T10:48:01.296 INFO:tasks.workunit.client.0.smithi060.stdout:4/590: dwrite d0/d27/f83 [0,4194304] 0 2024-02-26T10:48:01.296 INFO:tasks.workunit.client.0.smithi060.stdout:8/563: dwrite d9/d15/d29/f65 [0,4194304] 0 2024-02-26T10:48:01.296 INFO:tasks.workunit.client.0.smithi060.stdout:8/564: truncate d9/d15/d29/f64 361322 0 2024-02-26T10:48:01.303 INFO:tasks.workunit.client.0.smithi060.stdout:4/591: symlink d0/d7/d29/d60/lb7 0 2024-02-26T10:48:01.303 INFO:tasks.workunit.client.0.smithi060.stdout:4/592: fdatasync d0/da/d3b/d16/f3f 0 2024-02-26T10:48:01.303 INFO:tasks.workunit.client.0.smithi060.stdout:4/593: write d0/da/d3b/f95 [94048,80277] 0 2024-02-26T10:48:01.305 INFO:tasks.workunit.client.0.smithi060.stdout:4/594: symlink d0/da/d57/lb8 0 2024-02-26T10:48:01.305 INFO:tasks.workunit.client.0.smithi060.stdout:4/595: write d0/d27/d52/f71 [1380205,51729] 0 2024-02-26T10:48:01.309 INFO:tasks.workunit.client.0.smithi060.stdout:4/596: mkdir d0/da/d3b/db9 0 2024-02-26T10:48:01.309 INFO:tasks.workunit.client.0.smithi060.stdout:4/597: fsync d0/d27/d2f/f7d 0 2024-02-26T10:48:01.309 INFO:tasks.workunit.client.0.smithi060.stdout:4/598: readlink d0/da/l9b 0 2024-02-26T10:48:01.315 INFO:tasks.workunit.client.0.smithi060.stdout:4/599: mkdir d0/da/dba 0 2024-02-26T10:48:01.316 INFO:tasks.workunit.client.0.smithi060.stdout:1/643: dread d3/db/f40 [0,4194304] 0 2024-02-26T10:48:01.318 INFO:tasks.workunit.client.0.smithi060.stdout:1/644: rename d3/db/d1e/d38/d6c/fcb to d3/db/d1e/d39/fe3 0 2024-02-26T10:48:01.318 INFO:tasks.workunit.client.0.smithi060.stdout:1/645: readlink d3/db/d1e/d38/d6c/lb1 0 2024-02-26T10:48:01.319 INFO:tasks.workunit.client.0.smithi060.stdout:1/646: creat d3/d35/d5e/fe4 x:0 0 0 2024-02-26T10:48:01.323 INFO:tasks.workunit.client.0.smithi060.stdout:5/541: sync 2024-02-26T10:48:01.330 INFO:tasks.workunit.client.0.smithi060.stdout:4/600: write d0/f5 [1747702,53520] 0 2024-02-26T10:48:01.335 INFO:tasks.workunit.client.0.smithi060.stdout:7/707: dwrite da/d15/d21/d2f/d37/f38 [0,4194304] 0 2024-02-26T10:48:01.339 INFO:tasks.workunit.client.0.smithi060.stdout:8/565: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f8a [0,4194304] 0 2024-02-26T10:48:01.342 INFO:tasks.workunit.client.0.smithi060.stdout:4/601: symlink d0/d7/lbb 0 2024-02-26T10:48:01.346 INFO:tasks.workunit.client.0.smithi060.stdout:4/602: unlink d0/d27/d52/f62 0 2024-02-26T10:48:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.365+0000 7fa640a00640 1 -- 172.21.15.60:0/2096215889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 msgr2=0x7fa63c0772c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.365+0000 7fa640a00640 1 --2- 172.21.15.60:0/2096215889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 0x7fa63c0772c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa628009900 tx=0x7fa62802bc30 comp rx=0 tx=0).stop 2024-02-26T10:48:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.366+0000 7fa640a00640 1 -- 172.21.15.60:0/2096215889 shutdown_connections 2024-02-26T10:48:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.366+0000 7fa640a00640 1 --2- 172.21.15.60:0/2096215889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c077800 0x7fa63c0780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.366+0000 7fa640a00640 1 --2- 172.21.15.60:0/2096215889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 0x7fa63c0772c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.366+0000 7fa640a00640 1 --2- 172.21.15.60:0/2096215889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa63c075c20 0x7fa63c076040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.366+0000 7fa640a00640 1 -- 172.21.15.60:0/2096215889 >> 172.21.15.60:0/2096215889 conn(0x7fa63c0fea50 msgr2=0x7fa63c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.366+0000 7fa640a00640 1 -- 172.21.15.60:0/2096215889 shutdown_connections 2024-02-26T10:48:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.366+0000 7fa640a00640 1 -- 172.21.15.60:0/2096215889 wait complete. 2024-02-26T10:48:01.361 INFO:tasks.workunit.client.0.smithi060.stdout:2/521: dwrite dd/d43/d60/d41/d37/d4c/d76/fa3 [0,4194304] 0 2024-02-26T10:48:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.368+0000 7fa640a00640 1 Processor -- start 2024-02-26T10:48:01.367 INFO:tasks.workunit.client.0.smithi060.stdout:4/603: link d0/d27/d52/l88 d0/d7/d29/d8e/lbc 0 2024-02-26T10:48:01.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.374+0000 7fa640a00640 1 -- start start 2024-02-26T10:48:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa640a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa63c075c20 0x7fa63c070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa640a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 0x7fa63c0713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa640a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c077800 0x7fa63c072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa640a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c06b540 con 0x7fa63c075c20 2024-02-26T10:48:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa640a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c06b3c0 con 0x7fa63c077800 2024-02-26T10:48:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa640a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c06b6c0 con 0x7fa63c076e60 2024-02-26T10:48:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa639400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 0x7fa63c0713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa639400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 0x7fa63c0713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:32988/0 (socket says 172.21.15.60:32988) 2024-02-26T10:48:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa639400640 1 -- 172.21.15.60:0/3443968176 learned_addr learned my addr 172.21.15.60:0/3443968176 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa63a800640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c077800 0x7fa63c072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.375+0000 7fa639e00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa63c075c20 0x7fa63c070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.376+0000 7fa639400640 1 -- 172.21.15.60:0/3443968176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c077800 msgr2=0x7fa63c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.376+0000 7fa639400640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c077800 0x7fa63c072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.376+0000 7fa639400640 1 -- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa63c075c20 msgr2=0x7fa63c070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.376+0000 7fa639400640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa63c075c20 0x7fa63c070eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.376+0000 7fa639400640 1 -- 172.21.15.60:0/3443968176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c072aa0 con 0x7fa63c076e60 2024-02-26T10:48:01.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.376+0000 7fa639400640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 0x7fa63c0713f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa628005ab0 tx=0x7fa62802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.376+0000 7fa632a00640 1 -- 172.21.15.60:0/3443968176 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628043020 con 0x7fa63c076e60 2024-02-26T10:48:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.376+0000 7fa632a00640 1 -- 172.21.15.60:0/3443968176 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6280054e0 con 0x7fa63c076e60 2024-02-26T10:48:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.376+0000 7fa632a00640 1 -- 172.21.15.60:0/3443968176 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628003c70 con 0x7fa63c076e60 2024-02-26T10:48:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.380+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c072d30 con 0x7fa63c076e60 2024-02-26T10:48:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.380+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c073230 con 0x7fa63c076e60 2024-02-26T10:48:01.378 INFO:tasks.workunit.client.0.smithi060.stdout:0/533: sync 2024-02-26T10:48:01.378 INFO:tasks.workunit.client.0.smithi060.stdout:9/643: sync 2024-02-26T10:48:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.384+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c10ffb0 con 0x7fa63c076e60 2024-02-26T10:48:01.379 INFO:tasks.workunit.client.0.smithi060.stdout:1/647: dwrite d3/f2a [0,4194304] 0 2024-02-26T10:48:01.379 INFO:tasks.workunit.client.0.smithi060.stdout:9/644: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8/dcd/fda [747373,121774] 0 2024-02-26T10:48:01.380 INFO:tasks.workunit.client.0.smithi060.stdout:9/645: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fdc [743929,120159] 0 2024-02-26T10:48:01.380 INFO:tasks.workunit.client.0.smithi060.stdout:6/591: sync 2024-02-26T10:48:01.380 INFO:tasks.workunit.client.0.smithi060.stdout:0/534: creat d0/dd/d39/d93/fb6 x:0 0 0 2024-02-26T10:48:01.380 INFO:tasks.workunit.client.0.smithi060.stdout:0/535: chown d0/dd/d39/d40/f34 597948 1 2024-02-26T10:48:01.380 INFO:tasks.workunit.client.0.smithi060.stdout:0/536: stat d0/dd/d53/d87/d94/da6/la1 0 2024-02-26T10:48:01.381 INFO:tasks.workunit.client.0.smithi060.stdout:3/448: sync 2024-02-26T10:48:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.387+0000 7fa632a00640 1 -- 172.21.15.60:0/3443968176 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa628046020 con 0x7fa63c076e60 2024-02-26T10:48:01.383 INFO:tasks.workunit.client.0.smithi060.stdout:9/646: mkdir d3/d1e/d93/da2/db7/ddd 0 2024-02-26T10:48:01.383 INFO:tasks.workunit.client.0.smithi060.stdout:9/647: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/d47/fb4 [756370,75569] 0 2024-02-26T10:48:01.383 INFO:tasks.workunit.client.0.smithi060.stdout:9/648: readlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/l4d 0 2024-02-26T10:48:01.383 INFO:tasks.workunit.client.0.smithi060.stdout:9/649: dread - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/fbd zero size 2024-02-26T10:48:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.388+0000 7fa632a00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa6200a1e80 0x7fa6200a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.388+0000 7fa632a00640 1 -- 172.21.15.60:0/3443968176 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa6280efca0 con 0x7fa63c076e60 2024-02-26T10:48:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.390+0000 7fa639e00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa6200a1e80 0x7fa6200a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:01.384 INFO:tasks.workunit.client.0.smithi060.stdout:1/648: mknod d3/db/d1e/d6d/dd9/ce5 0 2024-02-26T10:48:01.386 INFO:tasks.workunit.client.0.smithi060.stdout:9/650: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/dde 0 2024-02-26T10:48:01.386 INFO:tasks.workunit.client.0.smithi060.stdout:9/651: fsync d3/d1e/d57/f79 0 2024-02-26T10:48:01.386 INFO:tasks.workunit.client.0.smithi060.stdout:1/649: dread - d3/db/d1e/d38/d6c/f88 zero size 2024-02-26T10:48:01.386 INFO:tasks.workunit.client.0.smithi060.stdout:1/650: chown d3/db/d2e/f4d 258779554 1 2024-02-26T10:48:01.387 INFO:tasks.workunit.client.0.smithi060.stdout:3/449: link d5/f27 d5/d45/d61/d80/f86 0 2024-02-26T10:48:01.387 INFO:tasks.workunit.client.0.smithi060.stdout:3/450: stat d5/de/f58 0 2024-02-26T10:48:01.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.394+0000 7fa639e00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa6200a1e80 0x7fa6200a4340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa62c005fd0 tx=0x7fa62c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:01.388 INFO:tasks.workunit.client.0.smithi060.stdout:1/651: truncate d3/f23 1568384 0 2024-02-26T10:48:01.388 INFO:tasks.workunit.client.0.smithi060.stdout:3/451: dread - d5/d76/f78 zero size 2024-02-26T10:48:01.388 INFO:tasks.workunit.client.0.smithi060.stdout:1/652: dread - d3/db/d2e/d51/fc7 zero size 2024-02-26T10:48:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.394+0000 7fa632a00640 1 -- 172.21.15.60:0/3443968176 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa62804d070 con 0x7fa63c076e60 2024-02-26T10:48:01.389 INFO:tasks.workunit.client.0.smithi060.stdout:1/653: chown d3/d35/f65 1961189 1 2024-02-26T10:48:01.389 INFO:tasks.workunit.client.0.smithi060.stdout:3/452: rename d5/l26 to d5/d76/l87 0 2024-02-26T10:48:01.390 INFO:tasks.workunit.client.0.smithi060.stdout:3/453: write d5/de/f58 [810435,124844] 0 2024-02-26T10:48:01.390 INFO:tasks.workunit.client.0.smithi060.stdout:3/454: fsync d5/de/df/d11/f6e 0 2024-02-26T10:48:01.390 INFO:tasks.workunit.client.0.smithi060.stdout:1/654: mkdir d3/d35/de6 0 2024-02-26T10:48:01.395 INFO:tasks.workunit.client.0.smithi060.stdout:9/652: dread d3/d1e/d5e/d35/d87/d9b/f8d [0,4194304] 0 2024-02-26T10:48:01.398 INFO:tasks.workunit.client.0.smithi060.stdout:0/537: rmdir d0 39 2024-02-26T10:48:01.398 INFO:tasks.workunit.client.0.smithi060.stdout:0/538: chown d0/d42/d7a/d7f/lac 21647080 1 2024-02-26T10:48:01.401 INFO:tasks.workunit.client.0.smithi060.stdout:5/542: dwrite d0/de/d15/f5a [0,4194304] 0 2024-02-26T10:48:01.401 INFO:tasks.workunit.client.0.smithi060.stdout:5/543: mknod d0/d4b/d7b/cbe 0 2024-02-26T10:48:01.402 INFO:tasks.workunit.client.0.smithi060.stdout:0/539: mkdir d0/dd/d53/d87/d94/db7 0 2024-02-26T10:48:01.405 INFO:tasks.workunit.client.0.smithi060.stdout:8/566: dwrite d9/f23 [0,4194304] 0 2024-02-26T10:48:01.405 INFO:tasks.workunit.client.0.smithi060.stdout:8/567: fsync d9/f23 0 2024-02-26T10:48:01.405 INFO:tasks.workunit.client.0.smithi060.stdout:5/544: creat d0/de/fbf x:0 0 0 2024-02-26T10:48:01.405 INFO:tasks.workunit.client.0.smithi060.stdout:5/545: write d0/de/d20/d32/f89 [4914545,5077] 0 2024-02-26T10:48:01.405 INFO:tasks.workunit.client.0.smithi060.stdout:0/540: creat d0/dd/d39/fb8 x:0 0 0 2024-02-26T10:48:01.407 INFO:tasks.workunit.client.0.smithi060.stdout:7/708: dwrite da/f24 [0,4194304] 0 2024-02-26T10:48:01.407 INFO:tasks.workunit.client.0.smithi060.stdout:7/709: write da/d66/f75 [5061382,39225] 0 2024-02-26T10:48:01.409 INFO:tasks.workunit.client.0.smithi060.stdout:8/568: creat d9/d15/d3e/fb2 x:0 0 0 2024-02-26T10:48:01.415 INFO:tasks.workunit.client.0.smithi060.stdout:0/541: link d0/dd/d39/d93/c95 d0/dd/d53/d87/d94/cb9 0 2024-02-26T10:48:01.415 INFO:tasks.workunit.client.0.smithi060.stdout:6/592: link d6/d7/d25/d1c/d1f/d23/d2b/c32 d6/d7/d2e/d56/d73/d8c/cc2 0 2024-02-26T10:48:01.415 INFO:tasks.workunit.client.0.smithi060.stdout:7/710: fdatasync da/d15/d21/d2f/d78/dd3/d67/f85 0 2024-02-26T10:48:01.415 INFO:tasks.workunit.client.0.smithi060.stdout:2/522: dread dd/d11/f4e [0,4194304] 0 2024-02-26T10:48:01.416 INFO:tasks.workunit.client.0.smithi060.stdout:2/523: creat dd/d43/d60/d41/d42/fb2 x:0 0 0 2024-02-26T10:48:01.417 INFO:tasks.workunit.client.0.smithi060.stdout:1/655: write d3/f23 [1420490,87586] 0 2024-02-26T10:48:01.418 INFO:tasks.workunit.client.0.smithi060.stdout:2/524: creat dd/d43/d55/d5e/d86/fb3 x:0 0 0 2024-02-26T10:48:01.419 INFO:tasks.workunit.client.0.smithi060.stdout:2/525: mkdir dd/d43/d60/d41/d37/d9a/db4 0 2024-02-26T10:48:01.419 INFO:tasks.workunit.client.0.smithi060.stdout:2/526: readlink dd/d43/d60/l9b 0 2024-02-26T10:48:01.432 INFO:tasks.workunit.client.0.smithi060.stdout:9/653: rename d3/d1e/d5e/d35/d87/d9b/d8a/dbe/f54 to d3/fdf 0 2024-02-26T10:48:01.436 INFO:tasks.workunit.client.0.smithi060.stdout:1/656: rename d3/db/d1e/d6d/da3/fbd to d3/db/d1e/d6d/dd1/fe7 0 2024-02-26T10:48:01.442 INFO:tasks.workunit.client.0.smithi060.stdout:5/546: dread d0/f1d [0,4194304] 0 2024-02-26T10:48:01.442 INFO:tasks.workunit.client.0.smithi060.stdout:5/547: read - d0/de/da4/fb3 zero size 2024-02-26T10:48:01.442 INFO:tasks.workunit.client.0.smithi060.stdout:5/548: write d0/f14 [1092424,15699] 0 2024-02-26T10:48:01.442 INFO:tasks.workunit.client.0.smithi060.stdout:5/549: fdatasync d0/de/d20/f24 0 2024-02-26T10:48:01.443 INFO:tasks.workunit.client.0.smithi060.stdout:7/711: dread da/d15/d21/f2c [0,4194304] 0 2024-02-26T10:48:01.443 INFO:tasks.workunit.client.0.smithi060.stdout:7/712: fdatasync da/d15/d21/d41/f97 0 2024-02-26T10:48:01.443 INFO:tasks.workunit.client.0.smithi060.stdout:7/713: dread - da/d66/dad/dd8/f82 zero size 2024-02-26T10:48:01.443 INFO:tasks.workunit.client.0.smithi060.stdout:7/714: chown da/d15/fcc 11915 1 2024-02-26T10:48:01.446 INFO:tasks.workunit.client.0.smithi060.stdout:5/550: mkdir d0/dc0 0 2024-02-26T10:48:01.452 INFO:tasks.workunit.client.0.smithi060.stdout:1/657: unlink d3/db/dc8/dbe/ce0 0 2024-02-26T10:48:01.454 INFO:tasks.workunit.client.0.smithi060.stdout:1/658: symlink d3/db/d1e/d6d/le8 0 2024-02-26T10:48:01.454 INFO:tasks.workunit.client.0.smithi060.stdout:1/659: write d3/d35/d5e/f70 [1233889,112270] 0 2024-02-26T10:48:01.458 INFO:tasks.workunit.client.0.smithi060.stdout:1/660: unlink d3/d35/c3c 0 2024-02-26T10:48:01.458 INFO:tasks.workunit.client.0.smithi060.stdout:1/661: chown d3/d35/d53/d68/d6b/dc9 1831690 1 2024-02-26T10:48:01.459 INFO:tasks.workunit.client.0.smithi060.stdout:1/662: mkdir d3/d54/de9 0 2024-02-26T10:48:01.474 INFO:tasks.workunit.client.0.smithi060.stdout:2/527: dwrite dd/d43/d60/d41/f6d [0,4194304] 0 2024-02-26T10:48:01.476 INFO:tasks.workunit.client.0.smithi060.stdout:5/551: dread d0/de/d15/f18 [4194304,4194304] 0 2024-02-26T10:48:01.476 INFO:tasks.workunit.client.0.smithi060.stdout:5/552: readlink d0/lac 0 2024-02-26T10:48:01.477 INFO:tasks.workunit.client.0.smithi060.stdout:5/553: rmdir d0/de/d70/d84/d92 39 2024-02-26T10:48:01.479 INFO:tasks.workunit.client.0.smithi060.stdout:5/554: unlink d0/de/d15/d26/f48 0 2024-02-26T10:48:01.480 INFO:tasks.workunit.client.0.smithi060.stdout:2/528: unlink dd/d43/d60/d41/d37/d38/d57/c67 0 2024-02-26T10:48:01.481 INFO:tasks.workunit.client.0.smithi060.stdout:5/555: mknod d0/de/d15/d65/d86/cc1 0 2024-02-26T10:48:01.481 INFO:tasks.workunit.client.0.smithi060.stdout:5/556: stat d0/de/d70/d84 0 2024-02-26T10:48:01.484 INFO:tasks.workunit.client.0.smithi060.stdout:5/557: symlink d0/de/d20/d32/d4c/lc2 0 2024-02-26T10:48:01.484 INFO:tasks.workunit.client.0.smithi060.stdout:5/558: readlink d0/de/d15/d65/lae 0 2024-02-26T10:48:01.484 INFO:tasks.workunit.client.0.smithi060.stdout:5/559: stat d0/d33/c3c 0 2024-02-26T10:48:01.485 INFO:tasks.workunit.client.0.smithi060.stdout:5/560: symlink d0/de/d70/d84/lc3 0 2024-02-26T10:48:01.494 INFO:tasks.workunit.client.0.smithi060.stdout:5/561: symlink d0/de/d20/d32/d4d/lc4 0 2024-02-26T10:48:01.503 INFO:tasks.workunit.client.0.smithi060.stdout:5/562: link d0/de/d15/c68 d0/de/d70/d84/d92/cc5 0 2024-02-26T10:48:01.503 INFO:tasks.workunit.client.0.smithi060.stdout:4/604: dwrite d0/d27/d52/f71 [0,4194304] 0 2024-02-26T10:48:01.503 INFO:tasks.workunit.client.0.smithi060.stdout:4/605: getdents d0/d7/d29/d8e/db3 0 2024-02-26T10:48:01.508 INFO:tasks.workunit.client.0.smithi060.stdout:4/606: rename d0/d7/d29 to d0/d27/d76/dbd 0 2024-02-26T10:48:01.508 INFO:tasks.workunit.client.0.smithi060.stdout:4/607: dread - d0/d27/d76/f91 zero size 2024-02-26T10:48:01.510 INFO:tasks.workunit.client.0.smithi060.stdout:0/542: dwrite d0/dd/f77 [0,4194304] 0 2024-02-26T10:48:01.512 INFO:tasks.workunit.client.0.smithi060.stdout:0/543: getdents d0/d37/d44 0 2024-02-26T10:48:01.512 INFO:tasks.workunit.client.0.smithi060.stdout:0/544: chown d0/dd/d39/d40/dad 26687 1 2024-02-26T10:48:01.513 INFO:tasks.workunit.client.0.smithi060.stdout:7/715: dwrite da/d15/d21/d2f/d78/fda [0,4194304] 0 2024-02-26T10:48:01.520 INFO:tasks.workunit.client.0.smithi060.stdout:2/529: write dd/d43/d60/d41/f6d [3689906,68026] 0 2024-02-26T10:48:01.522 INFO:tasks.workunit.client.0.smithi060.stdout:2/530: mknod dd/d43/d60/d41/d37/d4c/cb5 0 2024-02-26T10:48:01.522 INFO:tasks.workunit.client.0.smithi060.stdout:2/531: chown dd/l2c 8620459 1 2024-02-26T10:48:01.523 INFO:tasks.workunit.client.0.smithi060.stdout:2/532: unlink dd/d11/c5f 0 2024-02-26T10:48:01.525 INFO:tasks.workunit.client.0.smithi060.stdout:2/533: creat dd/d43/d55/d5e/d74/fb6 x:0 0 0 2024-02-26T10:48:01.525 INFO:tasks.workunit.client.0.smithi060.stdout:5/563: link d0/de/d20/d32/d4d/d97/l9a d0/de/d70/d84/lc6 0 2024-02-26T10:48:01.525 INFO:tasks.workunit.client.0.smithi060.stdout:5/564: dread - d0/de/da4/fbd zero size 2024-02-26T10:48:01.526 INFO:tasks.workunit.client.0.smithi060.stdout:5/565: rmdir d0/d4b/d7b 39 2024-02-26T10:48:01.526 INFO:tasks.workunit.client.0.smithi060.stdout:5/566: fdatasync d0/d33/f63 0 2024-02-26T10:48:01.527 INFO:tasks.workunit.client.0.smithi060.stdout:2/534: creat dd/d43/d60/d41/d37/d9a/fb7 x:0 0 0 2024-02-26T10:48:01.527 INFO:tasks.workunit.client.0.smithi060.stdout:2/535: truncate dd/d43/d55/d5e/d86/fb3 879042 0 2024-02-26T10:48:01.535 INFO:tasks.workunit.client.0.smithi060.stdout:1/663: dwrite d3/db/dc8/dbe/fd3 [0,4194304] 0 2024-02-26T10:48:01.535 INFO:tasks.workunit.client.0.smithi060.stdout:1/664: readlink d3/d54/l84 0 2024-02-26T10:48:01.549 INFO:tasks.workunit.client.0.smithi060.stdout:7/716: read da/d66/dad/f56 [1399345,83236] 0 2024-02-26T10:48:01.549 INFO:tasks.workunit.client.0.smithi060.stdout:8/569: dread d9/d5a/f69 [0,4194304] 0 2024-02-26T10:48:01.550 INFO:tasks.workunit.client.0.smithi060.stdout:8/570: chown d9/d15/c5b 27277 1 2024-02-26T10:48:01.550 INFO:tasks.workunit.client.0.smithi060.stdout:8/571: chown d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89 29860 1 2024-02-26T10:48:01.550 INFO:tasks.workunit.client.0.smithi060.stdout:6/593: read d6/d7/d25/d1c/f6d [1117146,36465] 0 2024-02-26T10:48:01.551 INFO:tasks.workunit.client.0.smithi060.stdout:3/455: dwrite d5/de/df/f5b [0,4194304] 0 2024-02-26T10:48:01.554 INFO:tasks.workunit.client.0.smithi060.stdout:3/456: rmdir d5/de/df/d57/d7a 39 2024-02-26T10:48:01.554 INFO:tasks.workunit.client.0.smithi060.stdout:3/457: chown d5/de/df/d11/d63/f64 12942119 1 2024-02-26T10:48:01.556 INFO:tasks.workunit.client.0.smithi060.stdout:8/572: link d9/d15/d29/d2f/d34/d72/d50/d6a/l47 d9/d15/d29/d2f/lb3 0 2024-02-26T10:48:01.556 INFO:tasks.workunit.client.0.smithi060.stdout:8/573: write d9/d5a/f68 [327509,126664] 0 2024-02-26T10:48:01.556 INFO:tasks.workunit.client.0.smithi060.stdout:6/594: unlink d6/d7/d25/d1c/d39/c45 0 2024-02-26T10:48:01.556 INFO:tasks.workunit.client.0.smithi060.stdout:6/595: chown d6/d7/d25/d80 6309 1 2024-02-26T10:48:01.556 INFO:tasks.workunit.client.0.smithi060.stdout:6/596: fsync d6/d7/d47/d69/f82 0 2024-02-26T10:48:01.557 INFO:tasks.workunit.client.0.smithi060.stdout:2/536: write dd/d43/d60/d41/d37/fa0 [7003266,42146] 0 2024-02-26T10:48:01.557 INFO:tasks.workunit.client.0.smithi060.stdout:6/597: write d6/d7/d25/d1c/d1f/d23/d28/f33 [18312,72779] 0 2024-02-26T10:48:01.558 INFO:tasks.workunit.client.0.smithi060.stdout:2/537: creat dd/d43/d55/d6e/fb8 x:0 0 0 2024-02-26T10:48:01.559 INFO:tasks.workunit.client.0.smithi060.stdout:3/458: unlink d5/de/f2b 0 2024-02-26T10:48:01.586 INFO:tasks.workunit.client.0.smithi060.stdout:0/545: dwrite d0/dd/d39/d40/f17 [8388608,4194304] 0 2024-02-26T10:48:01.591 INFO:tasks.workunit.client.0.smithi060.stdout:0/546: symlink d0/dd/d39/d40/lba 0 2024-02-26T10:48:01.591 INFO:tasks.workunit.client.0.smithi060.stdout:0/547: dread - d0/dd/d33/d6a/d6c/fab zero size 2024-02-26T10:48:01.591 INFO:tasks.workunit.client.0.smithi060.stdout:0/548: fdatasync d0/f78 0 2024-02-26T10:48:01.594 INFO:tasks.workunit.client.0.smithi060.stdout:8/574: rename d9/d15/d29/d2f/d34/d72/d50/fb0 to d9/d15/d29/d2f/d34/d72/d50/d6a/fb4 0 2024-02-26T10:48:01.595 INFO:tasks.workunit.client.0.smithi060.stdout:5/567: dwrite d0/de/f35 [0,4194304] 0 2024-02-26T10:48:01.603 INFO:tasks.workunit.client.0.smithi060.stdout:6/598: rename d6/d7/d25/d1c/d39 to d6/d7/d25/d1c/d1f/d23/d7a/dc3 0 2024-02-26T10:48:01.603 INFO:tasks.workunit.client.0.smithi060.stdout:6/599: truncate d6/d7/d25/d1c/d1f/d23/f50 696650 0 2024-02-26T10:48:01.603 INFO:tasks.workunit.client.0.smithi060.stdout:9/654: dwrite d3/d1e/d93/da2/db7/fcf [0,4194304] 0 2024-02-26T10:48:01.610 INFO:tasks.workunit.client.0.smithi060.stdout:6/600: dread d6/d7/d47/f9b [0,4194304] 0 2024-02-26T10:48:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:8/575: rename d9/d15/d29/d2f/d34/d72/d94/ca2 to d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/cb5 0 2024-02-26T10:48:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:8/576: creat d9/d15/d3e/fb6 x:0 0 0 2024-02-26T10:48:01.627 INFO:tasks.workunit.client.0.smithi060.stdout:0/549: write d0/f5f [3720600,2105] 0 2024-02-26T10:48:01.627 INFO:tasks.workunit.client.0.smithi060.stdout:0/550: chown d0/dd 9 1 2024-02-26T10:48:01.638 INFO:tasks.workunit.client.0.smithi060.stdout:3/459: dwrite d5/de/df/d4c/f6b [0,4194304] 0 2024-02-26T10:48:01.638 INFO:tasks.workunit.client.0.smithi060.stdout:5/568: dread d0/f90 [4194304,4194304] 0 2024-02-26T10:48:01.638 INFO:tasks.workunit.client.0.smithi060.stdout:4/608: dwrite d0/da/d57/f77 [0,4194304] 0 2024-02-26T10:48:01.639 INFO:tasks.workunit.client.0.smithi060.stdout:4/609: write d0/da/d57/f69 [325286,90028] 0 2024-02-26T10:48:01.643 INFO:tasks.workunit.client.0.smithi060.stdout:9/655: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/d47/d7a/fb5 [0,4194304] 0 2024-02-26T10:48:01.643 INFO:tasks.workunit.client.0.smithi060.stdout:6/601: write d6/f16 [742200,42378] 0 2024-02-26T10:48:01.655 INFO:tasks.workunit.client.0.smithi060.stdout:5/569: mknod d0/de/d70/d84/d92/cc7 0 2024-02-26T10:48:01.703 INFO:tasks.workunit.client.0.smithi060.stdout:9/656: dwrite d3/d1e/d5e/d35/f3e [0,4194304] 0 2024-02-26T10:48:01.703 INFO:tasks.workunit.client.0.smithi060.stdout:9/657: truncate d3/d68/fae 936134 0 2024-02-26T10:48:01.708 INFO:tasks.workunit.client.0.smithi060.stdout:9/658: link d3/f6b d3/d1e/d5e/d35/d87/d9b/d8a/fe0 0 2024-02-26T10:48:01.715 INFO:tasks.workunit.client.0.smithi060.stdout:4/610: dwrite d0/da/fb1 [0,4194304] 0 2024-02-26T10:48:01.724 INFO:tasks.workunit.client.0.smithi060.stdout:8/577: dwrite d9/d15/d29/d2f/d34/d72/f42 [0,4194304] 0 2024-02-26T10:48:01.724 INFO:tasks.workunit.client.0.smithi060.stdout:4/611: dread d0/d27/f5e [0,4194304] 0 2024-02-26T10:48:01.725 INFO:tasks.workunit.client.0.smithi060.stdout:8/578: write d9/d15/d3e/f9b [740478,104933] 0 2024-02-26T10:48:01.729 INFO:tasks.workunit.client.0.smithi060.stdout:4/612: creat d0/d27/d76/dbd/fbe x:0 0 0 2024-02-26T10:48:01.729 INFO:tasks.workunit.client.0.smithi060.stdout:3/460: dwrite d5/de/df/d57/d60/f3a [0,4194304] 0 2024-02-26T10:48:01.729 INFO:tasks.workunit.client.0.smithi060.stdout:3/461: stat d5/de/df/d11/f6e 0 2024-02-26T10:48:01.743 INFO:tasks.workunit.client.0.smithi060.stdout:0/551: dwrite d0/dd/f4e [0,4194304] 0 2024-02-26T10:48:01.745 INFO:tasks.workunit.client.0.smithi060.stdout:5/570: dwrite d0/de/d20/d32/d4d/fb4 [0,4194304] 0 2024-02-26T10:48:01.749 INFO:tasks.workunit.client.0.smithi060.stdout:5/571: mkdir d0/de/d20/d32/d4d/da3/da5/dc8 0 2024-02-26T10:48:01.749 INFO:tasks.workunit.client.0.smithi060.stdout:6/602: dwrite d6/d7/d47/f55 [0,4194304] 0 2024-02-26T10:48:01.751 INFO:tasks.workunit.client.0.smithi060.stdout:6/603: mknod d6/d7/d25/d1c/d1f/d23/d2b/d4d/cc4 0 2024-02-26T10:48:01.753 INFO:tasks.workunit.client.0.smithi060.stdout:6/604: truncate d6/d7/d25/d1c/f9e 957010 0 2024-02-26T10:48:01.753 INFO:tasks.workunit.client.0.smithi060.stdout:6/605: chown d6/f18 668 1 2024-02-26T10:48:01.753 INFO:tasks.workunit.client.0.smithi060.stdout:6/606: readlink d6/d7/d25/d1c/d1f/d23/d2b/d4d/l6a 0 2024-02-26T10:48:01.754 INFO:tasks.workunit.client.0.smithi060.stdout:6/607: write d6/d7/d25/d1c/d1f/d23/f50 [1347253,17998] 0 2024-02-26T10:48:01.755 INFO:tasks.workunit.client.0.smithi060.stdout:6/608: symlink d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/lc5 0 2024-02-26T10:48:01.755 INFO:tasks.workunit.client.0.smithi060.stdout:6/609: chown d6/d7/d2e/d56/d73 42385 1 2024-02-26T10:48:01.782 INFO:tasks.workunit.client.0.smithi060.stdout:9/659: dwrite d3/f6b [4194304,4194304] 0 2024-02-26T10:48:01.785 INFO:tasks.workunit.client.0.smithi060.stdout:9/660: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fe1 x:0 0 0 2024-02-26T10:48:01.787 INFO:tasks.workunit.client.0.smithi060.stdout:6/610: dread d6/d7/d25/d1c/d1f/d23/d2b/d67/f8a [0,4194304] 0 2024-02-26T10:48:01.787 INFO:tasks.workunit.client.0.smithi060.stdout:6/611: fsync d6/d7/d25/f49 0 2024-02-26T10:48:01.787 INFO:tasks.workunit.client.0.smithi060.stdout:1/665: sync 2024-02-26T10:48:01.788 INFO:tasks.workunit.client.0.smithi060.stdout:2/538: sync 2024-02-26T10:48:01.788 INFO:tasks.workunit.client.0.smithi060.stdout:2/539: stat dd/d43/d60/d41/d37/d38/l61 0 2024-02-26T10:48:01.789 INFO:tasks.workunit.client.0.smithi060.stdout:7/717: sync 2024-02-26T10:48:01.790 INFO:tasks.workunit.client.0.smithi060.stdout:6/612: creat d6/d7/d25/d1c/d1f/d23/d7a/dc3/fc6 x:0 0 0 2024-02-26T10:48:01.792 INFO:tasks.workunit.client.0.smithi060.stdout:7/718: creat da/df1/dd7/ff3 x:0 0 0 2024-02-26T10:48:01.792 INFO:tasks.workunit.client.0.smithi060.stdout:7/719: dread - da/d15/d21/fb2 zero size 2024-02-26T10:48:01.793 INFO:tasks.workunit.client.0.smithi060.stdout:2/540: mkdir dd/d43/d55/d5e/d86/d97/db9 0 2024-02-26T10:48:01.793 INFO:tasks.workunit.client.0.smithi060.stdout:2/541: dread - dd/d43/d60/d41/d42/d6f/fb1 zero size 2024-02-26T10:48:01.793 INFO:tasks.workunit.client.0.smithi060.stdout:6/613: symlink d6/lc7 0 2024-02-26T10:48:01.793 INFO:tasks.workunit.client.0.smithi060.stdout:6/614: dread - d6/d6e/d6f/fa1 zero size 2024-02-26T10:48:01.794 INFO:tasks.workunit.client.0.smithi060.stdout:8/579: dwrite d9/d15/d29/d2f/d34/d72/f4c [0,4194304] 0 2024-02-26T10:48:01.794 INFO:tasks.workunit.client.0.smithi060.stdout:8/580: truncate d9/d15/d3e/fb2 49466 0 2024-02-26T10:48:01.794 INFO:tasks.workunit.client.0.smithi060.stdout:6/615: link d6/d7/l1e d6/d6e/d6f/lc8 0 2024-02-26T10:48:01.795 INFO:tasks.workunit.client.0.smithi060.stdout:8/581: write d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/f96 [187680,99088] 0 2024-02-26T10:48:01.795 INFO:tasks.workunit.client.0.smithi060.stdout:6/616: creat d6/d6e/fc9 x:0 0 0 2024-02-26T10:48:01.796 INFO:tasks.workunit.client.0.smithi060.stdout:8/582: creat d9/d15/d29/d2f/fb7 x:0 0 0 2024-02-26T10:48:01.799 INFO:tasks.workunit.client.0.smithi060.stdout:6/617: mknod d6/d7/d25/d1c/d1f/d23/d7a/da0/cca 0 2024-02-26T10:48:01.799 INFO:tasks.workunit.client.0.smithi060.stdout:2/542: truncate dd/f68 3783796 0 2024-02-26T10:48:01.799 INFO:tasks.workunit.client.0.smithi060.stdout:2/543: chown lb 771066 1 2024-02-26T10:48:01.799 INFO:tasks.workunit.client.0.smithi060.stdout:2/544: dread - dd/d43/d55/d6e/fb8 zero size 2024-02-26T10:48:01.802 INFO:tasks.workunit.client.0.smithi060.stdout:4/613: dwrite d0/da/f14 [4194304,4194304] 0 2024-02-26T10:48:01.805 INFO:tasks.workunit.client.0.smithi060.stdout:5/572: dwrite d0/de/d20/d82/f87 [0,4194304] 0 2024-02-26T10:48:01.805 INFO:tasks.workunit.client.0.smithi060.stdout:5/573: write d0/de/d20/f58 [1129669,112362] 0 2024-02-26T10:48:01.805 INFO:tasks.workunit.client.0.smithi060.stdout:4/614: symlink d0/d27/d76/dbd/d8e/lbf 0 2024-02-26T10:48:01.814 INFO:tasks.workunit.client.0.smithi060.stdout:8/583: symlink d9/d5a/d9c/lb8 0 2024-02-26T10:48:01.815 INFO:tasks.workunit.client.0.smithi060.stdout:5/574: rmdir d0/de/d20/d32/d4d/da3 39 2024-02-26T10:48:01.816 INFO:tasks.workunit.client.0.smithi060.stdout:5/575: creat d0/fc9 x:0 0 0 2024-02-26T10:48:01.816 INFO:tasks.workunit.client.0.smithi060.stdout:5/576: read - d0/de/d20/d32/d4c/f74 zero size 2024-02-26T10:48:01.817 INFO:tasks.workunit.client.0.smithi060.stdout:5/577: creat d0/de/fca x:0 0 0 2024-02-26T10:48:01.826 INFO:tasks.workunit.client.0.smithi060.stdout:8/584: mknod d9/d15/d29/d95/cb9 0 2024-02-26T10:48:01.826 INFO:tasks.workunit.client.0.smithi060.stdout:4/615: rename d0/d7 to d0/d27/d7f/db0/db5/dc0 0 2024-02-26T10:48:01.826 INFO:tasks.workunit.client.0.smithi060.stdout:4/616: dread - d0/d27/d76/dbd/fbe zero size 2024-02-26T10:48:01.826 INFO:tasks.workunit.client.0.smithi060.stdout:8/585: write d9/f20 [5498277,19439] 0 2024-02-26T10:48:01.826 INFO:tasks.workunit.client.0.smithi060.stdout:8/586: write d9/d5a/d58/fa0 [635300,77538] 0 2024-02-26T10:48:01.827 INFO:tasks.workunit.client.0.smithi060.stdout:4/617: creat d0/da/d3b/d1b/fc1 x:0 0 0 2024-02-26T10:48:01.828 INFO:tasks.workunit.client.0.smithi060.stdout:2/545: write dd/d43/f6c [1311078,64260] 0 2024-02-26T10:48:01.829 INFO:tasks.workunit.client.0.smithi060.stdout:2/546: write dd/d43/d60/d41/d37/d38/d57/f79 [332250,37718] 0 2024-02-26T10:48:01.830 INFO:tasks.workunit.client.0.smithi060.stdout:2/547: mknod dd/d43/d55/d5e/d74/da5/cba 0 2024-02-26T10:48:01.834 INFO:tasks.workunit.client.0.smithi060.stdout:8/587: getdents d9/d5a/d9c 0 2024-02-26T10:48:01.841 INFO:tasks.workunit.client.0.smithi060.stdout:6/618: dwrite d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/fb2 [0,4194304] 0 2024-02-26T10:48:01.844 INFO:tasks.workunit.client.0.smithi060.stdout:5/578: rename d0/de/d20/d32/d4d/l50 to d0/de/d15/d65/d86/lcb 0 2024-02-26T10:48:01.852 INFO:tasks.workunit.client.0.smithi060.stdout:1/666: dread d3/d35/d5e/f66 [0,4194304] 0 2024-02-26T10:48:01.855 INFO:tasks.workunit.client.0.smithi060.stdout:5/579: dread d0/de/d15/f2d [0,4194304] 0 2024-02-26T10:48:01.863 INFO:tasks.workunit.client.0.smithi060.stdout:4/618: dread d0/da/d57/f96 [0,4194304] 0 2024-02-26T10:48:01.863 INFO:tasks.workunit.client.0.smithi060.stdout:4/619: chown d0/da/d3b/d1b/l3d 50448 1 2024-02-26T10:48:01.864 INFO:tasks.workunit.client.0.smithi060.stdout:4/620: dread - d0/da/f9a zero size 2024-02-26T10:48:01.865 INFO:tasks.workunit.client.0.smithi060.stdout:1/667: rename d3/d35/d53/d68/d85/c41 to d3/db/d1e/d39/cea 0 2024-02-26T10:48:01.867 INFO:tasks.workunit.client.0.smithi060.stdout:7/720: dwrite da/d15/d88/dce/faf [0,4194304] 0 2024-02-26T10:48:01.873 INFO:tasks.workunit.client.0.smithi060.stdout:8/588: dread d9/fd [0,4194304] 0 2024-02-26T10:48:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.888+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa63c02ce30 con 0x7fa63c076e60 2024-02-26T10:48:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:48:01.887 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:47:47.060274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":80,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":80,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:48:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.890+0000 7fa632a00640 1 -- 172.21.15.60:0/3443968176 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11365 (secure 0 0 0) 0x7fa6280bccc0 con 0x7fa63c076e60 2024-02-26T10:48:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-02-26T10:48:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.893+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa6200a1e80 msgr2=0x7fa6200a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.893+0000 7fa640a00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa6200a1e80 0x7fa6200a4340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa62c005fd0 tx=0x7fa62c005850 comp rx=0 tx=0).stop 2024-02-26T10:48:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 msgr2=0x7fa63c0713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 0x7fa63c0713f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa628005ab0 tx=0x7fa62802c6c0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 shutdown_connections 2024-02-26T10:48:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa6200a1e80 0x7fa6200a4340 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c077800 0x7fa63c072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa63c076e60 0x7fa63c0713f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 --2- 172.21.15.60:0/3443968176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa63c075c20 0x7fa63c070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 >> 172.21.15.60:0/3443968176 conn(0x7fa63c0fea50 msgr2=0x7fa63c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 shutdown_connections 2024-02-26T10:48:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:01.894+0000 7fa640a00640 1 -- 172.21.15.60:0/3443968176 wait complete. 2024-02-26T10:48:01.897 INFO:tasks.workunit.client.0.smithi060.stdout:7/721: dread da/d66/fcd [0,4194304] 0 2024-02-26T10:48:01.898 INFO:tasks.workunit.client.0.smithi060.stdout:8/589: mkdir d9/d15/d29/d2f/d34/d72/d94/dba 0 2024-02-26T10:48:01.901 INFO:tasks.workunit.client.0.smithi060.stdout:7/722: readlink da/d15/d21/d2f/d78/dd3/l7c 0 2024-02-26T10:48:01.901 INFO:tasks.workunit.client.0.smithi060.stdout:7/723: getdents da/d66/dad/d8f 0 2024-02-26T10:48:01.901 INFO:tasks.workunit.client.0.smithi060.stdout:7/724: fsync da/d15/d21/d2f/f48 0 2024-02-26T10:48:01.913 INFO:tasks.workunit.client.0.smithi060.stdout:5/580: write d0/f3a [2860826,23390] 0 2024-02-26T10:48:01.913 INFO:tasks.workunit.client.0.smithi060.stdout:6/619: dwrite d6/d7/d2e/d56/f61 [0,4194304] 0 2024-02-26T10:48:01.913 INFO:tasks.workunit.client.0.smithi060.stdout:6/620: fsync d6/f11 0 2024-02-26T10:48:01.914 INFO:tasks.workunit.client.0.smithi060.stdout:6/621: write d6/d7/d25/d1c/d1f/d23/d2b/d67/f8a [3678025,117255] 0 2024-02-26T10:48:01.915 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:48:01.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T10:48:01.916 INFO:tasks.workunit.client.0.smithi060.stdout:6/622: fdatasync d6/f16 0 2024-02-26T10:48:01.918 INFO:tasks.workunit.client.0.smithi060.stdout:4/621: dwrite d0/d27/d76/dbd/d60/fb6 [0,4194304] 0 2024-02-26T10:48:01.924 INFO:tasks.workunit.client.0.smithi060.stdout:4/622: mkdir d0/da/d3b/d1b/dc2 0 2024-02-26T10:48:01.924 INFO:tasks.workunit.client.0.smithi060.stdout:4/623: stat d0/da/fb1 0 2024-02-26T10:48:01.924 INFO:tasks.workunit.client.0.smithi060.stdout:4/624: write d0/da/d3b/d1b/fc1 [304469,12990] 0 2024-02-26T10:48:01.924 INFO:tasks.workunit.client.0.smithi060.stdout:4/625: symlink d0/d27/d7f/db0/lc3 0 2024-02-26T10:48:01.924 INFO:tasks.workunit.client.0.smithi060.stdout:4/626: link d0/d27/d7f/db0/db5/dc0/c50 d0/d27/d7f/cc4 0 2024-02-26T10:48:01.936 INFO:tasks.workunit.client.0.smithi060.stdout:1/668: dwrite d3/d35/d53/d68/d85/fd0 [0,4194304] 0 2024-02-26T10:48:01.936 INFO:tasks.workunit.client.0.smithi060.stdout:4/627: rmdir d0/d27/d76/da3 0 2024-02-26T10:48:01.936 INFO:tasks.workunit.client.0.smithi060.stdout:4/628: getdents d0/d27/d76/dbd/d8e/db3 0 2024-02-26T10:48:01.936 INFO:tasks.workunit.client.0.smithi060.stdout:4/629: write d0/d27/d76/dbd/d8e/fac [809093,59666] 0 2024-02-26T10:48:01.936 INFO:tasks.workunit.client.0.smithi060.stdout:4/630: dread - d0/d27/d76/dbd/d8e/fb4 zero size 2024-02-26T10:48:01.936 INFO:tasks.workunit.client.0.smithi060.stdout:6/623: unlink d6/d7/d25/d1c/d1f/d23/d7a/dc3/c9d 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:1/669: creat d3/db/d1e/d6d/da3/feb x:0 0 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:1/670: chown d3/d35/l75 1 1 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:8/590: dread d9/fc [0,4194304] 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:8/591: dread - d9/d15/d29/d2f/f98 zero size 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:4/631: mkdir d0/da/d3b/db9/dc5 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:6/624: symlink d6/d7/d2e/d56/d73/lcb 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:6/625: write d6/d7/d25/d1c/d1f/d23/d28/f33 [1383380,14898] 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:6/626: fsync d6/d7/d25/d1c/d1f/d23/d7a/f88 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:8/592: symlink d9/d15/d29/d2f/d34/lbb 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:8/593: chown d9/d15/d29/f7d 97 1 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:4/632: creat d0/da/d57/fc6 x:0 0 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:2/548: dwrite dd/d43/d60/d41/d37/d38/d57/f79 [0,4194304] 0 2024-02-26T10:48:01.937 INFO:tasks.workunit.client.0.smithi060.stdout:8/594: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/fbc x:0 0 0 2024-02-26T10:48:01.938 INFO:tasks.workunit.client.0.smithi060.stdout:2/549: unlink dd/d52/l54 0 2024-02-26T10:48:01.938 INFO:tasks.workunit.client.0.smithi060.stdout:2/550: chown dd/d43/d60/d41/d37/d38/d57/l71 0 1 2024-02-26T10:48:01.942 INFO:tasks.workunit.client.0.smithi060.stdout:2/551: truncate dd/d43/d60/f3f 4411574 0 2024-02-26T10:48:01.943 INFO:tasks.workunit.client.0.smithi060.stdout:1/671: dread d3/db/d2e/f95 [0,4194304] 0 2024-02-26T10:48:01.945 INFO:tasks.workunit.client.0.smithi060.stdout:4/633: rename d0/d27/d7f to d0/da/dc7 0 2024-02-26T10:48:01.953 INFO:tasks.workunit.client.0.smithi060.stdout:8/595: link d9/d15/d29/d2f/l87 d9/d5a/d58/lbd 0 2024-02-26T10:48:01.953 INFO:tasks.workunit.client.0.smithi060.stdout:8/596: truncate d9/d15/d29/f7d 689704 0 2024-02-26T10:48:01.953 INFO:tasks.workunit.client.0.smithi060.stdout:2/552: link dd/d43/d60/l4d dd/d43/d96/lbb 0 2024-02-26T10:48:01.953 INFO:tasks.workunit.client.0.smithi060.stdout:4/634: creat d0/d27/d76/dbd/d8e/db3/fc8 x:0 0 0 2024-02-26T10:48:01.954 INFO:tasks.workunit.client.0.smithi060.stdout:2/553: mkdir dd/d43/d60/d41/d42/d6f/d8f/dbc 0 2024-02-26T10:48:01.954 INFO:tasks.workunit.client.0.smithi060.stdout:2/554: chown dd/d43/d60/d41/d37/d38/l61 250088853 1 2024-02-26T10:48:01.958 INFO:tasks.workunit.client.0.smithi060.stdout:4/635: mknod d0/da/cc9 0 2024-02-26T10:48:01.958 INFO:tasks.workunit.client.0.smithi060.stdout:2/555: creat dd/d43/d60/fbd x:0 0 0 2024-02-26T10:48:01.958 INFO:tasks.workunit.client.0.smithi060.stdout:1/672: dread d3/db/d1e/d38/f8e [0,4194304] 0 2024-02-26T10:48:01.958 INFO:tasks.workunit.client.0.smithi060.stdout:1/673: fdatasync d3/db/d1e/d6d/da3/feb 0 2024-02-26T10:48:01.967 INFO:tasks.workunit.client.0.smithi060.stdout:5/581: dwrite d0/de/d15/f2d [0,4194304] 0 2024-02-26T10:48:01.979 INFO:tasks.workunit.client.0.smithi060.stdout:8/597: read d9/d15/d29/d2f/d34/d72/f42 [610396,106340] 0 2024-02-26T10:48:01.979 INFO:tasks.workunit.client.0.smithi060.stdout:4/636: dread d0/d27/d76/dbd/d8e/f99 [0,4194304] 0 2024-02-26T10:48:01.981 INFO:tasks.workunit.client.0.smithi060.stdout:5/582: write d0/de/d20/f31 [630785,122274] 0 2024-02-26T10:48:01.981 INFO:tasks.workunit.client.0.smithi060.stdout:8/598: getdents d9/d15/d29/d2f/d34 0 2024-02-26T10:48:01.982 INFO:tasks.workunit.client.0.smithi060.stdout:6/627: dwrite d6/f8e [0,4194304] 0 2024-02-26T10:48:01.984 INFO:tasks.workunit.client.0.smithi060.stdout:5/583: mkdir d0/de/d15/d65/d86/dcc 0 2024-02-26T10:48:01.984 INFO:tasks.workunit.client.0.smithi060.stdout:5/584: chown d0/de/f5d 3932 1 2024-02-26T10:48:01.988 INFO:tasks.workunit.client.0.smithi060.stdout:6/628: rename d6/d7/d25/d1c/d1f/d23/d7a to d6/d8d/dcc 0 2024-02-26T10:48:01.989 INFO:tasks.workunit.client.0.smithi060.stdout:5/585: mknod d0/de/d20/d32/d4d/d97/ccd 0 2024-02-26T10:48:02.002 INFO:tasks.workunit.client.0.smithi060.stdout:6/629: rename d6/d7/d25/d1c/f60 to d6/d6e/d6f/fcd 0 2024-02-26T10:48:02.004 INFO:tasks.workunit.client.0.smithi060.stdout:6/630: rename d6/d6e/d6f/fa1 to d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac/db5/fce 0 2024-02-26T10:48:02.004 INFO:tasks.workunit.client.0.smithi060.stdout:6/631: dread - d6/d7/d47/d69/fb1 zero size 2024-02-26T10:48:02.019 INFO:tasks.workunit.client.0.smithi060.stdout:2/556: dwrite dd/d43/d60/fbd [0,4194304] 0 2024-02-26T10:48:02.019 INFO:tasks.workunit.client.0.smithi060.stdout:2/557: chown dd/d43/d60/d41 60 1 2024-02-26T10:48:02.023 INFO:tasks.workunit.client.0.smithi060.stdout:5/586: dwrite d0/de/d15/d65/f6c [0,4194304] 0 2024-02-26T10:48:02.027 INFO:tasks.workunit.client.0.smithi060.stdout:4/637: write d0/da/d3b/d16/f18 [3691014,20031] 0 2024-02-26T10:48:02.030 INFO:tasks.workunit.client.0.smithi060.stdout:5/587: creat d0/de/d15/d26/fce x:0 0 0 2024-02-26T10:48:02.030 INFO:tasks.workunit.client.0.smithi060.stdout:5/588: write d0/de/d20/f31 [2046833,86381] 0 2024-02-26T10:48:02.031 INFO:tasks.workunit.client.0.smithi060.stdout:4/638: mkdir d0/da/d3b/dca 0 2024-02-26T10:48:02.032 INFO:tasks.workunit.client.0.smithi060.stdout:5/589: chown d0/de/d20/d32/d4d/da3 62936142 1 2024-02-26T10:48:02.032 INFO:tasks.workunit.client.0.smithi060.stdout:5/590: chown d0/de/d70/d84/d92/c8f 10 1 2024-02-26T10:48:02.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/599: dwrite d9/d15/d29/d2f/d34/d72/f42 [0,4194304] 0 2024-02-26T10:48:02.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/600: truncate d9/d15/d29/f59 285135 0 2024-02-26T10:48:02.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/601: fsync d9/f20 0 2024-02-26T10:48:02.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/602: readlink d9/d15/d3e/l97 0 2024-02-26T10:48:02.046 INFO:tasks.workunit.client.0.smithi060.stdout:1/674: dwrite d3/d35/d53/d57/f83 [0,4194304] 0 2024-02-26T10:48:02.047 INFO:tasks.workunit.client.0.smithi060.stdout:2/558: write dd/d43/d60/d41/f95 [3537130,60020] 0 2024-02-26T10:48:02.056 INFO:tasks.workunit.client.0.smithi060.stdout:6/632: dwrite d6/d8d/dcc/f88 [0,4194304] 0 2024-02-26T10:48:02.056 INFO:tasks.workunit.client.0.smithi060.stdout:6/633: truncate d6/d7/d25/d1c/d1f/f35 2582199 0 2024-02-26T10:48:02.056 INFO:tasks.workunit.client.0.smithi060.stdout:6/634: write d6/d7/d25/f49 [2866127,106444] 0 2024-02-26T10:48:02.057 INFO:tasks.workunit.client.0.smithi060.stdout:6/635: truncate d6/d6e/d6f/fcd 908683 0 2024-02-26T10:48:02.063 INFO:tasks.workunit.client.0.smithi060.stdout:1/675: creat d3/d35/d53/d68/d85/fec x:0 0 0 2024-02-26T10:48:02.063 INFO:tasks.workunit.client.0.smithi060.stdout:4/639: write d0/da/dc7/db0/db5/dc0/f6a [3017982,6131] 0 2024-02-26T10:48:02.063 INFO:tasks.workunit.client.0.smithi060.stdout:5/591: dread d0/d33/f62 [0,4194304] 0 2024-02-26T10:48:02.063 INFO:tasks.workunit.client.0.smithi060.stdout:4/640: creat d0/da/d3b/db9/fcb x:0 0 0 2024-02-26T10:48:02.064 INFO:tasks.workunit.client.0.smithi060.stdout:5/592: truncate d0/de/d15/f5a 3923875 0 2024-02-26T10:48:02.070 INFO:tasks.workunit.client.0.smithi060.stdout:6/636: stat d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/l9f 0 2024-02-26T10:48:02.080 INFO:tasks.workunit.client.0.smithi060.stdout:2/559: write dd/d43/d60/d41/f31 [2491709,32449] 0 2024-02-26T10:48:02.088 INFO:tasks.workunit.client.0.smithi060.stdout:2/560: dread dd/d43/d60/f3f [4194304,4194304] 0 2024-02-26T10:48:02.090 INFO:tasks.workunit.client.0.smithi060.stdout:2/561: creat dd/d43/d60/d41/d42/d6f/d8f/dbc/fbe x:0 0 0 2024-02-26T10:48:02.091 INFO:tasks.workunit.client.0.smithi060.stdout:2/562: truncate dd/f68 1267669 0 2024-02-26T10:48:02.092 INFO:tasks.workunit.client.0.smithi060.stdout:3/462: sync 2024-02-26T10:48:02.092 INFO:tasks.workunit.client.0.smithi060.stdout:0/552: sync 2024-02-26T10:48:02.092 INFO:tasks.workunit.client.0.smithi060.stdout:7/725: sync 2024-02-26T10:48:02.092 INFO:tasks.workunit.client.0.smithi060.stdout:7/726: chown da/d15/d21/d2f/d78/dd3/d67/f77 122604659 1 2024-02-26T10:48:02.092 INFO:tasks.workunit.client.0.smithi060.stdout:0/553: truncate d0/dd/f77 4886259 0 2024-02-26T10:48:02.092 INFO:tasks.workunit.client.0.smithi060.stdout:3/463: write d5/d37/f41 [1298489,16080] 0 2024-02-26T10:48:02.093 INFO:tasks.workunit.client.0.smithi060.stdout:0/554: dread - d0/dd/d39/d63/f67 zero size 2024-02-26T10:48:02.093 INFO:tasks.workunit.client.0.smithi060.stdout:9/661: sync 2024-02-26T10:48:02.093 INFO:tasks.workunit.client.0.smithi060.stdout:0/555: write d0/dd/fb0 [463576,66508] 0 2024-02-26T10:48:02.095 INFO:tasks.workunit.client.0.smithi060.stdout:2/563: chown dd/d43/d55/ca8 206840 1 2024-02-26T10:48:02.096 INFO:tasks.workunit.client.0.smithi060.stdout:5/593: creat d0/de/d20/d32/d4d/da3/fcf x:0 0 0 2024-02-26T10:48:02.097 INFO:tasks.workunit.client.0.smithi060.stdout:1/676: dread d3/f34 [0,4194304] 0 2024-02-26T10:48:02.097 INFO:tasks.workunit.client.0.smithi060.stdout:2/564: symlink dd/d43/d60/d41/d42/d6f/d8f/dbc/lbf 0 2024-02-26T10:48:02.097 INFO:tasks.workunit.client.0.smithi060.stdout:1/677: chown d3/d35/d53/cdb 31 1 2024-02-26T10:48:02.098 INFO:tasks.workunit.client.0.smithi060.stdout:1/678: symlink d3/db/d1e/d6d/d78/led 0 2024-02-26T10:48:02.102 INFO:tasks.workunit.client.0.smithi060.stdout:5/594: rename d0/de/d20/d32/d4c/fb0 to d0/de/d15/d65/fd0 0 2024-02-26T10:48:02.102 INFO:tasks.workunit.client.0.smithi060.stdout:5/595: chown d0/de/f29 1189 1 2024-02-26T10:48:02.102 INFO:tasks.workunit.client.0.smithi060.stdout:6/637: creat d6/d8d/fcf x:0 0 0 2024-02-26T10:48:02.105 INFO:tasks.workunit.client.0.smithi060.stdout:2/565: rename dd/d43/d60/d41/d37/d4c/l5b to dd/d43/d60/d41/d42/d6f/d8f/dbc/lc0 0 2024-02-26T10:48:02.107 INFO:tasks.workunit.client.0.smithi060.stdout:2/566: rename dd/d43/d60/d41/l2a to dd/d43/d60/d41/d42/d6f/lc1 0 2024-02-26T10:48:02.110 INFO:tasks.workunit.client.0.smithi060.stdout:7/727: mknod da/d15/d21/d41/cf4 0 2024-02-26T10:48:02.110 INFO:tasks.workunit.client.0.smithi060.stdout:6/638: link d6/d7/d25/d1c/l64 d6/d7/d25/d1c/d1f/d23/d2b/d4d/ld0 0 2024-02-26T10:48:02.111 INFO:tasks.workunit.client.0.smithi060.stdout:3/464: creat d5/de/df/d4c/f88 x:0 0 0 2024-02-26T10:48:02.114 INFO:tasks.workunit.client.0.smithi060.stdout:7/728: symlink da/df1/dd7/lf5 0 2024-02-26T10:48:02.119 INFO:tasks.workunit.client.0.smithi060.stdout:3/465: link d5/de/df/d4c/f88 d5/de/df/d57/d7a/f89 0 2024-02-26T10:48:02.122 INFO:tasks.workunit.client.0.smithi060.stdout:4/641: dwrite d0/d27/d2f/f7d [0,4194304] 0 2024-02-26T10:48:02.138 INFO:tasks.workunit.client.0.smithi060.stdout:3/466: truncate d5/de/df/d11/f1c 1193268 0 2024-02-26T10:48:02.141 INFO:tasks.workunit.client.0.smithi060.stdout:1/679: write d3/d54/f7c [1389253,29117] 0 2024-02-26T10:48:02.145 INFO:tasks.workunit.client.0.smithi060.stdout:9/662: symlink d3/d1e/d93/da2/le2 0 2024-02-26T10:48:02.153 INFO:tasks.workunit.client.0.smithi060.stdout:4/642: creat d0/da/fcc x:0 0 0 2024-02-26T10:48:02.153 INFO:tasks.workunit.client.0.smithi060.stdout:0/556: dwrite d0/dd/d53/d87/d94/da6/f9d [0,4194304] 0 2024-02-26T10:48:02.153 INFO:tasks.workunit.client.0.smithi060.stdout:7/729: mknod da/d15/d21/d2f/d78/dd3/cf6 0 2024-02-26T10:48:02.154 INFO:tasks.workunit.client.0.smithi060.stdout:3/467: creat d5/d76/d7d/f8a x:0 0 0 2024-02-26T10:48:02.158 INFO:tasks.workunit.client.0.smithi060.stdout:0/557: rmdir d0/dd/d39/d93 39 2024-02-26T10:48:02.159 INFO:tasks.workunit.client.0.smithi060.stdout:3/468: creat d5/de/df/d57/d7a/f8b x:0 0 0 2024-02-26T10:48:02.169 INFO:tasks.workunit.client.0.smithi060.stdout:0/558: creat d0/dd/fbb x:0 0 0 2024-02-26T10:48:02.169 INFO:tasks.workunit.client.0.smithi060.stdout:0/559: chown d0/dd/d39/d93/c95 23125 1 2024-02-26T10:48:02.177 INFO:tasks.workunit.client.0.smithi060.stdout:7/730: rename da/d15/d21/d2f/d78/dd3/f93 to da/d15/d21/ff7 0 2024-02-26T10:48:02.177 INFO:tasks.workunit.client.0.smithi060.stdout:1/680: write d3/f2b [856724,10806] 0 2024-02-26T10:48:02.177 INFO:tasks.workunit.client.0.smithi060.stdout:9/663: stat d3/d1e/d93/da2 0 2024-02-26T10:48:02.177 INFO:tasks.workunit.client.0.smithi060.stdout:2/567: dwrite dd/d43/d55/d5e/d74/fb6 [0,4194304] 0 2024-02-26T10:48:02.177 INFO:tasks.workunit.client.0.smithi060.stdout:2/568: chown dd/d43/d60/d41/d42 604335 1 2024-02-26T10:48:02.177 INFO:tasks.workunit.client.0.smithi060.stdout:2/569: write dd/d43/d60/f53 [2461096,17020] 0 2024-02-26T10:48:02.178 INFO:tasks.workunit.client.0.smithi060.stdout:9/664: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/d98/fe3 x:0 0 0 2024-02-26T10:48:02.178 INFO:tasks.workunit.client.0.smithi060.stdout:9/665: chown d3/d1e 1430 1 2024-02-26T10:48:02.178 INFO:tasks.workunit.client.0.smithi060.stdout:0/560: mknod d0/d42/cbc 0 2024-02-26T10:48:02.178 INFO:tasks.workunit.client.0.smithi060.stdout:0/561: stat d0/d37/d44 0 2024-02-26T10:48:02.178 INFO:tasks.workunit.client.0.smithi060.stdout:0/562: write d0/f82 [2528860,113272] 0 2024-02-26T10:48:02.178 INFO:tasks.workunit.client.0.smithi060.stdout:1/681: getdents d3/db 0 2024-02-26T10:48:02.178 INFO:tasks.workunit.client.0.smithi060.stdout:1/682: fdatasync d3/d35/d76/faf 0 2024-02-26T10:48:02.178 INFO:tasks.workunit.client.0.smithi060.stdout:2/570: creat dd/d43/d60/d41/d37/d4c/fc2 x:0 0 0 2024-02-26T10:48:02.178 INFO:tasks.workunit.client.0.smithi060.stdout:4/643: write d0/da/d57/f77 [2125123,24346] 0 2024-02-26T10:48:02.179 INFO:tasks.workunit.client.0.smithi060.stdout:5/596: dwrite d0/d33/f43 [4194304,4194304] 0 2024-02-26T10:48:02.179 INFO:tasks.workunit.client.0.smithi060.stdout:5/597: chown d0/d33/fa1 0 1 2024-02-26T10:48:02.179 INFO:tasks.workunit.client.0.smithi060.stdout:3/469: dread d5/de/df/d11/f36 [0,4194304] 0 2024-02-26T10:48:02.179 INFO:tasks.workunit.client.0.smithi060.stdout:3/470: fdatasync d5/de/df/d11/f1c 0 2024-02-26T10:48:02.179 INFO:tasks.workunit.client.0.smithi060.stdout:3/471: chown d5/de/df/d11/f62 12451405 1 2024-02-26T10:48:02.179 INFO:tasks.workunit.client.0.smithi060.stdout:2/571: creat dd/d43/d55/d5e/d74/fc3 x:0 0 0 2024-02-26T10:48:02.179 INFO:tasks.workunit.client.0.smithi060.stdout:2/572: chown dd/d43/d60/d41/f6d 840590986 1 2024-02-26T10:48:02.180 INFO:tasks.workunit.client.0.smithi060.stdout:5/598: creat d0/de/d20/d32/fd1 x:0 0 0 2024-02-26T10:48:02.180 INFO:tasks.workunit.client.0.smithi060.stdout:0/563: unlink d0/dd/fb0 0 2024-02-26T10:48:02.180 INFO:tasks.workunit.client.0.smithi060.stdout:0/564: dread - d0/d42/d7a/d7f/fb1 zero size 2024-02-26T10:48:02.181 INFO:tasks.workunit.client.0.smithi060.stdout:0/565: chown d0/c3b 92686677 1 2024-02-26T10:48:02.185 INFO:tasks.workunit.client.0.smithi060.stdout:3/472: rename d5/de/df/d11/c18 to d5/de/df/d11/c8c 0 2024-02-26T10:48:02.185 INFO:tasks.workunit.client.0.smithi060.stdout:0/566: mknod d0/dd/d39/d40/d83/cbd 0 2024-02-26T10:48:02.186 INFO:tasks.workunit.client.0.smithi060.stdout:1/683: link d3/db/d2e/d51/fac d3/d35/dc2/fee 0 2024-02-26T10:48:02.193 INFO:tasks.workunit.client.0.smithi060.stdout:0/567: rename d0/c57 to d0/dd/d39/d40/dad/cbe 0 2024-02-26T10:48:02.193 INFO:tasks.workunit.client.0.smithi060.stdout:3/473: mknod d5/de/df/c8d 0 2024-02-26T10:48:02.195 INFO:tasks.workunit.client.0.smithi060.stdout:2/573: mknod dd/d11/cc4 0 2024-02-26T10:48:02.197 INFO:tasks.workunit.client.0.smithi060.stdout:2/574: creat dd/d11/fc5 x:0 0 0 2024-02-26T10:48:02.211 INFO:tasks.workunit.client.0.smithi060.stdout:4/644: getdents d0/da/d3b 0 2024-02-26T10:48:02.211 INFO:tasks.workunit.client.0.smithi060.stdout:0/568: dread d0/f78 [0,4194304] 0 2024-02-26T10:48:02.212 INFO:tasks.workunit.client.0.smithi060.stdout:4/645: getdents d0/da/d57 0 2024-02-26T10:48:02.214 INFO:tasks.workunit.client.0.smithi060.stdout:0/569: creat d0/dd/d39/d63/fbf x:0 0 0 2024-02-26T10:48:02.214 INFO:tasks.workunit.client.0.smithi060.stdout:0/570: fdatasync d0/f55 0 2024-02-26T10:48:02.224 INFO:tasks.workunit.client.0.smithi060.stdout:4/646: creat d0/da/dba/fcd x:0 0 0 2024-02-26T10:48:02.224 INFO:tasks.workunit.client.0.smithi060.stdout:4/647: write d0/da/d57/f69 [1398397,60629] 0 2024-02-26T10:48:02.226 INFO:tasks.workunit.client.0.smithi060.stdout:5/599: dwrite d0/de/fbf [0,4194304] 0 2024-02-26T10:48:02.230 INFO:tasks.workunit.client.0.smithi060.stdout:0/571: dread d0/dd/f5d [0,4194304] 0 2024-02-26T10:48:02.230 INFO:tasks.workunit.client.0.smithi060.stdout:0/572: write d0/dd/d39/d40/f34 [8661598,6777] 0 2024-02-26T10:48:02.230 INFO:tasks.workunit.client.0.smithi060.stdout:2/575: dread dd/d43/d60/d41/d37/d4c/d76/fa3 [0,4194304] 0 2024-02-26T10:48:02.231 INFO:tasks.workunit.client.0.smithi060.stdout:0/573: mkdir d0/dd/d39/d40/d83/dc0 0 2024-02-26T10:48:02.232 INFO:tasks.workunit.client.0.smithi060.stdout:0/574: getdents d0/dd/d39/d40/d83/dc0 0 2024-02-26T10:48:02.234 INFO:tasks.workunit.client.0.smithi060.stdout:0/575: mknod d0/dd/d39/d40/cc1 0 2024-02-26T10:48:02.236 INFO:tasks.workunit.client.0.smithi060.stdout:7/731: dwrite da/d15/d21/d2f/f48 [0,4194304] 0 2024-02-26T10:48:02.238 INFO:tasks.workunit.client.0.smithi060.stdout:4/648: creat d0/da/d3b/fce x:0 0 0 2024-02-26T10:48:02.238 INFO:tasks.workunit.client.0.smithi060.stdout:4/649: getdents d0/d27/d2f 0 2024-02-26T10:48:02.239 INFO:tasks.workunit.client.0.smithi060.stdout:2/576: rename dd/d52 to dd/d43/d55/d5e/dc6 0 2024-02-26T10:48:02.240 INFO:tasks.workunit.client.0.smithi060.stdout:2/577: symlink dd/d43/lc7 0 2024-02-26T10:48:02.242 INFO:tasks.workunit.client.0.smithi060.stdout:2/578: write dd/d43/d60/d41/d37/d38/f89 [574062,96204] 0 2024-02-26T10:48:02.242 INFO:tasks.workunit.client.0.smithi060.stdout:2/579: chown dd/d43/d60/d41/d37/d4c/fc2 131316294 1 2024-02-26T10:48:02.244 INFO:tasks.workunit.client.0.smithi060.stdout:0/576: mknod d0/dd/d53/d87/d94/cc2 0 2024-02-26T10:48:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.259+0000 7eff6ea00640 1 -- 172.21.15.60:0/1693295987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 msgr2=0x7eff68107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.259+0000 7eff6ea00640 1 --2- 172.21.15.60:0/1693295987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 0x7eff68107b80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009900 tx=0x7eff5c02bc60 comp rx=0 tx=0).stop 2024-02-26T10:48:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.260+0000 7eff6ea00640 1 -- 172.21.15.60:0/1693295987 shutdown_connections 2024-02-26T10:48:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.260+0000 7eff6ea00640 1 --2- 172.21.15.60:0/1693295987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff68109310 0x7eff6810fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.260+0000 7eff6ea00640 1 --2- 172.21.15.60:0/1693295987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff68108950 0x7eff68108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.260+0000 7eff6ea00640 1 --2- 172.21.15.60:0/1693295987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 0x7eff68107b80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.260+0000 7eff6ea00640 1 -- 172.21.15.60:0/1693295987 >> 172.21.15.60:0/1693295987 conn(0x7eff68076700 msgr2=0x7eff68078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.260+0000 7eff6ea00640 1 -- 172.21.15.60:0/1693295987 shutdown_connections 2024-02-26T10:48:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.260+0000 7eff6ea00640 1 -- 172.21.15.60:0/1693295987 wait complete. 2024-02-26T10:48:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.264+0000 7eff6ea00640 1 Processor -- start 2024-02-26T10:48:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.264+0000 7eff6ea00640 1 -- start start 2024-02-26T10:48:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 0x7eff68072770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff68108950 0x7eff68072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff68109310 0x7eff68070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6ea00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6806b6d0 con 0x7eff68108950 2024-02-26T10:48:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6806b550 con 0x7eff68107760 2024-02-26T10:48:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6806b850 con 0x7eff68109310 2024-02-26T10:48:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 0x7eff68072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 0x7eff68072770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51088/0 (socket says 172.21.15.60:51088) 2024-02-26T10:48:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6d600640 1 -- 172.21.15.60:0/537337539 learned_addr learned my addr 172.21.15.60:0/537337539 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.265+0000 7eff6d600640 1 -- 172.21.15.60:0/537337539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff68109310 msgr2=0x7eff68070db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-26T10:48:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.266+0000 7eff6d600640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff68109310 0x7eff68070db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.266+0000 7eff6d600640 1 -- 172.21.15.60:0/537337539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff68108950 msgr2=0x7eff68072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.266+0000 7eff6d600640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff68108950 0x7eff68072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.266+0000 7eff6d600640 1 -- 172.21.15.60:0/537337539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff68071530 con 0x7eff68107760 2024-02-26T10:48:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.266+0000 7eff6d600640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 0x7eff68072770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7eff5c02efd0 tx=0x7eff5c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.266+0000 7eff66000640 1 -- 172.21.15.60:0/537337539 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c041070 con 0x7eff68107760 2024-02-26T10:48:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.266+0000 7eff66000640 1 -- 172.21.15.60:0/537337539 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff5c0044c0 con 0x7eff68107760 2024-02-26T10:48:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.266+0000 7eff66000640 1 -- 172.21.15.60:0/537337539 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0346a0 con 0x7eff68107760 2024-02-26T10:48:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.267+0000 7eff6ea00640 1 -- 172.21.15.60:0/537337539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff680717c0 con 0x7eff68107760 2024-02-26T10:48:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.267+0000 7eff6ea00640 1 -- 172.21.15.60:0/537337539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff681ae7a0 con 0x7eff68107760 2024-02-26T10:48:02.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.270+0000 7eff66000640 1 -- 172.21.15.60:0/537337539 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7eff5c044070 con 0x7eff68107760 2024-02-26T10:48:02.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.275+0000 7eff66000640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff500a1e00 0x7eff500a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.268 INFO:tasks.workunit.client.0.smithi060.stdout:7/732: rename da/d15/d21/d41/d58/f70 to da/df1/ff8 0 2024-02-26T10:48:02.270 INFO:tasks.workunit.client.0.smithi060.stdout:0/577: getdents d0/dd/d53 0 2024-02-26T10:48:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.275+0000 7eff66000640 1 -- 172.21.15.60:0/537337539 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7eff5c0efd90 con 0x7eff68107760 2024-02-26T10:48:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.275+0000 7eff6cc00640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff500a1e00 0x7eff500a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.273 INFO:tasks.workunit.client.0.smithi060.stdout:0/578: symlink d0/dd/d39/d40/d83/lc3 0 2024-02-26T10:48:02.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.277+0000 7eff6cc00640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff500a1e00 0x7eff500a42c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7eff680724e0 tx=0x7eff54006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.281+0000 7eff73028740 1 Processor -- start 2024-02-26T10:48:02.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.281+0000 7eff73028740 1 -- start start 2024-02-26T10:48:02.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.282+0000 7eff73028740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.282+0000 7eff73028740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b9830 0x55fe2d1be330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.282+0000 7eff6d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.283+0000 7eff6d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51104/0 (socket says 172.21.15.60:51104) 2024-02-26T10:48:02.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.283+0000 7eff6cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b9830 0x55fe2d1be330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.283+0000 7eff6d600640 1 -- 172.21.15.60:0/1655753734 learned_addr learned my addr 172.21.15.60:0/1655753734 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.283+0000 7eff73028740 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.284+0000 7eff73028740 1 -- 172.21.15.60:0/1655753734 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe2d11dd30 con 0x55fe2d1b9830 2024-02-26T10:48:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.284+0000 7eff73028740 1 -- 172.21.15.60:0/1655753734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe2d0c6cc0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.284+0000 7eff73028740 1 -- 172.21.15.60:0/1655753734 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe2d119dd0 con 0x55fe2d1be9a0 2024-02-26T10:48:02.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.284+0000 7eff6e000640 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.285+0000 7eff6d600640 1 -- 172.21.15.60:0/1655753734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 msgr2=0x55fe2d1c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.285+0000 7eff6d600640 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.285+0000 7eff6d600640 1 -- 172.21.15.60:0/1655753734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b9830 msgr2=0x55fe2d1be330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.285+0000 7eff6d600640 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b9830 0x55fe2d1be330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.285+0000 7eff6d600640 1 -- 172.21.15.60:0/1655753734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe2ce323f0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.285+0000 7eff6e000640 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:48:02.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.285+0000 7eff6cc00640 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b9830 0x55fe2d1be330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:48:02.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.286+0000 7eff6d600640 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b9260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0c4920 tx=0x7eff5c02bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb2ab11472f77a9a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.286+0000 7eff5a600640 1 -- 172.21.15.60:0/1655753734 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c045e90 con 0x55fe2d1b8e40 2024-02-26T10:48:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.286+0000 7eff5a600640 1 -- 172.21.15.60:0/1655753734 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff5c0bcdb0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.286+0000 7eff5a600640 1 -- 172.21.15.60:0/1655753734 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0baa40 con 0x55fe2d1b8e40 2024-02-26T10:48:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.287+0000 7eff73028740 1 -- 172.21.15.60:0/1655753734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 msgr2=0x55fe2d1b9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.287+0000 7eff73028740 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b9260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0c4920 tx=0x7eff5c02bc60 comp rx=0 tx=0).stop 2024-02-26T10:48:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.288+0000 7eff73028740 1 -- 172.21.15.60:0/1655753734 shutdown_connections 2024-02-26T10:48:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.288+0000 7eff73028740 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.288+0000 7eff73028740 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b9830 0x55fe2d1be330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.288+0000 7eff73028740 1 --2- 172.21.15.60:0/1655753734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b9260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.288+0000 7eff73028740 1 -- 172.21.15.60:0/1655753734 >> 172.21.15.60:0/1655753734 conn(0x55fe2d1b43b0 msgr2=0x55fe2d1b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.288+0000 7eff73028740 1 -- 172.21.15.60:0/1655753734 shutdown_connections 2024-02-26T10:48:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.288+0000 7eff73028740 1 -- 172.21.15.60:0/1655753734 wait complete. 2024-02-26T10:48:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.289+0000 7eff73028740 1 -- start start 2024-02-26T10:48:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.289+0000 7eff73028740 1 Processor -- start 2024-02-26T10:48:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.290+0000 7eff73028740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.290+0000 7eff73028740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.290+0000 7eff73028740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.291+0000 7eff6d600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.291+0000 7eff6d600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46246/0 (socket says 172.21.15.60:46246) 2024-02-26T10:48:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.291+0000 7eff6cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.291+0000 7eff6cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51106/0 (socket says 172.21.15.60:51106) 2024-02-26T10:48:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.291+0000 7eff73028740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe2d11dd30 con 0x55fe2d1b8e40 2024-02-26T10:48:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.291+0000 7eff73028740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe2d0bc1f0 con 0x55fe2d1b9830 2024-02-26T10:48:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.291+0000 7eff73028740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe2d0c6cc0 con 0x55fe2d1be9a0 2024-02-26T10:48:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.292+0000 7eff6d600640 1 -- 172.21.15.60:0/2455623285 learned_addr learned my addr 172.21.15.60:0/2455623285 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.292+0000 7eff6e000640 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.292+0000 7eff6d600640 1 -- 172.21.15.60:0/2455623285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 msgr2=0x55fe2d1ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.292+0000 7eff6d600640 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.292+0000 7eff6d600640 1 -- 172.21.15.60:0/2455623285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b9830 msgr2=0x55fe2d1c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.292+0000 7eff6d600640 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.292+0000 7eff6cc00640 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-02-26T10:48:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.292+0000 7eff6d600640 1 -- 172.21.15.60:0/2455623285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe2cefc7d0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.292+0000 7eff73028740 10 client.0 _ll_register_callbacks cb 0x55fe2ce31be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.293+0000 7eff73028740 1 -- 172.21.15.60:0/2455623285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff5c0bcdb0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.293+0000 7eff6d600640 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c2e60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0bc640 tx=0x7eff5c0c4ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b794c69034f5e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.293+0000 7eff4ea00640 10 client.15939 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:48:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.294+0000 7eff4ea00640 1 -- 172.21.15.60:0/2455623285 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0d51b0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.294+0000 7eff4ea00640 1 -- 172.21.15.60:0/2455623285 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7eff5c0038d0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.295+0000 7eff4ea00640 1 -- 172.21.15.60:0/2455623285 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff5c0bc6b0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.295+0000 7eff73028740 10 client.15939 fetch_fsmap learned FSMap version 29 2024-02-26T10:48:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.295+0000 7eff73028740 1 -- 172.21.15.60:0/2455623285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55fe2d1cd210 con 0x55fe2d1b8e40 2024-02-26T10:48:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.296+0000 7eff4ea00640 1 -- 172.21.15.60:0/2455623285 <== mon.0 v2:172.21.15.60:3300/0 4 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7eff5c0dc080 con 0x55fe2d1b8e40 2024-02-26T10:48:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.296+0000 7eff73028740 10 client.15939 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:48:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.296+0000 7eff73028740 10 client.15939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-26T10:48:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.296+0000 7eff73028740 20 client.15939 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.296+0000 7eff73028740 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55fe2d1cea10 0x55fe2d1d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.296+0000 7eff73028740 4 client.15939 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:48:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.296+0000 7eff73028740 1 -- 172.21.15.60:0/2455623285 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe2d12b270 con 0x55fe2d1cea10 2024-02-26T10:48:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.296+0000 7eff6cc00640 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55fe2d1cea10 0x55fe2d1d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.297+0000 7eff4ea00640 10 client.15939 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:48:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.297+0000 7eff6cc00640 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55fe2d1cea10 0x55fe2d1d0ed0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b4f8a5118c81aea server_cookie=bf98918a7496c341 in_seq=0 out_seq=0 2024-02-26T10:48:02.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.298+0000 7eff4ea00640 1 -- 172.21.15.60:0/2455623285 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe2d12b270 con 0x55fe2d1cea10 2024-02-26T10:48:02.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.298+0000 7eff4ea00640 10 client.15939 handle_command_reply: tid=0 2024-02-26T10:48:02.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.298+0000 7eff73028740 1 client.15939 shutdown 2024-02-26T10:48:02.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.299+0000 7eff73028740 1 -- 172.21.15.60:0/2455623285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 msgr2=0x55fe2d1c2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.299+0000 7eff73028740 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c2e60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0bc640 tx=0x7eff5c0c4ef0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.299+0000 7eff73028740 1 -- 172.21.15.60:0/2455623285 shutdown_connections 2024-02-26T10:48:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.299+0000 7eff73028740 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55fe2d1cea10 0x55fe2d1d0ed0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.299+0000 7eff73028740 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.299+0000 7eff73028740 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.299+0000 7eff4ea00640 0 client.15939 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:48:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.299+0000 7eff73028740 1 --2- 172.21.15.60:0/2455623285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c2e60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.299+0000 7eff73028740 1 -- 172.21.15.60:0/2455623285 >> 172.21.15.60:0/2455623285 conn(0x55fe2d1b43b0 msgr2=0x55fe2d1b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.295 INFO:tasks.workunit.client.0.smithi060.stdout:4/650: dwrite d0/d27/d76/f91 [0,4194304] 0 2024-02-26T10:48:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.302+0000 7eff73028740 1 -- 172.21.15.60:0/2455623285 shutdown_connections 2024-02-26T10:48:02.299 INFO:tasks.workunit.client.0.smithi060.stdout:2/580: dwrite dd/d43/d60/d41/f95 [0,4194304] 0 2024-02-26T10:48:02.299 INFO:tasks.workunit.client.0.smithi060.stdout:5/600: dwrite d0/de/f46 [0,4194304] 0 2024-02-26T10:48:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.303+0000 7eff73028740 1 -- 172.21.15.60:0/2455623285 wait complete. 2024-02-26T10:48:02.299 INFO:tasks.workunit.client.0.smithi060.stdout:4/651: symlink d0/d27/d76/dbd/lcf 0 2024-02-26T10:48:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.305+0000 7eff73028740 20 client.15939 trim_cache size 0 max 16384 2024-02-26T10:48:02.300 INFO:tasks.workunit.client.0.smithi060.stdout:2/581: rename dd/l14 to dd/d43/d55/d5e/d86/d97/d9c/lc8 0 2024-02-26T10:48:02.300 INFO:tasks.workunit.client.0.smithi060.stdout:1/684: dwrite d3/d35/d53/d68/d6b/f9e [0,4194304] 0 2024-02-26T10:48:02.302 INFO:tasks.workunit.client.0.smithi060.stdout:4/652: creat d0/da/dc7/db0/db5/fd0 x:0 0 0 2024-02-26T10:48:02.302 INFO:tasks.workunit.client.0.smithi060.stdout:4/653: fdatasync d0/da/f8b 0 2024-02-26T10:48:02.311 INFO:tasks.workunit.client.0.smithi060.stdout:3/474: dwrite d5/de/df/d4c/f4f [4194304,4194304] 0 2024-02-26T10:48:02.311 INFO:tasks.workunit.client.0.smithi060.stdout:7/733: dwrite da/d15/d21/d29/fe4 [0,4194304] 0 2024-02-26T10:48:02.321 INFO:tasks.workunit.client.0.smithi060.stdout:4/654: rename d0/da/l9b to d0/da/d3b/ld1 0 2024-02-26T10:48:02.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff73028740 1 Processor -- start 2024-02-26T10:48:02.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff73028740 1 -- start start 2024-02-26T10:48:02.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff73028740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff73028740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 0x55fe2d1ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff73028740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff73028740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681baf30 con 0x55fe2d1b8e40 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff73028740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681b8a70 con 0x55fe2d1be9a0 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff73028740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681b6b10 con 0x55fe2d1b9830 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff6e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff6e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51110/0 (socket says 172.21.15.60:51110) 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.329+0000 7eff6e000640 1 -- 172.21.15.60:0/4119773380 learned_addr learned my addr 172.21.15.60:0/4119773380 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.330+0000 7eff6cc00640 1 --2- 172.21.15.60:0/4119773380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 0x55fe2d1ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.330+0000 7eff6e000640 1 -- 172.21.15.60:0/4119773380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 msgr2=0x55fe2d1ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.330+0000 7eff6e000640 1 --2- 172.21.15.60:0/4119773380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 0x55fe2d1ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.330+0000 7eff6e000640 1 -- 172.21.15.60:0/4119773380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 msgr2=0x55fe2d1b7d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:48:02.324 INFO:tasks.workunit.client.0.smithi060.stdout:3/475: rename d5/fa to d5/de/df/d4c/d68/f8e 0 2024-02-26T10:48:02.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.330+0000 7eff6d600640 1 --2- 172.21.15.60:0/4119773380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.330+0000 7eff6e000640 1 --2- 172.21.15.60:0/4119773380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.330+0000 7eff6e000640 1 -- 172.21.15.60:0/4119773380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe2d1cb290 con 0x55fe2d1be9a0 2024-02-26T10:48:02.325 INFO:tasks.workunit.client.0.smithi060.stdout:3/476: creat d5/d45/f8f x:0 0 0 2024-02-26T10:48:02.325 INFO:tasks.workunit.client.0.smithi060.stdout:3/477: stat d5/de/d16/f6d 0 2024-02-26T10:48:02.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.331+0000 7eff6e000640 1 --2- 172.21.15.60:0/4119773380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c2480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7eff60008d10 tx=0x7eff6000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2921a175995879f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.326 INFO:tasks.workunit.client.0.smithi060.stdout:2/582: write dd/d43/d60/fbd [688391,112629] 0 2024-02-26T10:48:02.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.331+0000 7eff59c00640 1 -- 172.21.15.60:0/4119773380 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60004b20 con 0x55fe2d1be9a0 2024-02-26T10:48:02.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.331+0000 7eff59c00640 1 -- 172.21.15.60:0/4119773380 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff60004cc0 con 0x55fe2d1be9a0 2024-02-26T10:48:02.327 INFO:tasks.workunit.client.0.smithi060.stdout:5/601: write d0/f5 [3433720,56324] 0 2024-02-26T10:48:02.327 INFO:tasks.workunit.client.0.smithi060.stdout:5/602: fdatasync d0/de/d15/d26/f45 0 2024-02-26T10:48:02.327 INFO:tasks.workunit.client.0.smithi060.stdout:5/603: readlink d0/de/d70/d84/lc3 0 2024-02-26T10:48:02.327 INFO:tasks.workunit.client.0.smithi060.stdout:3/478: mknod d5/d45/d61/d80/c90 0 2024-02-26T10:48:02.327 INFO:tasks.workunit.client.0.smithi060.stdout:3/479: chown d5/de/df/d11/c8c 1224355 1 2024-02-26T10:48:02.327 INFO:tasks.workunit.client.0.smithi060.stdout:3/480: getdents d5/d76 0 2024-02-26T10:48:02.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.332+0000 7eff59c00640 1 -- 172.21.15.60:0/4119773380 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60019880 con 0x55fe2d1be9a0 2024-02-26T10:48:02.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.332+0000 7eff73028740 1 -- 172.21.15.60:0/4119773380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1be9a0 msgr2=0x55fe2d1c2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.332+0000 7eff73028740 1 --2- 172.21.15.60:0/4119773380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c2480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7eff60008d10 tx=0x7eff6000f790 comp rx=0 tx=0).stop 2024-02-26T10:48:02.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.332+0000 7eff73028740 1 -- 172.21.15.60:0/4119773380 shutdown_connections 2024-02-26T10:48:02.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.332+0000 7eff73028740 1 --2- 172.21.15.60:0/4119773380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1be9a0 0x55fe2d1c2480 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.332+0000 7eff73028740 1 --2- 172.21.15.60:0/4119773380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 0x55fe2d1ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.332+0000 7eff73028740 1 --2- 172.21.15.60:0/4119773380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.332+0000 7eff73028740 1 -- 172.21.15.60:0/4119773380 >> 172.21.15.60:0/4119773380 conn(0x55fe2d1b43b0 msgr2=0x55fe2d1b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.333+0000 7eff73028740 1 -- 172.21.15.60:0/4119773380 shutdown_connections 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.333+0000 7eff73028740 1 -- 172.21.15.60:0/4119773380 wait complete. 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.333+0000 7eff73028740 1 -- start start 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.334+0000 7eff73028740 1 Processor -- start 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.334+0000 7eff73028740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.334+0000 7eff73028740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.334+0000 7eff73028740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1be9a0 0x55fe2d224fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.334+0000 7eff73028740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681b7110 con 0x55fe2d1be9a0 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.334+0000 7eff73028740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681b8410 con 0x55fe2d1b8e40 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.334+0000 7eff73028740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681b7750 con 0x55fe2d1b9830 2024-02-26T10:48:02.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.334+0000 7eff73028740 10 client.0 _ll_register_callbacks cb 0x7eff5c0b8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:02.330 INFO:tasks.workunit.client.0.smithi060.stdout:3/481: creat d5/de/df/d4c/f91 x:0 0 0 2024-02-26T10:48:02.330 INFO:tasks.workunit.client.0.smithi060.stdout:3/482: chown d5/d45/f48 234973329 1 2024-02-26T10:48:02.330 INFO:tasks.workunit.client.0.smithi060.stdout:3/483: readlink d5/de/df/d4c/l71 0 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.335+0000 7eff6d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.335+0000 7eff6d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51122/0 (socket says 172.21.15.60:51122) 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.335+0000 7eff6e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1be9a0 0x55fe2d224fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.336+0000 7eff6e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1be9a0 0x55fe2d224fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46262/0 (socket says 172.21.15.60:46262) 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.336+0000 7eff6e000640 1 -- 172.21.15.60:0/2358274621 learned_addr learned my addr 172.21.15.60:0/2358274621 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.336+0000 7eff6cc00640 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.336+0000 7eff6d600640 1 -- 172.21.15.60:0/2358274621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 msgr2=0x55fe2d1c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.336+0000 7eff6d600640 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.336+0000 7eff6d600640 1 -- 172.21.15.60:0/2358274621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1be9a0 msgr2=0x55fe2d224fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.337+0000 7eff6d600640 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1be9a0 0x55fe2d224fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.332 INFO:tasks.workunit.client.0.smithi060.stdout:3/484: mkdir d5/d45/d61/d92 0 2024-02-26T10:48:02.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.337+0000 7eff6d600640 1 -- 172.21.15.60:0/2358274621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe2d225520 con 0x55fe2d1b8e40 2024-02-26T10:48:02.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.337+0000 7eff6e000640 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1be9a0 0x55fe2d224fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:48:02.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.338+0000 7eff6d600640 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c9300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55fe2d1d1140 tx=0x7eff5c003c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c43800e1642a55 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.338+0000 7eff4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:02.333 INFO:tasks.workunit.client.0.smithi060.stdout:3/485: mkdir d5/de/df/d11/d63/d93 0 2024-02-26T10:48:02.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.339+0000 7eff4ea00640 1 -- 172.21.15.60:0/2358274621 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0bc010 con 0x55fe2d1b8e40 2024-02-26T10:48:02.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.339+0000 7eff4ea00640 1 -- 172.21.15.60:0/2358274621 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7eff5c0bd9e0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.339+0000 7eff73028740 1 -- 172.21.15.60:0/2358274621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff60004cc0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.340+0000 7eff4ea00640 1 -- 172.21.15.60:0/2358274621 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff5c0d51b0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.340+0000 7eff73028740 10 client.26075 fetch_fsmap learned FSMap version 29 2024-02-26T10:48:02.334 INFO:tasks.workunit.client.0.smithi060.stdout:3/486: rename d5/de/df/f5b to d5/de/df/d57/f94 0 2024-02-26T10:48:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.340+0000 7eff73028740 1 -- 172.21.15.60:0/2358274621 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fe2d225520 con 0x55fe2d1b8e40 2024-02-26T10:48:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.341+0000 7eff4ea00640 1 -- 172.21.15.60:0/2358274621 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff60004cc0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.341+0000 7eff4ea00640 1 -- 172.21.15.60:0/2358274621 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7eff5c0c49c0 con 0x55fe2d1b8e40 2024-02-26T10:48:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.341+0000 7eff73028740 10 client.26075 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:48:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.341+0000 7eff73028740 10 client.26075 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-26T10:48:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.341+0000 7eff73028740 20 client.26075 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.341+0000 7eff73028740 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55fe2d226dc0 0x55fe2d2291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.341+0000 7eff73028740 4 client.26075 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:48:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.341+0000 7eff73028740 1 -- 172.21.15.60:0/2358274621 --> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe2d1255c0 con 0x55fe2d226dc0 2024-02-26T10:48:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.341+0000 7eff6cc00640 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55fe2d226dc0 0x55fe2d2291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.342+0000 7eff6cc00640 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55fe2d226dc0 0x55fe2d2291e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7ae33edbcf3ebe server_cookie=b7c1fbcae1ee735c in_seq=0 out_seq=0 2024-02-26T10:48:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.342+0000 7eff4ea00640 10 client.26075 ms_handle_connect on v2:172.21.15.100:6836/1347855255 2024-02-26T10:48:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.342+0000 7eff4ea00640 1 -- 172.21.15.60:0/2358274621 <== mds.2 v2:172.21.15.100:6836/1347855255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17740 (crc 0 0 0) 0x55fe2d1255c0 con 0x55fe2d226dc0 2024-02-26T10:48:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.342+0000 7eff4ea00640 10 client.26075 handle_command_reply: tid=0 2024-02-26T10:48:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.342+0000 7eff73028740 1 client.26075 shutdown 2024-02-26T10:48:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.343+0000 7eff73028740 1 -- 172.21.15.60:0/2358274621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 msgr2=0x55fe2d1c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.343+0000 7eff73028740 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c9300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55fe2d1d1140 tx=0x7eff5c003c70 comp rx=0 tx=0).stop 2024-02-26T10:48:02.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff73028740 1 -- 172.21.15.60:0/2358274621 shutdown_connections 2024-02-26T10:48:02.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff73028740 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.100:6836/1347855255,v1:172.21.15.100:6837/1347855255] conn(0x55fe2d226dc0 0x55fe2d2291e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff73028740 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fe2d1be9a0 0x55fe2d224fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff73028740 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fe2d1b9830 0x55fe2d1c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff73028740 1 --2- 172.21.15.60:0/2358274621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe2d1b8e40 0x55fe2d1c9300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff73028740 1 -- 172.21.15.60:0/2358274621 >> 172.21.15.60:0/2358274621 conn(0x55fe2d1b43b0 msgr2=0x55fe2d1b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff4ea00640 0 client.26075 ms_handle_reset on v2:172.21.15.100:6836/1347855255 2024-02-26T10:48:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff73028740 1 -- 172.21.15.60:0/2358274621 shutdown_connections 2024-02-26T10:48:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff73028740 1 -- 172.21.15.60:0/2358274621 wait complete. 2024-02-26T10:48:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.344+0000 7eff73028740 20 client.26075 trim_cache size 0 max 16384 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 110263384, 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632246, 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.h" 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 131814612, 2024-02-26T10:48:02.339 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632232, 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.c" 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 134314085, 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632262, 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.h" 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 238714452, 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632243, 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.h" 2024-02-26T10:48:02.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 258406227, 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631818, 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory_event.c" 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 260251041, 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632237, 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.h" 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 312299559, 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632251, 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.c" 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 323069430, 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632253, 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.h" 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 396168315, 2024-02-26T10:48:02.342 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631835, 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.am" 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 564954656, 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632248, 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.c" 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 574974244, 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632252, 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.in" 2024-02-26T10:48:02.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 590058427, 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631832, 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/ipmi_thres_test.c" 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 641580283, 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631819, 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/posix_vlog.c" 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.344 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 666764040, 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632235, 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.h" 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 774133568, 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632238, 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.c" 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 774636814, 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631831, 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t" 2024-02-26T10:48:02.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 786971472, 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632258, 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.c" 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 853175377, 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631828, 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_rdrs.c" 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 877530795, 2024-02-26T10:48:02.346 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631807, 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_close.c" 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 910782487, 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631833, 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.in" 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 912661216, 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631816, 2024-02-26T10:48:02.347 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/sync.c" 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 936784625, 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631808, 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/hotswap.c" 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 974138311, 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632240, 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.c" 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.348 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1004001644, 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631829, 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_util.c" 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1005562385, 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631812, 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_controls.c" 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1017100718, 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631810, 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_connection.c" 2024-02-26T10:48:02.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1206973673, 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631830, 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_slot_rdrs.c" 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1378444870, 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631806, 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor_event.c" 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.350 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1568423140, 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631827, 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_fru.c" 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1587503938, 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632261, 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.am" 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1694173494, 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632254, 2024-02-26T10:48:02.351 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.c" 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1803236338, 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631822, 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity.c" 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1814211515, 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632257, 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.c" 2024-02-26T10:48:02.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1816675952, 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631805, 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_vshm_rdrs.c" 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1911667334, 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632245, 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.h" 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1924325864, 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631826, 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.in" 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2042364000, 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631834, 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.am" 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2188378041, 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631804, 2024-02-26T10:48:02.354 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity_event.c" 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2311929523, 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632231, 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.c" 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2402170823, 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632234, 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.h" 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.355 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2442647225, 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632250, 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.c" 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2466564704, 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632233, 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.h" 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2505124305, 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632260, 2024-02-26T10:48:02.356 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.c" 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2838620444, 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632247, 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.h" 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2941559868, 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631809, 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_control_event.c" 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.357 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2942935778, 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632239, 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.h" 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2988220750, 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632241, 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.h" 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2992808727, 2024-02-26T10:48:02.358 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632259, 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.h" 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3146753425, 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631836, 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor.c" 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3211847093, 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631824, 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_mc_event.c" 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3477867673, 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631821, 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_event.c" 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3519655788, 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631811, 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ekeyfru.h" 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3587507200, 2024-02-26T10:48:02.360 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631815, 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.sym" 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3593897016, 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631814, 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory.c" 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3697749965, 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631820, 2024-02-26T10:48:02.361 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.c" 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3750612703, 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632255, 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.h" 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3875400324, 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632244, 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.c" 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3952858350, 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631825, 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_fru_rdrs.c" 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4104788736, 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632249, 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.c" 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4145953613, 2024-02-26T10:48:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631817, 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_drv.c" 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4170445418, 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632236, 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.c" 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4184984181, 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631813, 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.h" 2024-02-26T10:48:02.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.365 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.365 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4217264982, 2024-02-26T10:48:02.365 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632256, 2024-02-26T10:48:02.365 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.h" 2024-02-26T10:48:02.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.365 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4258376643, 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632242, 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.c" 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4286717593, 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631823, 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sel.c" 2024-02-26T10:48:02.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:48:02.367 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-02-26T10:48:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.346+0000 7eff6ea00640 1 -- 172.21.15.60:0/537337539 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff500a1e00 msgr2=0x7eff500a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.346+0000 7eff6ea00640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff500a1e00 0x7eff500a42c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7eff680724e0 tx=0x7eff54006c40 comp rx=0 tx=0).stop 2024-02-26T10:48:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.346+0000 7eff6ea00640 1 -- 172.21.15.60:0/537337539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 msgr2=0x7eff68072770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.346+0000 7eff6ea00640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 0x7eff68072770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7eff5c02efd0 tx=0x7eff5c0040c0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.347+0000 7eff6ea00640 1 -- 172.21.15.60:0/537337539 shutdown_connections 2024-02-26T10:48:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.347+0000 7eff6ea00640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7eff500a1e00 0x7eff500a42c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.347+0000 7eff6ea00640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff68109310 0x7eff68070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.347+0000 7eff6ea00640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff68108950 0x7eff68072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.347+0000 7eff6ea00640 1 --2- 172.21.15.60:0/537337539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff68107760 0x7eff68072770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.347+0000 7eff6ea00640 1 -- 172.21.15.60:0/537337539 >> 172.21.15.60:0/537337539 conn(0x7eff68076700 msgr2=0x7eff6810bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.347+0000 7eff6ea00640 1 -- 172.21.15.60:0/537337539 shutdown_connections 2024-02-26T10:48:02.369 INFO:tasks.workunit.client.0.smithi060.stdout:6/639: sync 2024-02-26T10:48:02.369 INFO:tasks.workunit.client.0.smithi060.stdout:8/603: sync 2024-02-26T10:48:02.369 INFO:tasks.workunit.client.0.smithi060.stdout:8/604: read d9/f13 [219192,46274] 0 2024-02-26T10:48:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.348+0000 7eff6ea00640 1 -- 172.21.15.60:0/537337539 wait complete. 2024-02-26T10:48:02.370 INFO:tasks.workunit.client.0.smithi060.stdout:5/604: dread d0/de/d15/d65/d86/d9b/f75 [0,4194304] 0 2024-02-26T10:48:02.370 INFO:tasks.workunit.client.0.smithi060.stdout:8/605: symlink d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/lbe 0 2024-02-26T10:48:02.370 INFO:tasks.workunit.client.0.smithi060.stdout:8/606: chown d9/f1e 667 1 2024-02-26T10:48:02.370 INFO:tasks.workunit.client.0.smithi060.stdout:5/605: mkdir d0/de/d20/d32/d4c/dd2 0 2024-02-26T10:48:02.371 INFO:tasks.workunit.client.0.smithi060.stdout:5/606: chown d0/de/d70/d84/lc6 490856 1 2024-02-26T10:48:02.371 INFO:tasks.workunit.client.0.smithi060.stdout:6/640: write d6/d7/d25/d1c/d1f/d23/d2b/d4d/f58 [174321,39955] 0 2024-02-26T10:48:02.371 INFO:tasks.workunit.client.0.smithi060.stdout:6/641: rmdir d6/d7/d2e/d56/d73/d8c 39 2024-02-26T10:48:02.371 INFO:tasks.workunit.client.0.smithi060.stdout:6/642: truncate d6/d7/f53 1058265 0 2024-02-26T10:48:02.371 INFO:tasks.workunit.client.0.smithi060.stdout:6/643: chown d6/d7/d47/lb4 4307 1 2024-02-26T10:48:02.371 INFO:tasks.workunit.client.0.smithi060.stdout:4/655: write d0/d27/d76/f93 [3170804,81216] 0 2024-02-26T10:48:02.371 INFO:tasks.workunit.client.0.smithi060.stdout:4/656: truncate d0/da/d3b/fce 823553 0 2024-02-26T10:48:02.371 INFO:tasks.workunit.client.0.smithi060.stdout:8/607: unlink d9/c5d 0 2024-02-26T10:48:02.371 INFO:tasks.workunit.client.0.smithi060.stdout:3/487: read d5/de/df/f81 [2285759,29911] 0 2024-02-26T10:48:02.372 INFO:tasks.workunit.client.0.smithi060.stdout:7/734: dwrite da/d15/d88/d3b/f9d [0,4194304] 0 2024-02-26T10:48:02.372 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:48:02.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T10:48:02.375 INFO:tasks.workunit.client.0.smithi060.stdout:5/607: rmdir d0 39 2024-02-26T10:48:02.375 INFO:tasks.workunit.client.0.smithi060.stdout:5/608: write d0/de/d15/d65/d86/d9b/f72 [689073,63557] 0 2024-02-26T10:48:02.381 INFO:tasks.workunit.client.0.smithi060.stdout:7/735: truncate da/d15/d88/dce/fbe 892648 0 2024-02-26T10:48:02.403 INFO:tasks.workunit.client.0.smithi060.stdout:6/644: dwrite d6/f11 [4194304,4194304] 0 2024-02-26T10:48:02.403 INFO:tasks.workunit.client.0.smithi060.stdout:6/645: write d6/d7/d25/f49 [1041001,51832] 0 2024-02-26T10:48:02.403 INFO:tasks.workunit.client.0.smithi060.stdout:6/646: readlink d6/d7/d25/d1c/d1f/lbf 0 2024-02-26T10:48:02.403 INFO:tasks.workunit.client.0.smithi060.stdout:6/647: dread - d6/d7/d25/d1c/d1f/fbb zero size 2024-02-26T10:48:02.403 INFO:tasks.workunit.client.0.smithi060.stdout:6/648: dread - d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f74 zero size 2024-02-26T10:48:02.408 INFO:tasks.workunit.client.0.smithi060.stdout:4/657: dwrite d0/da/d3b/f95 [0,4194304] 0 2024-02-26T10:48:02.414 INFO:tasks.workunit.client.0.smithi060.stdout:7/736: creat da/d15/d21/d2f/d37/ff9 x:0 0 0 2024-02-26T10:48:02.414 INFO:tasks.workunit.client.0.smithi060.stdout:3/488: write d5/de/f84 [3228175,46941] 0 2024-02-26T10:48:02.414 INFO:tasks.workunit.client.0.smithi060.stdout:7/737: write da/d66/fcd [2777198,8937] 0 2024-02-26T10:48:02.414 INFO:tasks.workunit.client.0.smithi060.stdout:7/738: fdatasync da/d66/dad/dd8/fe7 0 2024-02-26T10:48:02.415 INFO:tasks.workunit.client.0.smithi060.stdout:6/649: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/fd1 x:0 0 0 2024-02-26T10:48:02.417 INFO:tasks.workunit.client.0.smithi060.stdout:3/489: mknod d5/de/df/d11/d63/c95 0 2024-02-26T10:48:02.419 INFO:tasks.workunit.client.0.smithi060.stdout:6/650: link d6/d8d/dcc/da0/cca d6/d8d/dcc/da0/cd2 0 2024-02-26T10:48:02.419 INFO:tasks.workunit.client.0.smithi060.stdout:3/490: truncate d5/de/df/f81 3829360 0 2024-02-26T10:48:02.419 INFO:tasks.workunit.client.0.smithi060.stdout:7/739: mknod da/d66/dad/dd8/cfa 0 2024-02-26T10:48:02.421 INFO:tasks.workunit.client.0.smithi060.stdout:6/651: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/dc0/fd3 x:0 0 0 2024-02-26T10:48:02.422 INFO:tasks.workunit.client.0.smithi060.stdout:3/491: rmdir d5/d45/d61/d80 39 2024-02-26T10:48:02.422 INFO:tasks.workunit.client.0.smithi060.stdout:3/492: readlink d5/d76/l87 0 2024-02-26T10:48:02.423 INFO:tasks.workunit.client.0.smithi060.stdout:3/493: chown d5/de/df/f21 3426 1 2024-02-26T10:48:02.424 INFO:tasks.workunit.client.0.smithi060.stdout:7/740: write da/d15/d21/d2f/d78/dd3/f50 [530013,94514] 0 2024-02-26T10:48:02.424 INFO:tasks.workunit.client.0.smithi060.stdout:7/741: chown da/f60 821 1 2024-02-26T10:48:02.434 INFO:tasks.workunit.client.0.smithi060.stdout:8/608: dwrite d9/d15/d29/d2f/d34/d72/f4c [4194304,4194304] 0 2024-02-26T10:48:02.434 INFO:tasks.workunit.client.0.smithi060.stdout:8/609: truncate d9/d15/d29/d2f/d34/d72/d50/d6a/fb4 389166 0 2024-02-26T10:48:02.435 INFO:tasks.workunit.client.0.smithi060.stdout:6/652: rename d6/d6e/f96 to d6/d7/d25/d1c/d1f/d23/d28/fd4 0 2024-02-26T10:48:02.435 INFO:tasks.workunit.client.0.smithi060.stdout:6/653: write d6/d7/d25/d1c/d1f/d23/d2b/f3b [3269593,105339] 0 2024-02-26T10:48:02.436 INFO:tasks.workunit.client.0.smithi060.stdout:7/742: dread da/d15/f2e [0,4194304] 0 2024-02-26T10:48:02.436 INFO:tasks.workunit.client.0.smithi060.stdout:7/743: read - da/d15/d88/d3b/fd0 zero size 2024-02-26T10:48:02.436 INFO:tasks.workunit.client.0.smithi060.stdout:7/744: dread - da/d15/d21/d41/fd5 zero size 2024-02-26T10:48:02.440 INFO:tasks.workunit.client.0.smithi060.stdout:8/610: mknod d9/d15/d29/d2f/d34/d72/d50/d6a/cbf 0 2024-02-26T10:48:02.442 INFO:tasks.workunit.client.0.smithi060.stdout:6/654: link d6/d7/d25/d1c/f6d d6/d7/d25/d1c/d1f/d23/d28/fd5 0 2024-02-26T10:48:02.443 INFO:tasks.workunit.client.0.smithi060.stdout:6/655: readlink d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/l9f 0 2024-02-26T10:48:02.443 INFO:tasks.workunit.client.0.smithi060.stdout:6/656: write d6/d7/d25/d1c/d1f/d23/d28/fd5 [2169704,19948] 0 2024-02-26T10:48:02.447 INFO:tasks.workunit.client.0.smithi060.stdout:6/657: mknod d6/d7/d25/d1c/d1f/d23/cd6 0 2024-02-26T10:48:02.476 INFO:tasks.workunit.client.0.smithi060.stdout:9/666: sync 2024-02-26T10:48:02.480 INFO:tasks.workunit.client.0.smithi060.stdout:7/745: dread da/d15/d21/f2c [0,4194304] 0 2024-02-26T10:48:02.488 INFO:tasks.workunit.client.0.smithi060.stdout:4/658: dwrite d0/da/d57/fc6 [0,4194304] 0 2024-02-26T10:48:02.495 INFO:tasks.workunit.client.0.smithi060.stdout:3/494: dwrite d5/f24 [0,4194304] 0 2024-02-26T10:48:02.497 INFO:tasks.workunit.client.0.smithi060.stdout:8/611: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/f45 [0,4194304] 0 2024-02-26T10:48:02.509 INFO:tasks.workunit.client.0.smithi060.stdout:6/658: dwrite d6/d7/d2e/fb6 [0,4194304] 0 2024-02-26T10:48:02.512 INFO:tasks.workunit.client.0.smithi060.stdout:8/612: truncate d9/d15/d29/d2f/d34/d72/d94/f9e 3512873 0 2024-02-26T10:48:02.514 INFO:tasks.workunit.client.0.smithi060.stdout:8/613: mkdir d9/d15/d29/d2f/d34/d72/d50/d6a/dc0 0 2024-02-26T10:48:02.515 INFO:tasks.workunit.client.0.smithi060.stdout:8/614: mkdir d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/dc1 0 2024-02-26T10:48:02.517 INFO:tasks.workunit.client.0.smithi060.stdout:8/615: truncate d9/d5a/f69 1013564 0 2024-02-26T10:48:02.520 INFO:tasks.workunit.client.0.smithi060.stdout:8/616: link d9/d15/d29/d2f/d34/d72/d50/d6a/f71 d9/d15/d29/d2f/d34/d72/d50/d6a/d63/fc2 0 2024-02-26T10:48:02.521 INFO:tasks.workunit.client.0.smithi060.stdout:8/617: mknod d9/da4/cc3 0 2024-02-26T10:48:02.544 INFO:tasks.workunit.client.0.smithi060.stdout:7/746: dwrite da/d15/d88/d3b/f8c [0,4194304] 0 2024-02-26T10:48:02.554 INFO:tasks.workunit.client.0.smithi060.stdout:7/747: rename da/d15/d21/d2f/d78/dd3/dc8 to da/d15/d88/dce/dea/dfb 0 2024-02-26T10:48:02.558 INFO:tasks.workunit.client.0.smithi060.stdout:7/748: rename da/df1/dd7/ff3 to da/d15/ffc 0 2024-02-26T10:48:02.558 INFO:tasks.workunit.client.0.smithi060.stdout:7/749: chown da/d94/fb5 8117 1 2024-02-26T10:48:02.566 INFO:tasks.workunit.client.0.smithi060.stdout:3/495: dwrite d5/d37/f41 [0,4194304] 0 2024-02-26T10:48:02.569 INFO:tasks.workunit.client.0.smithi060.stdout:4/659: dwrite d0/da/d3b/d1b/f86 [0,4194304] 0 2024-02-26T10:48:02.571 INFO:tasks.workunit.client.0.smithi060.stdout:4/660: write d0/da/fcc [133462,68643] 0 2024-02-26T10:48:02.572 INFO:tasks.workunit.client.0.smithi060.stdout:6/659: dwrite d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/f94 [0,4194304] 0 2024-02-26T10:48:02.572 INFO:tasks.workunit.client.0.smithi060.stdout:6/660: write d6/d7/d2e/d56/d73/fa4 [5086806,65888] 0 2024-02-26T10:48:02.574 INFO:tasks.workunit.client.0.smithi060.stdout:8/618: dread d9/d15/d3e/f88 [0,4194304] 0 2024-02-26T10:48:02.575 INFO:tasks.workunit.client.0.smithi060.stdout:8/619: chown d9/d15/d29/d2f/d34/d72/d50/d6a/f74 87579943 1 2024-02-26T10:48:02.576 INFO:tasks.workunit.client.0.smithi060.stdout:8/620: mknod d9/d15/d29/d2f/d34/d72/d50/d6a/d63/cc4 0 2024-02-26T10:48:02.576 INFO:tasks.workunit.client.0.smithi060.stdout:7/750: dread da/d15/d21/d41/f97 [0,4194304] 0 2024-02-26T10:48:02.579 INFO:tasks.workunit.client.0.smithi060.stdout:7/751: symlink da/d15/d21/d2f/d78/dd3/d67/da8/lfd 0 2024-02-26T10:48:02.579 INFO:tasks.workunit.client.0.smithi060.stdout:7/752: write da/d15/d21/d2f/d37/feb [2517,105510] 0 2024-02-26T10:48:02.581 INFO:tasks.workunit.client.0.smithi060.stdout:7/753: mknod da/df1/dd7/cfe 0 2024-02-26T10:48:02.581 INFO:tasks.workunit.client.0.smithi060.stdout:0/579: sync 2024-02-26T10:48:02.581 INFO:tasks.workunit.client.0.smithi060.stdout:7/754: truncate da/d66/dad/dd8/fe7 81369 0 2024-02-26T10:48:02.581 INFO:tasks.workunit.client.0.smithi060.stdout:0/580: write d0/dd/d39/d40/d83/f8f [344937,83669] 0 2024-02-26T10:48:02.584 INFO:tasks.workunit.client.0.smithi060.stdout:0/581: unlink d0/dd/d33/d6a/d6c/f97 0 2024-02-26T10:48:02.584 INFO:tasks.workunit.client.0.smithi060.stdout:0/582: read - d0/dd/d33/d6a/d6c/f9a zero size 2024-02-26T10:48:02.585 INFO:tasks.workunit.client.0.smithi060.stdout:0/583: truncate d0/dd/faf 4913854 0 2024-02-26T10:48:02.587 INFO:tasks.workunit.client.0.smithi060.stdout:0/584: truncate d0/dd/d53/f64 618112 0 2024-02-26T10:48:02.587 INFO:tasks.workunit.client.0.smithi060.stdout:0/585: chown d0/dd/c18 2893 1 2024-02-26T10:48:02.589 INFO:tasks.workunit.client.0.smithi060.stdout:8/621: dread d9/d15/d3e/fb2 [0,4194304] 0 2024-02-26T10:48:02.590 INFO:tasks.workunit.client.0.smithi060.stdout:3/496: dread d5/f27 [0,4194304] 0 2024-02-26T10:48:02.591 INFO:tasks.workunit.client.0.smithi060.stdout:4/661: link d0/da/d3b/d16/c46 d0/d27/d52/cd2 0 2024-02-26T10:48:02.592 INFO:tasks.workunit.client.0.smithi060.stdout:4/662: stat d0/da/dc7/cc4 0 2024-02-26T10:48:02.593 INFO:tasks.workunit.client.0.smithi060.stdout:8/622: mkdir d9/d15/d29/d2f/d34/d72/d50/d6a/dc5 0 2024-02-26T10:48:02.593 INFO:tasks.workunit.client.0.smithi060.stdout:8/623: chown d9/d15/d29/d2f/caa 5 1 2024-02-26T10:48:02.596 INFO:tasks.workunit.client.0.smithi060.stdout:8/624: truncate d9/fc 1924387 0 2024-02-26T10:48:02.596 INFO:tasks.workunit.client.0.smithi060.stdout:8/625: truncate d9/d5a/d58/f8f 289922 0 2024-02-26T10:48:02.599 INFO:tasks.workunit.client.0.smithi060.stdout:0/586: dread d0/dd/d39/f7c [0,4194304] 0 2024-02-26T10:48:02.600 INFO:tasks.workunit.client.0.smithi060.stdout:8/626: symlink d9/d5a/d9c/lc6 0 2024-02-26T10:48:02.603 INFO:tasks.workunit.client.0.smithi060.stdout:0/587: truncate d0/dd/d53/da2/faa 753718 0 2024-02-26T10:48:02.604 INFO:tasks.workunit.client.0.smithi060.stdout:0/588: creat d0/dd/d53/d87/d94/db7/fc4 x:0 0 0 2024-02-26T10:48:02.609 INFO:tasks.workunit.client.0.smithi060.stdout:7/755: dread da/d94/f98 [0,4194304] 0 2024-02-26T10:48:02.628 INFO:tasks.workunit.client.0.smithi060.stdout:7/756: dread f2 [0,4194304] 0 2024-02-26T10:48:02.629 INFO:tasks.workunit.client.0.smithi060.stdout:7/757: getdents da/d66/dbd 0 2024-02-26T10:48:02.631 INFO:tasks.workunit.client.0.smithi060.stdout:7/758: unlink l8 0 2024-02-26T10:48:02.631 INFO:tasks.workunit.client.0.smithi060.stdout:7/759: chown da/d66/dad/dd8/cfa 52 1 2024-02-26T10:48:02.636 INFO:tasks.workunit.client.0.smithi060.stdout:5/609: sync 2024-02-26T10:48:02.636 INFO:tasks.workunit.client.0.smithi060.stdout:2/583: sync 2024-02-26T10:48:02.636 INFO:tasks.workunit.client.0.smithi060.stdout:1/685: sync 2024-02-26T10:48:02.645 INFO:tasks.workunit.client.0.smithi060.stdout:2/584: dread dd/d11/f62 [0,4194304] 0 2024-02-26T10:48:02.645 INFO:tasks.workunit.client.0.smithi060.stdout:2/585: truncate dd/d43/d60/f53 4906388 0 2024-02-26T10:48:02.645 INFO:tasks.workunit.client.0.smithi060.stdout:2/586: readlink dd/d43/d60/d41/l93 0 2024-02-26T10:48:02.646 INFO:tasks.workunit.client.0.smithi060.stdout:1/686: dread d3/f2b [0,4194304] 0 2024-02-26T10:48:02.648 INFO:tasks.workunit.client.0.smithi060.stdout:1/687: mkdir d3/d35/d53/d68/d6b/def 0 2024-02-26T10:48:02.651 INFO:tasks.workunit.client.0.smithi060.stdout:1/688: symlink d3/db/d1e/d38/lf0 0 2024-02-26T10:48:02.651 INFO:tasks.workunit.client.0.smithi060.stdout:1/689: write d3/db/d2e/f4d [2267576,109041] 0 2024-02-26T10:48:02.652 INFO:tasks.workunit.client.0.smithi060.stdout:0/589: dwrite d0/dd/d53/d87/d94/da6/f9d [0,4194304] 0 2024-02-26T10:48:02.654 INFO:tasks.workunit.client.0.smithi060.stdout:0/590: symlink d0/dd/d39/lc5 0 2024-02-26T10:48:02.659 INFO:tasks.workunit.client.0.smithi060.stdout:2/587: link c8 dd/d43/d55/d5e/dc6/cc9 0 2024-02-26T10:48:02.665 INFO:tasks.workunit.client.0.smithi060.stdout:2/588: symlink dd/d43/d55/d5e/d86/d97/d9c/lca 0 2024-02-26T10:48:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.680+0000 7f77df600640 1 -- 172.21.15.60:0/2814331588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 msgr2=0x7f77d810d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.680+0000 7f77df600640 1 --2- 172.21.15.60:0/2814331588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 0x7f77d810d2a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77c00074e0 tx=0x7f77c002c8b0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.681+0000 7f77df600640 1 -- 172.21.15.60:0/2814331588 shutdown_connections 2024-02-26T10:48:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.681+0000 7f77df600640 1 --2- 172.21.15.60:0/2814331588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 0x7f77d810d2a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.681+0000 7f77df600640 1 --2- 172.21.15.60:0/2814331588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d810d930 0x7f77d810fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.681+0000 7f77df600640 1 --2- 172.21.15.60:0/2814331588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d81084d0 0x7f77d810a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.681+0000 7f77df600640 1 -- 172.21.15.60:0/2814331588 >> 172.21.15.60:0/2814331588 conn(0x7f77d806e0c0 msgr2=0x7f77d8070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.681+0000 7f77df600640 1 -- 172.21.15.60:0/2814331588 shutdown_connections 2024-02-26T10:48:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.681+0000 7f77df600640 1 -- 172.21.15.60:0/2814331588 wait complete. 2024-02-26T10:48:02.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.682+0000 7f77df600640 1 Processor -- start 2024-02-26T10:48:02.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.683+0000 7f77df600640 1 -- start start 2024-02-26T10:48:02.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.683+0000 7f77df600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d81084d0 0x7f77d81191b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.683+0000 7f77df600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 0x7f77d81196f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.683+0000 7f77df600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d810d930 0x7f77d8117840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.684+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d81084d0 0x7f77d81191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.684+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d81084d0 0x7f77d81191b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:33056/0 (socket says 172.21.15.60:33056) 2024-02-26T10:48:02.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.684+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 0x7f77d81196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.684+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 0x7f77d81196f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51136/0 (socket says 172.21.15.60:51136) 2024-02-26T10:48:02.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.684+0000 7f77dcc00640 1 -- 172.21.15.60:0/530719751 learned_addr learned my addr 172.21.15.60:0/530719751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.684+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81121d0 con 0x7f77d810d930 2024-02-26T10:48:02.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.684+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8112050 con 0x7f77d810ae90 2024-02-26T10:48:02.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.684+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8112350 con 0x7f77d81084d0 2024-02-26T10:48:02.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.685+0000 7f77dd600640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d810d930 0x7f77d8117840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.685+0000 7f77cfe00640 1 -- 172.21.15.60:0/530719751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d81084d0 msgr2=0x7f77d81191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.685+0000 7f77cfe00640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d81084d0 0x7f77d81191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.685+0000 7f77cfe00640 1 -- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d810d930 msgr2=0x7f77d8117840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.685+0000 7f77cfe00640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d810d930 0x7f77d8117840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.685+0000 7f77cfe00640 1 -- 172.21.15.60:0/530719751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8117ee0 con 0x7f77d810ae90 2024-02-26T10:48:02.683 INFO:tasks.workunit.client.0.smithi060.stdout:5/610: dwrite d0/de/d15/d65/f6c [0,4194304] 0 2024-02-26T10:48:02.683 INFO:tasks.workunit.client.0.smithi060.stdout:5/611: chown d0/de/da4/cb2 1 1 2024-02-26T10:48:02.683 INFO:tasks.workunit.client.0.smithi060.stdout:7/760: dwrite da/df1/dd7/de0/fe6 [0,4194304] 0 2024-02-26T10:48:02.683 INFO:tasks.workunit.client.0.smithi060.stdout:7/761: fsync da/d15/d21/d2f/fbf 0 2024-02-26T10:48:02.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.687+0000 7f77cfe00640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 0x7f77d81196f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f77d8073460 tx=0x7f77c00045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.686 INFO:tasks.workunit.client.0.smithi060.stdout:5/612: mknod d0/de/d20/d32/cd3 0 2024-02-26T10:48:02.686 INFO:tasks.workunit.client.0.smithi060.stdout:8/627: dwrite d9/d15/d29/d2f/fa3 [0,4194304] 0 2024-02-26T10:48:02.686 INFO:tasks.workunit.client.0.smithi060.stdout:8/628: chown d9/d15/d29/d2f/d34/l8d 57188938 1 2024-02-26T10:48:02.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.694+0000 7f77d6000640 1 -- 172.21.15.60:0/530719751 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0040070 con 0x7f77d810ae90 2024-02-26T10:48:02.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.695+0000 7f77d6000640 1 -- 172.21.15.60:0/530719751 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c000a6d0 con 0x7f77d810ae90 2024-02-26T10:48:02.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.695+0000 7f77d6000640 1 -- 172.21.15.60:0/530719751 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0003b40 con 0x7f77d810ae90 2024-02-26T10:48:02.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.695+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d8118170 con 0x7f77d810ae90 2024-02-26T10:48:02.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.696+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d8118640 con 0x7f77d810ae90 2024-02-26T10:48:02.694 INFO:tasks.workunit.client.0.smithi060.stdout:8/629: dread f8 [0,4194304] 0 2024-02-26T10:48:02.696 INFO:tasks.workunit.client.0.smithi060.stdout:5/613: dread d0/d4b/f57 [0,4194304] 0 2024-02-26T10:48:02.696 INFO:tasks.workunit.client.0.smithi060.stdout:8/630: creat d9/d5a/d9c/fc7 x:0 0 0 2024-02-26T10:48:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.705+0000 7f77d6000640 1 -- 172.21.15.60:0/530719751 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f77c0039070 con 0x7f77d810ae90 2024-02-26T10:48:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.707+0000 7f77e23df740 1 Processor -- start 2024-02-26T10:48:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.708+0000 7f77e23df740 1 -- start start 2024-02-26T10:48:02.705 INFO:tasks.workunit.client.0.smithi060.stdout:2/589: write dd/d43/d55/d5e/dc6/f7a [1016618,75301] 0 2024-02-26T10:48:02.706 INFO:tasks.workunit.client.0.smithi060.stdout:0/591: dwrite d0/dd/d33/f4a [0,4194304] 0 2024-02-26T10:48:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.710+0000 7f77d6000640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f77bc0a1e80 0x7f77bc0a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.710+0000 7f77d6000640 1 -- 172.21.15.60:0/530719751 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f77c00ef3d0 con 0x7f77d810ae90 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.713+0000 7f77e23df740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7beec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.713+0000 7f77e23df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7bef1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.713+0000 7f77e23df740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7bef3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.713+0000 7f77e23df740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d7be50d30 con 0x555d7beebc70 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.713+0000 7f77e23df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d7bdf9cc0 con 0x555d7beec660 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.713+0000 7f77e23df740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d7be4cdd0 con 0x555d7bef17d0 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.713+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7beec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.714+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7beec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46296/0 (socket says 172.21.15.60:46296) 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.714+0000 7f77dcc00640 1 -- 172.21.15.60:0/2583799875 learned_addr learned my addr 172.21.15.60:0/2583799875 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.714+0000 7f77cfe00640 1 --2- 172.21.15.60:0/2583799875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7bef1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.714+0000 7f77dcc00640 1 -- 172.21.15.60:0/2583799875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 msgr2=0x555d7bef3c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.714+0000 7f77dcc00640 1 --2- 172.21.15.60:0/2583799875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7bef3c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.714+0000 7f77dcc00640 1 -- 172.21.15.60:0/2583799875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 msgr2=0x555d7bef1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.714+0000 7f77dcc00640 1 --2- 172.21.15.60:0/2583799875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7bef1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.714+0000 7f77dcc00640 1 -- 172.21.15.60:0/2583799875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d7bb653f0 con 0x555d7beebc70 2024-02-26T10:48:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.715+0000 7f77dd600640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f77bc0a1e80 0x7f77bc0a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.715+0000 7f77dcc00640 1 --2- 172.21.15.60:0/2583799875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7beec090 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f77c8011490 tx=0x7f77c8011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c00e52e2d8a8053 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.715+0000 7f77cdc00640 1 -- 172.21.15.60:0/2583799875 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8012770 con 0x555d7beebc70 2024-02-26T10:48:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.716+0000 7f77cdc00640 1 -- 172.21.15.60:0/2583799875 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c8012df0 con 0x555d7beebc70 2024-02-26T10:48:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.716+0000 7f77cdc00640 1 -- 172.21.15.60:0/2583799875 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80193a0 con 0x555d7beebc70 2024-02-26T10:48:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.717+0000 7f77e23df740 1 -- 172.21.15.60:0/2583799875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 msgr2=0x555d7beec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.717+0000 7f77e23df740 1 --2- 172.21.15.60:0/2583799875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7beec090 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f77c8011490 tx=0x7f77c8011950 comp rx=0 tx=0).stop 2024-02-26T10:48:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.717+0000 7f77e23df740 1 -- 172.21.15.60:0/2583799875 shutdown_connections 2024-02-26T10:48:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.717+0000 7f77e23df740 1 --2- 172.21.15.60:0/2583799875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7bef1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.717+0000 7f77e23df740 1 --2- 172.21.15.60:0/2583799875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7bef3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.717+0000 7f77e23df740 1 --2- 172.21.15.60:0/2583799875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7beec090 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.717+0000 7f77e23df740 1 -- 172.21.15.60:0/2583799875 >> 172.21.15.60:0/2583799875 conn(0x555d7bee71e0 msgr2=0x555d7bee75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.717+0000 7f77e23df740 1 -- 172.21.15.60:0/2583799875 shutdown_connections 2024-02-26T10:48:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.718+0000 7f77e23df740 1 -- 172.21.15.60:0/2583799875 wait complete. 2024-02-26T10:48:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.718+0000 7f77e23df740 1 -- start start 2024-02-26T10:48:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.719+0000 7f77e23df740 1 Processor -- start 2024-02-26T10:48:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.719+0000 7f77dd600640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f77bc0a1e80 0x7f77bc0a4340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77d806af50 tx=0x7f77d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.713 INFO:tasks.workunit.client.0.smithi060.stdout:1/690: dwrite d3/d35/dc2/fee [0,4194304] 0 2024-02-26T10:48:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.721+0000 7f77e23df740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7bef6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.721+0000 7f77e23df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7bef65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.721+0000 7f77e23df740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7befd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.722+0000 7f77e23df740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d7bef4780 con 0x555d7beebc70 2024-02-26T10:48:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.722+0000 7f77e23df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d7bdef1f0 con 0x555d7beec660 2024-02-26T10:48:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.722+0000 7f77e23df740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d7bef4900 con 0x555d7bef17d0 2024-02-26T10:48:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.722+0000 7f77e23df740 10 client.0 _ll_register_callbacks cb 0x555d7bb64be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.722+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7bef6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.722+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7bef6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46308/0 (socket says 172.21.15.60:46308) 2024-02-26T10:48:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.722+0000 7f77dcc00640 1 -- 172.21.15.60:0/3131862393 learned_addr learned my addr 172.21.15.60:0/3131862393 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.722+0000 7f77dcc00640 1 -- 172.21.15.60:0/3131862393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 msgr2=0x555d7befd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.722+0000 7f77dcc00640 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7befd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.723+0000 7f77dcc00640 1 -- 172.21.15.60:0/3131862393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 msgr2=0x555d7bef65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.723+0000 7f77dcc00640 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7bef65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.723+0000 7f77dcc00640 1 -- 172.21.15.60:0/3131862393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d7bc2f7d0 con 0x555d7beebc70 2024-02-26T10:48:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.723+0000 7f77dcc00640 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7bef6070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f77c801c040 tx=0x7f77c8009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a676a3f8d482f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.724+0000 7f77c6000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:48:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.724+0000 7f77c6000640 1 -- 172.21.15.60:0/3131862393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8028070 con 0x555d7beebc70 2024-02-26T10:48:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.724+0000 7f77c6000640 1 -- 172.21.15.60:0/3131862393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f77c8003970 con 0x555d7beebc70 2024-02-26T10:48:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.725+0000 7f77e23df740 1 -- 172.21.15.60:0/3131862393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77c8012df0 con 0x555d7beebc70 2024-02-26T10:48:02.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.725+0000 7f77c6000640 1 -- 172.21.15.60:0/3131862393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77c8032050 con 0x555d7beebc70 2024-02-26T10:48:02.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.725+0000 7f77e23df740 10 client.15948 fetch_fsmap learned FSMap version 29 2024-02-26T10:48:02.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.725+0000 7f77e23df740 1 -- 172.21.15.60:0/3131862393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555d7bf00960 con 0x555d7beebc70 2024-02-26T10:48:02.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.726+0000 7f77c6000640 1 -- 172.21.15.60:0/3131862393 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c8012df0 con 0x555d7beebc70 2024-02-26T10:48:02.720 INFO:tasks.workunit.client.0.smithi060.stdout:9/667: sync 2024-02-26T10:48:02.720 INFO:tasks.workunit.client.0.smithi060.stdout:8/631: rename d9/d15/f27 to d9/d15/d29/d2f/d34/d72/d50/d6a/d63/fc8 0 2024-02-26T10:48:02.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.727+0000 7f77c6000640 1 -- 172.21.15.60:0/3131862393 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f77c800f040 con 0x555d7beebc70 2024-02-26T10:48:02.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.728+0000 7f77e23df740 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:48:02.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.728+0000 7f77e23df740 10 client.15948 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T10:48:02.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.728+0000 7f77e23df740 20 client.15948 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:02.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.728+0000 7f77e23df740 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x555d7bf01dc0 0x555d7bf041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.728+0000 7f77e23df740 4 client.15948 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:48:02.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.728+0000 7f77e23df740 1 -- 172.21.15.60:0/3131862393 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d7be5e270 con 0x555d7bf01dc0 2024-02-26T10:48:02.722 INFO:tasks.workunit.client.0.smithi060.stdout:9/668: symlink d3/d1e/d93/da2/le4 0 2024-02-26T10:48:02.722 INFO:tasks.workunit.client.0.smithi060.stdout:9/669: readlink d3/d1e/d5e/d35/d87/lca 0 2024-02-26T10:48:02.722 INFO:tasks.workunit.client.0.smithi060.stdout:0/592: write d0/f3 [63706,87637] 0 2024-02-26T10:48:02.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.729+0000 7f77cfe00640 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x555d7bf01dc0 0x555d7bf041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.730+0000 7f77cfe00640 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x555d7bf01dc0 0x555d7bf041f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d8bfd666d23758f server_cookie=390d1586afcd0bd8 in_seq=0 out_seq=0 2024-02-26T10:48:02.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.730+0000 7f77c6000640 10 client.15948 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:48:02.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.731+0000 7f77c6000640 1 -- 172.21.15.60:0/3131862393 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d7be5e270 con 0x555d7bf01dc0 2024-02-26T10:48:02.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.731+0000 7f77c6000640 10 client.15948 handle_command_reply: tid=0 2024-02-26T10:48:02.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.731+0000 7f77e23df740 1 client.15948 shutdown 2024-02-26T10:48:02.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.732+0000 7f77e23df740 1 -- 172.21.15.60:0/3131862393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 msgr2=0x555d7bef6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.732+0000 7f77e23df740 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7bef6070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f77c801c040 tx=0x7f77c8009cb0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.726 INFO:tasks.workunit.client.0.smithi060.stdout:5/614: dread d0/de/d20/f24 [0,4194304] 0 2024-02-26T10:48:02.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.732+0000 7f77e23df740 1 -- 172.21.15.60:0/3131862393 shutdown_connections 2024-02-26T10:48:02.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.732+0000 7f77e23df740 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7bef65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.732+0000 7f77e23df740 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x555d7bf01dc0 0x555d7bf041f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.732+0000 7f77e23df740 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7befd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.732+0000 7f77e23df740 1 --2- 172.21.15.60:0/3131862393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7bef6070 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.732+0000 7f77c6000640 0 client.15948 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:48:02.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.732+0000 7f77e23df740 1 -- 172.21.15.60:0/3131862393 >> 172.21.15.60:0/3131862393 conn(0x555d7bee71e0 msgr2=0x555d7bee6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.727 INFO:tasks.workunit.client.0.smithi060.stdout:5/615: getdents d0/de/d20/d32/d4d/da3/da5 0 2024-02-26T10:48:02.727 INFO:tasks.workunit.client.0.smithi060.stdout:0/593: rename d0/d37/d44 to d0/dd/d39/d40/d83/dc6 0 2024-02-26T10:48:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.734+0000 7f77e23df740 1 -- 172.21.15.60:0/3131862393 shutdown_connections 2024-02-26T10:48:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.735+0000 7f77e23df740 1 -- 172.21.15.60:0/3131862393 wait complete. 2024-02-26T10:48:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.735+0000 7f77e23df740 20 client.15948 trim_cache size 0 max 16384 2024-02-26T10:48:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:7/762: dwrite da/d15/d21/d2f/d37/ff9 [0,4194304] 0 2024-02-26T10:48:02.738 INFO:tasks.workunit.client.0.smithi060.stdout:7/763: creat da/df1/fff x:0 0 0 2024-02-26T10:48:02.738 INFO:tasks.workunit.client.0.smithi060.stdout:2/590: dread dd/d43/d60/d41/f95 [0,4194304] 0 2024-02-26T10:48:02.739 INFO:tasks.workunit.client.0.smithi060.stdout:2/591: write f5 [1491813,58662] 0 2024-02-26T10:48:02.739 INFO:tasks.workunit.client.0.smithi060.stdout:9/670: link d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f7f d3/d1e/d5e/d35/d87/d9b/d8a/dbe/fe5 0 2024-02-26T10:48:02.739 INFO:tasks.workunit.client.0.smithi060.stdout:5/616: mknod d0/de/d15/cd4 0 2024-02-26T10:48:02.739 INFO:tasks.workunit.client.0.smithi060.stdout:9/671: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f4e [1301124,77667] 0 2024-02-26T10:48:02.739 INFO:tasks.workunit.client.0.smithi060.stdout:1/691: dread d3/db/f40 [0,4194304] 0 2024-02-26T10:48:02.739 INFO:tasks.workunit.client.0.smithi060.stdout:2/592: creat dd/d43/d55/d5e/dc6/fcb x:0 0 0 2024-02-26T10:48:02.740 INFO:tasks.workunit.client.0.smithi060.stdout:7/764: rename da/d15/d88/d3b/c5e to da/d94/c100 0 2024-02-26T10:48:02.740 INFO:tasks.workunit.client.0.smithi060.stdout:7/765: stat da/d66/cf2 0 2024-02-26T10:48:02.740 INFO:tasks.workunit.client.0.smithi060.stdout:7/766: chown da/d15/d21/d2f/f91 0 1 2024-02-26T10:48:02.741 INFO:tasks.workunit.client.0.smithi060.stdout:7/767: readlink da/d66/dbd/ld1 0 2024-02-26T10:48:02.741 INFO:tasks.workunit.client.0.smithi060.stdout:1/692: creat d3/db/d1e/ff1 x:0 0 0 2024-02-26T10:48:02.741 INFO:tasks.workunit.client.0.smithi060.stdout:2/593: creat dd/fcc x:0 0 0 2024-02-26T10:48:02.741 INFO:tasks.workunit.client.0.smithi060.stdout:2/594: write dd/d43/d60/d41/f1a [7379701,34173] 0 2024-02-26T10:48:02.741 INFO:tasks.workunit.client.0.smithi060.stdout:7/768: read da/d15/d88/d3b/f71 [284365,48432] 0 2024-02-26T10:48:02.741 INFO:tasks.workunit.client.0.smithi060.stdout:9/672: symlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/le6 0 2024-02-26T10:48:02.741 INFO:tasks.workunit.client.0.smithi060.stdout:7/769: dread - da/d15/d21/d2f/d37/fdf zero size 2024-02-26T10:48:02.742 INFO:tasks.workunit.client.0.smithi060.stdout:9/673: dread - d3/d1e/d93/da2/fb3 zero size 2024-02-26T10:48:02.745 INFO:tasks.workunit.client.0.smithi060.stdout:1/693: symlink d3/lf2 0 2024-02-26T10:48:02.745 INFO:tasks.workunit.client.0.smithi060.stdout:9/674: mknod d3/d1e/ce7 0 2024-02-26T10:48:02.745 INFO:tasks.workunit.client.0.smithi060.stdout:5/617: rename d0/de/d20/d82/f87 to d0/de/d15/d65/d86/dcc/fd5 0 2024-02-26T10:48:02.745 INFO:tasks.workunit.client.0.smithi060.stdout:5/618: creat d0/de/d20/d32/d4d/da3/da5/dc8/fd6 x:0 0 0 2024-02-26T10:48:02.745 INFO:tasks.workunit.client.0.smithi060.stdout:5/619: read d0/de/f29 [3382744,54136] 0 2024-02-26T10:48:02.745 INFO:tasks.workunit.client.0.smithi060.stdout:2/595: rename dd/d43/d60/d41/d42/d6f/fb1 to dd/d43/d60/d41/d42/d6f/d8f/fcd 0 2024-02-26T10:48:02.745 INFO:tasks.workunit.client.0.smithi060.stdout:2/596: chown dd/d43/d60/d41/d37/l75 789473916 1 2024-02-26T10:48:02.747 INFO:tasks.workunit.client.0.smithi060.stdout:9/675: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/de8 0 2024-02-26T10:48:02.748 INFO:tasks.workunit.client.0.smithi060.stdout:2/597: write dd/d43/d55/f5a [930412,61870] 0 2024-02-26T10:48:02.751 INFO:tasks.workunit.client.0.smithi060.stdout:2/598: getdents dd/d43/d96 0 2024-02-26T10:48:02.751 INFO:tasks.workunit.client.0.smithi060.stdout:5/620: rmdir d0/de/d15/db9 0 2024-02-26T10:48:02.752 INFO:tasks.workunit.client.0.smithi060.stdout:5/621: creat d0/de/da4/fd7 x:0 0 0 2024-02-26T10:48:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.760+0000 7f77e23df740 1 Processor -- start 2024-02-26T10:48:02.754 INFO:tasks.workunit.client.0.smithi060.stdout:5/622: creat d0/de/fd8 x:0 0 0 2024-02-26T10:48:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.761+0000 7f77e23df740 1 -- start start 2024-02-26T10:48:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.761+0000 7f77e23df740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7bee5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.761+0000 7f77e23df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7befd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.761+0000 7f77e23df740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7bee6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.761+0000 7f77e23df740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81c37a0 con 0x555d7beebc70 2024-02-26T10:48:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.761+0000 7f77e23df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81c12e0 con 0x555d7beec660 2024-02-26T10:48:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.761+0000 7f77e23df740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81bf380 con 0x555d7bef17d0 2024-02-26T10:48:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.763+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7befd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.763+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7befd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51168/0 (socket says 172.21.15.60:51168) 2024-02-26T10:48:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.763+0000 7f77cfe00640 1 -- 172.21.15.60:0/2575247236 learned_addr learned my addr 172.21.15.60:0/2575247236 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.763+0000 7f77dd600640 1 --2- 172.21.15.60:0/2575247236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7bee6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.764+0000 7f77cfe00640 1 -- 172.21.15.60:0/2575247236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 msgr2=0x555d7bee6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.764+0000 7f77cfe00640 1 --2- 172.21.15.60:0/2575247236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7bee6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.765+0000 7f77cfe00640 1 -- 172.21.15.60:0/2575247236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 msgr2=0x555d7bee5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.765+0000 7f77cfe00640 1 --2- 172.21.15.60:0/2575247236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7bee5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.765+0000 7f77cfe00640 1 -- 172.21.15.60:0/2575247236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d7bee7f50 con 0x555d7beec660 2024-02-26T10:48:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.765+0000 7f77dd600640 1 --2- 172.21.15.60:0/2575247236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7bee6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:02.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.765+0000 7f77cfe00640 1 --2- 172.21.15.60:0/2575247236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7befd5b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77c0008b30 tx=0x7f77c00ec990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f65d925318f5733 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.765+0000 7f77cd200640 1 -- 172.21.15.60:0/2575247236 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00d3070 con 0x555d7beec660 2024-02-26T10:48:02.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.766+0000 7f77cd200640 1 -- 172.21.15.60:0/2575247236 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c00bc3f0 con 0x555d7beec660 2024-02-26T10:48:02.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.766+0000 7f77cd200640 1 -- 172.21.15.60:0/2575247236 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00b9ff0 con 0x555d7beec660 2024-02-26T10:48:02.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.767+0000 7f77e23df740 1 -- 172.21.15.60:0/2575247236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 msgr2=0x555d7befd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.767+0000 7f77e23df740 1 --2- 172.21.15.60:0/2575247236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7befd5b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77c0008b30 tx=0x7f77c00ec990 comp rx=0 tx=0).stop 2024-02-26T10:48:02.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.768+0000 7f77e23df740 1 -- 172.21.15.60:0/2575247236 shutdown_connections 2024-02-26T10:48:02.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.768+0000 7f77e23df740 1 --2- 172.21.15.60:0/2575247236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7beec660 0x555d7befd5b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.768+0000 7f77e23df740 1 --2- 172.21.15.60:0/2575247236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7bef17d0 0x555d7bee6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.768+0000 7f77e23df740 1 --2- 172.21.15.60:0/2575247236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beebc70 0x555d7bee5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.768+0000 7f77e23df740 1 -- 172.21.15.60:0/2575247236 >> 172.21.15.60:0/2575247236 conn(0x555d7bee71e0 msgr2=0x555d7beedb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.768+0000 7f77e23df740 1 -- 172.21.15.60:0/2575247236 shutdown_connections 2024-02-26T10:48:02.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.768+0000 7f77e23df740 1 -- 172.21.15.60:0/2575247236 wait complete. 2024-02-26T10:48:02.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.769+0000 7f77e23df740 1 -- start start 2024-02-26T10:48:02.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.769+0000 7f77e23df740 1 Processor -- start 2024-02-26T10:48:02.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.770+0000 7f77e23df740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7beebc70 0x555d7befd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.770+0000 7f77e23df740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beec660 0x555d7befbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.770+0000 7f77e23df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7bef17d0 0x555d7befc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.770+0000 7f77e23df740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81bf980 con 0x555d7beec660 2024-02-26T10:48:02.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.770+0000 7f77e23df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81c0c80 con 0x555d7bef17d0 2024-02-26T10:48:02.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.770+0000 7f77e23df740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81bffc0 con 0x555d7beebc70 2024-02-26T10:48:02.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.770+0000 7f77e23df740 10 client.0 _ll_register_callbacks cb 0x7f77c8009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:02.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beec660 0x555d7befbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beec660 0x555d7befbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46328/0 (socket says 172.21.15.60:46328) 2024-02-26T10:48:02.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77cfe00640 1 -- 172.21.15.60:0/2101179887 learned_addr learned my addr 172.21.15.60:0/2101179887 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:02.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7beebc70 0x555d7befd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77dd600640 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7bef17d0 0x555d7befc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77cfe00640 1 -- 172.21.15.60:0/2101179887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7beebc70 msgr2=0x555d7befd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77cfe00640 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7beebc70 0x555d7befd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77cfe00640 1 -- 172.21.15.60:0/2101179887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7bef17d0 msgr2=0x555d7befc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:02.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77cfe00640 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7bef17d0 0x555d7befc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.771+0000 7f77cfe00640 1 -- 172.21.15.60:0/2101179887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d7befc8e0 con 0x555d7beec660 2024-02-26T10:48:02.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.772+0000 7f77cfe00640 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beec660 0x555d7befbb10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f77c00bb9f0 tx=0x7f77c00072c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fe663ef5de2e8f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:02.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.772+0000 7f77c6000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:48:02.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.772+0000 7f77c6000640 1 -- 172.21.15.60:0/2101179887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00d3030 con 0x555d7beec660 2024-02-26T10:48:02.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.772+0000 7f77c6000640 1 -- 172.21.15.60:0/2101179887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f77c000a880 con 0x555d7beec660 2024-02-26T10:48:02.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.772+0000 7f77e23df740 1 -- 172.21.15.60:0/2101179887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77c00bc3f0 con 0x555d7beec660 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.773+0000 7f77c6000640 1 -- 172.21.15.60:0/2101179887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77c00da050 con 0x555d7beec660 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.773+0000 7f77e23df740 10 client.15951 fetch_fsmap learned FSMap version 29 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.773+0000 7f77e23df740 1 -- 172.21.15.60:0/2101179887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555d7bf59430 con 0x555d7beec660 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.773+0000 7f77c6000640 1 -- 172.21.15.60:0/2101179887 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c00bc3f0 con 0x555d7beec660 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.774+0000 7f77c6000640 1 -- 172.21.15.60:0/2101179887 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f77c00bb140 con 0x555d7beec660 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.774+0000 7f77e23df740 10 client.15951 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.774+0000 7f77e23df740 10 client.15951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.774+0000 7f77e23df740 20 client.15951 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.774+0000 7f77e23df740 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x555d7bf59db0 0x555d7bf5c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.774+0000 7f77e23df740 4 client.15951 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:48:02.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.774+0000 7f77e23df740 1 -- 172.21.15.60:0/2101179887 --> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555d7be585c0 con 0x555d7bf59db0 2024-02-26T10:48:02.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.776+0000 7f77dcc00640 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x555d7bf59db0 0x555d7bf5c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:02.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.779+0000 7f77dcc00640 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x555d7bf59db0 0x555d7bf5c1d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb34970ea685c58 server_cookie=b076434806a00d02 in_seq=0 out_seq=0 2024-02-26T10:48:02.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.779+0000 7f77c6000640 10 client.15951 ms_handle_connect on v2:172.21.15.113:6838/2057195735 2024-02-26T10:48:02.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.780+0000 7f77c6000640 1 -- 172.21.15.60:0/2101179887 <== mds.1 v2:172.21.15.113:6838/2057195735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+28862 (crc 0 0 0) 0x555d7be585c0 con 0x555d7bf59db0 2024-02-26T10:48:02.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.780+0000 7f77c6000640 10 client.15951 handle_command_reply: tid=0 2024-02-26T10:48:02.774 INFO:tasks.workunit.client.0.smithi060.stdout:9/676: dread d3/d1e/d5e/d35/d87/fc1 [0,4194304] 0 2024-02-26T10:48:02.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.781+0000 7f77e23df740 1 client.15951 shutdown 2024-02-26T10:48:02.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.781+0000 7f77e23df740 1 -- 172.21.15.60:0/2101179887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beec660 msgr2=0x555d7befbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.781+0000 7f77e23df740 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beec660 0x555d7befbb10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f77c00bb9f0 tx=0x7f77c00072c0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77e23df740 1 -- 172.21.15.60:0/2101179887 shutdown_connections 2024-02-26T10:48:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77e23df740 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d7bef17d0 0x555d7befc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77e23df740 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.113:6838/2057195735,v1:172.21.15.113:6839/2057195735] conn(0x555d7bf59db0 0x555d7bf5c1d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77e23df740 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d7beec660 0x555d7befbb10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77e23df740 1 --2- 172.21.15.60:0/2101179887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555d7beebc70 0x555d7befd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77c6000640 0 client.15951 ms_handle_reset on v2:172.21.15.113:6838/2057195735 2024-02-26T10:48:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77e23df740 1 -- 172.21.15.60:0/2101179887 >> 172.21.15.60:0/2101179887 conn(0x555d7bee71e0 msgr2=0x555d7beea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77e23df740 1 -- 172.21.15.60:0/2101179887 shutdown_connections 2024-02-26T10:48:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77e23df740 1 -- 172.21.15.60:0/2101179887 wait complete. 2024-02-26T10:48:02.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.782+0000 7f77e23df740 20 client.15951 trim_cache size 0 max 16384 2024-02-26T10:48:02.777 INFO:tasks.workunit.client.0.smithi060.stdout:9/677: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/de9 0 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 52580559, 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631710, 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_013.c" 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 120857269, 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631720, 2024-02-26T10:48:02.778 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_006.c" 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 180073712, 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632636, 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/8-1.c" 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 234077096, 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631746, 2024-02-26T10:48:02.779 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_028.c" 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 263346481, 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631712, 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.am" 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 265565002, 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631733, 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_012.c" 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 271409304, 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631723, 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_001.c" 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 381949674, 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631695, 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_enum_utils_test.c" 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 387111911, 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631768, 2024-02-26T10:48:02.781 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_005.c" 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 402152707, 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631694, 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rdr_test.c" 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 492971224, 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632635, 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/1-1.c" 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.782 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 538945458, 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631685, 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_text_utils_test.c" 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 550800747, 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631737, 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_036.c" 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 555072297, 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631766, 2024-02-26T10:48:02.783 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_023.c" 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 611946816, 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631715, 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_003.c" 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 615871376, 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632633, 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/5-1.c" 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.784 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 663414329, 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632640, 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/11-1.c" 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 711809447, 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632639, 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/4-1.c" 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 726846847, 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631740, 2024-02-26T10:48:02.785 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_034.c" 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 756205454, 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631730, 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_026.c" 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 817577950, 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632634, 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/6-1.c" 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.786 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 898363847, 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631743, 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_022.c" 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 900457148, 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631759, 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.in" 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 908851937, 2024-02-26T10:48:02.787 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631741, 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_013.c" 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 926641769, 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631687, 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_time_utils_test.c" 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 928041557, 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631749, 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_007.c" 2024-02-26T10:48:02.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 999210131, 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631754, 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_044.c" 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1053644585, 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632638, 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/7-1.c" 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.789 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1151552889, 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631731, 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_010.c" 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1178077232, 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632265, 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.in" 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1191267456, 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632632, 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/2-1.c" 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1252260674, 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631739, 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_027.c" 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1268476878, 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632630, 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/9-1.c" 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1403426037, 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631760, 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_008.c" 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1403667332, 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631729, 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_011.c" 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1419860331, 2024-02-26T10:48:02.792 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631724, 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.am" 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1447726230, 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631756, 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test.h" 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1485634572, 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631719, 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_005.c" 2024-02-26T10:48:02.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1493930249, 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631689, 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_encode_test.c" 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1497142873, 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631752, 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_020.c" 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.794 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1527432246, 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632637, 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/assertions.xml" 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1534601639, 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631758, 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_031.c" 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1549274681, 2024-02-26T10:48:02.795 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631721, 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_002.c" 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1553313812, 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631684, 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrfield_test.c" 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1557285089, 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631683, 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_utils_test.c" 2024-02-26T10:48:02.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1626856606, 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631763, 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_009.c" 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1652340322, 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631722, 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.in" 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.797 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1736527868, 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631700, 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_012.c" 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1775686753, 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631682, 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrinfo_test.c" 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1797847575, 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631753, 2024-02-26T10:48:02.798 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_014.c" 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1813030090, 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631728, 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_038.c" 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1868082986, 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631693, 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.am" 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.799 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1925191709, 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631717, 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_008.c" 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1955814977, 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632629, 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/3-1.c" 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2001273979, 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631706, 2024-02-26T10:48:02.800 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_000.c" 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2021206997, 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631703, 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_007.c" 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2085036102, 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631691, 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpiatca_enum_utils_test.c" 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.801 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2091078123, 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631708, 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_005.c" 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2091857103, 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631755, 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_006.c" 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2098473037, 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631705, 2024-02-26T10:48:02.802 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_001.c" 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2162824008, 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631745, 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_004.c" 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2320293415, 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631774, 2024-02-26T10:48:02.803 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.am" 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2368359441, 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631736, 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_033.c" 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2485152224, 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631697, 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_006.c" 2024-02-26T10:48:02.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2544021302, 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631692, 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_utils_test.c" 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2646328350, 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631761, 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_045.c" 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2651055715, 2024-02-26T10:48:02.805 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631735, 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_035.c" 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2711375382, 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631773, 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_003.c" 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2717686744, 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631702, 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_010.c" 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.806 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2730820050, 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631767, 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_018.c" 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2739625087, 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632266, 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/sysfs2hpi.c" 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2749592871, 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631711, 2024-02-26T10:48:02.807 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_004.c" 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2801937141, 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631698, 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_002.c" 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2828780396, 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631688, 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrarea_test.c" 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2938119167, 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631751, 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_compare.c" 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2969732743, 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631771, 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_041.c" 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3020383070, 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631748, 2024-02-26T10:48:02.809 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_019.c" 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3069902837, 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631772, 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_043.c" 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3081292958, 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631704, 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_003.c" 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.810 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3129068830, 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631734, 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_032.c" 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3153934479, 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631732, 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_016.c" 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3164325210, 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631750, 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_030.c" 2024-02-26T10:48:02.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3314644624, 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631718, 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_007.c" 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3353787575, 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631707, 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.in" 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3362085298, 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631764, 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_040.c" 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3375262909, 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631765, 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_015.c" 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3387810885, 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631686, 2024-02-26T10:48:02.813 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rptentry_test.c" 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3391971065, 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631709, 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_011.c" 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3403058285, 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632267, 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.am" 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.814 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3513157897, 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631757, 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_037.c" 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3565254938, 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631742, 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_021.c" 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3602528510, 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632631, 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/10-1.c" 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3613618268, 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631747, 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_025.c" 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3633258708, 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631770, 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_029.c" 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3644241078, 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631769, 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_001.c" 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3660927375, 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631738, 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_039.c" 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3663831718, 2024-02-26T10:48:02.817 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631690, 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.in" 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3674078795, 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631744, 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_002.c" 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3723166584, 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631727, 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_017.c" 2024-02-26T10:48:02.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 3928623611, 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631716, 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_004.c" 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4072896960, 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631699, 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_009.c" 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.819 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4113138317, 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631762, 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_042.c" 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4150004916, 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631726, 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_024.c" 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.820 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4170341107, 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511631701, 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_008.c" 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4282624035, 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511632264, 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/README" 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:48:02.821 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-02-26T10:48:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f77bc0a1e80 msgr2=0x7f77bc0a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f77bc0a1e80 0x7f77bc0a4340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77d806af50 tx=0x7f77d0008040 comp rx=0 tx=0).stop 2024-02-26T10:48:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 msgr2=0x7f77d81196f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 0x7f77d81196f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f77d8073460 tx=0x7f77c00045e0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 shutdown_connections 2024-02-26T10:48:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f77bc0a1e80 0x7f77bc0a4340 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d810ae90 0x7f77d81196f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d810d930 0x7f77d8117840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 --2- 172.21.15.60:0/530719751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d81084d0 0x7f77d81191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.783+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 >> 172.21.15.60:0/530719751 conn(0x7f77d806e0c0 msgr2=0x7f77d810cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.784+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 shutdown_connections 2024-02-26T10:48:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:02.784+0000 7f77df600640 1 -- 172.21.15.60:0/530719751 wait complete. 2024-02-26T10:48:02.825 INFO:tasks.workunit.client.0.smithi060.stdout:9/678: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/fea x:0 0 0 2024-02-26T10:48:02.825 INFO:tasks.workunit.client.0.smithi060.stdout:0/594: dread d0/dd/d39/d40/d83/dc6/f79 [0,4194304] 0 2024-02-26T10:48:02.825 INFO:tasks.workunit.client.0.smithi060.stdout:0/595: stat d0/d42/cbc 0 2024-02-26T10:48:02.825 INFO:tasks.workunit.client.0.smithi060.stdout:1/694: read d3/d35/f65 [1429247,16052] 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:0/596: rmdir d0/dd/d53/da2 39 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:0/597: mkdir d0/dd/d53/d87/d94/da6/dc7 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:1/695: unlink d3/d35/l4a 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:0/598: truncate d0/d37/f6d 48523 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:1/696: creat d3/db/ff3 x:0 0 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:0/599: mkdir d0/d42/d7a/dc8 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:5/623: dwrite d0/de/d15/d26/f78 [0,4194304] 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:0/600: symlink d0/d42/lc9 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:5/624: creat d0/de/d15/d65/d86/dcc/fd9 x:0 0 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:7/770: dwrite da/d15/d21/d2f/d78/fac [0,4194304] 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:7/771: stat da/df1/ff8 0 2024-02-26T10:48:02.826 INFO:tasks.workunit.client.0.smithi060.stdout:5/625: symlink d0/de/d20/lda 0 2024-02-26T10:48:02.827 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T10:48:02.828 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T10:48:02.828 INFO:tasks.workunit.client.0.smithi060.stdout:9/679: rename d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/d47 to d3/d1e/d5e/d35/deb 0 2024-02-26T10:48:02.828 INFO:tasks.workunit.client.0.smithi060.stdout:9/680: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc 14 1 2024-02-26T10:48:02.828 INFO:tasks.workunit.client.0.smithi060.stdout:7/772: dread da/d15/d21/f28 [0,4194304] 0 2024-02-26T10:48:02.828 INFO:tasks.workunit.client.0.smithi060.stdout:6/661: sync 2024-02-26T10:48:02.830 INFO:tasks.workunit.client.0.smithi060.stdout:2/599: dwrite f5 [4194304,4194304] 0 2024-02-26T10:48:02.830 INFO:tasks.workunit.client.0.smithi060.stdout:2/600: read - dd/fcc zero size 2024-02-26T10:48:02.831 INFO:tasks.workunit.client.0.smithi060.stdout:6/662: symlink d6/d7/d2e/d56/d73/d8c/ld7 0 2024-02-26T10:48:02.831 INFO:tasks.workunit.client.0.smithi060.stdout:6/663: chown d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac/cba 0 1 2024-02-26T10:48:02.831 INFO:tasks.workunit.client.0.smithi060.stdout:2/601: creat dd/d43/d55/d5e/d86/d97/fce x:0 0 0 2024-02-26T10:48:02.831 INFO:tasks.workunit.client.0.smithi060.stdout:2/602: truncate dd/d43/fb0 6864228 0 2024-02-26T10:48:02.831 INFO:tasks.workunit.client.0.smithi060.stdout:6/664: rmdir d6/d8d/dcc/dbc 0 2024-02-26T10:48:02.831 INFO:tasks.workunit.client.0.smithi060.stdout:2/603: rmdir dd/d43/d60/d41/d37 39 2024-02-26T10:48:02.834 INFO:tasks.workunit.client.0.smithi060.stdout:2/604: readlink dd/d11/l98 0 2024-02-26T10:48:02.835 INFO:tasks.workunit.client.0.smithi060.stdout:2/605: stat dd/d43/d55/d5e/d86/la1 0 2024-02-26T10:48:02.835 INFO:tasks.workunit.client.0.smithi060.stdout:2/606: stat dd/d43/d55 0 2024-02-26T10:48:02.835 INFO:tasks.workunit.client.0.smithi060.stdout:2/607: link dd/f8d dd/d43/d60/d41/d37/d9a/db4/fcf 0 2024-02-26T10:48:02.835 INFO:tasks.workunit.client.0.smithi060.stdout:2/608: fsync dd/d43/d55/d6e/fac 0 2024-02-26T10:48:02.835 INFO:tasks.workunit.client.0.smithi060.stdout:6/665: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/dc0/dd8 0 2024-02-26T10:48:02.836 INFO:tasks.workunit.client.0.smithi060.stdout:5/626: write d0/de/f35 [540272,76177] 0 2024-02-26T10:48:02.836 INFO:tasks.workunit.client.0.smithi060.stdout:2/609: fsync dd/d43/d60/d41/d37/d9a/db4/fcf 0 2024-02-26T10:48:02.837 INFO:tasks.workunit.client.0.smithi060.stdout:6/666: symlink d6/d8d/ld9 0 2024-02-26T10:48:02.837 INFO:tasks.workunit.client.0.smithi060.stdout:6/667: chown d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/lc5 11079722 1 2024-02-26T10:48:02.838 INFO:tasks.workunit.client.0.smithi060.stdout:5/627: creat d0/de/d20/d32/d4d/d97/fdb x:0 0 0 2024-02-26T10:48:02.839 INFO:tasks.workunit.client.0.smithi060.stdout:6/668: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/fda x:0 0 0 2024-02-26T10:48:02.841 INFO:tasks.workunit.client.0.smithi060.stdout:5/628: symlink d0/dc0/ldc 0 2024-02-26T10:48:02.858 INFO:tasks.workunit.client.0.smithi060.stdout:2/610: read dd/d43/d60/d41/d37/d69/f72 [118180,81824] 0 2024-02-26T10:48:02.858 INFO:tasks.workunit.client.0.smithi060.stdout:2/611: fdatasync dd/d43/d60/d41/d37/d38/f40 0 2024-02-26T10:48:02.859 INFO:tasks.workunit.client.0.smithi060.stdout:7/773: dread da/d15/d88/d3b/f9d [0,4194304] 0 2024-02-26T10:48:02.860 INFO:tasks.workunit.client.0.smithi060.stdout:2/612: readlink dd/d11/l17 0 2024-02-26T10:48:02.860 INFO:tasks.workunit.client.0.smithi060.stdout:0/601: dwrite d0/dd/d53/d87/d94/da6/f9d [0,4194304] 0 2024-02-26T10:48:02.861 INFO:tasks.workunit.client.0.smithi060.stdout:2/613: creat dd/d43/d60/d41/d37/d9a/fd0 x:0 0 0 2024-02-26T10:48:02.861 INFO:tasks.workunit.client.0.smithi060.stdout:2/614: fdatasync dd/d43/d60/d41/d37/d38/f40 0 2024-02-26T10:48:02.862 INFO:tasks.workunit.client.0.smithi060.stdout:2/615: mkdir dd/dd1 0 2024-02-26T10:48:02.864 INFO:tasks.workunit.client.0.smithi060.stdout:2/616: unlink dd/d11/f25 0 2024-02-26T10:48:02.868 INFO:tasks.workunit.client.0.smithi060.stdout:1/697: dwrite d3/d35/d53/d68/dad/fb6 [0,4194304] 0 2024-02-26T10:48:02.873 INFO:tasks.workunit.client.0.smithi060.stdout:0/602: dread d0/dd/d39/d40/d83/dc6/d58/f5b [0,4194304] 0 2024-02-26T10:48:02.888 INFO:tasks.workunit.client.0.smithi060.stdout:0/603: mknod d0/dd/d53/da2/cca 0 2024-02-26T10:48:02.889 INFO:tasks.workunit.client.0.smithi060.stdout:6/669: dwrite d6/d7/d25/d1c/d1f/d23/d28/fa3 [0,4194304] 0 2024-02-26T10:48:02.890 INFO:tasks.workunit.client.0.smithi060.stdout:0/604: creat d0/dd/d39/d40/dad/fcb x:0 0 0 2024-02-26T10:48:02.890 INFO:tasks.workunit.client.0.smithi060.stdout:0/605: truncate d0/dd/d39/d40/f98 535676 0 2024-02-26T10:48:02.891 INFO:tasks.workunit.client.0.smithi060.stdout:0/606: write d0/dd/faf [5817160,109570] 0 2024-02-26T10:48:02.891 INFO:tasks.workunit.client.0.smithi060.stdout:0/607: fsync d0/dd/d33/d6a/d6c/fab 0 2024-02-26T10:48:02.891 INFO:tasks.workunit.client.0.smithi060.stdout:7/774: dread da/d15/d21/f2c [0,4194304] 0 2024-02-26T10:48:02.892 INFO:tasks.workunit.client.0.smithi060.stdout:7/775: chown da/d15 25 1 2024-02-26T10:48:02.892 INFO:tasks.workunit.client.0.smithi060.stdout:0/608: creat d0/dd/d39/d63/fcc x:0 0 0 2024-02-26T10:48:02.892 INFO:tasks.workunit.client.0.smithi060.stdout:0/609: chown d0/dd/d39/d40/d83/ca5 17860 1 2024-02-26T10:48:02.892 INFO:tasks.workunit.client.0.smithi060.stdout:0/610: write d0/dd/d53/d87/fa9 [2649744,95418] 0 2024-02-26T10:48:02.896 INFO:tasks.workunit.client.0.smithi060.stdout:7/776: rename da/d15/f46 to da/d15/d21/d2f/f101 0 2024-02-26T10:48:02.897 INFO:tasks.workunit.client.0.smithi060.stdout:7/777: write da/d15/f2e [1366716,103746] 0 2024-02-26T10:48:02.897 INFO:tasks.workunit.client.0.smithi060.stdout:7/778: fsync da/d15/d21/d2f/d37/ff9 0 2024-02-26T10:48:02.898 INFO:tasks.workunit.client.0.smithi060.stdout:0/611: rename d0/dd/d39/d63/fbf to d0/dd/d33/d6a/fcd 0 2024-02-26T10:48:02.898 INFO:tasks.workunit.client.0.smithi060.stdout:0/612: chown d0/dd/d39/d63/fcc 13177 1 2024-02-26T10:48:02.902 INFO:tasks.workunit.client.0.smithi060.stdout:7/779: truncate da/d15/d88/dce/faf 3641206 0 2024-02-26T10:48:02.903 INFO:tasks.workunit.client.0.smithi060.stdout:7/780: rename da/d66/dbd/fe1 to da/d66/dbd/f102 0 2024-02-26T10:48:02.908 INFO:tasks.workunit.client.0.smithi060.stdout:5/629: dwrite d0/de/d15/f18 [0,4194304] 0 2024-02-26T10:48:02.913 INFO:tasks.workunit.client.0.smithi060.stdout:5/630: mkdir d0/de/d15/d26/ddd 0 2024-02-26T10:48:02.921 INFO:tasks.workunit.client.0.smithi060.stdout:7/781: rename da/d94/fd6 to da/d15/d21/d2f/d78/f103 0 2024-02-26T10:48:02.923 INFO:tasks.workunit.client.0.smithi060.stdout:4/663: sync 2024-02-26T10:48:02.923 INFO:tasks.workunit.client.0.smithi060.stdout:4/664: write d0/da/dc7/db0/db5/dc0/f8 [5426627,110067] 0 2024-02-26T10:48:02.923 INFO:tasks.workunit.client.0.smithi060.stdout:3/497: sync 2024-02-26T10:48:02.923 INFO:tasks.workunit.client.0.smithi060.stdout:8/632: sync 2024-02-26T10:48:02.925 INFO:tasks.workunit.client.0.smithi060.stdout:4/665: creat d0/d27/d76/fd3 x:0 0 0 2024-02-26T10:48:02.925 INFO:tasks.workunit.client.0.smithi060.stdout:8/633: creat d9/d15/d29/d2f/fc9 x:0 0 0 2024-02-26T10:48:02.925 INFO:tasks.workunit.client.0.smithi060.stdout:8/634: chown d9/d15/d29/d2f/d34/d72/d50/d6a/f8e 44082 1 2024-02-26T10:48:02.926 INFO:tasks.workunit.client.0.smithi060.stdout:3/498: rmdir d5/d45/d61 39 2024-02-26T10:48:02.933 INFO:tasks.workunit.client.0.smithi060.stdout:8/635: symlink d9/d15/d29/d2f/d34/d72/d50/lca 0 2024-02-26T10:48:02.947 INFO:tasks.workunit.client.0.smithi060.stdout:6/670: dwrite d6/f8e [0,4194304] 0 2024-02-26T10:48:02.949 INFO:tasks.workunit.client.0.smithi060.stdout:2/617: dwrite dd/f2b [4194304,4194304] 0 2024-02-26T10:48:02.956 INFO:tasks.workunit.client.0.smithi060.stdout:6/671: getdents d6/d7/d47/d69 0 2024-02-26T10:48:02.958 INFO:tasks.workunit.client.0.smithi060.stdout:6/672: creat d6/d7/d2e/d56/d73/fdb x:0 0 0 2024-02-26T10:48:02.959 INFO:tasks.workunit.client.0.smithi060.stdout:1/698: dwrite d3/f34 [0,4194304] 0 2024-02-26T10:48:02.959 INFO:tasks.workunit.client.0.smithi060.stdout:6/673: creat d6/d6e/fdc x:0 0 0 2024-02-26T10:48:02.961 INFO:tasks.workunit.client.0.smithi060.stdout:1/699: link d3/db/d2e/fd5 d3/db/ff4 0 2024-02-26T10:48:02.965 INFO:tasks.workunit.client.0.smithi060.stdout:6/674: rename d6/d7/d47/lb4 to d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/ldd 0 2024-02-26T10:48:02.965 INFO:tasks.workunit.client.0.smithi060.stdout:1/700: rmdir d3/db/dc8/dbe 39 2024-02-26T10:48:02.965 INFO:tasks.workunit.client.0.smithi060.stdout:1/701: getdents d3/d54/dc0 0 2024-02-26T10:48:02.965 INFO:tasks.workunit.client.0.smithi060.stdout:3/499: dread d5/de/df/f33 [0,4194304] 0 2024-02-26T10:48:02.972 INFO:tasks.workunit.client.0.smithi060.stdout:1/702: unlink d3/f34 0 2024-02-26T10:48:02.975 INFO:tasks.workunit.client.0.smithi060.stdout:1/703: getdents d3/db/d2e 0 2024-02-26T10:48:02.975 INFO:tasks.workunit.client.0.smithi060.stdout:1/704: readlink d3/db/ldd 0 2024-02-26T10:48:02.975 INFO:tasks.workunit.client.0.smithi060.stdout:1/705: write d3/d54/f7c [3971436,28378] 0 2024-02-26T10:48:02.978 INFO:tasks.workunit.client.0.smithi060.stdout:2/618: write dd/d43/d60/d41/d37/d4c/d76/f84 [292606,102533] 0 2024-02-26T10:48:02.983 INFO:tasks.workunit.client.0.smithi060.stdout:5/631: dread d0/de/d20/f24 [0,4194304] 0 2024-02-26T10:48:02.983 INFO:tasks.workunit.client.0.smithi060.stdout:5/632: truncate d0/de/d20/f24 4836723 0 2024-02-26T10:48:02.985 INFO:tasks.workunit.client.0.smithi060.stdout:0/613: dwrite d0/dd/d39/f7e [0,4194304] 0 2024-02-26T10:48:02.986 INFO:tasks.workunit.client.0.smithi060.stdout:5/633: creat d0/de/d20/d32/d4c/dd2/fde x:0 0 0 2024-02-26T10:48:02.987 INFO:tasks.workunit.client.0.smithi060.stdout:5/634: symlink d0/de/d70/d84/ldf 0 2024-02-26T10:48:02.989 INFO:tasks.workunit.client.0.smithi060.stdout:5/635: rename d0/dc0/ldc to d0/de/d15/d65/d86/le0 0 2024-02-26T10:48:02.990 INFO:tasks.workunit.client.0.smithi060.stdout:0/614: truncate d0/dd/f77 4161397 0 2024-02-26T10:48:02.990 INFO:tasks.workunit.client.0.smithi060.stdout:0/615: readlink d0/dd/d39/d40/d83/lc3 0 2024-02-26T10:48:02.991 INFO:tasks.workunit.client.0.smithi060.stdout:5/636: mknod d0/de/d15/d65/d86/ce1 0 2024-02-26T10:48:02.993 INFO:tasks.workunit.client.0.smithi060.stdout:0/616: rename d0/dd/c2a to d0/dd/d39/d40/d83/cce 0 2024-02-26T10:48:03.012 INFO:tasks.workunit.client.0.smithi060.stdout:9/681: sync 2024-02-26T10:48:03.017 INFO:tasks.workunit.client.0.smithi060.stdout:9/682: mknod d3/d1e/d5e/d35/d87/d9b/d8a/cec 0 2024-02-26T10:48:03.017 INFO:tasks.workunit.client.0.smithi060.stdout:1/706: dread d3/f5c [0,4194304] 0 2024-02-26T10:48:03.024 INFO:tasks.workunit.client.0.smithi060.stdout:3/500: dwrite d5/f4a [4194304,4194304] 0 2024-02-26T10:48:03.031 INFO:tasks.workunit.client.0.smithi060.stdout:1/707: rename d3/d35/d53/c6a to d3/d35/d53/d68/d90/cf5 0 2024-02-26T10:48:03.040 INFO:tasks.workunit.client.0.smithi060.stdout:6/675: dwrite d6/f40 [0,4194304] 0 2024-02-26T10:48:03.040 INFO:tasks.workunit.client.0.smithi060.stdout:0/617: read d0/f55 [819961,59108] 0 2024-02-26T10:48:03.040 INFO:tasks.workunit.client.0.smithi060.stdout:0/618: chown d0/dd/fa8 7 1 2024-02-26T10:48:03.040 INFO:tasks.workunit.client.0.smithi060.stdout:3/501: creat d5/de/df/d11/d63/d93/f96 x:0 0 0 2024-02-26T10:48:03.041 INFO:tasks.workunit.client.0.smithi060.stdout:4/666: dwrite d0/d27/d76/f93 [0,4194304] 0 2024-02-26T10:48:03.042 INFO:tasks.workunit.client.0.smithi060.stdout:0/619: mknod d0/dd/d39/ccf 0 2024-02-26T10:48:03.043 INFO:tasks.workunit.client.0.smithi060.stdout:1/708: rename d3/db/d1e/d39/la9 to d3/db/d1e/lf6 0 2024-02-26T10:48:03.043 INFO:tasks.workunit.client.0.smithi060.stdout:3/502: mkdir d5/de/df/d11/d63/d93/d97 0 2024-02-26T10:48:03.046 INFO:tasks.workunit.client.0.smithi060.stdout:4/667: getdents d0/d27/d52 0 2024-02-26T10:48:03.046 INFO:tasks.workunit.client.0.smithi060.stdout:0/620: creat d0/d42/fd0 x:0 0 0 2024-02-26T10:48:03.046 INFO:tasks.workunit.client.0.smithi060.stdout:1/709: stat d3/db/dc8/dbe/cc5 0 2024-02-26T10:48:03.047 INFO:tasks.workunit.client.0.smithi060.stdout:0/621: creat d0/dd/fd1 x:0 0 0 2024-02-26T10:48:03.047 INFO:tasks.workunit.client.0.smithi060.stdout:0/622: stat d0/dd/d39/d40/c2e 0 2024-02-26T10:48:03.050 INFO:tasks.workunit.client.0.smithi060.stdout:1/710: rmdir d3/db/d2e/d51/d63 39 2024-02-26T10:48:03.053 INFO:tasks.workunit.client.0.smithi060.stdout:2/619: dwrite dd/d43/d60/d41/d37/d9a/fb7 [0,4194304] 0 2024-02-26T10:48:03.053 INFO:tasks.workunit.client.0.smithi060.stdout:2/620: fdatasync dd/d43/d60/d41/d42/fb2 0 2024-02-26T10:48:03.054 INFO:tasks.workunit.client.0.smithi060.stdout:8/636: dwrite d9/d15/d3e/fb2 [0,4194304] 0 2024-02-26T10:48:03.054 INFO:tasks.workunit.client.0.smithi060.stdout:2/621: mkdir dd/d43/dd2 0 2024-02-26T10:48:03.054 INFO:tasks.workunit.client.0.smithi060.stdout:0/623: getdents d0/dd/d53/d87/d94 0 2024-02-26T10:48:03.054 INFO:tasks.workunit.client.0.smithi060.stdout:2/622: fdatasync dd/d43/d60/d41/d37/d4c/d76/fa3 0 2024-02-26T10:48:03.058 INFO:tasks.workunit.client.0.smithi060.stdout:0/624: creat d0/dd/d53/d87/fd2 x:0 0 0 2024-02-26T10:48:03.060 INFO:tasks.workunit.client.0.smithi060.stdout:0/625: write d0/d42/fd0 [13183,130542] 0 2024-02-26T10:48:03.062 INFO:tasks.workunit.client.0.smithi060.stdout:8/637: rename d9/d15/d29/d2f/l32 to d9/d15/d3e/lcb 0 2024-02-26T10:48:03.067 INFO:tasks.workunit.client.0.smithi060.stdout:2/623: rename dd/d43/d60/d41/d42 to dd/d43/d55/d6e/dd3 0 2024-02-26T10:48:03.068 INFO:tasks.workunit.client.0.smithi060.stdout:2/624: creat dd/d43/d55/d5e/fd4 x:0 0 0 2024-02-26T10:48:03.069 INFO:tasks.workunit.client.0.smithi060.stdout:8/638: unlink d9/d6f/c85 0 2024-02-26T10:48:03.073 INFO:tasks.workunit.client.0.smithi060.stdout:8/639: creat d9/d5a/fcc x:0 0 0 2024-02-26T10:48:03.073 INFO:tasks.workunit.client.0.smithi060.stdout:5/637: dwrite d0/de/fbf [0,4194304] 0 2024-02-26T10:48:03.073 INFO:tasks.workunit.client.0.smithi060.stdout:5/638: write d0/de/d15/f2f [1454595,56481] 0 2024-02-26T10:48:03.073 INFO:tasks.workunit.client.0.smithi060.stdout:9/683: dwrite d3/d1e/d5e/d1b/d23/f36 [0,4194304] 0 2024-02-26T10:48:03.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:02 smithi113 ceph-mon[36126]: pgmap v487: 129 pgs: 129 active+clean; 972 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 146 MiB/s wr, 804 op/s 2024-02-26T10:48:03.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:02 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3443968176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:48:03.097 INFO:tasks.workunit.client.0.smithi060.stdout:8/640: rmdir d9/d15/d29 39 2024-02-26T10:48:03.099 INFO:tasks.workunit.client.0.smithi060.stdout:8/641: creat d9/d5a/fcd x:0 0 0 2024-02-26T10:48:03.100 INFO:tasks.workunit.client.0.smithi060.stdout:9/684: unlink d3/d1e/dbb/dc2/fc3 0 2024-02-26T10:48:03.102 INFO:tasks.workunit.client.0.smithi060.stdout:9/685: rename d3/d1e/d5e/ca1 to d3/d1e/d5e/d35/d87/ced 0 2024-02-26T10:48:03.102 INFO:tasks.workunit.client.0.smithi060.stdout:9/686: chown d3/d1e/d5e/d35/d87/lca 23818 1 2024-02-26T10:48:03.103 INFO:tasks.workunit.client.0.smithi060.stdout:1/711: dwrite d3/d54/f7c [0,4194304] 0 2024-02-26T10:48:03.105 INFO:tasks.workunit.client.0.smithi060.stdout:1/712: mkdir d3/db/d1e/d39/d9b/df7 0 2024-02-26T10:48:03.105 INFO:tasks.workunit.client.0.smithi060.stdout:1/713: dread - d3/db/d2e/d51/d63/f92 zero size 2024-02-26T10:48:03.106 INFO:tasks.workunit.client.0.smithi060.stdout:9/687: symlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/lee 0 2024-02-26T10:48:03.107 INFO:tasks.workunit.client.0.smithi060.stdout:1/714: creat d3/d35/d53/ff8 x:0 0 0 2024-02-26T10:48:03.108 INFO:tasks.workunit.client.0.smithi060.stdout:9/688: rmdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8 39 2024-02-26T10:48:03.110 INFO:tasks.workunit.client.0.smithi060.stdout:1/715: symlink d3/d35/d5e/lf9 0 2024-02-26T10:48:03.112 INFO:tasks.workunit.client.0.smithi060.stdout:1/716: creat d3/d35/d53/ffa x:0 0 0 2024-02-26T10:48:03.113 INFO:tasks.workunit.client.0.smithi060.stdout:9/689: unlink d3/d1e/d93/da2/db7/fcf 0 2024-02-26T10:48:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:02 smithi060 ceph-mon[31539]: pgmap v487: 129 pgs: 129 active+clean; 972 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 146 MiB/s wr, 804 op/s 2024-02-26T10:48:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:02 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3443968176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:48:03.114 INFO:tasks.workunit.client.0.smithi060.stdout:1/717: symlink d3/d35/d53/d68/d85/lfb 0 2024-02-26T10:48:03.114 INFO:tasks.workunit.client.0.smithi060.stdout:1/718: chown d3/db/d1e/d6d/dd9 6898349 1 2024-02-26T10:48:03.114 INFO:tasks.workunit.client.0.smithi060.stdout:9/690: readlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/l14 0 2024-02-26T10:48:03.121 INFO:tasks.workunit.client.0.smithi060.stdout:1/719: rename d3/db/d1e/c2f to d3/db/d1e/d38/cfc 0 2024-02-26T10:48:03.121 INFO:tasks.workunit.client.0.smithi060.stdout:1/720: write d3/d35/d53/d57/f71 [2450935,65779] 0 2024-02-26T10:48:03.121 INFO:tasks.workunit.client.0.smithi060.stdout:1/721: stat d3/d35/d53/d68/d85/f89 0 2024-02-26T10:48:03.122 INFO:tasks.workunit.client.0.smithi060.stdout:9/691: creat d3/d1e/d5e/d35/deb/fef x:0 0 0 2024-02-26T10:48:03.122 INFO:tasks.workunit.client.0.smithi060.stdout:1/722: readlink d3/db/d1e/d38/d6c/l7b 0 2024-02-26T10:48:03.122 INFO:tasks.workunit.client.0.smithi060.stdout:1/723: chown d3/db/d1e/d39/f80 645754 1 2024-02-26T10:48:03.122 INFO:tasks.workunit.client.0.smithi060.stdout:1/724: fdatasync d3/db/d1e/d38/f79 0 2024-02-26T10:48:03.122 INFO:tasks.workunit.client.0.smithi060.stdout:1/725: chown d3/d35/d5e/lf9 3887 1 2024-02-26T10:48:03.122 INFO:tasks.workunit.client.0.smithi060.stdout:1/726: chown d3/db/d1e/d6d/da3/daa/dd6 49 1 2024-02-26T10:48:03.122 INFO:tasks.workunit.client.0.smithi060.stdout:2/625: dwrite dd/fcc [0,4194304] 0 2024-02-26T10:48:03.122 INFO:tasks.workunit.client.0.smithi060.stdout:2/626: chown dd/d43/d60/d41/d37/d38/l61 47556320 1 2024-02-26T10:48:03.122 INFO:tasks.workunit.client.0.smithi060.stdout:1/727: truncate d3/db/d1e/d39/f58 1500905 0 2024-02-26T10:48:03.123 INFO:tasks.workunit.client.0.smithi060.stdout:1/728: fsync d3/d35/fa5 0 2024-02-26T10:48:03.123 INFO:tasks.workunit.client.0.smithi060.stdout:9/692: link d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/f8f d3/d1e/d5e/d35/d87/d9b/d8a/dbe/ff0 0 2024-02-26T10:48:03.123 INFO:tasks.workunit.client.0.smithi060.stdout:2/627: creat dd/d43/d55/d6e/fd5 x:0 0 0 2024-02-26T10:48:03.123 INFO:tasks.workunit.client.0.smithi060.stdout:2/628: chown f4 1505228595 1 2024-02-26T10:48:03.123 INFO:tasks.workunit.client.0.smithi060.stdout:9/693: symlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/de8/lf1 0 2024-02-26T10:48:03.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:02 smithi100 ceph-mon[36051]: pgmap v487: 129 pgs: 129 active+clean; 972 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 146 MiB/s wr, 804 op/s 2024-02-26T10:48:03.125 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:02 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3443968176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:48:03.136 INFO:tasks.workunit.client.0.smithi060.stdout:8/642: link d9/d15/d29/d2f/d34/d72/d50/d6a/l47 d9/d15/lce 0 2024-02-26T10:48:03.136 INFO:tasks.workunit.client.0.smithi060.stdout:8/643: dread - d9/d15/d29/d2f/fc9 zero size 2024-02-26T10:48:03.136 INFO:tasks.workunit.client.0.smithi060.stdout:9/694: truncate d3/d1e/f58 914489 0 2024-02-26T10:48:03.136 INFO:tasks.workunit.client.0.smithi060.stdout:6/676: dwrite f4 [0,4194304] 0 2024-02-26T10:48:03.136 INFO:tasks.workunit.client.0.smithi060.stdout:6/677: dread - d6/d7/d25/d1c/d1f/d23/d2b/d67/dc0/fd3 zero size 2024-02-26T10:48:03.136 INFO:tasks.workunit.client.0.smithi060.stdout:9/695: link d3/d1e/ce7 d3/d1e/d5e/dc9/cf2 0 2024-02-26T10:48:03.140 INFO:tasks.workunit.client.0.smithi060.stdout:9/696: creat d3/d1e/d57/ff3 x:0 0 0 2024-02-26T10:48:03.146 INFO:tasks.workunit.client.0.smithi060.stdout:7/782: sync 2024-02-26T10:48:03.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.159+0000 7f8ec1c00640 1 -- 172.21.15.60:0/477211356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 msgr2=0x7f8ebc104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:03.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.159+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/477211356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 0x7f8ebc104b10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009860 tx=0x7f8ea802bb90 comp rx=0 tx=0).stop 2024-02-26T10:48:03.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.160+0000 7f8ec1c00640 1 -- 172.21.15.60:0/477211356 shutdown_connections 2024-02-26T10:48:03.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.160+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/477211356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc105050 0x7f8ebc10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.160+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/477211356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 0x7f8ebc104b10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.160+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/477211356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ebc1034a0 0x7f8ebc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.160+0000 7f8ec1c00640 1 -- 172.21.15.60:0/477211356 >> 172.21.15.60:0/477211356 conn(0x7f8ebc0fea10 msgr2=0x7f8ebc100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:03.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.160+0000 7f8ec1c00640 1 -- 172.21.15.60:0/477211356 shutdown_connections 2024-02-26T10:48:03.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.160+0000 7f8ec1c00640 1 -- 172.21.15.60:0/477211356 wait complete. 2024-02-26T10:48:03.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.161+0000 7f8ec1c00640 1 Processor -- start 2024-02-26T10:48:03.155 INFO:tasks.workunit.client.0.smithi060.stdout:2/629: dread dd/d43/d60/d41/f31 [0,4194304] 0 2024-02-26T10:48:03.155 INFO:tasks.workunit.client.0.smithi060.stdout:2/630: fdatasync dd/d43/d60/f53 0 2024-02-26T10:48:03.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.161+0000 7f8ec1c00640 1 -- start start 2024-02-26T10:48:03.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8ec1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc1034a0 0x7f8ebc199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8ec1c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 0x7f8ebc199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8ec1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ebc105050 0x7f8ebc1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8ec1c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc10f130 con 0x7f8ebc104690 2024-02-26T10:48:03.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8ec1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc10efb0 con 0x7f8ebc1034a0 2024-02-26T10:48:03.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8ec1c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc10f2b0 con 0x7f8ebc105050 2024-02-26T10:48:03.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8eba000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 0x7f8ebc199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8eba000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 0x7f8ebc199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46340/0 (socket says 172.21.15.60:46340) 2024-02-26T10:48:03.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8eba000640 1 -- 172.21.15.60:0/3398692482 learned_addr learned my addr 172.21.15.60:0/3398692482 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:03.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.162+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ebc105050 0x7f8ebc1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.164+0000 7f8ebaa00640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc1034a0 0x7f8ebc199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.164+0000 7f8eba000640 1 -- 172.21.15.60:0/3398692482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ebc105050 msgr2=0x7f8ebc1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:03.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.164+0000 7f8eba000640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ebc105050 0x7f8ebc1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.164+0000 7f8eba000640 1 -- 172.21.15.60:0/3398692482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc1034a0 msgr2=0x7f8ebc199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:03.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.164+0000 7f8eba000640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc1034a0 0x7f8ebc199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.164+0000 7f8eba000640 1 -- 172.21.15.60:0/3398692482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc1a13a0 con 0x7f8ebc104690 2024-02-26T10:48:03.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.165+0000 7f8eba000640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 0x7f8ebc199cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8034980 tx=0x7f8ea80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:03.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.167+0000 7f8eb3400640 1 -- 172.21.15.60:0/3398692482 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8041020 con 0x7f8ebc104690 2024-02-26T10:48:03.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.167+0000 7f8eb3400640 1 -- 172.21.15.60:0/3398692482 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ea8034720 con 0x7f8ebc104690 2024-02-26T10:48:03.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.167+0000 7f8eb3400640 1 -- 172.21.15.60:0/3398692482 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8039040 con 0x7f8ebc104690 2024-02-26T10:48:03.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.167+0000 7f8ec1c00640 1 -- 172.21.15.60:0/3398692482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc1a1630 con 0x7f8ebc104690 2024-02-26T10:48:03.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.167+0000 7f8ec1c00640 1 -- 172.21.15.60:0/3398692482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc1a1ad0 con 0x7f8ebc104690 2024-02-26T10:48:03.161 INFO:tasks.workunit.client.0.smithi060.stdout:0/626: dwrite d0/dd/d39/d63/f67 [0,4194304] 0 2024-02-26T10:48:03.161 INFO:tasks.workunit.client.0.smithi060.stdout:8/644: dread d9/f13 [0,4194304] 0 2024-02-26T10:48:03.161 INFO:tasks.workunit.client.0.smithi060.stdout:0/627: dread - d0/dd/d53/d87/fd2 zero size 2024-02-26T10:48:03.162 INFO:tasks.workunit.client.0.smithi060.stdout:0/628: readlink d0/dd/l13 0 2024-02-26T10:48:03.167 INFO:tasks.workunit.client.0.smithi060.stdout:7/783: link da/d94/f98 da/df1/dd7/de0/f104 0 2024-02-26T10:48:03.167 INFO:tasks.workunit.client.0.smithi060.stdout:7/784: fdatasync da/d15/d21/d2f/d37/ff9 0 2024-02-26T10:48:03.167 INFO:tasks.workunit.client.0.smithi060.stdout:7/785: truncate da/d66/dbd/f102 1952254 0 2024-02-26T10:48:03.167 INFO:tasks.workunit.client.0.smithi060.stdout:4/668: dwrite d0/d27/d76/f93 [0,4194304] 0 2024-02-26T10:48:03.167 INFO:tasks.workunit.client.0.smithi060.stdout:9/697: dread d3/d1e/d57/f79 [0,4194304] 0 2024-02-26T10:48:03.167 INFO:tasks.workunit.client.0.smithi060.stdout:4/669: truncate d0/d27/d76/dbd/d8e/fb4 185834 0 2024-02-26T10:48:03.167 INFO:tasks.workunit.client.0.smithi060.stdout:9/698: dread - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/d98/fe3 zero size 2024-02-26T10:48:03.167 INFO:tasks.workunit.client.0.smithi060.stdout:9/699: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/ff4 x:0 0 0 2024-02-26T10:48:03.171 INFO:tasks.workunit.client.0.smithi060.stdout:0/629: rename d0/dd/f5d to d0/dd/d39/d40/d83/dc6/d58/fd3 0 2024-02-26T10:48:03.171 INFO:tasks.workunit.client.0.smithi060.stdout:0/630: creat d0/dd/d53/d87/fd4 x:0 0 0 2024-02-26T10:48:03.171 INFO:tasks.workunit.client.0.smithi060.stdout:3/503: dwrite d5/f24 [0,4194304] 0 2024-02-26T10:48:03.171 INFO:tasks.workunit.client.0.smithi060.stdout:5/639: dread d0/de/d15/d26/f78 [0,4194304] 0 2024-02-26T10:48:03.171 INFO:tasks.workunit.client.0.smithi060.stdout:1/729: dwrite d3/d35/d5e/fe4 [0,4194304] 0 2024-02-26T10:48:03.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.174+0000 7f8eb3400640 1 -- 172.21.15.60:0/3398692482 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8ea8044020 con 0x7f8ebc104690 2024-02-26T10:48:03.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.175+0000 7f8ec4a45740 1 Processor -- start 2024-02-26T10:48:03.172 INFO:tasks.workunit.client.0.smithi060.stdout:3/504: mkdir d5/de/df/d11/d63/d93/d97/d98 0 2024-02-26T10:48:03.172 INFO:tasks.workunit.client.0.smithi060.stdout:0/631: symlink d0/dd/d53/d87/d94/db7/ld5 0 2024-02-26T10:48:03.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.178+0000 7f8ec4a45740 1 -- start start 2024-02-26T10:48:03.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.178+0000 7f8eb3400640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ea00a1f50 0x7f8ea00a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.178+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 0x55913dacb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.178+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913dad0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.178+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913dad3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.178+0000 7f8ec4a45740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913da27d30 con 0x55913dacb480 2024-02-26T10:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.178+0000 7f8ec4a45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913d9d0cc0 con 0x55913dad0fe0 2024-02-26T10:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.178+0000 7f8ec4a45740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913da23dd0 con 0x55913dacbe70 2024-02-26T10:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.179+0000 7f8eb3400640 1 -- 172.21.15.60:0/3398692482 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f8ea80efb10 con 0x7f8ebc104690 2024-02-26T10:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.180+0000 7f8eba000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913dad0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.180+0000 7f8eba000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913dad0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:33108/0 (socket says 172.21.15.60:33108) 2024-02-26T10:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.180+0000 7f8eba000640 1 -- 172.21.15.60:0/3887200473 learned_addr learned my addr 172.21.15.60:0/3887200473 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.180+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3887200473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913dad3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.180+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ea00a1f50 0x7f8ea00a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.183 INFO:tasks.workunit.client.0.smithi060.stdout:3/505: symlink d5/de/df/d4c/l99 0 2024-02-26T10:48:03.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.182+0000 7f8ebb400640 1 -- 172.21.15.60:0/3887200473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 msgr2=0x55913dad0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.182+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3887200473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913dad0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.182+0000 7f8ebb400640 1 -- 172.21.15.60:0/3887200473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 msgr2=0x55913dacb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.182+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3887200473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 0x55913dacb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.182+0000 7f8ebb400640 1 -- 172.21.15.60:0/3887200473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55913d73c3f0 con 0x55913dad0fe0 2024-02-26T10:48:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.190+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ea00a1f50 0x7f8ea00a4410 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4005fd0 tx=0x7f8eb4009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.190+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3887200473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913dad3470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4012fd0 tx=0x7f8eb401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d2112e5383bfae8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.191+0000 7f8ea7e00640 1 -- 172.21.15.60:0/3887200473 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400c800 con 0x55913dad0fe0 2024-02-26T10:48:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.191+0000 7f8ea7e00640 1 -- 172.21.15.60:0/3887200473 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eb40203f0 con 0x55913dad0fe0 2024-02-26T10:48:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.191+0000 7f8ea7e00640 1 -- 172.21.15.60:0/3887200473 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb401e5f0 con 0x55913dad0fe0 2024-02-26T10:48:03.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.192+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3887200473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 msgr2=0x55913dad3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.192+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3887200473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913dad3470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4012fd0 tx=0x7f8eb401c000 comp rx=0 tx=0).stop 2024-02-26T10:48:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.194+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3887200473 shutdown_connections 2024-02-26T10:48:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.194+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3887200473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913dad3470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.194+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3887200473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913dad0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.194+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3887200473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 0x55913dacb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.194+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3887200473 >> 172.21.15.60:0/3887200473 conn(0x55913dac69f0 msgr2=0x55913dac6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.194+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3887200473 shutdown_connections 2024-02-26T10:48:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.194+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3887200473 wait complete. 2024-02-26T10:48:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.195+0000 7f8ec4a45740 1 -- start start 2024-02-26T10:48:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.195+0000 7f8ec4a45740 1 Processor -- start 2024-02-26T10:48:03.190 INFO:tasks.workunit.client.0.smithi060.stdout:8/645: dread d9/d15/d29/d2f/d34/d72/f42 [0,4194304] 0 2024-02-26T10:48:03.190 INFO:tasks.workunit.client.0.smithi060.stdout:0/632: write d0/dd/d39/f7c [450230,46825] 0 2024-02-26T10:48:03.190 INFO:tasks.workunit.client.0.smithi060.stdout:0/633: getdents d0/dd/d39/d40/d83/dc6 0 2024-02-26T10:48:03.190 INFO:tasks.workunit.client.0.smithi060.stdout:0/634: chown d0/dd/l71 41925 1 2024-02-26T10:48:03.190 INFO:tasks.workunit.client.0.smithi060.stdout:0/635: readlink d0/d42/d7a/d7f/lac 0 2024-02-26T10:48:03.190 INFO:tasks.workunit.client.0.smithi060.stdout:0/636: chown d0/dd/d39/d40/f19 0 1 2024-02-26T10:48:03.191 INFO:tasks.workunit.client.0.smithi060.stdout:8/646: symlink d9/d15/d29/lcf 0 2024-02-26T10:48:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.197+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacb480 0x55913dae69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913daedf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8ec4a45740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913dae4f40 con 0x55913dacbe70 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8ec4a45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913d9c61f0 con 0x55913dad0fe0 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8ec4a45740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913dae50c0 con 0x55913dacb480 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8ec4a45740 10 client.0 _ll_register_callbacks cb 0x55913d73bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8eba000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8eba000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46358/0 (socket says 172.21.15.60:46358) 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8eba000640 1 -- 172.21.15.60:0/1654822327 learned_addr learned my addr 172.21.15.60:0/1654822327 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8eba000640 1 -- 172.21.15.60:0/1654822327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacb480 msgr2=0x55913dae69d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8eba000640 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacb480 0x55913dae69d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.198+0000 7f8eba000640 1 -- 172.21.15.60:0/1654822327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 msgr2=0x55913daedf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.199+0000 7f8eba000640 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913daedf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.199+0000 7f8eba000640 1 -- 172.21.15.60:0/1654822327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55913d8067d0 con 0x55913dacbe70 2024-02-26T10:48:03.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.199+0000 7f8eba000640 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8000c00 tx=0x7f8ea80bbd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62bd78817f9ad5d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.199+0000 7f8ea4c00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:48:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.200+0000 7f8ea4c00640 1 -- 172.21.15.60:0/1654822327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea803f0d0 con 0x55913dacbe70 2024-02-26T10:48:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.200+0000 7f8ea4c00640 1 -- 172.21.15.60:0/1654822327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f8ea80ed310 con 0x55913dacbe70 2024-02-26T10:48:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.200+0000 7f8ec4a45740 1 -- 172.21.15.60:0/1654822327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eb40203f0 con 0x55913dacbe70 2024-02-26T10:48:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.201+0000 7f8ea4c00640 1 -- 172.21.15.60:0/1654822327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ea80b86d0 con 0x55913dacbe70 2024-02-26T10:48:03.196 INFO:tasks.workunit.client.0.smithi060.stdout:0/637: rename d0/dd/d33/d6a to d0/dd/d39/d40/dad/dd6 0 2024-02-26T10:48:03.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.203+0000 7f8ec4a45740 10 client.15960 fetch_fsmap learned FSMap version 29 2024-02-26T10:48:03.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.203+0000 7f8ec4a45740 1 -- 172.21.15.60:0/1654822327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55913daf12c0 con 0x55913dacbe70 2024-02-26T10:48:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.204+0000 7f8ea4c00640 1 -- 172.21.15.60:0/1654822327 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eb40203f0 con 0x55913dacbe70 2024-02-26T10:48:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.205+0000 7f8ea4c00640 1 -- 172.21.15.60:0/1654822327 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8ea803c030 con 0x55913dacbe70 2024-02-26T10:48:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.205+0000 7f8ec4a45740 10 client.15960 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:48:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.205+0000 7f8ec4a45740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.205+0000 7f8ec4a45740 20 client.15960 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.206+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55913daf2a20 0x55913daf4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.206+0000 7f8ec4a45740 4 client.15960 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:48:03.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.206+0000 7f8ec4a45740 1 -- 172.21.15.60:0/1654822327 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55913da35270 con 0x55913daf2a20 2024-02-26T10:48:03.200 INFO:tasks.workunit.client.0.smithi060.stdout:8/647: truncate d9/d15/d29/d2f/d34/d72/d94/f9e 2414302 0 2024-02-26T10:48:03.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.206+0000 7f8ebaa00640 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55913daf2a20 0x55913daf4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.208+0000 7f8ebaa00640 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55913daf2a20 0x55913daf4e80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c21c602aa16ce362 server_cookie=6eacb0e8dc74181f in_seq=0 out_seq=0 2024-02-26T10:48:03.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.208+0000 7f8ea4c00640 10 client.15960 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:03.202 INFO:tasks.workunit.client.0.smithi060.stdout:8/648: chown d9/c17 1107 1 2024-02-26T10:48:03.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.209+0000 7f8ea4c00640 1 -- 172.21.15.60:0/1654822327 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55913da35270 con 0x55913daf2a20 2024-02-26T10:48:03.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.209+0000 7f8ea4c00640 10 client.15960 handle_command_reply: tid=0 2024-02-26T10:48:03.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.209+0000 7f8ec4a45740 1 client.15960 shutdown 2024-02-26T10:48:03.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.209+0000 7f8ec4a45740 1 -- 172.21.15.60:0/1654822327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 msgr2=0x55913dae6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:03.204 INFO:tasks.workunit.client.0.smithi060.stdout:8/649: truncate d9/f20 6183715 0 2024-02-26T10:48:03.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.209+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8000c00 tx=0x7f8ea80bbd90 comp rx=0 tx=0).stop 2024-02-26T10:48:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.211+0000 7f8ec4a45740 1 -- 172.21.15.60:0/1654822327 shutdown_connections 2024-02-26T10:48:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.211+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55913daf2a20 0x55913daf4e80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.211+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913daedf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.211+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.211+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/1654822327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacb480 0x55913dae69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.211+0000 7f8ec4a45740 1 -- 172.21.15.60:0/1654822327 >> 172.21.15.60:0/1654822327 conn(0x55913dac69f0 msgr2=0x55913dabd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:03.206 INFO:tasks.workunit.client.0.smithi060.stdout:8/650: rmdir d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9 39 2024-02-26T10:48:03.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.212+0000 7f8ea4c00640 0 client.15960 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.212+0000 7f8ec4a45740 1 -- 172.21.15.60:0/1654822327 shutdown_connections 2024-02-26T10:48:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.212+0000 7f8ec4a45740 1 -- 172.21.15.60:0/1654822327 wait complete. 2024-02-26T10:48:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.212+0000 7f8ec4a45740 20 client.15960 trim_cache size 0 max 16384 2024-02-26T10:48:03.208 INFO:tasks.workunit.client.0.smithi060.stdout:8/651: symlink d9/d5a/d58/ld0 0 2024-02-26T10:48:03.209 INFO:tasks.workunit.client.0.smithi060.stdout:8/652: chown d9/d15/d29/d2f/d34/d72/d50/d6a/f46 377602 1 2024-02-26T10:48:03.209 INFO:tasks.workunit.client.0.smithi060.stdout:8/653: creat d9/d15/d29/d2f/fd1 x:0 0 0 2024-02-26T10:48:03.210 INFO:tasks.workunit.client.0.smithi060.stdout:8/654: fsync d9/d15/d29/d2f/d34/d72/d50/d6a/fb4 0 2024-02-26T10:48:03.212 INFO:tasks.workunit.client.0.smithi060.stdout:2/631: dread dd/d43/d55/f5a [0,4194304] 0 2024-02-26T10:48:03.212 INFO:tasks.workunit.client.0.smithi060.stdout:2/632: fdatasync dd/d11/f44 0 2024-02-26T10:48:03.225 INFO:tasks.workunit.client.0.smithi060.stdout:6/678: dwrite d6/d7/d25/f5c [0,4194304] 0 2024-02-26T10:48:03.226 INFO:tasks.workunit.client.0.smithi060.stdout:2/633: rmdir dd/d43/d55/d5e/d74/da5 39 2024-02-26T10:48:03.227 INFO:tasks.workunit.client.0.smithi060.stdout:2/634: chown f4 1661 1 2024-02-26T10:48:03.227 INFO:tasks.workunit.client.0.smithi060.stdout:2/635: readlink l9 0 2024-02-26T10:48:03.227 INFO:tasks.workunit.client.0.smithi060.stdout:2/636: fdatasync dd/d43/d55/d5e/d74/da5/fa7 0 2024-02-26T10:48:03.227 INFO:tasks.workunit.client.0.smithi060.stdout:2/637: readlink dd/d43/d60/d41/d37/d4c/lab 0 2024-02-26T10:48:03.232 INFO:tasks.workunit.client.0.smithi060.stdout:8/655: dread d9/d5a/f69 [0,4194304] 0 2024-02-26T10:48:03.235 INFO:tasks.workunit.client.0.smithi060.stdout:8/656: stat d9/d15/d3e/fac 0 2024-02-26T10:48:03.240 INFO:tasks.workunit.client.0.smithi060.stdout:5/640: dwrite d0/d33/f5f [0,4194304] 0 2024-02-26T10:48:03.248 INFO:tasks.workunit.client.0.smithi060.stdout:8/657: link d9/d15/d29/f64 d9/d15/d29/d2f/d34/d72/d7e/fd2 0 2024-02-26T10:48:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.256+0000 7f8ec4a45740 1 Processor -- start 2024-02-26T10:48:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.256+0000 7f8ec4a45740 1 -- start start 2024-02-26T10:48:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.257+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dacb480 0x55913dae69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.257+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.257+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dad0fe0 0x55913daedf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.257+0000 7f8ec4a45740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc072f70 con 0x55913dacbe70 2024-02-26T10:48:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.258+0000 7f8ebaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dacb480 0x55913dae69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.258+0000 7f8ebaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dacb480 0x55913dae69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51212/0 (socket says 172.21.15.60:51212) 2024-02-26T10:48:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.258+0000 7f8ebaa00640 1 -- 172.21.15.60:0/788478481 learned_addr learned my addr 172.21.15.60:0/788478481 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.258+0000 7f8ebb400640 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dad0fe0 0x55913daedf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.258+0000 7f8ec4a45740 1 -- 172.21.15.60:0/788478481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc070ab0 con 0x55913dacb480 2024-02-26T10:48:03.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.258+0000 7f8ec4a45740 1 -- 172.21.15.60:0/788478481 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc06eb50 con 0x55913dad0fe0 2024-02-26T10:48:03.253 INFO:tasks.workunit.client.0.smithi060.stdout:5/641: read d0/de/d15/d65/fa0 [1937,76552] 0 2024-02-26T10:48:03.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.259+0000 7f8eba000640 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.259+0000 7f8ebb400640 1 -- 172.21.15.60:0/788478481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dacb480 msgr2=0x55913dae69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.259+0000 7f8ebb400640 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dacb480 0x55913dae69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.259+0000 7f8ebb400640 1 -- 172.21.15.60:0/788478481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 msgr2=0x55913dae6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.259+0000 7f8ebb400640 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.254 INFO:tasks.workunit.client.0.smithi060.stdout:5/642: creat d0/de/d15/d65/fe2 x:0 0 0 2024-02-26T10:48:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.260+0000 7f8ebb400640 1 -- 172.21.15.60:0/788478481 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55913dae5240 con 0x55913dad0fe0 2024-02-26T10:48:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.260+0000 7f8eba000640 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.260+0000 7f8ebaa00640 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dacb480 0x55913dae69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.260+0000 7f8ebb400640 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dad0fe0 0x55913daedf10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4025000 tx=0x7f8eb4003c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f23cd7f5281c966e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:03.256 INFO:tasks.workunit.client.0.smithi060.stdout:5/643: creat d0/d33/fe3 x:0 0 0 2024-02-26T10:48:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.261+0000 7f8ea7400640 1 -- 172.21.15.60:0/788478481 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400ca00 con 0x55913dad0fe0 2024-02-26T10:48:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.261+0000 7f8ea7400640 1 -- 172.21.15.60:0/788478481 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eb4005bd0 con 0x55913dad0fe0 2024-02-26T10:48:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.262+0000 7f8ec4a45740 1 -- 172.21.15.60:0/788478481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dad0fe0 msgr2=0x55913daedf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.262+0000 7f8ea7400640 1 -- 172.21.15.60:0/788478481 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400ca00 con 0x55913dad0fe0 2024-02-26T10:48:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.262+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dad0fe0 0x55913daedf10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4025000 tx=0x7f8eb4003c70 comp rx=0 tx=0).stop 2024-02-26T10:48:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.264+0000 7f8ec4a45740 1 -- 172.21.15.60:0/788478481 shutdown_connections 2024-02-26T10:48:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.264+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dad0fe0 0x55913daedf10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.264+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacbe70 0x55913dae6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.264+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/788478481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dacb480 0x55913dae69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.264+0000 7f8ec4a45740 1 -- 172.21.15.60:0/788478481 >> 172.21.15.60:0/788478481 conn(0x55913dac69f0 msgr2=0x55913daf9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.264+0000 7f8ec4a45740 1 -- 172.21.15.60:0/788478481 shutdown_connections 2024-02-26T10:48:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.264+0000 7f8ec4a45740 1 -- 172.21.15.60:0/788478481 wait complete. 2024-02-26T10:48:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.265+0000 7f8ec4a45740 1 -- start start 2024-02-26T10:48:03.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.265+0000 7f8ec4a45740 1 Processor -- start 2024-02-26T10:48:03.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.265+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 0x55913daedd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.265+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913daec480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ec4a45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913daec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913daec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913daec9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51224/0 (socket says 172.21.15.60:51224) 2024-02-26T10:48:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebb400640 1 -- 172.21.15.60:0/3034479057 learned_addr learned my addr 172.21.15.60:0/3034479057 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ec4a45740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc06f150 con 0x55913dacb480 2024-02-26T10:48:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc070450 con 0x55913dad0fe0 2024-02-26T10:48:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc06f790 con 0x55913dacbe70 2024-02-26T10:48:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebaa00640 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 0x55913daedd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ec4a45740 10 client.0 _ll_register_callbacks cb 0x7f8eb4012750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:03.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebb400640 1 -- 172.21.15.60:0/3034479057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 msgr2=0x55913daec480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:03.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8eba000640 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913daec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.261 INFO:tasks.workunit.client.0.smithi060.stdout:0/638: dwrite d0/dd/d39/d40/f19 [0,4194304] 0 2024-02-26T10:48:03.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913daec480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebb400640 1 -- 172.21.15.60:0/3034479057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 msgr2=0x55913daedd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:03.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 0x55913daedd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebb400640 1 -- 172.21.15.60:0/3034479057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55913daed0b0 con 0x55913dad0fe0 2024-02-26T10:48:03.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.266+0000 7f8ebaa00640 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 0x55913daedd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:48:03.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.267+0000 7f8eba000640 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913daec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:48:03.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.267+0000 7f8ebb400640 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913daec9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009830 tx=0x7f8eb400cfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba5d54d1a164c70f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:03.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.267+0000 7f8ea4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:03.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.267+0000 7f8ea4c00640 1 -- 172.21.15.60:0/3034479057 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb403d060 con 0x55913dad0fe0 2024-02-26T10:48:03.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.267+0000 7f8ea4c00640 1 -- 172.21.15.60:0/3034479057 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f8eb400ca10 con 0x55913dad0fe0 2024-02-26T10:48:03.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.267+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eb4005bd0 con 0x55913dad0fe0 2024-02-26T10:48:03.263 INFO:tasks.workunit.client.0.smithi060.stdout:0/639: creat d0/d42/fd7 x:0 0 0 2024-02-26T10:48:03.263 INFO:tasks.workunit.client.0.smithi060.stdout:0/640: chown d0/dd/d39/d40/d83/ca5 5743 1 2024-02-26T10:48:03.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ea4c00640 1 -- 172.21.15.60:0/3034479057 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eb4044050 con 0x55913dad0fe0 2024-02-26T10:48:03.264 INFO:tasks.workunit.client.0.smithi060.stdout:0/641: readlink d0/dd/d39/d40/d83/dc6/l8a 0 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ec4a45740 10 client.26105 fetch_fsmap learned FSMap version 29 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55913db49e50 con 0x55913dad0fe0 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ea4c00640 1 -- 172.21.15.60:0/3034479057 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eb4005bd0 con 0x55913dad0fe0 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ea4c00640 1 -- 172.21.15.60:0/3034479057 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8eb401f070 con 0x55913dad0fe0 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ec4a45740 10 client.26105 fetch_fsmap finished waiting for FSMap version 29 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ec4a45740 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ec4a45740 20 client.26105 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55913db4a5c0 0x55913db4c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ec4a45740 4 client.26105 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55913da2f5c0 con 0x55913db4a5c0 2024-02-26T10:48:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.268+0000 7f8ebaa00640 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55913db4a5c0 0x55913db4c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:03.266 INFO:tasks.workunit.client.0.smithi060.stdout:0/642: mkdir d0/dd/d39/d40/dd8 0 2024-02-26T10:48:03.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.269+0000 7f8ea4c00640 10 client.26105 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:03.266 INFO:tasks.workunit.client.0.smithi060.stdout:8/658: dread d9/d15/d3e/f9b [0,4194304] 0 2024-02-26T10:48:03.266 INFO:tasks.workunit.client.0.smithi060.stdout:8/659: chown d9/d15/d29/f59 16 1 2024-02-26T10:48:03.266 INFO:tasks.workunit.client.0.smithi060.stdout:7/786: dwrite da/d15/f17 [0,4194304] 0 2024-02-26T10:48:03.267 INFO:tasks.workunit.client.0.smithi060.stdout:7/787: readlink da/l11 0 2024-02-26T10:48:03.267 INFO:tasks.workunit.client.0.smithi060.stdout:0/643: symlink d0/dd/d39/d40/dad/dd6/ld9 0 2024-02-26T10:48:03.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.270+0000 7f8ebaa00640 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55913db4a5c0 0x55913db4c9e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db1324161f0bd9a server_cookie=8c346ad6d9d54282 in_seq=0 out_seq=0 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.271+0000 7f8ea4c00640 1 -- 172.21.15.60:0/3034479057 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55913da2f5c0 con 0x55913db4a5c0 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.271+0000 7f8ea4c00640 10 client.26105 handle_command_reply: tid=0 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.271+0000 7f8ec4a45740 1 client.26105 shutdown 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.272+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 msgr2=0x55913daec9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.272+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913daec9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009830 tx=0x7f8eb400cfb0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.272+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 shutdown_connections 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.272+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55913db4a5c0 0x55913db4c9e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.272+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55913dad0fe0 0x55913daec9c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.272+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55913dacbe70 0x55913daec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.272+0000 7f8ea4c00640 0 client.26105 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.272+0000 7f8ec4a45740 1 --2- 172.21.15.60:0/3034479057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55913dacb480 0x55913daedd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.272+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 >> 172.21.15.60:0/3034479057 conn(0x55913dac69f0 msgr2=0x55913dae69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:03.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.273+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 shutdown_connections 2024-02-26T10:48:03.269 INFO:tasks.workunit.client.0.smithi060.stdout:0/644: mkdir d0/dd/d39/d40/dad/dda 0 2024-02-26T10:48:03.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.273+0000 7f8ec4a45740 1 -- 172.21.15.60:0/3034479057 wait complete. 2024-02-26T10:48:03.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.273+0000 7f8ec4a45740 20 client.26105 trim_cache size 0 max 16384 2024-02-26T10:48:03.270 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T10:48:03.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 -- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ea00a1f50 msgr2=0x7f8ea00a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:03.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ea00a1f50 0x7f8ea00a4410 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4005fd0 tx=0x7f8eb4009940 comp rx=0 tx=0).stop 2024-02-26T10:48:03.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 -- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 msgr2=0x7f8ebc199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:03.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 0x7f8ebc199cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8034980 tx=0x7f8ea80042f0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 -- 172.21.15.60:0/3398692482 shutdown_connections 2024-02-26T10:48:03.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f8ea00a1f50 0x7f8ea00a4410 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ebc105050 0x7f8ebc1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ebc104690 0x7f8ebc199cc0 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8034980 tx=0x7f8ea80042f0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 --2- 172.21.15.60:0/3398692482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc1034a0 0x7f8ebc199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:03.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.274+0000 7f8ec1c00640 1 -- 172.21.15.60:0/3398692482 >> 172.21.15.60:0/3398692482 conn(0x7f8ebc0fea10 msgr2=0x7f8ebc107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:03.271 INFO:tasks.workunit.client.0.smithi060.stdout:1/730: dread d3/d35/d53/d68/d85/fd0 [0,4194304] 0 2024-02-26T10:48:03.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.277+0000 7f8ec1c00640 1 -- 172.21.15.60:0/3398692482 shutdown_connections 2024-02-26T10:48:03.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:48:03.277+0000 7f8ec1c00640 1 -- 172.21.15.60:0/3398692482 wait complete. 2024-02-26T10:48:03.272 INFO:tasks.workunit.client.0.smithi060.stdout:8/660: mknod d9/d15/d29/d2f/d34/d72/d94/dba/cd3 0 2024-02-26T10:48:03.272 INFO:tasks.workunit.client.0.smithi060.stdout:8/661: fdatasync d9/d15/d29/d2f/d34/d72/f62 0 2024-02-26T10:48:03.272 INFO:tasks.workunit.client.0.smithi060.stdout:8/662: fdatasync d9/d15/d29/d2f/d34/d72/d94/f9e 0 2024-02-26T10:48:03.272 INFO:tasks.workunit.client.0.smithi060.stdout:3/506: dwrite d5/de/df/d11/f20 [0,4194304] 0 2024-02-26T10:48:03.272 INFO:tasks.workunit.client.0.smithi060.stdout:4/670: dwrite d0/d27/d2f/f7d [0,4194304] 0 2024-02-26T10:48:03.274 INFO:tasks.workunit.client.0.smithi060.stdout:1/731: read d3/db/d1e/d38/f8e [23698,93658] 0 2024-02-26T10:48:03.274 INFO:tasks.workunit.client.0.smithi060.stdout:1/732: dread - d3/db/d1e/d6d/dd1/fe7 zero size 2024-02-26T10:48:03.275 INFO:tasks.workunit.client.0.smithi060.stdout:1/733: creat d3/d35/d53/d68/dad/ffd x:0 0 0 2024-02-26T10:48:03.278 INFO:tasks.workunit.client.0.smithi060.stdout:1/734: mknod d3/db/d1e/d6d/dd9/cfe 0 2024-02-26T10:48:03.279 INFO:tasks.workunit.client.0.smithi060.stdout:1/735: truncate d3/d35/d53/d68/dad/fb6 131931 0 2024-02-26T10:48:03.279 INFO:tasks.workunit.client.0.smithi060.stdout:1/736: dread - d3/d35/d53/d68/fdc zero size 2024-02-26T10:48:03.285 INFO:tasks.workunit.client.0.smithi060.stdout:1/737: creat d3/db/d1e/d39/fff x:0 0 0 2024-02-26T10:48:03.287 INFO:tasks.workunit.client.0.smithi060.stdout:4/671: dread d0/d27/d76/dbd/f38 [0,4194304] 0 2024-02-26T10:48:03.292 INFO:tasks.workunit.client.0.smithi060.stdout:7/788: rename da/d15/d21/d29/c83 to da/d15/d21/d41/d6c/c105 0 2024-02-26T10:48:03.296 INFO:tasks.workunit.client.0.smithi060.stdout:7/789: mknod da/d15/d21/d29/c106 0 2024-02-26T10:48:03.296 INFO:tasks.workunit.client.0.smithi060.stdout:4/672: symlink d0/d27/d76/ld4 0 2024-02-26T10:48:03.296 INFO:tasks.workunit.client.0.smithi060.stdout:1/738: write d3/db/dc8/f9a [739421,109762] 0 2024-02-26T10:48:03.296 INFO:tasks.workunit.client.0.smithi060.stdout:1/739: stat d3/db/dc8/f9a 0 2024-02-26T10:48:03.297 INFO:tasks.workunit.client.0.smithi060.stdout:1/740: creat d3/db/d1e/d6d/da3/daa/f100 x:0 0 0 2024-02-26T10:48:03.297 INFO:tasks.workunit.client.0.smithi060.stdout:0/645: rename d0/dd/l13 to d0/dd/d53/d87/d94/ldb 0 2024-02-26T10:48:03.301 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 110263384, 'ino': 1099511632246, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.h'}, {'damage_type': 'backtrace', 'id': 131814612, 'ino': 1099511632232, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_sensor.c'}, {'damage_type': 'backtrace', 'id': 134314085, 'ino': 1099511632262, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.h'}, {'damage_type': 'backtrace', 'id': 238714452, 'ino': 1099511632243, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.h'}, {'damage_type': 'backtrace', 'id': 258406227, 'ino': 1099511631818, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory_event.c'}, {'damage_type': 'backtrace', 'id': 260251041, 'ino': 1099511632237, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.h'}, {'damage_type': 'backtrace', 'id': 312299559, 'ino': 1099511632251, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.c'}, {'damage_type': 'backtrace', 'id': 323069430, 'ino': 1099511632253, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.h'}, {'damage_type': 'backtrace', 'id': 396168315, 'ino': 1099511631835, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.am'}, {'damage_type': 'backtrace', 'id': 564954656, 'ino': 1099511632248, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.c'}, {'damage_type': 'backtrace', 'id': 574974244, 'ino': 1099511632252, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.in'}, {'damage_type': 'backtrace', 'id': 590058427, 'ino': 1099511631832, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/ipmi_thres_test.c'}, {'damage_type': 'backtrace', 'id': 641580283, 'ino': 1099511631819, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/posix_vlog.c'}, {'damage_type': 'backtrace', 'id': 666764040, 'ino': 1099511632235, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.h'}, {'damage_type': 'backtrace', 'id': 774133568, 'ino': 1099511632238, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_reset.c'}, {'damage_type': 'backtrace', 'id': 774636814, 'ino': 1099511631831, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t'}, {'damage_type': 'backtrace', 'id': 786971472, 'ino': 1099511632258, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.c'}, {'damage_type': 'backtrace', 'id': 853175377, 'ino': 1099511631828, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_rdrs.c'}, {'damage_type': 'backtrace', 'id': 877530795, 'ino': 1099511631807, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_close.c'}, {'damage_type': 'backtrace', 'id': 910782487, 'ino': 1099511631833, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.in'}, {'damage_type': 'backtrace', 'id': 912661216, 'ino': 1099511631816, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/sync.c'}, {'damage_type': 'backtrace', 'id': 936784625, 'ino': 1099511631808, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/hotswap.c'}, {'damage_type': 'backtrace', 'id': 974138311, 'ino': 1099511632240, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.c'}, {'damage_type': 'backtrace', 'id': 1004001644, 'ino': 1099511631829, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_util.c'}, {'damage_type': 'backtrace', 'id': 1005562385, 'ino': 1099511631812, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_controls.c'}, {'damage_type': 'backtrace', 'id': 1017100718, 'ino': 1099511631810, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_connection.c'}, {'damage_type': 'backtrace', 'id': 1206973673, 'ino': 1099511631830, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_slot_rdrs.c'}, {'damage_type': 'backtrace', 'id': 1378444870, 'ino': 1099511631806, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor_event.c'}, {'damage_type': 'backtrace', 'id': 1568423140, 'ino': 1099511631827, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_shelf_fru.c'}, {'damage_type': 'backtrace', 'id': 1587503938, 'ino': 1099511632261, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/Makefile.am'}, {'damage_type': 'backtrace', 'id': 1694173494, 'ino': 1099511632254, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.c'}, {'damage_type': 'backtrace', 'id': 1803236338, 'ino': 1099511631822, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity.c'}, {'damage_type': 'backtrace', 'id': 1814211515, 'ino': 1099511632257, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.c'}, {'damage_type': 'backtrace', 'id': 1816675952, 'ino': 1099511631805, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_vshm_rdrs.c'}, {'damage_type': 'backtrace', 'id': 1911667334, 'ino': 1099511632245, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.h'}, {'damage_type': 'backtrace', 'id': 1924325864, 'ino': 1099511631826, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/Makefile.in'}, {'damage_type': 'backtrace', 'id': 2042364000, 'ino': 1099511631834, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/t/Makefile.am'}, {'damage_type': 'backtrace', 'id': 2188378041, 'ino': 1099511631804, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_entity_event.c'}, {'damage_type': 'backtrace', 'id': 2311929523, 'ino': 1099511632231, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.c'}, {'damage_type': 'backtrace', 'id': 2402170823, 'ino': 1099511632234, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_indicator.h'}, {'damage_type': 'backtrace', 'id': 2442647225, 'ino': 1099511632250, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_power.c'}, {'damage_type': 'backtrace', 'id': 2466564704, 'ino': 1099511632233, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_hotswap.h'}, {'damage_type': 'backtrace', 'id': 2505124305, 'ino': 1099511632260, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.c'}, {'damage_type': 'backtrace', 'id': 2838620444, 'ino': 1099511632247, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_eventlog.h'}, {'damage_type': 'backtrace', 'id': 2941559868, 'ino': 1099511631809, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_control_event.c'}, {'damage_type': 'backtrace', 'id': 2942935778, 'ino': 1099511632239, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_annunciator.h'}, {'damage_type': 'backtrace', 'id': 2988220750, 'ino': 1099511632241, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_discover.h'}, {'damage_type': 'backtrace', 'id': 2992808727, 'ino': 1099511632259, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas.h'}, {'damage_type': 'backtrace', 'id': 3146753425, 'ino': 1099511631836, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sensor.c'}, {'damage_type': 'backtrace', 'id': 3211847093, 'ino': 1099511631824, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_mc_event.c'}, {'damage_type': 'backtrace', 'id': 3477867673, 'ino': 1099511631821, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_event.c'}, {'damage_type': 'backtrace', 'id': 3519655788, 'ino': 1099511631811, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ekeyfru.h'}, {'damage_type': 'backtrace', 'id': 3587507200, 'ino': 1099511631815, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.sym'}, {'damage_type': 'backtrace', 'id': 3593897016, 'ino': 1099511631814, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_inventory.c'}, {'damage_type': 'backtrace', 'id': 3697749965, 'ino': 1099511631820, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.c'}, {'damage_type': 'backtrace', 'id': 3750612703, 'ino': 1099511632255, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_control.h'}, {'damage_type': 'backtrace', 'id': 3875400324, 'ino': 1099511632244, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_inventory.c'}, {'damage_type': 'backtrace', 'id': 3952858350, 'ino': 1099511631825, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/atca_fru_rdrs.c'}, {'damage_type': 'backtrace', 'id': 4104788736, 'ino': 1099511632249, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_resource.c'}, {'damage_type': 'backtrace', 'id': 4145953613, 'ino': 1099511631817, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_drv.c'}, {'damage_type': 'backtrace', 'id': 4170445418, 'ino': 1099511632236, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_watchdog.c'}, {'damage_type': 'backtrace', 'id': 4184984181, 'ino': 1099511631813, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi.h'}, {'damage_type': 'backtrace', 'id': 4217264982, 'ino': 1099511632256, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_event.h'}, {'damage_type': 'backtrace', 'id': 4258376643, 'ino': 1099511632242, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/rtas/rtas_utils.c'}, {'damage_type': 'backtrace', 'id': 4286717593, 'ino': 1099511631823, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/ipmi/ipmi_sel.c'}] 2024-02-26T10:48:03.301 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 52580559, 'ino': 1099511631710, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_013.c'}, {'damage_type': 'backtrace', 'id': 120857269, 'ino': 1099511631720, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_006.c'}, {'damage_type': 'backtrace', 'id': 180073712, 'ino': 1099511632636, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/8-1.c'}, {'damage_type': 'backtrace', 'id': 234077096, 'ino': 1099511631746, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_028.c'}, {'damage_type': 'backtrace', 'id': 263346481, 'ino': 1099511631712, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.am'}, {'damage_type': 'backtrace', 'id': 265565002, 'ino': 1099511631733, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_012.c'}, {'damage_type': 'backtrace', 'id': 271409304, 'ino': 1099511631723, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_001.c'}, {'damage_type': 'backtrace', 'id': 381949674, 'ino': 1099511631695, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_enum_utils_test.c'}, {'damage_type': 'backtrace', 'id': 387111911, 'ino': 1099511631768, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_005.c'}, {'damage_type': 'backtrace', 'id': 402152707, 'ino': 1099511631694, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rdr_test.c'}, {'damage_type': 'backtrace', 'id': 492971224, 'ino': 1099511632635, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/1-1.c'}, {'damage_type': 'backtrace', 'id': 538945458, 'ino': 1099511631685, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_text_utils_test.c'}, {'damage_type': 'backtrace', 'id': 550800747, 'ino': 1099511631737, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_036.c'}, {'damage_type': 'backtrace', 'id': 555072297, 'ino': 1099511631766, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_023.c'}, {'damage_type': 'backtrace', 'id': 611946816, 'ino': 1099511631715, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_003.c'}, {'damage_type': 'backtrace', 'id': 615871376, 'ino': 1099511632633, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/5-1.c'}, {'damage_type': 'backtrace', 'id': 663414329, 'ino': 1099511632640, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/11-1.c'}, {'damage_type': 'backtrace', 'id': 711809447, 'ino': 1099511632639, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/4-1.c'}, {'damage_type': 'backtrace', 'id': 726846847, 'ino': 1099511631740, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_034.c'}, {'damage_type': 'backtrace', 'id': 756205454, 'ino': 1099511631730, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_026.c'}, {'damage_type': 'backtrace', 'id': 817577950, 'ino': 1099511632634, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/6-1.c'}, {'damage_type': 'backtrace', 'id': 898363847, 'ino': 1099511631743, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_022.c'}, {'damage_type': 'backtrace', 'id': 900457148, 'ino': 1099511631759, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.in'}, {'damage_type': 'backtrace', 'id': 908851937, 'ino': 1099511631741, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_013.c'}, {'damage_type': 'backtrace', 'id': 926641769, 'ino': 1099511631687, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_time_utils_test.c'}, {'damage_type': 'backtrace', 'id': 928041557, 'ino': 1099511631749, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_007.c'}, {'damage_type': 'backtrace', 'id': 999210131, 'ino': 1099511631754, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_044.c'}, {'damage_type': 'backtrace', 'id': 1053644585, 'ino': 1099511632638, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/7-1.c'}, {'damage_type': 'backtrace', 'id': 1151552889, 'ino': 1099511631731, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_010.c'}, {'damage_type': 'backtrace', 'id': 1178077232, 'ino': 1099511632265, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.in'}, {'damage_type': 'backtrace', 'id': 1191267456, 'ino': 1099511632632, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/2-1.c'}, {'damage_type': 'backtrace', 'id': 1252260674, 'ino': 1099511631739, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_027.c'}, {'damage_type': 'backtrace', 'id': 1268476878, 'ino': 1099511632630, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/9-1.c'}, {'damage_type': 'backtrace', 'id': 1403426037, 'ino': 1099511631760, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_008.c'}, {'damage_type': 'backtrace', 'id': 1403667332, 'ino': 1099511631729, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_011.c'}, {'damage_type': 'backtrace', 'id': 1419860331, 'ino': 1099511631724, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.am'}, {'damage_type': 'backtrace', 'id': 1447726230, 'ino': 1099511631756, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test.h'}, {'damage_type': 'backtrace', 'id': 1485634572, 'ino': 1099511631719, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_005.c'}, {'damage_type': 'backtrace', 'id': 1493930249, 'ino': 1099511631689, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_encode_test.c'}, {'damage_type': 'backtrace', 'id': 1497142873, 'ino': 1099511631752, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_020.c'}, {'damage_type': 'backtrace', 'id': 1527432246, 'ino': 1099511632637, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/assertions.xml'}, {'damage_type': 'backtrace', 'id': 1534601639, 'ino': 1099511631758, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_031.c'}, {'damage_type': 'backtrace', 'id': 1549274681, 'ino': 1099511631721, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_002.c'}, {'damage_type': 'backtrace', 'id': 1553313812, 'ino': 1099511631684, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrfield_test.c'}, {'damage_type': 'backtrace', 'id': 1557285089, 'ino': 1099511631683, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_utils_test.c'}, {'damage_type': 'backtrace', 'id': 1626856606, 'ino': 1099511631763, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_009.c'}, {'damage_type': 'backtrace', 'id': 1652340322, 'ino': 1099511631722, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/Makefile.in'}, {'damage_type': 'backtrace', 'id': 1736527868, 'ino': 1099511631700, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_012.c'}, {'damage_type': 'backtrace', 'id': 1775686753, 'ino': 1099511631682, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrinfo_test.c'}, {'damage_type': 'backtrace', 'id': 1797847575, 'ino': 1099511631753, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_014.c'}, {'damage_type': 'backtrace', 'id': 1813030090, 'ino': 1099511631728, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_038.c'}, {'damage_type': 'backtrace', 'id': 1868082986, 'ino': 1099511631693, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.am'}, {'damage_type': 'backtrace', 'id': 1925191709, 'ino': 1099511631717, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_008.c'}, {'damage_type': 'backtrace', 'id': 1955814977, 'ino': 1099511632629, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/3-1.c'}, {'damage_type': 'backtrace', 'id': 2001273979, 'ino': 1099511631706, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_000.c'}, {'damage_type': 'backtrace', 'id': 2021206997, 'ino': 1099511631703, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_007.c'}, {'damage_type': 'backtrace', 'id': 2085036102, 'ino': 1099511631691, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpiatca_enum_utils_test.c'}, {'damage_type': 'backtrace', 'id': 2091078123, 'ino': 1099511631708, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_005.c'}, {'damage_type': 'backtrace', 'id': 2091857103, 'ino': 1099511631755, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_006.c'}, {'damage_type': 'backtrace', 'id': 2098473037, 'ino': 1099511631705, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_001.c'}, {'damage_type': 'backtrace', 'id': 2162824008, 'ino': 1099511631745, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_004.c'}, {'damage_type': 'backtrace', 'id': 2320293415, 'ino': 1099511631774, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/Makefile.am'}, {'damage_type': 'backtrace', 'id': 2368359441, 'ino': 1099511631736, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_033.c'}, {'damage_type': 'backtrace', 'id': 2485152224, 'ino': 1099511631697, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_006.c'}, {'damage_type': 'backtrace', 'id': 2544021302, 'ino': 1099511631692, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_event_utils_test.c'}, {'damage_type': 'backtrace', 'id': 2646328350, 'ino': 1099511631761, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_045.c'}, {'damage_type': 'backtrace', 'id': 2651055715, 'ino': 1099511631735, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_035.c'}, {'damage_type': 'backtrace', 'id': 2711375382, 'ino': 1099511631773, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_003.c'}, {'damage_type': 'backtrace', 'id': 2717686744, 'ino': 1099511631702, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_010.c'}, {'damage_type': 'backtrace', 'id': 2730820050, 'ino': 1099511631767, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_018.c'}, {'damage_type': 'backtrace', 'id': 2739625087, 'ino': 1099511632266, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/sysfs2hpi.c'}, {'damage_type': 'backtrace', 'id': 2749592871, 'ino': 1099511631711, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_004.c'}, {'damage_type': 'backtrace', 'id': 2801937141, 'ino': 1099511631698, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_002.c'}, {'damage_type': 'backtrace', 'id': 2828780396, 'ino': 1099511631688, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_idrarea_test.c'}, {'damage_type': 'backtrace', 'id': 2938119167, 'ino': 1099511631751, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_compare.c'}, {'damage_type': 'backtrace', 'id': 2969732743, 'ino': 1099511631771, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_041.c'}, {'damage_type': 'backtrace', 'id': 3020383070, 'ino': 1099511631748, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_019.c'}, {'damage_type': 'backtrace', 'id': 3069902837, 'ino': 1099511631772, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_043.c'}, {'damage_type': 'backtrace', 'id': 3081292958, 'ino': 1099511631704, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_003.c'}, {'damage_type': 'backtrace', 'id': 3129068830, 'ino': 1099511631734, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_032.c'}, {'damage_type': 'backtrace', 'id': 3153934479, 'ino': 1099511631732, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_016.c'}, {'damage_type': 'backtrace', 'id': 3164325210, 'ino': 1099511631750, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_030.c'}, {'damage_type': 'backtrace', 'id': 3314644624, 'ino': 1099511631718, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_007.c'}, {'damage_type': 'backtrace', 'id': 3353787575, 'ino': 1099511631707, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/Makefile.in'}, {'damage_type': 'backtrace', 'id': 3362085298, 'ino': 1099511631764, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_040.c'}, {'damage_type': 'backtrace', 'id': 3375262909, 'ino': 1099511631765, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_015.c'}, {'damage_type': 'backtrace', 'id': 3387810885, 'ino': 1099511631686, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/sahpi_struct_rptentry_test.c'}, {'damage_type': 'backtrace', 'id': 3391971065, 'ino': 1099511631709, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_011.c'}, {'damage_type': 'backtrace', 'id': 3403058285, 'ino': 1099511632267, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/Makefile.am'}, {'damage_type': 'backtrace', 'id': 3513157897, 'ino': 1099511631757, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_037.c'}, {'damage_type': 'backtrace', 'id': 3565254938, 'ino': 1099511631742, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_021.c'}, {'damage_type': 'backtrace', 'id': 3602528510, 'ino': 1099511632631, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/definitions/mqueue_h/10-1.c'}, {'damage_type': 'backtrace', 'id': 3613618268, 'ino': 1099511631747, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_025.c'}, {'damage_type': 'backtrace', 'id': 3633258708, 'ino': 1099511631770, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_029.c'}, {'damage_type': 'backtrace', 'id': 3644241078, 'ino': 1099511631769, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_001.c'}, {'damage_type': 'backtrace', 'id': 3660927375, 'ino': 1099511631738, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_039.c'}, {'damage_type': 'backtrace', 'id': 3663831718, 'ino': 1099511631690, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/sahpi/Makefile.in'}, {'damage_type': 'backtrace', 'id': 3674078795, 'ino': 1099511631744, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_002.c'}, {'damage_type': 'backtrace', 'id': 3723166584, 'ino': 1099511631727, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_017.c'}, {'damage_type': 'backtrace', 'id': 3928623611, 'ino': 1099511631716, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/ann/ann_test_004.c'}, {'damage_type': 'backtrace', 'id': 4072896960, 'ino': 1099511631699, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_009.c'}, {'damage_type': 'backtrace', 'id': 4113138317, 'ino': 1099511631762, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_042.c'}, {'damage_type': 'backtrace', 'id': 4150004916, 'ino': 1099511631726, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/el/el_test_024.c'}, {'damage_type': 'backtrace', 'id': 4170341107, 'ino': 1099511631701, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/utils/t/uid/uid_utils_008.c'}, {'damage_type': 'backtrace', 'id': 4282624035, 'ino': 1099511632264, 'path': '/volumes/qa/sv_1/eb7728f5-bef9-4e6c-a969-ea3331134b72/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite/plugins/sysfs/README'}] 2024-02-26T10:48:03.301 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-02-26T10:48:03.302 INFO:tasks.workunit.client.0.smithi060.stdout:0/646: unlink d0/dd/d33/c36 0 2024-02-26T10:48:03.302 INFO:tasks.workunit.client.0.smithi060.stdout:0/647: truncate d0/dd/f77 3265091 0 2024-02-26T10:48:03.307 INFO:tasks.workunit.client.0.smithi060.stdout:5/644: dwrite d0/de/d15/d65/d86/dcc/fd5 [0,4194304] 0 2024-02-26T10:48:03.307 INFO:tasks.workunit.client.0.smithi060.stdout:5/645: write d0/d33/fe3 [7190,13136] 0 2024-02-26T10:48:03.327 INFO:tasks.workunit.client.0.smithi060.stdout:3/507: dwrite d5/de/df/d4c/f70 [0,4194304] 0 2024-02-26T10:48:03.327 INFO:tasks.workunit.client.0.smithi060.stdout:3/508: stat d5/d37 0 2024-02-26T10:48:03.327 INFO:tasks.workunit.client.0.smithi060.stdout:3/509: write d5/de/df/d4c/f88 [23045,99985] 0 2024-02-26T10:48:03.328 INFO:tasks.workunit.client.0.smithi060.stdout:4/673: write d0/da/d57/f96 [1486608,101507] 0 2024-02-26T10:48:03.352 INFO:tasks.workunit.client.0.smithi060.stdout:7/790: dwrite da/d66/dad/dd8/f81 [0,4194304] 0 2024-02-26T10:48:03.354 INFO:tasks.workunit.client.0.smithi060.stdout:8/663: dwrite d9/d15/d3e/fac [0,4194304] 0 2024-02-26T10:48:03.361 INFO:tasks.workunit.client.0.smithi060.stdout:0/648: dwrite d0/dd/d39/fb8 [0,4194304] 0 2024-02-26T10:48:03.361 INFO:tasks.workunit.client.0.smithi060.stdout:8/664: mkdir d9/d15/d29/d2f/d34/d72/d94/dd4 0 2024-02-26T10:48:03.368 INFO:tasks.workunit.client.0.smithi060.stdout:3/510: dread d5/de/df/d57/d60/f30 [0,4194304] 0 2024-02-26T10:48:03.371 INFO:tasks.workunit.client.0.smithi060.stdout:3/511: rename d5/de/df/d11/c8c to d5/de/df/c9a 0 2024-02-26T10:48:03.372 INFO:tasks.workunit.client.0.smithi060.stdout:3/512: truncate d5/d37/f56 660806 0 2024-02-26T10:48:03.372 INFO:tasks.workunit.client.0.smithi060.stdout:3/513: mknod d5/de/c9b 0 2024-02-26T10:48:03.381 INFO:tasks.workunit.client.0.smithi060.stdout:8/665: dread d9/d15/d29/d2f/d34/d72/f42 [0,4194304] 0 2024-02-26T10:48:03.382 INFO:tasks.workunit.client.0.smithi060.stdout:8/666: fsync d9/d15/f81 0 2024-02-26T10:48:03.383 INFO:tasks.workunit.client.0.smithi060.stdout:8/667: creat d9/d6f/fd5 x:0 0 0 2024-02-26T10:48:03.384 INFO:tasks.workunit.client.0.smithi060.stdout:8/668: stat d9/l38 0 2024-02-26T10:48:03.384 INFO:tasks.workunit.client.0.smithi060.stdout:1/741: dwrite d3/db/d1e/d39/fff [0,4194304] 0 2024-02-26T10:48:03.385 INFO:tasks.workunit.client.0.smithi060.stdout:8/669: mknod d9/d15/d29/d95/cd6 0 2024-02-26T10:48:03.390 INFO:tasks.workunit.client.0.smithi060.stdout:8/670: mknod d9/d15/d29/d2f/d34/d72/d50/cd7 0 2024-02-26T10:48:03.390 INFO:tasks.workunit.client.0.smithi060.stdout:1/742: unlink d3/db/d1e/d38/f79 0 2024-02-26T10:48:03.390 INFO:tasks.workunit.client.0.smithi060.stdout:1/743: chown d3/d35/dc2/fee 145683 1 2024-02-26T10:48:03.391 INFO:tasks.workunit.client.0.smithi060.stdout:1/744: readlink d3/db/d1e/d39/l9f 0 2024-02-26T10:48:03.391 INFO:tasks.workunit.client.0.smithi060.stdout:1/745: stat d3/d35/l75 0 2024-02-26T10:48:03.391 INFO:tasks.workunit.client.0.smithi060.stdout:1/746: chown d3/db/d1e/d38/c45 1104824 1 2024-02-26T10:48:03.391 INFO:tasks.workunit.client.0.smithi060.stdout:1/747: fdatasync d3/d35/d53/d57/f83 0 2024-02-26T10:48:03.391 INFO:tasks.workunit.client.0.smithi060.stdout:1/748: write d3/db/d2e/f30 [1254122,60380] 0 2024-02-26T10:48:03.393 INFO:tasks.workunit.client.0.smithi060.stdout:1/749: mknod d3/db/d1e/d6d/dd9/c101 0 2024-02-26T10:48:03.394 INFO:tasks.workunit.client.0.smithi060.stdout:1/750: dread - d3/db/d1e/d6d/dd1/fe7 zero size 2024-02-26T10:48:03.394 INFO:tasks.workunit.client.0.smithi060.stdout:1/751: chown d3/d54/cae 501234 1 2024-02-26T10:48:03.394 INFO:tasks.workunit.client.0.smithi060.stdout:1/752: dread - d3/db/d1e/d6d/da3/daa/dd6/fdf zero size 2024-02-26T10:48:03.396 INFO:tasks.workunit.client.0.smithi060.stdout:1/753: rmdir d3/db/d1e/d6d/da3/daa/dd6 39 2024-02-26T10:48:03.401 INFO:tasks.workunit.client.0.smithi060.stdout:1/754: getdents d3/db/d1e/d38 0 2024-02-26T10:48:03.401 INFO:tasks.workunit.client.0.smithi060.stdout:1/755: truncate d3/db/d1e/d38/f8e 1313589 0 2024-02-26T10:48:03.401 INFO:tasks.workunit.client.0.smithi060.stdout:1/756: stat d3/db/dc8/dce 0 2024-02-26T10:48:03.401 INFO:tasks.workunit.client.0.smithi060.stdout:1/757: chown d3/c67 447 1 2024-02-26T10:48:03.406 INFO:tasks.workunit.client.0.smithi060.stdout:1/758: link d3/d35/d53/d68/d85/fd0 d3/db/d1e/f102 0 2024-02-26T10:48:03.406 INFO:tasks.workunit.client.0.smithi060.stdout:9/700: sync 2024-02-26T10:48:03.407 INFO:tasks.workunit.client.0.smithi060.stdout:9/701: stat d3/d1e/f82 0 2024-02-26T10:48:03.407 INFO:tasks.workunit.client.0.smithi060.stdout:0/649: dwrite d0/f3 [0,4194304] 0 2024-02-26T10:48:03.407 INFO:tasks.workunit.client.0.smithi060.stdout:5/646: dwrite d0/de/d15/f3b [0,4194304] 0 2024-02-26T10:48:03.408 INFO:tasks.workunit.client.0.smithi060.stdout:5/647: write d0/d33/f63 [232899,127869] 0 2024-02-26T10:48:03.409 INFO:tasks.workunit.client.0.smithi060.stdout:9/702: truncate d3/d1e/d5e/d35/d87/d9b/d8a/fe0 6370202 0 2024-02-26T10:48:03.409 INFO:tasks.workunit.client.0.smithi060.stdout:9/703: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f63 [681330,102956] 0 2024-02-26T10:48:03.415 INFO:tasks.workunit.client.0.smithi060.stdout:7/791: dwrite da/d66/dbd/fca [0,4194304] 0 2024-02-26T10:48:03.415 INFO:tasks.workunit.client.0.smithi060.stdout:7/792: dread - da/d15/d21/fd2 zero size 2024-02-26T10:48:03.415 INFO:tasks.workunit.client.0.smithi060.stdout:7/793: chown da/d15/d21/d29/c106 8 1 2024-02-26T10:48:03.415 INFO:tasks.workunit.client.0.smithi060.stdout:7/794: chown da/d15/d21/ff7 133 1 2024-02-26T10:48:03.415 INFO:tasks.workunit.client.0.smithi060.stdout:4/674: dwrite d0/d27/d76/faa [0,4194304] 0 2024-02-26T10:48:03.416 INFO:tasks.workunit.client.0.smithi060.stdout:6/679: sync 2024-02-26T10:48:03.419 INFO:tasks.workunit.client.0.smithi060.stdout:2/638: sync 2024-02-26T10:48:03.420 INFO:tasks.workunit.client.0.smithi060.stdout:7/795: creat da/d15/d21/d41/d58/f107 x:0 0 0 2024-02-26T10:48:03.421 INFO:tasks.workunit.client.0.smithi060.stdout:2/639: creat dd/d43/d60/d41/d37/d9a/db4/fd6 x:0 0 0 2024-02-26T10:48:03.421 INFO:tasks.workunit.client.0.smithi060.stdout:7/796: chown da/d15/d21/d2f/d78/dd3/d67 875961605 1 2024-02-26T10:48:03.422 INFO:tasks.workunit.client.0.smithi060.stdout:2/640: mknod dd/d43/d55/d5e/d86/d97/cd7 0 2024-02-26T10:48:03.422 INFO:tasks.workunit.client.0.smithi060.stdout:4/675: symlink d0/da/dc7/db0/db5/dc0/ld5 0 2024-02-26T10:48:03.423 INFO:tasks.workunit.client.0.smithi060.stdout:2/641: symlink dd/d43/d60/d41/d37/d4c/d76/da6/ld8 0 2024-02-26T10:48:03.425 INFO:tasks.workunit.client.0.smithi060.stdout:2/642: symlink dd/d43/d60/d41/d37/d9a/db4/ld9 0 2024-02-26T10:48:03.425 INFO:tasks.workunit.client.0.smithi060.stdout:2/643: stat dd/f94 0 2024-02-26T10:48:03.426 INFO:tasks.workunit.client.0.smithi060.stdout:2/644: link dd/d43/d60/d41/d37/d69/cae dd/d43/d60/d41/d37/cda 0 2024-02-26T10:48:03.426 INFO:tasks.workunit.client.0.smithi060.stdout:2/645: truncate dd/d43/d60/d41/d37/d9a/db4/fd6 801492 0 2024-02-26T10:48:03.426 INFO:tasks.workunit.client.0.smithi060.stdout:2/646: chown dd/d43/d55/d5e/dc6/l63 1889689938 1 2024-02-26T10:48:03.428 INFO:tasks.workunit.client.0.smithi060.stdout:2/647: creat dd/d43/d60/d41/d37/fdb x:0 0 0 2024-02-26T10:48:03.428 INFO:tasks.workunit.client.0.smithi060.stdout:8/671: read d9/d15/d3e/fac [3098462,34660] 0 2024-02-26T10:48:03.429 INFO:tasks.workunit.client.0.smithi060.stdout:7/797: creat da/d15/d88/dce/dea/f108 x:0 0 0 2024-02-26T10:48:03.429 INFO:tasks.workunit.client.0.smithi060.stdout:2/648: creat dd/d43/d55/d5e/d86/d97/fdc x:0 0 0 2024-02-26T10:48:03.431 INFO:tasks.workunit.client.0.smithi060.stdout:2/649: write dd/d43/d55/f8e [32189,109816] 0 2024-02-26T10:48:03.431 INFO:tasks.workunit.client.0.smithi060.stdout:7/798: write da/d15/d21/d41/dc7/fef [618344,25975] 0 2024-02-26T10:48:03.431 INFO:tasks.workunit.client.0.smithi060.stdout:7/799: stat da/f60 0 2024-02-26T10:48:03.436 INFO:tasks.workunit.client.0.smithi060.stdout:2/650: mkdir dd/d43/d60/d41/d37/d4c/d76/da6/ddd 0 2024-02-26T10:48:03.436 INFO:tasks.workunit.client.0.smithi060.stdout:5/648: read d0/de/d20/d32/d4d/f55 [4233807,1132] 0 2024-02-26T10:48:03.436 INFO:tasks.workunit.client.0.smithi060.stdout:5/649: chown d0/de/d15/d65/d86/cc1 51148 1 2024-02-26T10:48:03.437 INFO:tasks.workunit.client.0.smithi060.stdout:1/759: dread d3/d35/d53/d68/f8c [0,4194304] 0 2024-02-26T10:48:03.437 INFO:tasks.workunit.client.0.smithi060.stdout:1/760: write d3/db/d1e/d38/d6c/fb9 [1386953,107084] 0 2024-02-26T10:48:03.437 INFO:tasks.workunit.client.0.smithi060.stdout:0/650: dread d0/dd/d39/d40/d83/f8f [0,4194304] 0 2024-02-26T10:48:03.438 INFO:tasks.workunit.client.0.smithi060.stdout:9/704: rename d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/db9 to d3/d1e/dbb/df5 0 2024-02-26T10:48:03.444 INFO:tasks.workunit.client.0.smithi060.stdout:3/514: dwrite d5/de/f46 [0,4194304] 0 2024-02-26T10:48:03.444 INFO:tasks.workunit.client.0.smithi060.stdout:3/515: write d5/d37/f56 [1425617,74874] 0 2024-02-26T10:48:03.444 INFO:tasks.workunit.client.0.smithi060.stdout:1/761: rmdir d3/d35/d53/d68/d90 39 2024-02-26T10:48:03.444 INFO:tasks.workunit.client.0.smithi060.stdout:1/762: stat d3/d35/d5e/lf9 0 2024-02-26T10:48:03.445 INFO:tasks.workunit.client.0.smithi060.stdout:6/680: dread d6/d7/d25/d1c/f9e [0,4194304] 0 2024-02-26T10:48:03.446 INFO:tasks.workunit.client.0.smithi060.stdout:7/800: dread da/d15/f2e [0,4194304] 0 2024-02-26T10:48:03.450 INFO:tasks.workunit.client.0.smithi060.stdout:6/681: mkdir d6/d7/d47/dde 0 2024-02-26T10:48:03.451 INFO:tasks.workunit.client.0.smithi060.stdout:6/682: mknod d6/d7/d25/d1c/d6b/cdf 0 2024-02-26T10:48:03.462 INFO:tasks.workunit.client.0.smithi060.stdout:8/672: write d9/d15/f81 [467291,48153] 0 2024-02-26T10:48:03.464 INFO:tasks.workunit.client.0.smithi060.stdout:9/705: rename d3/d1e/d5e/d9e/laa to d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/lf6 0 2024-02-26T10:48:03.465 INFO:tasks.workunit.client.0.smithi060.stdout:8/673: rmdir d9/d15/d29/d2f/d34/d72/d94 39 2024-02-26T10:48:03.466 INFO:tasks.workunit.client.0.smithi060.stdout:3/516: rename d5/de/d16/l39 to d5/de/df/d11/d63/l9c 0 2024-02-26T10:48:03.469 INFO:tasks.workunit.client.0.smithi060.stdout:9/706: rename d3/d1e/ca6 to d3/d1e/dbb/cf7 0 2024-02-26T10:48:03.469 INFO:tasks.workunit.client.0.smithi060.stdout:9/707: readlink d3/d1e/d5e/d1b/l92 0 2024-02-26T10:48:03.469 INFO:tasks.workunit.client.0.smithi060.stdout:9/708: chown d3/d68/l77 252308 1 2024-02-26T10:48:03.470 INFO:tasks.workunit.client.0.smithi060.stdout:3/517: mknod d5/c9d 0 2024-02-26T10:48:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:3/518: symlink d5/d76/d7d/l9e 0 2024-02-26T10:48:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:3/519: dread - d5/de/df/d4c/f6f zero size 2024-02-26T10:48:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:3/520: truncate d5/de/df/d11/d63/f7f 677023 0 2024-02-26T10:48:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:3/521: write d5/de/df/d11/f62 [439970,100837] 0 2024-02-26T10:48:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:2/651: dread dd/d43/d55/d5e/d74/fb6 [0,4194304] 0 2024-02-26T10:48:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:7/801: write da/d15/d21/d2f/fbf [615872,124070] 0 2024-02-26T10:48:03.491 INFO:tasks.workunit.client.0.smithi060.stdout:8/674: write d9/d5a/f6e [338403,118653] 0 2024-02-26T10:48:03.498 INFO:tasks.workunit.client.0.smithi060.stdout:9/709: write d3/d1e/d5e/d35/d87/d9b/d8a/fe0 [4273548,23142] 0 2024-02-26T10:48:03.498 INFO:tasks.workunit.client.0.smithi060.stdout:9/710: chown d3/l4 103375 1 2024-02-26T10:48:03.506 INFO:tasks.workunit.client.0.smithi060.stdout:6/683: write d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/fb2 [692348,77094] 0 2024-02-26T10:48:03.506 INFO:tasks.workunit.client.0.smithi060.stdout:8/675: creat d9/d6f/fd8 x:0 0 0 2024-02-26T10:48:03.507 INFO:tasks.workunit.client.0.smithi060.stdout:9/711: truncate d3/d1e/d5e/d35/d87/d9b/d8a/dbe/fb6 200362 0 2024-02-26T10:48:03.507 INFO:tasks.workunit.client.0.smithi060.stdout:6/684: creat d6/fe0 x:0 0 0 2024-02-26T10:48:03.507 INFO:tasks.workunit.client.0.smithi060.stdout:8/676: symlink d9/d15/d3e/ld9 0 2024-02-26T10:48:03.507 INFO:tasks.workunit.client.0.smithi060.stdout:8/677: dread - d9/d15/d29/d2f/f98 zero size 2024-02-26T10:48:03.507 INFO:tasks.workunit.client.0.smithi060.stdout:1/763: dread d3/d35/d53/d57/f83 [0,4194304] 0 2024-02-26T10:48:03.510 INFO:tasks.workunit.client.0.smithi060.stdout:8/678: mknod d9/d15/d29/d2f/d34/d72/d50/d6a/d63/cda 0 2024-02-26T10:48:03.510 INFO:tasks.workunit.client.0.smithi060.stdout:1/764: mknod d3/db/d1e/d6d/dd1/c103 0 2024-02-26T10:48:03.510 INFO:tasks.workunit.client.0.smithi060.stdout:1/765: write d3/f9 [105430,12744] 0 2024-02-26T10:48:03.511 INFO:tasks.workunit.client.0.smithi060.stdout:2/652: dread f4 [0,4194304] 0 2024-02-26T10:48:03.511 INFO:tasks.workunit.client.0.smithi060.stdout:2/653: write dd/d43/d55/d6e/dd3/d6f/d8f/fcd [110291,103338] 0 2024-02-26T10:48:03.512 INFO:tasks.workunit.client.0.smithi060.stdout:2/654: dread - dd/d11/fc5 zero size 2024-02-26T10:48:03.512 INFO:tasks.workunit.client.0.smithi060.stdout:5/650: dwrite d0/d33/f63 [4194304,4194304] 0 2024-02-26T10:48:03.512 INFO:tasks.workunit.client.0.smithi060.stdout:2/655: stat dd/d43/d55/d5e/fd4 0 2024-02-26T10:48:03.512 INFO:tasks.workunit.client.0.smithi060.stdout:5/651: write d0/de/da4/fd7 [300018,21254] 0 2024-02-26T10:48:03.514 INFO:tasks.workunit.client.0.smithi060.stdout:5/652: write d0/de/f12 [45547,98135] 0 2024-02-26T10:48:03.514 INFO:tasks.workunit.client.0.smithi060.stdout:8/679: creat d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/fdb x:0 0 0 2024-02-26T10:48:03.514 INFO:tasks.workunit.client.0.smithi060.stdout:6/685: link d6/d7/d25/d1c/d1f/d23/d28/f48 d6/d8d/dcc/dc3/d98/fe1 0 2024-02-26T10:48:03.514 INFO:tasks.workunit.client.0.smithi060.stdout:2/656: mkdir dd/d43/d55/d5e/d86/d97/db9/dde 0 2024-02-26T10:48:03.514 INFO:tasks.workunit.client.0.smithi060.stdout:2/657: symlink dd/d43/d60/d41/d37/d4c/ldf 0 2024-02-26T10:48:03.515 INFO:tasks.workunit.client.0.smithi060.stdout:1/766: mkdir d3/db/d104 0 2024-02-26T10:48:03.515 INFO:tasks.workunit.client.0.smithi060.stdout:1/767: truncate d3/db/d1e/d39/f48 4282620 0 2024-02-26T10:48:03.515 INFO:tasks.workunit.client.0.smithi060.stdout:8/680: mkdir d9/d6f/ddc 0 2024-02-26T10:48:03.517 INFO:tasks.workunit.client.0.smithi060.stdout:5/653: creat d0/dc0/fe4 x:0 0 0 2024-02-26T10:48:03.519 INFO:tasks.workunit.client.0.smithi060.stdout:1/768: mkdir d3/db/d1e/d6d/dd9/d105 0 2024-02-26T10:48:03.519 INFO:tasks.workunit.client.0.smithi060.stdout:9/712: mkdir d3/d1e/dbb/dc2/df8 0 2024-02-26T10:48:03.519 INFO:tasks.workunit.client.0.smithi060.stdout:9/713: readlink d3/d1e/d5e/d1b/d23/l26 0 2024-02-26T10:48:03.519 INFO:tasks.workunit.client.0.smithi060.stdout:6/686: mknod d6/d8d/dcc/da0/ce2 0 2024-02-26T10:48:03.519 INFO:tasks.workunit.client.0.smithi060.stdout:2/658: symlink dd/d43/d60/d41/d37/le0 0 2024-02-26T10:48:03.522 INFO:tasks.workunit.client.0.smithi060.stdout:2/659: creat dd/d11/fe1 x:0 0 0 2024-02-26T10:48:03.522 INFO:tasks.workunit.client.0.smithi060.stdout:6/687: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/dc0/dd8/fe3 x:0 0 0 2024-02-26T10:48:03.522 INFO:tasks.workunit.client.0.smithi060.stdout:6/688: chown d6/d7/d25/d1c/l64 7 1 2024-02-26T10:48:03.522 INFO:tasks.workunit.client.0.smithi060.stdout:2/660: creat dd/d43/d55/fe2 x:0 0 0 2024-02-26T10:48:03.522 INFO:tasks.workunit.client.0.smithi060.stdout:1/769: creat d3/d35/d53/f106 x:0 0 0 2024-02-26T10:48:03.526 INFO:tasks.workunit.client.0.smithi060.stdout:1/770: creat d3/d35/d53/d68/d6b/f107 x:0 0 0 2024-02-26T10:48:03.526 INFO:tasks.workunit.client.0.smithi060.stdout:2/661: mkdir dd/d11/de3 0 2024-02-26T10:48:03.526 INFO:tasks.workunit.client.0.smithi060.stdout:2/662: chown dd/d43/d60/d41/d37/d38/l4b 718531392 1 2024-02-26T10:48:03.526 INFO:tasks.workunit.client.0.smithi060.stdout:4/676: dwrite d0/da/d3b/f59 [0,4194304] 0 2024-02-26T10:48:03.528 INFO:tasks.workunit.client.0.smithi060.stdout:4/677: mkdir d0/da/dd6 0 2024-02-26T10:48:03.528 INFO:tasks.workunit.client.0.smithi060.stdout:4/678: readlink d0/d27/d76/l7a 0 2024-02-26T10:48:03.535 INFO:tasks.workunit.client.0.smithi060.stdout:2/663: getdents dd/d43/d60/d41/d37/d9a/db4 0 2024-02-26T10:48:03.537 INFO:tasks.workunit.client.0.smithi060.stdout:0/651: dwrite d0/dd/d39/d40/dad/fcb [0,4194304] 0 2024-02-26T10:48:03.538 INFO:tasks.workunit.client.0.smithi060.stdout:2/664: creat dd/d43/d60/d41/d37/d69/fe4 x:0 0 0 2024-02-26T10:48:03.538 INFO:tasks.workunit.client.0.smithi060.stdout:2/665: chown dd/d43/d55/d5e/d74/da5/cba 985482 1 2024-02-26T10:48:03.538 INFO:tasks.workunit.client.0.smithi060.stdout:2/666: truncate dd/d43/d55/d5e/fd4 748719 0 2024-02-26T10:48:03.538 INFO:tasks.workunit.client.0.smithi060.stdout:2/667: fdatasync dd/d43/d60/d41/d37/d69/f72 0 2024-02-26T10:48:03.538 INFO:tasks.workunit.client.0.smithi060.stdout:2/668: chown dd/d11/l51 19226 1 2024-02-26T10:48:03.538 INFO:tasks.workunit.client.0.smithi060.stdout:2/669: readlink dd/d43/d60/d41/l30 0 2024-02-26T10:48:03.542 INFO:tasks.workunit.client.0.smithi060.stdout:0/652: mkdir d0/dd/d39/d40/d83/dc6/ddc 0 2024-02-26T10:48:03.542 INFO:tasks.workunit.client.0.smithi060.stdout:5/654: dread d0/de/d20/f85 [0,4194304] 0 2024-02-26T10:48:03.542 INFO:tasks.workunit.client.0.smithi060.stdout:0/653: truncate d0/dd/d39/d40/d83/dc6/f66 735508 0 2024-02-26T10:48:03.543 INFO:tasks.workunit.client.0.smithi060.stdout:0/654: stat d0/l56 0 2024-02-26T10:48:03.543 INFO:tasks.workunit.client.0.smithi060.stdout:0/655: chown d0/dd/d39/d40/d83/ca5 1275362 1 2024-02-26T10:48:03.543 INFO:tasks.workunit.client.0.smithi060.stdout:0/656: truncate d0/dd/d39/d40/f98 1559904 0 2024-02-26T10:48:03.543 INFO:tasks.workunit.client.0.smithi060.stdout:9/714: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f6c [0,4194304] 0 2024-02-26T10:48:03.547 INFO:tasks.workunit.client.0.smithi060.stdout:2/670: creat dd/d43/d60/d41/d37/d4c/fe5 x:0 0 0 2024-02-26T10:48:03.547 INFO:tasks.workunit.client.0.smithi060.stdout:0/657: fdatasync d0/dd/fbb 0 2024-02-26T10:48:03.549 INFO:tasks.workunit.client.0.smithi060.stdout:0/658: mkdir d0/d37/ddd 0 2024-02-26T10:48:03.549 INFO:tasks.workunit.client.0.smithi060.stdout:9/715: unlink d3/l1c 0 2024-02-26T10:48:03.549 INFO:tasks.workunit.client.0.smithi060.stdout:0/659: fsync d0/dd/d39/d40/d83/f8f 0 2024-02-26T10:48:03.549 INFO:tasks.workunit.client.0.smithi060.stdout:0/660: truncate d0/dd/d39/d93/fb6 74656 0 2024-02-26T10:48:03.550 INFO:tasks.workunit.client.0.smithi060.stdout:9/716: symlink d3/d1e/d93/lf9 0 2024-02-26T10:48:03.555 INFO:tasks.workunit.client.0.smithi060.stdout:8/681: dread d9/d15/f60 [0,4194304] 0 2024-02-26T10:48:03.555 INFO:tasks.workunit.client.0.smithi060.stdout:8/682: symlink d9/d15/d29/ldd 0 2024-02-26T10:48:03.557 INFO:tasks.workunit.client.0.smithi060.stdout:8/683: creat d9/d15/d29/d2f/d34/d72/d7e/fde x:0 0 0 2024-02-26T10:48:03.561 INFO:tasks.workunit.client.0.smithi060.stdout:8/684: creat d9/d15/d29/d2f/d34/fdf x:0 0 0 2024-02-26T10:48:03.561 INFO:tasks.workunit.client.0.smithi060.stdout:8/685: fsync d9/d5a/faf 0 2024-02-26T10:48:03.561 INFO:tasks.workunit.client.0.smithi060.stdout:8/686: write d9/d5a/d58/fab [352612,30953] 0 2024-02-26T10:48:03.565 INFO:tasks.workunit.client.0.smithi060.stdout:1/771: read d3/db/d1e/d38/d6c/fb9 [341933,49687] 0 2024-02-26T10:48:03.565 INFO:tasks.workunit.client.0.smithi060.stdout:1/772: read - d3/db/d1e/d6d/da3/daa/dd6/fdf zero size 2024-02-26T10:48:03.565 INFO:tasks.workunit.client.0.smithi060.stdout:8/687: rmdir d9/d15/d29/d2f/d34/d72/d50 39 2024-02-26T10:48:03.566 INFO:tasks.workunit.client.0.smithi060.stdout:1/773: unlink d3/d35/d53/d68/d85/c4c 0 2024-02-26T10:48:03.567 INFO:tasks.workunit.client.0.smithi060.stdout:8/688: readlink d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/lbe 0 2024-02-26T10:48:03.567 INFO:tasks.workunit.client.0.smithi060.stdout:8/689: truncate d9/d5a/f49 157192 0 2024-02-26T10:48:03.567 INFO:tasks.workunit.client.0.smithi060.stdout:8/690: chown d9/f14 1356 1 2024-02-26T10:48:03.567 INFO:tasks.workunit.client.0.smithi060.stdout:1/774: getdents d3/db/d2e/d51 0 2024-02-26T10:48:03.570 INFO:tasks.workunit.client.0.smithi060.stdout:7/802: dwrite da/d15/d21/d2f/d78/f7d [0,4194304] 0 2024-02-26T10:48:03.570 INFO:tasks.workunit.client.0.smithi060.stdout:8/691: symlink d9/d5a/le0 0 2024-02-26T10:48:03.570 INFO:tasks.workunit.client.0.smithi060.stdout:1/775: rename d3/db/f59 to d3/d54/de9/f108 0 2024-02-26T10:48:03.576 INFO:tasks.workunit.client.0.smithi060.stdout:7/803: creat da/d15/d21/d41/d6c/f109 x:0 0 0 2024-02-26T10:48:03.576 INFO:tasks.workunit.client.0.smithi060.stdout:7/804: chown da/d15/la6 96 1 2024-02-26T10:48:03.580 INFO:tasks.workunit.client.0.smithi060.stdout:2/671: dread dd/fcc [0,4194304] 0 2024-02-26T10:48:03.585 INFO:tasks.workunit.client.0.smithi060.stdout:4/679: dread d0/da/d3b/d1b/fa7 [0,4194304] 0 2024-02-26T10:48:03.587 INFO:tasks.workunit.client.0.smithi060.stdout:8/692: mknod d9/ce1 0 2024-02-26T10:48:03.592 INFO:tasks.workunit.client.0.smithi060.stdout:8/693: creat d9/d15/d29/d2f/d34/d72/d94/dd4/fe2 x:0 0 0 2024-02-26T10:48:03.598 INFO:tasks.workunit.client.0.smithi060.stdout:6/689: dwrite d6/d7/fd [0,4194304] 0 2024-02-26T10:48:03.600 INFO:tasks.workunit.client.0.smithi060.stdout:0/661: dwrite d0/dd/d53/f64 [0,4194304] 0 2024-02-26T10:48:03.600 INFO:tasks.workunit.client.0.smithi060.stdout:6/690: rmdir d6/d8d/dcc/dc3/d98 39 2024-02-26T10:48:03.602 INFO:tasks.workunit.client.0.smithi060.stdout:0/662: rename d0/d42/d7a/d7f to d0/dd/d39/d40/dde 0 2024-02-26T10:48:03.603 INFO:tasks.workunit.client.0.smithi060.stdout:6/691: fdatasync d6/d7/d25/d1c/d1f/d23/d28/fd4 0 2024-02-26T10:48:03.604 INFO:tasks.workunit.client.0.smithi060.stdout:0/663: creat d0/dd/d39/d40/dad/dd6/d6c/fdf x:0 0 0 2024-02-26T10:48:03.604 INFO:tasks.workunit.client.0.smithi060.stdout:0/664: stat d0/dd/d39/d40/d83/cb2 0 2024-02-26T10:48:03.606 INFO:tasks.workunit.client.0.smithi060.stdout:7/805: write da/d15/d21/d2f/d78/f7d [3810346,47587] 0 2024-02-26T10:48:03.606 INFO:tasks.workunit.client.0.smithi060.stdout:6/692: link d6/d7/d25/d1c/d1f/d23/d2b/c43 d6/d6e/ce4 0 2024-02-26T10:48:03.606 INFO:tasks.workunit.client.0.smithi060.stdout:0/665: dread - d0/dd/fd1 zero size 2024-02-26T10:48:03.607 INFO:tasks.workunit.client.0.smithi060.stdout:6/693: mknod d6/d7/d25/d80/ce5 0 2024-02-26T10:48:03.607 INFO:tasks.workunit.client.0.smithi060.stdout:7/806: creat da/d15/d21/d2f/d78/f10a x:0 0 0 2024-02-26T10:48:03.608 INFO:tasks.workunit.client.0.smithi060.stdout:7/807: truncate da/d15/d21/f28 2059127 0 2024-02-26T10:48:03.608 INFO:tasks.workunit.client.0.smithi060.stdout:7/808: fsync da/d15/ffc 0 2024-02-26T10:48:03.608 INFO:tasks.workunit.client.0.smithi060.stdout:6/694: write d6/d8d/dcc/fc1 [640642,38552] 0 2024-02-26T10:48:03.609 INFO:tasks.workunit.client.0.smithi060.stdout:7/809: mkdir da/d66/dbd/d10b 0 2024-02-26T10:48:03.609 INFO:tasks.workunit.client.0.smithi060.stdout:7/810: chown da/d15/d88/lb7 14 1 2024-02-26T10:48:03.610 INFO:tasks.workunit.client.0.smithi060.stdout:6/695: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d4d/db9/de6 0 2024-02-26T10:48:03.610 INFO:tasks.workunit.client.0.smithi060.stdout:6/696: write d6/d7/d25/d1c/d1f/d23/f9c [425679,3199] 0 2024-02-26T10:48:03.610 INFO:tasks.workunit.client.0.smithi060.stdout:7/811: truncate da/df1/dd7/de0/fe6 2680247 0 2024-02-26T10:48:03.618 INFO:tasks.workunit.client.0.smithi060.stdout:7/812: unlink da/d15/d21/cbb 0 2024-02-26T10:48:03.619 INFO:tasks.workunit.client.0.smithi060.stdout:7/813: stat da/d15/d21/d41/d6c/cc1 0 2024-02-26T10:48:03.620 INFO:tasks.workunit.client.0.smithi060.stdout:1/776: dwrite d3/db/d1e/d38/d6c/fb9 [0,4194304] 0 2024-02-26T10:48:03.620 INFO:tasks.workunit.client.0.smithi060.stdout:7/814: chown da/d15/d88/dce/f63 123 1 2024-02-26T10:48:03.621 INFO:tasks.workunit.client.0.smithi060.stdout:1/777: mknod d3/d35/d76/c109 0 2024-02-26T10:48:03.621 INFO:tasks.workunit.client.0.smithi060.stdout:1/778: readlink d3/d35/l44 0 2024-02-26T10:48:03.622 INFO:tasks.workunit.client.0.smithi060.stdout:6/697: rmdir d6 39 2024-02-26T10:48:03.650 INFO:tasks.workunit.client.0.smithi060.stdout:8/694: dread d9/f1e [4194304,4194304] 0 2024-02-26T10:48:03.650 INFO:tasks.workunit.client.0.smithi060.stdout:8/695: fsync d9/d15/d29/d2f/d34/fdf 0 2024-02-26T10:48:03.651 INFO:tasks.workunit.client.0.smithi060.stdout:8/696: link d9/d15/d29/d2f/d34/d72/f3a d9/d15/d29/d2f/d34/d72/d94/dd4/fe3 0 2024-02-26T10:48:03.657 INFO:tasks.workunit.client.0.smithi060.stdout:7/815: dread da/f24 [0,4194304] 0 2024-02-26T10:48:03.669 INFO:tasks.workunit.client.0.smithi060.stdout:0/666: dread d0/dd/d53/f64 [0,4194304] 0 2024-02-26T10:48:03.670 INFO:tasks.workunit.client.0.smithi060.stdout:2/672: dwrite dd/d43/d60/d41/d37/d4c/f92 [0,4194304] 0 2024-02-26T10:48:03.670 INFO:tasks.workunit.client.0.smithi060.stdout:6/698: dwrite d6/d7/d25/d1c/d1f/f35 [0,4194304] 0 2024-02-26T10:48:03.673 INFO:tasks.workunit.client.0.smithi060.stdout:2/673: symlink dd/d11/le6 0 2024-02-26T10:48:03.674 INFO:tasks.workunit.client.0.smithi060.stdout:2/674: symlink dd/d43/d60/d41/le7 0 2024-02-26T10:48:03.679 INFO:tasks.workunit.client.0.smithi060.stdout:2/675: getdents dd/d43/d60/d41/d37/d4c 0 2024-02-26T10:48:03.679 INFO:tasks.workunit.client.0.smithi060.stdout:2/676: write dd/d43/d60/d41/d37/d69/f7e [1112027,98002] 0 2024-02-26T10:48:03.696 INFO:tasks.workunit.client.0.smithi060.stdout:1/779: dwrite d3/d35/d53/d68/d90/fbc [4194304,4194304] 0 2024-02-26T10:48:03.696 INFO:tasks.workunit.client.0.smithi060.stdout:1/780: fdatasync d3/d35/d53/d57/f87 0 2024-02-26T10:48:03.704 INFO:tasks.workunit.client.0.smithi060.stdout:1/781: read d3/f2b [787632,61711] 0 2024-02-26T10:48:03.704 INFO:tasks.workunit.client.0.smithi060.stdout:1/782: write d3/db/d2e/d51/fac [5195766,90226] 0 2024-02-26T10:48:03.713 INFO:tasks.workunit.client.0.smithi060.stdout:8/697: dwrite d9/d15/d29/d2f/fb7 [0,4194304] 0 2024-02-26T10:48:03.713 INFO:tasks.workunit.client.0.smithi060.stdout:8/698: chown d9/d5a/le0 315 1 2024-02-26T10:48:03.713 INFO:tasks.workunit.client.0.smithi060.stdout:2/677: dread dd/d43/d55/f8e [0,4194304] 0 2024-02-26T10:48:03.713 INFO:tasks.workunit.client.0.smithi060.stdout:2/678: write dd/d43/d60/d41/d37/fdb [362074,938] 0 2024-02-26T10:48:03.714 INFO:tasks.workunit.client.0.smithi060.stdout:8/699: rename d9/f13 to d9/d15/d29/d2f/fe4 0 2024-02-26T10:48:03.714 INFO:tasks.workunit.client.0.smithi060.stdout:6/699: dwrite d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/fd1 [0,4194304] 0 2024-02-26T10:48:03.715 INFO:tasks.workunit.client.0.smithi060.stdout:7/816: dwrite da/d15/d21/d2f/d37/f38 [4194304,4194304] 0 2024-02-26T10:48:03.720 INFO:tasks.workunit.client.0.smithi060.stdout:8/700: creat d9/d15/fe5 x:0 0 0 2024-02-26T10:48:03.724 INFO:tasks.workunit.client.0.smithi060.stdout:7/817: write da/d15/d21/d41/dc7/fef [494956,76584] 0 2024-02-26T10:48:03.725 INFO:tasks.workunit.client.0.smithi060.stdout:6/700: dread d6/d8d/fb3 [0,4194304] 0 2024-02-26T10:48:03.725 INFO:tasks.workunit.client.0.smithi060.stdout:6/701: stat d6/d7/d25/d1c/d1f/d23/d2b/f3b 0 2024-02-26T10:48:03.726 INFO:tasks.workunit.client.0.smithi060.stdout:7/818: symlink da/d66/dad/d8f/l10c 0 2024-02-26T10:48:03.726 INFO:tasks.workunit.client.0.smithi060.stdout:6/702: symlink d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/da7/le7 0 2024-02-26T10:48:03.726 INFO:tasks.workunit.client.0.smithi060.stdout:7/819: creat da/d15/d21/d41/dc7/f10d x:0 0 0 2024-02-26T10:48:03.728 INFO:tasks.workunit.client.0.smithi060.stdout:7/820: rename da/df1/fff to da/d15/d21/d2f/d78/dd3/d67/f10e 0 2024-02-26T10:48:03.743 INFO:tasks.workunit.client.0.smithi060.stdout:6/703: dread d6/d8d/dcc/dc3/d98/fe1 [0,4194304] 0 2024-02-26T10:48:03.746 INFO:tasks.workunit.client.0.smithi060.stdout:6/704: mknod d6/d7/d25/d1c/d1f/d23/d2b/d67/ce8 0 2024-02-26T10:48:03.748 INFO:tasks.workunit.client.0.smithi060.stdout:1/783: dwrite d3/d35/d53/d68/d85/fd0 [0,4194304] 0 2024-02-26T10:48:03.754 INFO:tasks.workunit.client.0.smithi060.stdout:7/821: dread da/d15/d88/dce/fab [0,4194304] 0 2024-02-26T10:48:03.760 INFO:tasks.workunit.client.0.smithi060.stdout:1/784: getdents d3/db/d1e 0 2024-02-26T10:48:03.763 INFO:tasks.workunit.client.0.smithi060.stdout:1/785: creat d3/db/d1e/d39/d9b/df7/f10a x:0 0 0 2024-02-26T10:48:03.764 INFO:tasks.workunit.client.0.smithi060.stdout:1/786: write d3/db/d1e/d39/f8f [987423,29504] 0 2024-02-26T10:48:03.764 INFO:tasks.workunit.client.0.smithi060.stdout:0/667: dwrite d0/dd/f2d [0,4194304] 0 2024-02-26T10:48:03.766 INFO:tasks.workunit.client.0.smithi060.stdout:0/668: symlink d0/dd/d39/d40/d83/le0 0 2024-02-26T10:48:03.767 INFO:tasks.workunit.client.0.smithi060.stdout:0/669: readlink d0/dd/d39/d40/d83/le0 0 2024-02-26T10:48:03.769 INFO:tasks.workunit.client.0.smithi060.stdout:0/670: rename d0/dd/d39/d40/c21 to d0/dd/d53/d87/d94/db7/ce1 0 2024-02-26T10:48:03.773 INFO:tasks.workunit.client.0.smithi060.stdout:2/679: dwrite dd/d11/fe1 [0,4194304] 0 2024-02-26T10:48:03.773 INFO:tasks.workunit.client.0.smithi060.stdout:8/701: dread d9/d15/d3e/fb2 [0,4194304] 0 2024-02-26T10:48:03.773 INFO:tasks.workunit.client.0.smithi060.stdout:0/671: rename d0/dd/d39/d40/cc1 to d0/dd/d39/d40/ce2 0 2024-02-26T10:48:03.774 INFO:tasks.workunit.client.0.smithi060.stdout:1/787: mknod d3/d35/d53/d57/c10b 0 2024-02-26T10:48:03.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/717: sync 2024-02-26T10:48:03.775 INFO:tasks.workunit.client.0.smithi060.stdout:3/522: sync 2024-02-26T10:48:03.775 INFO:tasks.workunit.client.0.smithi060.stdout:4/680: sync 2024-02-26T10:48:03.775 INFO:tasks.workunit.client.0.smithi060.stdout:4/681: fdatasync d0/da/f4c 0 2024-02-26T10:48:03.775 INFO:tasks.workunit.client.0.smithi060.stdout:4/682: chown d0/da/dc7 368786 1 2024-02-26T10:48:03.775 INFO:tasks.workunit.client.0.smithi060.stdout:8/702: link f8 d9/d15/d29/fe6 0 2024-02-26T10:48:03.776 INFO:tasks.workunit.client.0.smithi060.stdout:5/655: sync 2024-02-26T10:48:03.776 INFO:tasks.workunit.client.0.smithi060.stdout:2/680: creat dd/d43/d60/d41/d37/d38/d57/fe8 x:0 0 0 2024-02-26T10:48:03.776 INFO:tasks.workunit.client.0.smithi060.stdout:2/681: write dd/d43/d60/d41/d37/d4c/f82 [646065,98601] 0 2024-02-26T10:48:03.777 INFO:tasks.workunit.client.0.smithi060.stdout:1/788: creat d3/db/d1e/d6d/d78/f10c x:0 0 0 2024-02-26T10:48:03.777 INFO:tasks.workunit.client.0.smithi060.stdout:0/672: unlink d0/dd/d39/d40/d83/lc3 0 2024-02-26T10:48:03.781 INFO:tasks.workunit.client.0.smithi060.stdout:9/718: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/ffa x:0 0 0 2024-02-26T10:48:03.781 INFO:tasks.workunit.client.0.smithi060.stdout:3/523: getdents d5/de/df/d57/d7a 0 2024-02-26T10:48:03.782 INFO:tasks.workunit.client.0.smithi060.stdout:0/673: creat d0/dd/d53/d87/d94/da6/dc7/fe3 x:0 0 0 2024-02-26T10:48:03.785 INFO:tasks.workunit.client.0.smithi060.stdout:1/789: rename d3/db/dc8/dbe to d3/db/d104/d10d 0 2024-02-26T10:48:03.788 INFO:tasks.workunit.client.0.smithi060.stdout:8/703: getdents d9/d15/d29 0 2024-02-26T10:48:03.788 INFO:tasks.workunit.client.0.smithi060.stdout:0/674: rmdir d0/dd/d33 39 2024-02-26T10:48:03.789 INFO:tasks.workunit.client.0.smithi060.stdout:8/704: creat d9/d5a/fe7 x:0 0 0 2024-02-26T10:48:03.795 INFO:tasks.workunit.client.0.smithi060.stdout:0/675: unlink d0/dd/c74 0 2024-02-26T10:48:03.796 INFO:tasks.workunit.client.0.smithi060.stdout:8/705: unlink d9/d15/d29/d2f/d34/d72/d94/dd4/fe2 0 2024-02-26T10:48:03.797 INFO:tasks.workunit.client.0.smithi060.stdout:0/676: write d0/dd/d39/d40/d83/f8f [1311059,97029] 0 2024-02-26T10:48:03.798 INFO:tasks.workunit.client.0.smithi060.stdout:8/706: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/fe8 x:0 0 0 2024-02-26T10:48:03.805 INFO:tasks.workunit.client.0.smithi060.stdout:5/656: dread d0/f5 [0,4194304] 0 2024-02-26T10:48:03.805 INFO:tasks.workunit.client.0.smithi060.stdout:5/657: stat d0/de/d20/d32/d4d/da3/da5/dc8/fd6 0 2024-02-26T10:48:03.806 INFO:tasks.workunit.client.0.smithi060.stdout:5/658: truncate d0/de/d15/d65/d86/d9b/f72 636488 0 2024-02-26T10:48:03.808 INFO:tasks.workunit.client.0.smithi060.stdout:5/659: rename d0/d33/c52 to d0/de/d20/d32/d4c/dd2/ce5 0 2024-02-26T10:48:03.810 INFO:tasks.workunit.client.0.smithi060.stdout:5/660: creat d0/de/d20/d32/d4c/fe6 x:0 0 0 2024-02-26T10:48:03.810 INFO:tasks.workunit.client.0.smithi060.stdout:5/661: fdatasync d0/de/d20/d32/d4c/f74 0 2024-02-26T10:48:03.810 INFO:tasks.workunit.client.0.smithi060.stdout:5/662: dread - d0/de/d20/d32/d4c/dd2/fde zero size 2024-02-26T10:48:03.812 INFO:tasks.workunit.client.0.smithi060.stdout:8/707: creat d9/d6f/fe9 x:0 0 0 2024-02-26T10:48:03.814 INFO:tasks.workunit.client.0.smithi060.stdout:3/524: dread d5/d37/f41 [0,4194304] 0 2024-02-26T10:48:03.814 INFO:tasks.workunit.client.0.smithi060.stdout:8/708: symlink d9/da4/lea 0 2024-02-26T10:48:03.815 INFO:tasks.workunit.client.0.smithi060.stdout:5/663: creat d0/de/d70/d84/d92/fe7 x:0 0 0 2024-02-26T10:48:03.815 INFO:tasks.workunit.client.0.smithi060.stdout:5/664: read - d0/de/da4/fb3 zero size 2024-02-26T10:48:03.815 INFO:tasks.workunit.client.0.smithi060.stdout:1/790: dread d3/db/d1e/d38/d6c/fb9 [0,4194304] 0 2024-02-26T10:48:03.816 INFO:tasks.workunit.client.0.smithi060.stdout:8/709: mkdir d9/da4/deb 0 2024-02-26T10:48:03.816 INFO:tasks.workunit.client.0.smithi060.stdout:8/710: chown d9/f24 314 1 2024-02-26T10:48:03.816 INFO:tasks.workunit.client.0.smithi060.stdout:8/711: fsync d9/d5a/fcc 0 2024-02-26T10:48:03.816 INFO:tasks.workunit.client.0.smithi060.stdout:5/665: creat d0/de/d20/d82/fe8 x:0 0 0 2024-02-26T10:48:03.816 INFO:tasks.workunit.client.0.smithi060.stdout:5/666: truncate d0/de/d15/d26/fce 534562 0 2024-02-26T10:48:03.817 INFO:tasks.workunit.client.0.smithi060.stdout:5/667: symlink d0/de/le9 0 2024-02-26T10:48:03.817 INFO:tasks.workunit.client.0.smithi060.stdout:1/791: fdatasync d3/db/d1e/d6d/da3/daa/dd6/fdf 0 2024-02-26T10:48:03.817 INFO:tasks.workunit.client.0.smithi060.stdout:1/792: truncate d3/db/d2e/f33 4837078 0 2024-02-26T10:48:03.822 INFO:tasks.workunit.client.0.smithi060.stdout:1/793: unlink d3/db/d2e/fd5 0 2024-02-26T10:48:03.830 INFO:tasks.workunit.client.0.smithi060.stdout:7/822: dwrite da/d15/d21/d2f/f101 [4194304,4194304] 0 2024-02-26T10:48:03.830 INFO:tasks.workunit.client.0.smithi060.stdout:6/705: dwrite d6/d7/d25/d1c/d1f/d23/d28/f7f [0,4194304] 0 2024-02-26T10:48:03.831 INFO:tasks.workunit.client.0.smithi060.stdout:6/706: chown d6/d7/d25/l92 74635 1 2024-02-26T10:48:03.831 INFO:tasks.workunit.client.0.smithi060.stdout:9/719: write d3/f6b [3544057,106926] 0 2024-02-26T10:48:03.831 INFO:tasks.workunit.client.0.smithi060.stdout:9/720: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f6d [871785,63413] 0 2024-02-26T10:48:03.831 INFO:tasks.workunit.client.0.smithi060.stdout:9/721: readlink d3/d68/l77 0 2024-02-26T10:48:03.836 INFO:tasks.workunit.client.0.smithi060.stdout:6/707: creat d6/d6e/d6f/fe9 x:0 0 0 2024-02-26T10:48:03.836 INFO:tasks.workunit.client.0.smithi060.stdout:6/708: dread - d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac/db5/fce zero size 2024-02-26T10:48:03.844 INFO:tasks.workunit.client.0.smithi060.stdout:7/823: mknod da/d15/c10f 0 2024-02-26T10:48:03.844 INFO:tasks.workunit.client.0.smithi060.stdout:7/824: chown da/d15/d21/d29/c106 328091 1 2024-02-26T10:48:03.849 INFO:tasks.workunit.client.0.smithi060.stdout:6/709: rename d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/l9f to d6/lea 0 2024-02-26T10:48:03.849 INFO:tasks.workunit.client.0.smithi060.stdout:4/683: dwrite d0/da/d57/f65 [0,4194304] 0 2024-02-26T10:48:03.855 INFO:tasks.workunit.client.0.smithi060.stdout:4/684: symlink d0/da/d3b/d1b/ld7 0 2024-02-26T10:48:03.860 INFO:tasks.workunit.client.0.smithi060.stdout:6/710: truncate d6/d7/d47/d69/f82 1443207 0 2024-02-26T10:48:03.863 INFO:tasks.workunit.client.0.smithi060.stdout:8/712: write d9/d15/d29/d2f/d34/d72/d50/d6a/d63/fc8 [3527714,63888] 0 2024-02-26T10:48:03.863 INFO:tasks.workunit.client.0.smithi060.stdout:6/711: creat d6/d7/d2e/feb x:0 0 0 2024-02-26T10:48:03.870 INFO:tasks.workunit.client.0.smithi060.stdout:6/712: stat d6/d7/d2e/l6c 0 2024-02-26T10:48:03.870 INFO:tasks.workunit.client.0.smithi060.stdout:2/682: dwrite dd/d43/d60/d41/f36 [0,4194304] 0 2024-02-26T10:48:03.870 INFO:tasks.workunit.client.0.smithi060.stdout:8/713: creat d9/d15/d29/d2f/d34/d72/d7e/fec x:0 0 0 2024-02-26T10:48:03.871 INFO:tasks.workunit.client.0.smithi060.stdout:0/677: dread d0/dd/f2d [0,4194304] 0 2024-02-26T10:48:03.871 INFO:tasks.workunit.client.0.smithi060.stdout:0/678: chown d0/dd/d39/d40/f34 14812 1 2024-02-26T10:48:03.871 INFO:tasks.workunit.client.0.smithi060.stdout:0/679: fsync d0/dd/d53/da2/faa 0 2024-02-26T10:48:03.878 INFO:tasks.workunit.client.0.smithi060.stdout:2/683: creat dd/d43/d55/d6e/fe9 x:0 0 0 2024-02-26T10:48:03.879 INFO:tasks.workunit.client.0.smithi060.stdout:2/684: chown dd/d43/d60/d41/d37/d38/l61 6 1 2024-02-26T10:48:03.879 INFO:tasks.workunit.client.0.smithi060.stdout:2/685: write dd/d43/d60/d41/d37/d69/f72 [546052,20672] 0 2024-02-26T10:48:03.887 INFO:tasks.workunit.client.0.smithi060.stdout:5/668: dwrite d0/de/d20/d32/f89 [0,4194304] 0 2024-02-26T10:48:03.894 INFO:tasks.workunit.client.0.smithi060.stdout:3/525: dwrite d5/de/df/d57/d60/f1e [0,4194304] 0 2024-02-26T10:48:03.895 INFO:tasks.workunit.client.0.smithi060.stdout:8/714: rename d9/d15/c5b to d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/dc1/ced 0 2024-02-26T10:48:03.897 INFO:tasks.workunit.client.0.smithi060.stdout:0/680: rename d0/dd/d39/d40/dd8 to d0/dd/d39/d40/dad/dda/de4 0 2024-02-26T10:48:03.899 INFO:tasks.workunit.client.0.smithi060.stdout:1/794: dwrite d3/d35/d53/d68/d85/f94 [0,4194304] 0 2024-02-26T10:48:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:2/686: rename dd/d43/d60/d41/d37/d9a/fd0 to dd/d43/d55/d5e/d86/d97/d9c/fea 0 2024-02-26T10:48:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:2/687: chown dd/d11/f44 74361 1 2024-02-26T10:48:03.917 INFO:tasks.workunit.client.0.smithi060.stdout:5/669: dread d0/de/d15/f5a [0,4194304] 0 2024-02-26T10:48:03.917 INFO:tasks.workunit.client.0.smithi060.stdout:5/670: fsync d0/f30 0 2024-02-26T10:48:03.918 INFO:tasks.workunit.client.0.smithi060.stdout:2/688: rename dd/d11/f44 to dd/d43/d55/d5e/d86/d97/db9/feb 0 2024-02-26T10:48:03.919 INFO:tasks.workunit.client.0.smithi060.stdout:9/722: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/f8f [0,4194304] 0 2024-02-26T10:48:03.919 INFO:tasks.workunit.client.0.smithi060.stdout:5/671: truncate d0/f17 337190 0 2024-02-26T10:48:03.920 INFO:tasks.workunit.client.0.smithi060.stdout:2/689: mknod dd/d43/d55/d6e/dd3/d6f/cec 0 2024-02-26T10:48:03.925 INFO:tasks.workunit.client.0.smithi060.stdout:9/723: rmdir d3/d1e/d5e/d35 39 2024-02-26T10:48:03.926 INFO:tasks.workunit.client.0.smithi060.stdout:9/724: dread - d3/d1e/d93/fce zero size 2024-02-26T10:48:03.926 INFO:tasks.workunit.client.0.smithi060.stdout:9/725: fsync d3/d1e/d5e/d35/fd7 0 2024-02-26T10:48:03.926 INFO:tasks.workunit.client.0.smithi060.stdout:9/726: fsync d3/d1e/d5e/d35/f3e 0 2024-02-26T10:48:03.935 INFO:tasks.workunit.client.0.smithi060.stdout:2/690: write dd/d43/d55/d5e/d74/da5/fa7 [6546473,84685] 0 2024-02-26T10:48:03.936 INFO:tasks.workunit.client.0.smithi060.stdout:2/691: symlink dd/d43/d60/d41/d37/d4c/led 0 2024-02-26T10:48:03.936 INFO:tasks.workunit.client.0.smithi060.stdout:2/692: readlink dd/d43/d60/d41/l93 0 2024-02-26T10:48:03.940 INFO:tasks.workunit.client.0.smithi060.stdout:2/693: rmdir dd/d43 39 2024-02-26T10:48:03.941 INFO:tasks.workunit.client.0.smithi060.stdout:2/694: getdents dd/d43/d60/d41/d37/d9a 0 2024-02-26T10:48:03.943 INFO:tasks.workunit.client.0.smithi060.stdout:2/695: rename dd/fcc to dd/d43/d60/d41/d37/d4c/fee 0 2024-02-26T10:48:03.943 INFO:tasks.workunit.client.0.smithi060.stdout:2/696: dread - dd/d43/d60/d41/d37/d69/fe4 zero size 2024-02-26T10:48:03.943 INFO:tasks.workunit.client.0.smithi060.stdout:6/713: read f4 [1159560,43891] 0 2024-02-26T10:48:03.943 INFO:tasks.workunit.client.0.smithi060.stdout:1/795: link d3/db/d1e/d39/c5d d3/d35/de6/c10e 0 2024-02-26T10:48:03.944 INFO:tasks.workunit.client.0.smithi060.stdout:4/685: dwrite d0/da/dc7/db0/db5/dc0/f6a [0,4194304] 0 2024-02-26T10:48:03.944 INFO:tasks.workunit.client.0.smithi060.stdout:2/697: mkdir dd/d43/d55/d5e/d74/da5/def 0 2024-02-26T10:48:03.944 INFO:tasks.workunit.client.0.smithi060.stdout:2/698: readlink dd/d43/d55/l73 0 2024-02-26T10:48:03.960 INFO:tasks.workunit.client.0.smithi060.stdout:1/796: rename d3/f15 to d3/db/d1e/d6d/da3/f10f 0 2024-02-26T10:48:03.973 INFO:tasks.workunit.client.0.smithi060.stdout:9/727: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fd9 [0,4194304] 0 2024-02-26T10:48:03.976 INFO:tasks.workunit.client.0.smithi060.stdout:2/699: dread dd/d43/d60/d41/d37/fa0 [4194304,4194304] 0 2024-02-26T10:48:03.976 INFO:tasks.workunit.client.0.smithi060.stdout:2/700: fdatasync dd/d43/d60/d41/f31 0 2024-02-26T10:48:03.977 INFO:tasks.workunit.client.0.smithi060.stdout:2/701: chown dd/d43/d55/d5e/d86/d97/d9c/fea 92 1 2024-02-26T10:48:03.981 INFO:tasks.workunit.client.0.smithi060.stdout:9/728: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f7e [387956,121116] 0 2024-02-26T10:48:03.981 INFO:tasks.workunit.client.0.smithi060.stdout:9/729: write d3/d1e/d57/f88 [873175,75145] 0 2024-02-26T10:48:03.981 INFO:tasks.workunit.client.0.smithi060.stdout:9/730: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f7e [275479,92663] 0 2024-02-26T10:48:03.981 INFO:tasks.workunit.client.0.smithi060.stdout:6/714: dwrite d6/d7/d25/d80/fae [0,4194304] 0 2024-02-26T10:48:03.981 INFO:tasks.workunit.client.0.smithi060.stdout:2/702: symlink dd/d11/lf0 0 2024-02-26T10:48:03.984 INFO:tasks.workunit.client.0.smithi060.stdout:2/703: rmdir dd/d11/de3 0 2024-02-26T10:48:03.986 INFO:tasks.workunit.client.0.smithi060.stdout:2/704: creat dd/d43/d60/d41/d37/d69/ff1 x:0 0 0 2024-02-26T10:48:03.987 INFO:tasks.workunit.client.0.smithi060.stdout:0/681: dwrite d0/d42/fd7 [0,4194304] 0 2024-02-26T10:48:03.988 INFO:tasks.workunit.client.0.smithi060.stdout:2/705: mkdir dd/d43/d96/df2 0 2024-02-26T10:48:03.991 INFO:tasks.workunit.client.0.smithi060.stdout:0/682: rename d0/dd/d33/f4a to d0/dd/d39/d63/fe5 0 2024-02-26T10:48:03.993 INFO:tasks.workunit.client.0.smithi060.stdout:0/683: creat d0/dd/d39/d40/d83/dc0/fe6 x:0 0 0 2024-02-26T10:48:03.995 INFO:tasks.workunit.client.0.smithi060.stdout:1/797: write d3/d35/d53/d68/d6b/f9e [899043,33555] 0 2024-02-26T10:48:03.995 INFO:tasks.workunit.client.0.smithi060.stdout:0/684: creat d0/dd/d53/d87/d94/da6/dc7/fe7 x:0 0 0 2024-02-26T10:48:03.998 INFO:tasks.workunit.client.0.smithi060.stdout:0/685: mknod d0/dd/d39/d40/dad/dda/de4/ce8 0 2024-02-26T10:48:03.998 INFO:tasks.workunit.client.0.smithi060.stdout:0/686: read - d0/dd/d53/d87/fd2 zero size 2024-02-26T10:48:04.000 INFO:tasks.workunit.client.0.smithi060.stdout:1/798: symlink d3/d35/dc2/l110 0 2024-02-26T10:48:04.002 INFO:tasks.workunit.client.0.smithi060.stdout:0/687: write d0/dd/f4e [5064075,26336] 0 2024-02-26T10:48:04.002 INFO:tasks.workunit.client.0.smithi060.stdout:1/799: stat d3/d35/dc2/cd4 0 2024-02-26T10:48:04.004 INFO:tasks.workunit.client.0.smithi060.stdout:0/688: rename d0/dd/d39/d63/c75 to d0/dd/d33/ce9 0 2024-02-26T10:48:04.005 INFO:tasks.workunit.client.0.smithi060.stdout:4/686: dwrite d0/d27/d2f/f55 [0,4194304] 0 2024-02-26T10:48:04.006 INFO:tasks.workunit.client.0.smithi060.stdout:0/689: creat d0/dd/d39/d40/dde/fea x:0 0 0 2024-02-26T10:48:04.007 INFO:tasks.workunit.client.0.smithi060.stdout:6/715: read d6/d7/d25/d1c/d1f/d23/d28/f34 [303993,99795] 0 2024-02-26T10:48:04.007 INFO:tasks.workunit.client.0.smithi060.stdout:9/731: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/f8f [0,4194304] 0 2024-02-26T10:48:04.008 INFO:tasks.workunit.client.0.smithi060.stdout:0/690: stat d0/dd/d53/d87/d94/ldb 0 2024-02-26T10:48:04.011 INFO:tasks.workunit.client.0.smithi060.stdout:1/800: rename d3/db to d3/db/dc8/d111 22 2024-02-26T10:48:04.011 INFO:tasks.workunit.client.0.smithi060.stdout:6/716: creat d6/d8d/dcc/fec x:0 0 0 2024-02-26T10:48:04.015 INFO:tasks.workunit.client.0.smithi060.stdout:1/801: creat d3/d35/d53/f112 x:0 0 0 2024-02-26T10:48:04.031 INFO:tasks.workunit.client.0.smithi060.stdout:4/687: write d0/da/f80 [2977006,67757] 0 2024-02-26T10:48:04.034 INFO:tasks.workunit.client.0.smithi060.stdout:4/688: rename d0/d27/d76/dbd/d8e/fb4 to d0/da/fd8 0 2024-02-26T10:48:04.034 INFO:tasks.workunit.client.0.smithi060.stdout:4/689: read d0/da/d3b/d16/f6c [1172561,38699] 0 2024-02-26T10:48:04.034 INFO:tasks.workunit.client.0.smithi060.stdout:4/690: readlink d0/da/d3b/d1b/l1c 0 2024-02-26T10:48:04.035 INFO:tasks.workunit.client.0.smithi060.stdout:4/691: rmdir d0/da/d3b/d1b/dc2 0 2024-02-26T10:48:04.038 INFO:tasks.workunit.client.0.smithi060.stdout:0/691: dread d0/d42/fd7 [0,4194304] 0 2024-02-26T10:48:04.040 INFO:tasks.workunit.client.0.smithi060.stdout:0/692: read d0/dd/d39/d40/f1f [4315220,121849] 0 2024-02-26T10:48:04.047 INFO:tasks.workunit.client.0.smithi060.stdout:2/706: dwrite dd/d43/d55/d5e/dc6/fcb [0,4194304] 0 2024-02-26T10:48:04.047 INFO:tasks.workunit.client.0.smithi060.stdout:2/707: getdents dd/d43/d60/d41/d37/d9a 0 2024-02-26T10:48:04.048 INFO:tasks.workunit.client.0.smithi060.stdout:9/732: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/f8f [0,4194304] 0 2024-02-26T10:48:04.048 INFO:tasks.workunit.client.0.smithi060.stdout:9/733: readlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/l12 0 2024-02-26T10:48:04.052 INFO:tasks.workunit.client.0.smithi060.stdout:6/717: dread d6/d7/d25/f2d [0,4194304] 0 2024-02-26T10:48:04.058 INFO:tasks.workunit.client.0.smithi060.stdout:6/718: symlink d6/d7/d47/dde/led 0 2024-02-26T10:48:04.060 INFO:tasks.workunit.client.0.smithi060.stdout:6/719: rename d6/d7/d25/fb0 to d6/d7/d25/d1c/d1f/d23/d2b/d4d/fee 0 2024-02-26T10:48:04.060 INFO:tasks.workunit.client.0.smithi060.stdout:6/720: chown d6/d7/d25/d1c/d1f/d23/faf 130 1 2024-02-26T10:48:04.062 INFO:tasks.workunit.client.0.smithi060.stdout:7/825: sync 2024-02-26T10:48:04.062 INFO:tasks.workunit.client.0.smithi060.stdout:6/721: mkdir d6/d7/d47/def 0 2024-02-26T10:48:04.065 INFO:tasks.workunit.client.0.smithi060.stdout:6/722: symlink d6/d7/d25/d1c/d1f/d23/d2b/d67/lf0 0 2024-02-26T10:48:04.066 INFO:tasks.workunit.client.0.smithi060.stdout:6/723: chown d6/d7/d25/d1c/f6d 5692 1 2024-02-26T10:48:04.066 INFO:tasks.workunit.client.0.smithi060.stdout:6/724: write d6/d7/d25/d1c/d1f/f37 [1345142,32342] 0 2024-02-26T10:48:04.067 INFO:tasks.workunit.client.0.smithi060.stdout:4/692: dread d0/da/d3b/f59 [0,4194304] 0 2024-02-26T10:48:04.069 INFO:tasks.workunit.client.0.smithi060.stdout:6/725: link d6/d8d/dcc/dc3/f44 d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/ff1 0 2024-02-26T10:48:04.069 INFO:tasks.workunit.client.0.smithi060.stdout:9/734: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8/dcd/fda [0,4194304] 0 2024-02-26T10:48:04.069 INFO:tasks.workunit.client.0.smithi060.stdout:9/735: truncate d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fe1 818034 0 2024-02-26T10:48:04.070 INFO:tasks.workunit.client.0.smithi060.stdout:4/693: creat d0/da/dc7/db0/fd9 x:0 0 0 2024-02-26T10:48:04.070 INFO:tasks.workunit.client.0.smithi060.stdout:6/726: symlink d6/d7/d25/d1c/d1f/d23/d2b/lf2 0 2024-02-26T10:48:04.072 INFO:tasks.workunit.client.0.smithi060.stdout:4/694: creat d0/da/dc7/db0/db5/dc0/fda x:0 0 0 2024-02-26T10:48:04.077 INFO:tasks.workunit.client.0.smithi060.stdout:6/727: unlink d6/d8d/fb3 0 2024-02-26T10:48:04.078 INFO:tasks.workunit.client.0.smithi060.stdout:4/695: unlink d0/d27/d76/dbd/f64 0 2024-02-26T10:48:04.078 INFO:tasks.workunit.client.0.smithi060.stdout:6/728: readlink d6/d7/d2e/d56/d73/lcb 0 2024-02-26T10:48:04.078 INFO:tasks.workunit.client.0.smithi060.stdout:4/696: write d0/d27/d76/dbd/d8e/db3/fc8 [22072,26571] 0 2024-02-26T10:48:04.080 INFO:tasks.workunit.client.0.smithi060.stdout:6/729: creat d6/d7/d47/dde/ff3 x:0 0 0 2024-02-26T10:48:04.080 INFO:tasks.workunit.client.0.smithi060.stdout:0/693: dread d0/dd/f31 [0,4194304] 0 2024-02-26T10:48:04.080 INFO:tasks.workunit.client.0.smithi060.stdout:1/802: dwrite d3/db/d1e/d39/f58 [0,4194304] 0 2024-02-26T10:48:04.080 INFO:tasks.workunit.client.0.smithi060.stdout:0/694: truncate d0/dd/d53/d87/fd2 883158 0 2024-02-26T10:48:04.080 INFO:tasks.workunit.client.0.smithi060.stdout:0/695: chown d0/c9c 41426 1 2024-02-26T10:48:04.080 INFO:tasks.workunit.client.0.smithi060.stdout:0/696: readlink d0/dd/d39/lc5 0 2024-02-26T10:48:04.081 INFO:tasks.workunit.client.0.smithi060.stdout:1/803: creat d3/db/d1e/d38/d6c/f113 x:0 0 0 2024-02-26T10:48:04.083 INFO:tasks.workunit.client.0.smithi060.stdout:4/697: read d0/da/d57/f58 [614941,124283] 0 2024-02-26T10:48:04.084 INFO:tasks.workunit.client.0.smithi060.stdout:6/730: truncate d6/d7/d25/f46 657580 0 2024-02-26T10:48:04.085 INFO:tasks.workunit.client.0.smithi060.stdout:7/826: write da/f18 [2560555,125032] 0 2024-02-26T10:48:04.086 INFO:tasks.workunit.client.0.smithi060.stdout:6/731: mknod d6/d7/d47/d69/cf4 0 2024-02-26T10:48:04.086 INFO:tasks.workunit.client.0.smithi060.stdout:6/732: chown d6/d7/d25/d1c/d1f/d23/d2b/d4d/f8b 262761488 1 2024-02-26T10:48:04.088 INFO:tasks.workunit.client.0.smithi060.stdout:1/804: symlink d3/db/d2e/l114 0 2024-02-26T10:48:04.093 INFO:tasks.workunit.client.0.smithi060.stdout:7/827: symlink da/d15/d21/d2f/d78/l110 0 2024-02-26T10:48:04.097 INFO:tasks.workunit.client.0.smithi060.stdout:2/708: dwrite dd/d43/d55/d5e/d86/d97/db9/feb [0,4194304] 0 2024-02-26T10:48:04.098 INFO:tasks.workunit.client.0.smithi060.stdout:2/709: fdatasync dd/d43/d55/d5e/d86/d97/fce 0 2024-02-26T10:48:04.105 INFO:tasks.workunit.client.0.smithi060.stdout:9/736: dread d3/d1e/d5e/d35/d87/fc1 [0,4194304] 0 2024-02-26T10:48:04.109 INFO:tasks.workunit.client.0.smithi060.stdout:4/698: write d0/ff [2835008,108216] 0 2024-02-26T10:48:04.110 INFO:tasks.workunit.client.0.smithi060.stdout:7/828: rename da/d15/d88/dce/dea to da/d66/dbd/d111 0 2024-02-26T10:48:04.110 INFO:tasks.workunit.client.0.smithi060.stdout:3/526: sync 2024-02-26T10:48:04.112 INFO:tasks.workunit.client.0.smithi060.stdout:2/710: rmdir dd/d43/d60/d41/d37/d69 39 2024-02-26T10:48:04.112 INFO:tasks.workunit.client.0.smithi060.stdout:2/711: read dd/d43/f6c [580487,58447] 0 2024-02-26T10:48:04.114 INFO:tasks.workunit.client.0.smithi060.stdout:4/699: unlink d0/da/d3b/d1b/l3d 0 2024-02-26T10:48:04.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:03 smithi060 ceph-mon[31539]: pgmap v488: 129 pgs: 129 active+clean; 1.0 GiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 135 MiB/s rd, 149 MiB/s wr, 748 op/s 2024-02-26T10:48:04.120 INFO:tasks.workunit.client.0.smithi060.stdout:7/829: dread da/d15/d21/d2f/d37/feb [0,4194304] 0 2024-02-26T10:48:04.123 INFO:tasks.workunit.client.0.smithi060.stdout:5/672: sync 2024-02-26T10:48:04.123 INFO:tasks.workunit.client.0.smithi060.stdout:8/715: sync 2024-02-26T10:48:04.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:03 smithi100 ceph-mon[36051]: pgmap v488: 129 pgs: 129 active+clean; 1.0 GiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 135 MiB/s rd, 149 MiB/s wr, 748 op/s 2024-02-26T10:48:04.127 INFO:tasks.workunit.client.0.smithi060.stdout:8/716: creat d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/fee x:0 0 0 2024-02-26T10:48:04.142 INFO:tasks.workunit.client.0.smithi060.stdout:0/697: dwrite d0/dd/d53/f64 [4194304,4194304] 0 2024-02-26T10:48:04.145 INFO:tasks.workunit.client.0.smithi060.stdout:0/698: symlink d0/dd/d53/d87/d94/leb 0 2024-02-26T10:48:04.145 INFO:tasks.workunit.client.0.smithi060.stdout:1/805: write d3/d35/d53/d68/d90/fbc [5130394,16895] 0 2024-02-26T10:48:04.145 INFO:tasks.workunit.client.0.smithi060.stdout:0/699: write d0/dd/d39/d93/fb6 [556243,121292] 0 2024-02-26T10:48:04.146 INFO:tasks.workunit.client.0.smithi060.stdout:1/806: truncate d3/d35/d53/d68/d6b/f107 824549 0 2024-02-26T10:48:04.146 INFO:tasks.workunit.client.0.smithi060.stdout:0/700: dread - d0/dd/d39/d40/dad/dd6/d6c/f9a zero size 2024-02-26T10:48:04.146 INFO:tasks.workunit.client.0.smithi060.stdout:7/830: dread da/d15/d88/d3b/f9d [0,4194304] 0 2024-02-26T10:48:04.146 INFO:tasks.workunit.client.0.smithi060.stdout:7/831: stat da/d15/d21/d29/c106 0 2024-02-26T10:48:04.147 INFO:tasks.workunit.client.0.smithi060.stdout:1/807: mknod d3/db/d1e/d6d/da3/daa/dcd/c115 0 2024-02-26T10:48:04.148 INFO:tasks.workunit.client.0.smithi060.stdout:0/701: unlink d0/dd/d53/d87/d94/leb 0 2024-02-26T10:48:04.148 INFO:tasks.workunit.client.0.smithi060.stdout:0/702: write d0/dd/d33/f88 [104718,30808] 0 2024-02-26T10:48:04.149 INFO:tasks.workunit.client.0.smithi060.stdout:7/832: mknod da/d94/c112 0 2024-02-26T10:48:04.149 INFO:tasks.workunit.client.0.smithi060.stdout:2/712: dread dd/d43/d60/d41/d37/fa0 [0,4194304] 0 2024-02-26T10:48:04.152 INFO:tasks.workunit.client.0.smithi060.stdout:0/703: mknod d0/dd/d53/d87/d94/da6/dc7/cec 0 2024-02-26T10:48:04.153 INFO:tasks.workunit.client.0.smithi060.stdout:1/808: unlink d3/d35/d53/d68/d85/lfb 0 2024-02-26T10:48:04.153 INFO:tasks.workunit.client.0.smithi060.stdout:1/809: truncate d3/db/dc8/f9a 379106 0 2024-02-26T10:48:04.154 INFO:tasks.workunit.client.0.smithi060.stdout:1/810: chown d3/d35/d53/d57 1 1 2024-02-26T10:48:04.155 INFO:tasks.workunit.client.0.smithi060.stdout:1/811: rename d3/d35/d53/ffa to d3/d35/d53/d68/f116 0 2024-02-26T10:48:04.155 INFO:tasks.workunit.client.0.smithi060.stdout:1/812: readlink d3/db/d1e/d6d/d78/led 0 2024-02-26T10:48:04.156 INFO:tasks.workunit.client.0.smithi060.stdout:1/813: write d3/db/d2e/d51/d63/f92 [999299,15669] 0 2024-02-26T10:48:04.158 INFO:tasks.workunit.client.0.smithi060.stdout:1/814: truncate d3/d35/d53/d57/f83 1016192 0 2024-02-26T10:48:04.161 INFO:tasks.workunit.client.0.smithi060.stdout:1/815: mkdir d3/d35/d53/d68/d117 0 2024-02-26T10:48:04.162 INFO:tasks.workunit.client.0.smithi060.stdout:1/816: write d3/d35/f65 [413528,83030] 0 2024-02-26T10:48:04.163 INFO:tasks.workunit.client.0.smithi060.stdout:8/717: dread d9/f20 [0,4194304] 0 2024-02-26T10:48:04.167 INFO:tasks.workunit.client.0.smithi060.stdout:1/817: link d3/d35/d5e/f97 d3/db/f118 0 2024-02-26T10:48:04.176 INFO:tasks.workunit.client.0.smithi060.stdout:6/733: dwrite d6/fe [0,4194304] 0 2024-02-26T10:48:04.176 INFO:tasks.workunit.client.0.smithi060.stdout:8/718: rename d9/c17 to d9/d5a/d9c/cef 0 2024-02-26T10:48:04.189 INFO:tasks.workunit.client.0.smithi060.stdout:6/734: dread d6/d8d/dcc/dc3/f77 [0,4194304] 0 2024-02-26T10:48:04.189 INFO:tasks.workunit.client.0.smithi060.stdout:6/735: fsync d6/d7/d25/d1c/d1f/d23/f3a 0 2024-02-26T10:48:04.190 INFO:tasks.workunit.client.0.smithi060.stdout:6/736: fsync d6/f9 0 2024-02-26T10:48:04.191 INFO:tasks.workunit.client.0.smithi060.stdout:6/737: link d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac/cba d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/cf5 0 2024-02-26T10:48:04.193 INFO:tasks.workunit.client.0.smithi060.stdout:0/704: write d0/dd/d39/d40/f19 [8033280,67577] 0 2024-02-26T10:48:04.193 INFO:tasks.workunit.client.0.smithi060.stdout:0/705: fdatasync d0/dd/d39/fb8 0 2024-02-26T10:48:04.196 INFO:tasks.workunit.client.0.smithi060.stdout:6/738: creat d6/d7/d25/d1c/ff6 x:0 0 0 2024-02-26T10:48:04.197 INFO:tasks.workunit.client.0.smithi060.stdout:6/739: rmdir d6/d7/d25/d1c/d6b 39 2024-02-26T10:48:04.203 INFO:tasks.workunit.client.0.smithi060.stdout:4/700: dwrite d0/d27/d76/f93 [0,4194304] 0 2024-02-26T10:48:04.203 INFO:tasks.workunit.client.0.smithi060.stdout:4/701: stat d0/d27/d76/f8a 0 2024-02-26T10:48:04.207 INFO:tasks.workunit.client.0.smithi060.stdout:7/833: dwrite f6 [0,4194304] 0 2024-02-26T10:48:04.208 INFO:tasks.workunit.client.0.smithi060.stdout:7/834: symlink da/d94/l113 0 2024-02-26T10:48:04.212 INFO:tasks.workunit.client.0.smithi060.stdout:7/835: symlink da/d15/d21/d41/d58/l114 0 2024-02-26T10:48:04.217 INFO:tasks.workunit.client.0.smithi060.stdout:7/836: dread da/d15/d21/d2f/d37/feb [0,4194304] 0 2024-02-26T10:48:04.219 INFO:tasks.workunit.client.0.smithi060.stdout:7/837: mknod da/d66/dad/d8f/c115 0 2024-02-26T10:48:04.220 INFO:tasks.workunit.client.0.smithi060.stdout:7/838: symlink da/d15/d21/l116 0 2024-02-26T10:48:04.221 INFO:tasks.workunit.client.0.smithi060.stdout:7/839: mknod da/d15/d88/dce/c117 0 2024-02-26T10:48:04.221 INFO:tasks.workunit.client.0.smithi060.stdout:7/840: dread - da/d15/d21/d2f/d78/dd3/d67/f10e zero size 2024-02-26T10:48:04.222 INFO:tasks.workunit.client.0.smithi060.stdout:5/673: dwrite d0/f3a [0,4194304] 0 2024-02-26T10:48:04.224 INFO:tasks.workunit.client.0.smithi060.stdout:5/674: mknod d0/de/d15/d26/ddd/cea 0 2024-02-26T10:48:04.226 INFO:tasks.workunit.client.0.smithi060.stdout:9/737: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f37 [0,4194304] 0 2024-02-26T10:48:04.226 INFO:tasks.workunit.client.0.smithi060.stdout:9/738: chown d3/d1e/d5e/ld6 441913624 1 2024-02-26T10:48:04.227 INFO:tasks.workunit.client.0.smithi060.stdout:5/675: link d0/de/d15/d65/d86/lcb d0/d33/leb 0 2024-02-26T10:48:04.227 INFO:tasks.workunit.client.0.smithi060.stdout:5/676: dread - d0/de/d15/d65/f9d zero size 2024-02-26T10:48:04.229 INFO:tasks.workunit.client.0.smithi060.stdout:5/677: symlink d0/d4b/d7b/lec 0 2024-02-26T10:48:04.231 INFO:tasks.workunit.client.0.smithi060.stdout:7/841: dread da/d66/f75 [4194304,4194304] 0 2024-02-26T10:48:04.233 INFO:tasks.workunit.client.0.smithi060.stdout:7/842: mknod da/df1/dd7/c118 0 2024-02-26T10:48:04.234 INFO:tasks.workunit.client.0.smithi060.stdout:7/843: readlink da/d15/d21/d2f/d78/l9f 0 2024-02-26T10:48:04.235 INFO:tasks.workunit.client.0.smithi060.stdout:0/706: fdatasync d0/dd/d33/f88 0 2024-02-26T10:48:04.241 INFO:tasks.workunit.client.0.smithi060.stdout:0/707: truncate d0/f55 1083385 0 2024-02-26T10:48:04.250 INFO:tasks.workunit.client.0.smithi060.stdout:8/719: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/fdb [0,4194304] 0 2024-02-26T10:48:04.253 INFO:tasks.workunit.client.0.smithi060.stdout:8/720: unlink d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/fe8 0 2024-02-26T10:48:04.256 INFO:tasks.workunit.client.0.smithi060.stdout:8/721: mkdir d9/d5a/d9c/df0 0 2024-02-26T10:48:04.256 INFO:tasks.workunit.client.0.smithi060.stdout:8/722: stat d9/d15/d29/f2c 0 2024-02-26T10:48:04.258 INFO:tasks.workunit.client.0.smithi060.stdout:8/723: unlink d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/f96 0 2024-02-26T10:48:04.259 INFO:tasks.workunit.client.0.smithi060.stdout:9/739: write d3/d1e/d5e/d1b/d23/f36 [187797,82351] 0 2024-02-26T10:48:04.259 INFO:tasks.workunit.client.0.smithi060.stdout:9/740: chown d3/d1e/d5e/d35/fd7 414 1 2024-02-26T10:48:04.259 INFO:tasks.workunit.client.0.smithi060.stdout:9/741: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/fcc 85 1 2024-02-26T10:48:04.260 INFO:tasks.workunit.client.0.smithi060.stdout:9/742: truncate d3/d1e/d5e/d35/d87/d9b/d8a/dbe/f38 2704406 0 2024-02-26T10:48:04.260 INFO:tasks.workunit.client.0.smithi060.stdout:9/743: write d3/f6b [4750877,105838] 0 2024-02-26T10:48:04.262 INFO:tasks.workunit.client.0.smithi060.stdout:9/744: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/dfb 0 2024-02-26T10:48:04.263 INFO:tasks.workunit.client.0.smithi060.stdout:9/745: truncate d3/d68/fae 315242 0 2024-02-26T10:48:04.264 INFO:tasks.workunit.client.0.smithi060.stdout:7/844: dread da/d66/dad/f56 [0,4194304] 0 2024-02-26T10:48:04.265 INFO:tasks.workunit.client.0.smithi060.stdout:7/845: readlink da/d15/d21/d2f/d37/l44 0 2024-02-26T10:48:04.265 INFO:tasks.workunit.client.0.smithi060.stdout:9/746: truncate d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/fcc 382899 0 2024-02-26T10:48:04.265 INFO:tasks.workunit.client.0.smithi060.stdout:4/702: write d0/d27/d76/dbd/f45 [3303019,62139] 0 2024-02-26T10:48:04.265 INFO:tasks.workunit.client.0.smithi060.stdout:4/703: chown d0/d27/d76/dbd/c79 771 1 2024-02-26T10:48:04.265 INFO:tasks.workunit.client.0.smithi060.stdout:4/704: fsync d0/da/f80 0 2024-02-26T10:48:04.265 INFO:tasks.workunit.client.0.smithi060.stdout:4/705: chown d0/d27/d76/dbd/c63 98 1 2024-02-26T10:48:04.267 INFO:tasks.workunit.client.0.smithi060.stdout:7/846: rename da/l11 to da/d66/dad/d8f/l119 0 2024-02-26T10:48:04.267 INFO:tasks.workunit.client.0.smithi060.stdout:6/740: dwrite d6/d7/ff [0,4194304] 0 2024-02-26T10:48:04.269 INFO:tasks.workunit.client.0.smithi060.stdout:5/678: dread d0/de/d15/d65/d86/dcc/fd5 [0,4194304] 0 2024-02-26T10:48:04.271 INFO:tasks.workunit.client.0.smithi060.stdout:7/847: rmdir da/d15/d21/d2f/d78/dd3/d67/da8/de3 39 2024-02-26T10:48:04.279 INFO:tasks.workunit.client.0.smithi060.stdout:6/741: truncate d6/d7/d25/d1c/f2c 1608268 0 2024-02-26T10:48:04.279 INFO:tasks.workunit.client.0.smithi060.stdout:5/679: link d0/de/d20/d32/d4c/f74 d0/de/d20/d32/fed 0 2024-02-26T10:48:04.279 INFO:tasks.workunit.client.0.smithi060.stdout:5/680: unlink d0/de/d70/d84/d92/fe7 0 2024-02-26T10:48:04.279 INFO:tasks.workunit.client.0.smithi060.stdout:9/747: read d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f80 [3467674,22079] 0 2024-02-26T10:48:04.279 INFO:tasks.workunit.client.0.smithi060.stdout:5/681: mknod d0/d33/cee 0 2024-02-26T10:48:04.279 INFO:tasks.workunit.client.0.smithi060.stdout:5/682: creat d0/d33/fef x:0 0 0 2024-02-26T10:48:04.283 INFO:tasks.workunit.client.0.smithi060.stdout:9/748: creat d3/d1e/d5e/d35/d87/d9b/ffc x:0 0 0 2024-02-26T10:48:04.286 INFO:tasks.workunit.client.0.smithi060.stdout:0/708: write d0/dd/d53/f64 [1555437,2159] 0 2024-02-26T10:48:04.287 INFO:tasks.workunit.client.0.smithi060.stdout:9/749: creat d3/d1e/d5e/d1b/d23/ffd x:0 0 0 2024-02-26T10:48:04.287 INFO:tasks.workunit.client.0.smithi060.stdout:9/750: stat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/de8 0 2024-02-26T10:48:04.287 INFO:tasks.workunit.client.0.smithi060.stdout:9/751: read d3/f6b [4254020,7361] 0 2024-02-26T10:48:04.287 INFO:tasks.workunit.client.0.smithi060.stdout:9/752: mknod d3/d1e/d93/da2/db7/cfe 0 2024-02-26T10:48:04.287 INFO:tasks.workunit.client.0.smithi060.stdout:9/753: write d3/d1e/d5e/d35/deb/d7a/fd1 [221247,41938] 0 2024-02-26T10:48:04.289 INFO:tasks.workunit.client.0.smithi060.stdout:5/683: rename d0/f90 to d0/de/d20/ff0 0 2024-02-26T10:48:04.289 INFO:tasks.workunit.client.0.smithi060.stdout:0/709: link d0/dd/d39/d40/d83/cb2 d0/dd/d39/d40/dad/dd6/ced 0 2024-02-26T10:48:04.290 INFO:tasks.workunit.client.0.smithi060.stdout:5/684: creat d0/de/d20/d32/d4c/ff1 x:0 0 0 2024-02-26T10:48:04.290 INFO:tasks.workunit.client.0.smithi060.stdout:5/685: readlink d0/de/l7e 0 2024-02-26T10:48:04.291 INFO:tasks.workunit.client.0.smithi060.stdout:0/710: rename d0/l22 to d0/lee 0 2024-02-26T10:48:04.292 INFO:tasks.workunit.client.0.smithi060.stdout:9/754: mknod d3/d1e/dbb/df5/cff 0 2024-02-26T10:48:04.293 INFO:tasks.workunit.client.0.smithi060.stdout:9/755: creat d3/d68/f100 x:0 0 0 2024-02-26T10:48:04.297 INFO:tasks.workunit.client.0.smithi060.stdout:5/686: write d0/de/d15/f2f [552711,17377] 0 2024-02-26T10:48:04.300 INFO:tasks.workunit.client.0.smithi060.stdout:6/742: dread d6/f16 [0,4194304] 0 2024-02-26T10:48:04.300 INFO:tasks.workunit.client.0.smithi060.stdout:5/687: truncate d0/de/d15/d26/fce 8614 0 2024-02-26T10:48:04.301 INFO:tasks.workunit.client.0.smithi060.stdout:6/743: creat d6/d7/d25/d1c/d1f/d23/ff7 x:0 0 0 2024-02-26T10:48:04.303 INFO:tasks.workunit.client.0.smithi060.stdout:6/744: mknod d6/d7/d47/cf8 0 2024-02-26T10:48:04.303 INFO:tasks.workunit.client.0.smithi060.stdout:6/745: chown d6/d7/d2e/d56/lad 277834 1 2024-02-26T10:48:04.303 INFO:tasks.workunit.client.0.smithi060.stdout:6/746: write d6/d6e/d6f/fcd [1023001,71703] 0 2024-02-26T10:48:04.303 INFO:tasks.workunit.client.0.smithi060.stdout:6/747: truncate d6/d7/d25/f59 299337 0 2024-02-26T10:48:04.308 INFO:tasks.workunit.client.0.smithi060.stdout:9/756: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f63 [0,4194304] 0 2024-02-26T10:48:04.314 INFO:tasks.workunit.client.0.smithi060.stdout:6/748: dread d6/d7/d25/d1c/d1f/d23/d28/f33 [0,4194304] 0 2024-02-26T10:48:04.316 INFO:tasks.workunit.client.0.smithi060.stdout:6/749: rename d6/d7/d25/d1c/d1f/d23/d2b/d67/dc0 to d6/d7/d25/d80/df9 0 2024-02-26T10:48:04.316 INFO:tasks.workunit.client.0.smithi060.stdout:6/750: fsync d6/d7/d25/d1c/f9e 0 2024-02-26T10:48:04.317 INFO:tasks.workunit.client.0.smithi060.stdout:6/751: fsync d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/f94 0 2024-02-26T10:48:04.327 INFO:tasks.workunit.client.0.smithi060.stdout:0/711: dwrite d0/dd/fbb [0,4194304] 0 2024-02-26T10:48:04.327 INFO:tasks.workunit.client.0.smithi060.stdout:0/712: chown d0/dd/d53/d87/d94/ldb 2480950 1 2024-02-26T10:48:04.328 INFO:tasks.workunit.client.0.smithi060.stdout:0/713: rmdir d0/dd/d39/d40 39 2024-02-26T10:48:04.331 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:03 smithi113 ceph-mon[36126]: pgmap v488: 129 pgs: 129 active+clean; 1.0 GiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 135 MiB/s rd, 149 MiB/s wr, 748 op/s 2024-02-26T10:48:04.332 INFO:tasks.workunit.client.0.smithi060.stdout:0/714: link d0/dd/c4c d0/dd/d39/d93/cef 0 2024-02-26T10:48:04.335 INFO:tasks.workunit.client.0.smithi060.stdout:0/715: truncate d0/f5f 25632 0 2024-02-26T10:48:04.344 INFO:tasks.workunit.client.0.smithi060.stdout:4/706: dwrite d0/da/dc7/db0/db5/dc0/f37 [4194304,4194304] 0 2024-02-26T10:48:04.345 INFO:tasks.workunit.client.0.smithi060.stdout:6/752: dread d6/d7/d25/d1c/d1f/d23/d28/f7f [0,4194304] 0 2024-02-26T10:48:04.353 INFO:tasks.workunit.client.0.smithi060.stdout:4/707: getdents d0/d27 0 2024-02-26T10:48:04.355 INFO:tasks.workunit.client.0.smithi060.stdout:6/753: creat d6/d7/d47/d69/ffa x:0 0 0 2024-02-26T10:48:04.356 INFO:tasks.workunit.client.0.smithi060.stdout:4/708: creat d0/da/dc7/db0/db5/dc0/fdb x:0 0 0 2024-02-26T10:48:04.357 INFO:tasks.workunit.client.0.smithi060.stdout:6/754: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/da7/dfb 0 2024-02-26T10:48:04.359 INFO:tasks.workunit.client.0.smithi060.stdout:4/709: unlink d0/da/d3b/d1b/la8 0 2024-02-26T10:48:04.359 INFO:tasks.workunit.client.0.smithi060.stdout:9/757: dwrite d3/d68/fae [0,4194304] 0 2024-02-26T10:48:04.362 INFO:tasks.workunit.client.0.smithi060.stdout:9/758: getdents d3/d1e/d57 0 2024-02-26T10:48:04.364 INFO:tasks.workunit.client.0.smithi060.stdout:3/527: sync 2024-02-26T10:48:04.366 INFO:tasks.workunit.client.0.smithi060.stdout:3/528: mknod d5/c9f 0 2024-02-26T10:48:04.370 INFO:tasks.workunit.client.0.smithi060.stdout:5/688: dread d0/de/d20/f24 [0,4194304] 0 2024-02-26T10:48:04.370 INFO:tasks.workunit.client.0.smithi060.stdout:5/689: chown d0/d33 20487168 1 2024-02-26T10:48:04.370 INFO:tasks.workunit.client.0.smithi060.stdout:7/848: dwrite da/d15/d21/d2f/f68 [0,4194304] 0 2024-02-26T10:48:04.379 INFO:tasks.workunit.client.0.smithi060.stdout:9/759: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f65 [1934376,94791] 0 2024-02-26T10:48:04.387 INFO:tasks.workunit.client.0.smithi060.stdout:6/755: read d6/d7/d25/d1c/d1f/f35 [1998342,96461] 0 2024-02-26T10:48:04.388 INFO:tasks.workunit.client.0.smithi060.stdout:6/756: chown d6/d8d/ld9 214 1 2024-02-26T10:48:04.388 INFO:tasks.workunit.client.0.smithi060.stdout:6/757: mkdir d6/d7/d25/d1c/dfc 0 2024-02-26T10:48:04.419 INFO:tasks.workunit.client.0.smithi060.stdout:5/690: write d0/de/d15/d26/f45 [3015302,59783] 0 2024-02-26T10:48:04.420 INFO:tasks.workunit.client.0.smithi060.stdout:7/849: dwrite da/d15/f53 [0,4194304] 0 2024-02-26T10:48:04.420 INFO:tasks.workunit.client.0.smithi060.stdout:7/850: chown da/d66/dbd/ld1 139974 1 2024-02-26T10:48:04.420 INFO:tasks.workunit.client.0.smithi060.stdout:0/716: dwrite d0/dd/d39/d40/dad/fcb [0,4194304] 0 2024-02-26T10:48:04.420 INFO:tasks.workunit.client.0.smithi060.stdout:0/717: fdatasync d0/f82 0 2024-02-26T10:48:04.420 INFO:tasks.workunit.client.0.smithi060.stdout:7/851: symlink da/df1/l11a 0 2024-02-26T10:48:04.421 INFO:tasks.workunit.client.0.smithi060.stdout:5/691: creat d0/de/d15/d26/ddd/ff2 x:0 0 0 2024-02-26T10:48:04.421 INFO:tasks.workunit.client.0.smithi060.stdout:0/718: creat d0/dd/d39/d40/d83/dc0/ff0 x:0 0 0 2024-02-26T10:48:04.423 INFO:tasks.workunit.client.0.smithi060.stdout:5/692: rename d0/de/da4/fb3 to d0/de/d15/d65/d86/ff3 0 2024-02-26T10:48:04.423 INFO:tasks.workunit.client.0.smithi060.stdout:5/693: chown d0/d4b/d7b/cbe 10965 1 2024-02-26T10:48:04.423 INFO:tasks.workunit.client.0.smithi060.stdout:5/694: write d0/de/da4/fd7 [1365009,113260] 0 2024-02-26T10:48:04.443 INFO:tasks.workunit.client.0.smithi060.stdout:4/710: dwrite d0/d27/d76/dbd/d8e/fa1 [0,4194304] 0 2024-02-26T10:48:04.445 INFO:tasks.workunit.client.0.smithi060.stdout:9/760: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/d98/fa4 [0,4194304] 0 2024-02-26T10:48:04.447 INFO:tasks.workunit.client.0.smithi060.stdout:9/761: mkdir d3/d68/d101 0 2024-02-26T10:48:04.451 INFO:tasks.workunit.client.0.smithi060.stdout:9/762: symlink d3/d1e/d5e/d1b/d23/l102 0 2024-02-26T10:48:04.457 INFO:tasks.workunit.client.0.smithi060.stdout:0/719: write d0/f3 [836575,14511] 0 2024-02-26T10:48:04.457 INFO:tasks.workunit.client.0.smithi060.stdout:0/720: write d0/dd/d39/d40/dad/dd6/d6c/fdf [22766,32604] 0 2024-02-26T10:48:04.458 INFO:tasks.workunit.client.0.smithi060.stdout:6/758: dwrite d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f99 [0,4194304] 0 2024-02-26T10:48:04.458 INFO:tasks.workunit.client.0.smithi060.stdout:6/759: write d6/d7/d25/d1c/d1f/d23/d28/f33 [2005652,31948] 0 2024-02-26T10:48:04.458 INFO:tasks.workunit.client.0.smithi060.stdout:0/721: rename d0/dd/f2d to d0/ff1 0 2024-02-26T10:48:04.460 INFO:tasks.workunit.client.0.smithi060.stdout:6/760: unlink d6/d7/d47/d69/ffa 0 2024-02-26T10:48:04.468 INFO:tasks.workunit.client.0.smithi060.stdout:7/852: dwrite da/d15/d21/d2f/d78/dd3/d67/f10e [0,4194304] 0 2024-02-26T10:48:04.468 INFO:tasks.workunit.client.0.smithi060.stdout:7/853: write da/d15/d21/d2f/fbf [5062976,17454] 0 2024-02-26T10:48:04.475 INFO:tasks.workunit.client.0.smithi060.stdout:7/854: creat da/d66/dad/d8f/f11b x:0 0 0 2024-02-26T10:48:04.476 INFO:tasks.workunit.client.0.smithi060.stdout:7/855: creat da/d15/f11c x:0 0 0 2024-02-26T10:48:04.479 INFO:tasks.workunit.client.0.smithi060.stdout:7/856: truncate da/d66/dad/f56 859129 0 2024-02-26T10:48:04.479 INFO:tasks.workunit.client.0.smithi060.stdout:7/857: write da/d94/fb5 [517782,69722] 0 2024-02-26T10:48:04.483 INFO:tasks.workunit.client.0.smithi060.stdout:7/858: unlink da/d94/c9e 0 2024-02-26T10:48:04.483 INFO:tasks.workunit.client.0.smithi060.stdout:7/859: chown da/d15/d88/dce/c117 1729520249 1 2024-02-26T10:48:04.492 INFO:tasks.workunit.client.0.smithi060.stdout:7/860: write da/d15/d21/d2f/d78/dd3/f50 [142712,45618] 0 2024-02-26T10:48:04.492 INFO:tasks.workunit.client.0.smithi060.stdout:7/861: dread - da/d15/d21/d2f/d78/f10a zero size 2024-02-26T10:48:04.492 INFO:tasks.workunit.client.0.smithi060.stdout:7/862: write da/d15/d88/d3b/fde [236088,66809] 0 2024-02-26T10:48:04.493 INFO:tasks.workunit.client.0.smithi060.stdout:7/863: chown da/df1/lcb 1149922543 1 2024-02-26T10:48:04.494 INFO:tasks.workunit.client.0.smithi060.stdout:7/864: symlink da/d94/l11d 0 2024-02-26T10:48:04.496 INFO:tasks.workunit.client.0.smithi060.stdout:7/865: creat da/d15/d88/dce/f11e x:0 0 0 2024-02-26T10:48:04.505 INFO:tasks.workunit.client.0.smithi060.stdout:6/761: dwrite d6/f71 [0,4194304] 0 2024-02-26T10:48:04.505 INFO:tasks.workunit.client.0.smithi060.stdout:6/762: fdatasync d6/d6e/fc9 0 2024-02-26T10:48:04.505 INFO:tasks.workunit.client.0.smithi060.stdout:6/763: chown d6/d8d/fcf 0 1 2024-02-26T10:48:04.505 INFO:tasks.workunit.client.0.smithi060.stdout:6/764: stat d6/d7/d47 0 2024-02-26T10:48:04.506 INFO:tasks.workunit.client.0.smithi060.stdout:6/765: mkdir d6/d7/d2e/d56/dfd 0 2024-02-26T10:48:04.507 INFO:tasks.workunit.client.0.smithi060.stdout:6/766: symlink d6/d7/d25/d80/df9/lfe 0 2024-02-26T10:48:04.507 INFO:tasks.workunit.client.0.smithi060.stdout:6/767: write d6/d7/d2e/fb6 [4719450,100053] 0 2024-02-26T10:48:04.510 INFO:tasks.workunit.client.0.smithi060.stdout:9/763: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f11 [0,4194304] 0 2024-02-26T10:48:04.527 INFO:tasks.workunit.client.0.smithi060.stdout:0/722: dwrite d0/dd/f4e [4194304,4194304] 0 2024-02-26T10:48:04.527 INFO:tasks.workunit.client.0.smithi060.stdout:0/723: dread - d0/dd/d39/d40/dad/dd6/fcd zero size 2024-02-26T10:48:04.527 INFO:tasks.workunit.client.0.smithi060.stdout:9/764: rename d3/d1e/d5e/d35/d87/d9b/d8a/cb2 to d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/c103 0 2024-02-26T10:48:04.527 INFO:tasks.workunit.client.0.smithi060.stdout:9/765: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/fb6 [402362,80624] 0 2024-02-26T10:48:04.529 INFO:tasks.workunit.client.0.smithi060.stdout:0/724: mkdir d0/d42/df2 0 2024-02-26T10:48:04.529 INFO:tasks.workunit.client.0.smithi060.stdout:9/766: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/f104 x:0 0 0 2024-02-26T10:48:04.530 INFO:tasks.workunit.client.0.smithi060.stdout:9/767: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f105 x:0 0 0 2024-02-26T10:48:04.536 INFO:tasks.workunit.client.0.smithi060.stdout:7/866: dwrite da/d15/d21/d2f/d78/f10a [0,4194304] 0 2024-02-26T10:48:04.538 INFO:tasks.workunit.client.0.smithi060.stdout:7/867: rename da/d66/dbd/f102 to da/df1/f11f 0 2024-02-26T10:48:04.538 INFO:tasks.workunit.client.0.smithi060.stdout:7/868: getdents da/df1/dd7/dec 0 2024-02-26T10:48:04.540 INFO:tasks.workunit.client.0.smithi060.stdout:7/869: read da/d94/fb5 [67074,56283] 0 2024-02-26T10:48:04.541 INFO:tasks.workunit.client.0.smithi060.stdout:9/768: link d3/d1e/d5e/d1b/c6e d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/c106 0 2024-02-26T10:48:04.542 INFO:tasks.workunit.client.0.smithi060.stdout:7/870: creat da/d66/dad/d8f/f120 x:0 0 0 2024-02-26T10:48:04.543 INFO:tasks.workunit.client.0.smithi060.stdout:9/769: unlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/fbd 0 2024-02-26T10:48:04.547 INFO:tasks.workunit.client.0.smithi060.stdout:9/770: mkdir d3/d1e/d5e/d35/deb/d107 0 2024-02-26T10:48:04.553 INFO:tasks.workunit.client.0.smithi060.stdout:7/871: link da/d15/l7e da/d15/d21/d2f/d78/l121 0 2024-02-26T10:48:04.555 INFO:tasks.workunit.client.0.smithi060.stdout:1/818: sync 2024-02-26T10:48:04.555 INFO:tasks.workunit.client.0.smithi060.stdout:2/713: sync 2024-02-26T10:48:04.555 INFO:tasks.workunit.client.0.smithi060.stdout:8/724: sync 2024-02-26T10:48:04.555 INFO:tasks.workunit.client.0.smithi060.stdout:8/725: stat d9/ce1 0 2024-02-26T10:48:04.555 INFO:tasks.workunit.client.0.smithi060.stdout:8/726: fdatasync d9/d15/d3e/f9b 0 2024-02-26T10:48:04.559 INFO:tasks.workunit.client.0.smithi060.stdout:6/768: dwrite d6/d7/d25/d1c/d1f/d23/f9c [0,4194304] 0 2024-02-26T10:48:04.561 INFO:tasks.workunit.client.0.smithi060.stdout:2/714: write dd/d43/d60/d41/d37/d4c/fa2 [244994,20321] 0 2024-02-26T10:48:04.562 INFO:tasks.workunit.client.0.smithi060.stdout:6/769: getdents d6/d7/d25/d1c/d1f/d23 0 2024-02-26T10:48:04.567 INFO:tasks.workunit.client.0.smithi060.stdout:8/727: write d9/d15/d29/d2f/d34/d72/d94/f9e [3329009,54977] 0 2024-02-26T10:48:04.567 INFO:tasks.workunit.client.0.smithi060.stdout:8/728: write d9/f24 [1083288,63175] 0 2024-02-26T10:48:04.569 INFO:tasks.workunit.client.0.smithi060.stdout:0/725: dwrite d0/d37/f6d [0,4194304] 0 2024-02-26T10:48:04.569 INFO:tasks.workunit.client.0.smithi060.stdout:0/726: readlink d0/d37/l81 0 2024-02-26T10:48:04.569 INFO:tasks.workunit.client.0.smithi060.stdout:0/727: chown d0/ff1 610 1 2024-02-26T10:48:04.571 INFO:tasks.workunit.client.0.smithi060.stdout:8/729: mknod d9/d6f/ddc/cf1 0 2024-02-26T10:48:04.571 INFO:tasks.workunit.client.0.smithi060.stdout:8/730: fdatasync d9/d5a/fcd 0 2024-02-26T10:48:04.575 INFO:tasks.workunit.client.0.smithi060.stdout:3/529: sync 2024-02-26T10:48:04.576 INFO:tasks.workunit.client.0.smithi060.stdout:6/770: dread d6/d7/f53 [0,4194304] 0 2024-02-26T10:48:04.578 INFO:tasks.workunit.client.0.smithi060.stdout:3/530: creat d5/de/df/d11/d63/d93/d97/fa0 x:0 0 0 2024-02-26T10:48:04.578 INFO:tasks.workunit.client.0.smithi060.stdout:3/531: truncate d5/de/df/d11/f20 4685354 0 2024-02-26T10:48:04.578 INFO:tasks.workunit.client.0.smithi060.stdout:3/532: truncate d5/de/df/d4c/f88 449308 0 2024-02-26T10:48:04.579 INFO:tasks.workunit.client.0.smithi060.stdout:9/771: dread d3/d1e/d5e/d1b/f56 [0,4194304] 0 2024-02-26T10:48:04.580 INFO:tasks.workunit.client.0.smithi060.stdout:3/533: mkdir d5/d45/d61/d92/da1 0 2024-02-26T10:48:04.581 INFO:tasks.workunit.client.0.smithi060.stdout:9/772: symlink d3/d1e/d5e/d35/deb/l108 0 2024-02-26T10:48:04.581 INFO:tasks.workunit.client.0.smithi060.stdout:9/773: dread - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/f8b zero size 2024-02-26T10:48:04.583 INFO:tasks.workunit.client.0.smithi060.stdout:9/774: symlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/dde/l109 0 2024-02-26T10:48:04.584 INFO:tasks.workunit.client.0.smithi060.stdout:9/775: mknod d3/d1e/d57/c10a 0 2024-02-26T10:48:04.584 INFO:tasks.workunit.client.0.smithi060.stdout:3/534: rmdir d5/de/df/d11 39 2024-02-26T10:48:04.585 INFO:tasks.workunit.client.0.smithi060.stdout:6/771: dread d6/f9 [0,4194304] 0 2024-02-26T10:48:04.586 INFO:tasks.workunit.client.0.smithi060.stdout:6/772: chown d6/d7/d47/dde/ff3 238 1 2024-02-26T10:48:04.586 INFO:tasks.workunit.client.0.smithi060.stdout:6/773: write d6/d8d/dcc/fc1 [1072809,38544] 0 2024-02-26T10:48:04.586 INFO:tasks.workunit.client.0.smithi060.stdout:6/774: write d6/d7/d25/d1c/d1f/d23/d28/f4c [1124069,18453] 0 2024-02-26T10:48:04.589 INFO:tasks.workunit.client.0.smithi060.stdout:9/776: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f10b x:0 0 0 2024-02-26T10:48:04.589 INFO:tasks.workunit.client.0.smithi060.stdout:9/777: chown d3/d1e/d5e/d1b/l92 639 1 2024-02-26T10:48:04.589 INFO:tasks.workunit.client.0.smithi060.stdout:9/778: truncate d3/d1e/d57/fd0 380852 0 2024-02-26T10:48:04.589 INFO:tasks.workunit.client.0.smithi060.stdout:9/779: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f4c [4242770,41967] 0 2024-02-26T10:48:04.589 INFO:tasks.workunit.client.0.smithi060.stdout:9/780: chown d3/d1e/d5e/d35/deb/d7a/fd1 60762 1 2024-02-26T10:48:04.590 INFO:tasks.workunit.client.0.smithi060.stdout:9/781: rmdir d3/d1e/d5e/d35/d87/d9b/d8a 39 2024-02-26T10:48:04.591 INFO:tasks.workunit.client.0.smithi060.stdout:9/782: mkdir d3/d1e/d57/d9c/d10c 0 2024-02-26T10:48:04.592 INFO:tasks.workunit.client.0.smithi060.stdout:9/783: creat d3/d1e/dbb/dc2/f10d x:0 0 0 2024-02-26T10:48:04.593 INFO:tasks.workunit.client.0.smithi060.stdout:9/784: mkdir d3/d1e/d93/da2/db7/ddd/d10e 0 2024-02-26T10:48:04.609 INFO:tasks.workunit.client.0.smithi060.stdout:6/775: write d6/d7/d25/d1c/d1f/d23/d2b/d4d/fee [1633072,86135] 0 2024-02-26T10:48:04.618 INFO:tasks.workunit.client.0.smithi060.stdout:2/715: dwrite dd/d43/d60/d41/d37/fdb [0,4194304] 0 2024-02-26T10:48:04.619 INFO:tasks.workunit.client.0.smithi060.stdout:0/728: dwrite d0/d37/f6d [0,4194304] 0 2024-02-26T10:48:04.621 INFO:tasks.workunit.client.0.smithi060.stdout:2/716: truncate dd/d43/d55/d5e/d74/fb6 1514980 0 2024-02-26T10:48:04.623 INFO:tasks.workunit.client.0.smithi060.stdout:3/535: dwrite d5/de/df/d57/f94 [4194304,4194304] 0 2024-02-26T10:48:04.623 INFO:tasks.workunit.client.0.smithi060.stdout:3/536: dread - d5/d76/d7d/f8a zero size 2024-02-26T10:48:04.624 INFO:tasks.workunit.client.0.smithi060.stdout:0/729: truncate d0/dd/d39/f7e 5928468 0 2024-02-26T10:48:04.625 INFO:tasks.workunit.client.0.smithi060.stdout:3/537: mkdir d5/de/df/d11/d63/d93/da2 0 2024-02-26T10:48:04.626 INFO:tasks.workunit.client.0.smithi060.stdout:0/730: link d0/dd/d53/d87/d94/ldb d0/dd/d53/d87/d94/db7/lf3 0 2024-02-26T10:48:04.628 INFO:tasks.workunit.client.0.smithi060.stdout:3/538: mknod d5/de/df/d11/d63/d93/d97/ca3 0 2024-02-26T10:48:04.628 INFO:tasks.workunit.client.0.smithi060.stdout:3/539: stat d5/de/df/d11/c43 0 2024-02-26T10:48:04.628 INFO:tasks.workunit.client.0.smithi060.stdout:3/540: chown d5/de/f14 3797562 1 2024-02-26T10:48:04.628 INFO:tasks.workunit.client.0.smithi060.stdout:9/785: dread d3/d1e/f82 [0,4194304] 0 2024-02-26T10:48:04.629 INFO:tasks.workunit.client.0.smithi060.stdout:9/786: read d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f6c [364270,14530] 0 2024-02-26T10:48:04.631 INFO:tasks.workunit.client.0.smithi060.stdout:9/787: unlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/f38 0 2024-02-26T10:48:04.634 INFO:tasks.workunit.client.0.smithi060.stdout:9/788: rmdir d3/d1e/dbb/dc2 39 2024-02-26T10:48:04.635 INFO:tasks.workunit.client.0.smithi060.stdout:9/789: rmdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32 39 2024-02-26T10:48:04.643 INFO:tasks.workunit.client.0.smithi060.stdout:0/731: dread d0/f55 [0,4194304] 0 2024-02-26T10:48:04.645 INFO:tasks.workunit.client.0.smithi060.stdout:0/732: mknod d0/dd/d33/cf4 0 2024-02-26T10:48:04.646 INFO:tasks.workunit.client.0.smithi060.stdout:0/733: mknod d0/d37/cf5 0 2024-02-26T10:48:04.646 INFO:tasks.workunit.client.0.smithi060.stdout:0/734: stat d0/la 0 2024-02-26T10:48:04.657 INFO:tasks.workunit.client.0.smithi060.stdout:6/776: dwrite d6/d7/d47/f9b [0,4194304] 0 2024-02-26T10:48:04.658 INFO:tasks.workunit.client.0.smithi060.stdout:3/541: dread d5/de/df/d4c/f4f [4194304,4194304] 0 2024-02-26T10:48:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:2/717: dwrite dd/d43/d55/d6e/dd3/d6f/fa9 [0,4194304] 0 2024-02-26T10:48:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:2/718: truncate dd/d11/f62 1609983 0 2024-02-26T10:48:04.679 INFO:tasks.workunit.client.0.smithi060.stdout:5/695: sync 2024-02-26T10:48:04.679 INFO:tasks.workunit.client.0.smithi060.stdout:4/711: sync 2024-02-26T10:48:04.680 INFO:tasks.workunit.client.0.smithi060.stdout:4/712: getdents d0/da/d3b/db9 0 2024-02-26T10:48:04.681 INFO:tasks.workunit.client.0.smithi060.stdout:5/696: creat d0/ff4 x:0 0 0 2024-02-26T10:48:04.682 INFO:tasks.workunit.client.0.smithi060.stdout:5/697: symlink d0/d4b/d7b/lf5 0 2024-02-26T10:48:04.682 INFO:tasks.workunit.client.0.smithi060.stdout:5/698: chown d0/de/d15/l94 12 1 2024-02-26T10:48:04.683 INFO:tasks.workunit.client.0.smithi060.stdout:5/699: mknod d0/d33/cf6 0 2024-02-26T10:48:04.683 INFO:tasks.workunit.client.0.smithi060.stdout:5/700: read - d0/de/d20/d82/fe8 zero size 2024-02-26T10:48:04.684 INFO:tasks.workunit.client.0.smithi060.stdout:5/701: creat d0/de/d15/d65/d86/d9b/ff7 x:0 0 0 2024-02-26T10:48:04.686 INFO:tasks.workunit.client.0.smithi060.stdout:5/702: rename d0/de/d20/l83 to d0/de/d20/d82/lf8 0 2024-02-26T10:48:04.686 INFO:tasks.workunit.client.0.smithi060.stdout:5/703: fsync d0/de/d20/d32/d4d/da3/fcf 0 2024-02-26T10:48:04.687 INFO:tasks.workunit.client.0.smithi060.stdout:5/704: creat d0/de/d20/d32/ff9 x:0 0 0 2024-02-26T10:48:04.688 INFO:tasks.workunit.client.0.smithi060.stdout:5/705: creat d0/de/d20/d32/d4d/da3/da5/dc8/ffa x:0 0 0 2024-02-26T10:48:04.698 INFO:tasks.workunit.client.0.smithi060.stdout:5/706: link d0/de/d70/d84/lc6 d0/de/d20/d32/d4c/dd2/lfb 0 2024-02-26T10:48:04.698 INFO:tasks.workunit.client.0.smithi060.stdout:5/707: read d0/d33/fe3 [16229,86672] 0 2024-02-26T10:48:04.699 INFO:tasks.workunit.client.0.smithi060.stdout:5/708: truncate d0/d4b/f51 3999180 0 2024-02-26T10:48:04.713 INFO:tasks.workunit.client.0.smithi060.stdout:5/709: link d0/de/d20/d32/d4d/da3/da5/la6 d0/de/d15/d65/d86/d9b/lfc 0 2024-02-26T10:48:04.719 INFO:tasks.workunit.client.0.smithi060.stdout:3/542: dwrite d5/de/df/d57/d60/f30 [0,4194304] 0 2024-02-26T10:48:04.719 INFO:tasks.workunit.client.0.smithi060.stdout:3/543: chown d5/d45/d61 589756274 1 2024-02-26T10:48:04.721 INFO:tasks.workunit.client.0.smithi060.stdout:3/544: stat d5/de/df/d57/d60/c59 0 2024-02-26T10:48:04.731 INFO:tasks.workunit.client.0.smithi060.stdout:2/719: read dd/d43/d60/d41/d37/d9a/fb7 [141147,109259] 0 2024-02-26T10:48:04.732 INFO:tasks.workunit.client.0.smithi060.stdout:3/545: symlink d5/de/df/d11/d63/d93/la4 0 2024-02-26T10:48:04.732 INFO:tasks.workunit.client.0.smithi060.stdout:3/546: chown d5/d45/c54 57562 1 2024-02-26T10:48:04.732 INFO:tasks.workunit.client.0.smithi060.stdout:6/777: dwrite d6/fe [0,4194304] 0 2024-02-26T10:48:04.733 INFO:tasks.workunit.client.0.smithi060.stdout:3/547: symlink d5/de/df/d57/d7a/la5 0 2024-02-26T10:48:04.734 INFO:tasks.workunit.client.0.smithi060.stdout:3/548: readlink d5/d76/d7d/l9e 0 2024-02-26T10:48:04.734 INFO:tasks.workunit.client.0.smithi060.stdout:4/713: dwrite d0/da/dc7/db0/db5/dc0/f8 [4194304,4194304] 0 2024-02-26T10:48:04.734 INFO:tasks.workunit.client.0.smithi060.stdout:4/714: stat d0/da/dd6 0 2024-02-26T10:48:04.734 INFO:tasks.workunit.client.0.smithi060.stdout:6/778: getdents d6/d7/d25/d1c/d1f/d23/d2b 0 2024-02-26T10:48:04.738 INFO:tasks.workunit.client.0.smithi060.stdout:3/549: creat d5/d45/d61/d80/fa6 x:0 0 0 2024-02-26T10:48:04.740 INFO:tasks.workunit.client.0.smithi060.stdout:6/779: creat d6/d7/d25/d1c/d1f/d23/d2b/d4d/fff x:0 0 0 2024-02-26T10:48:04.740 INFO:tasks.workunit.client.0.smithi060.stdout:4/715: mkdir d0/da/d3b/ddc 0 2024-02-26T10:48:04.741 INFO:tasks.workunit.client.0.smithi060.stdout:4/716: chown d0/da/d3b/ld1 1 1 2024-02-26T10:48:04.742 INFO:tasks.workunit.client.0.smithi060.stdout:7/872: sync 2024-02-26T10:48:04.742 INFO:tasks.workunit.client.0.smithi060.stdout:9/790: sync 2024-02-26T10:48:04.742 INFO:tasks.workunit.client.0.smithi060.stdout:0/735: sync 2024-02-26T10:48:04.743 INFO:tasks.workunit.client.0.smithi060.stdout:9/791: truncate d3/d1e/d5e/d35/deb/fb4 1465714 0 2024-02-26T10:48:04.743 INFO:tasks.workunit.client.0.smithi060.stdout:1/819: sync 2024-02-26T10:48:04.743 INFO:tasks.workunit.client.0.smithi060.stdout:4/717: creat d0/da/d3b/db9/fdd x:0 0 0 2024-02-26T10:48:04.743 INFO:tasks.workunit.client.0.smithi060.stdout:4/718: chown d0/d27/d76/dbd/f38 267785862 1 2024-02-26T10:48:04.747 INFO:tasks.workunit.client.0.smithi060.stdout:0/736: creat d0/dd/d39/d63/ff6 x:0 0 0 2024-02-26T10:48:04.747 INFO:tasks.workunit.client.0.smithi060.stdout:0/737: chown d0/d37/f6d 1 1 2024-02-26T10:48:04.747 INFO:tasks.workunit.client.0.smithi060.stdout:8/731: sync 2024-02-26T10:48:04.749 INFO:tasks.workunit.client.0.smithi060.stdout:4/719: mkdir d0/d27/dde 0 2024-02-26T10:48:04.750 INFO:tasks.workunit.client.0.smithi060.stdout:4/720: mknod d0/da/d3b/d1b/cdf 0 2024-02-26T10:48:04.756 INFO:tasks.workunit.client.0.smithi060.stdout:4/721: creat d0/da/fe0 x:0 0 0 2024-02-26T10:48:04.756 INFO:tasks.workunit.client.0.smithi060.stdout:2/720: dread dd/d43/d60/f1e [0,4194304] 0 2024-02-26T10:48:04.757 INFO:tasks.workunit.client.0.smithi060.stdout:2/721: link dd/d43/d60/d41/d37/fdb dd/d43/d60/ff3 0 2024-02-26T10:48:04.768 INFO:tasks.workunit.client.0.smithi060.stdout:2/722: symlink dd/d43/d96/df2/lf4 0 2024-02-26T10:48:04.768 INFO:tasks.workunit.client.0.smithi060.stdout:3/550: dread f3 [0,4194304] 0 2024-02-26T10:48:04.768 INFO:tasks.workunit.client.0.smithi060.stdout:3/551: symlink d5/d45/d61/la7 0 2024-02-26T10:48:04.769 INFO:tasks.workunit.client.0.smithi060.stdout:3/552: chown d5/de/df/d11/d63/f64 258 1 2024-02-26T10:48:04.770 INFO:tasks.workunit.client.0.smithi060.stdout:3/553: creat d5/de/df/d57/d60/fa8 x:0 0 0 2024-02-26T10:48:04.770 INFO:tasks.workunit.client.0.smithi060.stdout:5/710: dwrite d0/de/d20/d32/d4d/d97/fdb [0,4194304] 0 2024-02-26T10:48:04.771 INFO:tasks.workunit.client.0.smithi060.stdout:3/554: mknod d5/d76/d7d/ca9 0 2024-02-26T10:48:04.771 INFO:tasks.workunit.client.0.smithi060.stdout:3/555: fsync d5/d45/d61/d80/f86 0 2024-02-26T10:48:04.771 INFO:tasks.workunit.client.0.smithi060.stdout:5/711: creat d0/de/d20/d32/d4c/ffd x:0 0 0 2024-02-26T10:48:04.772 INFO:tasks.workunit.client.0.smithi060.stdout:5/712: chown d0/de/d20/d32/d4d/fb4 98635 1 2024-02-26T10:48:04.772 INFO:tasks.workunit.client.0.smithi060.stdout:5/713: truncate d0/de/f5d 336852 0 2024-02-26T10:48:04.772 INFO:tasks.workunit.client.0.smithi060.stdout:5/714: truncate d0/de/f35 5026307 0 2024-02-26T10:48:04.773 INFO:tasks.workunit.client.0.smithi060.stdout:5/715: symlink d0/de/d70/lfe 0 2024-02-26T10:48:04.773 INFO:tasks.workunit.client.0.smithi060.stdout:5/716: fdatasync d0/de/d20/d32/d4c/fe6 0 2024-02-26T10:48:04.777 INFO:tasks.workunit.client.0.smithi060.stdout:5/717: unlink d0/de/d20/d32/d4d/d97/l9a 0 2024-02-26T10:48:04.777 INFO:tasks.workunit.client.0.smithi060.stdout:5/718: chown d0/d33/l37 20 1 2024-02-26T10:48:04.780 INFO:tasks.workunit.client.0.smithi060.stdout:7/873: dread da/d15/d21/d29/f4a [0,4194304] 0 2024-02-26T10:48:04.781 INFO:tasks.workunit.client.0.smithi060.stdout:8/732: rename d9/d5a/f6e to d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/ff2 0 2024-02-26T10:48:04.784 INFO:tasks.workunit.client.0.smithi060.stdout:7/874: mknod da/d15/d21/d2f/d78/dd3/c122 0 2024-02-26T10:48:04.784 INFO:tasks.workunit.client.0.smithi060.stdout:7/875: readlink da/d15/d88/l65 0 2024-02-26T10:48:04.785 INFO:tasks.workunit.client.0.smithi060.stdout:6/780: dwrite d6/d7/d2e/feb [0,4194304] 0 2024-02-26T10:48:04.786 INFO:tasks.workunit.client.0.smithi060.stdout:7/876: creat da/f123 x:0 0 0 2024-02-26T10:48:04.786 INFO:tasks.workunit.client.0.smithi060.stdout:7/877: write da/d66/dad/d8f/f11b [230108,78138] 0 2024-02-26T10:48:04.786 INFO:tasks.workunit.client.0.smithi060.stdout:7/878: write da/d15/d88/d3b/fd0 [116564,7893] 0 2024-02-26T10:48:04.793 INFO:tasks.workunit.client.0.smithi060.stdout:8/733: dread d9/d5a/f69 [0,4194304] 0 2024-02-26T10:48:04.793 INFO:tasks.workunit.client.0.smithi060.stdout:8/734: write d9/d15/d29/d2f/fc9 [80585,54703] 0 2024-02-26T10:48:04.811 INFO:tasks.workunit.client.0.smithi060.stdout:1/820: dwrite d3/db/d1e/d38/f8e [0,4194304] 0 2024-02-26T10:48:04.811 INFO:tasks.workunit.client.0.smithi060.stdout:1/821: readlink d3/lf2 0 2024-02-26T10:48:04.811 INFO:tasks.workunit.client.0.smithi060.stdout:2/723: dread dd/d43/d55/d6e/dd3/d6f/d8f/fcd [0,4194304] 0 2024-02-26T10:48:04.811 INFO:tasks.workunit.client.0.smithi060.stdout:2/724: write dd/d43/d60/d41/d37/d38/d57/fe8 [850563,77569] 0 2024-02-26T10:48:04.814 INFO:tasks.workunit.client.0.smithi060.stdout:2/725: symlink dd/d43/d55/d5e/d86/d97/db9/lf5 0 2024-02-26T10:48:04.821 INFO:tasks.workunit.client.0.smithi060.stdout:1/822: read d3/d35/dc2/fee [5015623,106512] 0 2024-02-26T10:48:04.821 INFO:tasks.workunit.client.0.smithi060.stdout:1/823: chown d3/f1b 3125 1 2024-02-26T10:48:04.822 INFO:tasks.workunit.client.0.smithi060.stdout:1/824: dread - d3/db/d1e/d6d/da3/daa/fde zero size 2024-02-26T10:48:04.825 INFO:tasks.workunit.client.0.smithi060.stdout:1/825: truncate d3/d35/d53/d68/d90/fbc 5531198 0 2024-02-26T10:48:04.831 INFO:tasks.workunit.client.0.smithi060.stdout:4/722: dread d0/d27/d76/f93 [0,4194304] 0 2024-02-26T10:48:04.831 INFO:tasks.workunit.client.0.smithi060.stdout:6/781: read d6/d7/d25/d1c/d1f/d23/d28/fa3 [1379609,103006] 0 2024-02-26T10:48:04.834 INFO:tasks.workunit.client.0.smithi060.stdout:9/792: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/fa0 [0,4194304] 0 2024-02-26T10:48:04.834 INFO:tasks.workunit.client.0.smithi060.stdout:9/793: fsync d3/d1e/d5e/d1b/d23/f36 0 2024-02-26T10:48:04.834 INFO:tasks.workunit.client.0.smithi060.stdout:7/879: dread f2 [4194304,4194304] 0 2024-02-26T10:48:04.835 INFO:tasks.workunit.client.0.smithi060.stdout:9/794: mknod d3/d1e/dbb/dc2/c10f 0 2024-02-26T10:48:04.838 INFO:tasks.workunit.client.0.smithi060.stdout:7/880: rmdir da/d66/dbd/d10b 0 2024-02-26T10:48:04.841 INFO:tasks.workunit.client.0.smithi060.stdout:7/881: mkdir da/d15/d21/d124 0 2024-02-26T10:48:04.841 INFO:tasks.workunit.client.0.smithi060.stdout:4/723: mkdir d0/da/d3b/dca/de1 0 2024-02-26T10:48:04.841 INFO:tasks.workunit.client.0.smithi060.stdout:7/882: fsync da/d15/f19 0 2024-02-26T10:48:04.841 INFO:tasks.workunit.client.0.smithi060.stdout:4/724: chown d0/da/c15 62039683 1 2024-02-26T10:48:04.841 INFO:tasks.workunit.client.0.smithi060.stdout:7/883: write da/d66/dad/d8f/f120 [20000,104376] 0 2024-02-26T10:48:04.843 INFO:tasks.workunit.client.0.smithi060.stdout:5/719: dwrite d0/de/d20/d32/d4d/d97/fdb [0,4194304] 0 2024-02-26T10:48:04.845 INFO:tasks.workunit.client.0.smithi060.stdout:3/556: dread d5/d45/d61/d80/f86 [4194304,4194304] 0 2024-02-26T10:48:04.845 INFO:tasks.workunit.client.0.smithi060.stdout:3/557: write d5/d76/f78 [990371,114636] 0 2024-02-26T10:48:04.849 INFO:tasks.workunit.client.0.smithi060.stdout:9/795: link d3/d1e/d93/fce d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/f110 0 2024-02-26T10:48:04.849 INFO:tasks.workunit.client.0.smithi060.stdout:9/796: dread - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f105 zero size 2024-02-26T10:48:04.850 INFO:tasks.workunit.client.0.smithi060.stdout:9/797: symlink d3/d1e/d5e/dc9/l111 0 2024-02-26T10:48:04.851 INFO:tasks.workunit.client.0.smithi060.stdout:3/558: unlink d5/de/df/d4c/f88 0 2024-02-26T10:48:04.852 INFO:tasks.workunit.client.0.smithi060.stdout:9/798: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f6c [2006258,48845] 0 2024-02-26T10:48:04.878 INFO:tasks.workunit.client.0.smithi060.stdout:4/725: dread d0/da/d57/f96 [0,4194304] 0 2024-02-26T10:48:04.879 INFO:tasks.workunit.client.0.smithi060.stdout:2/726: dwrite dd/d43/d60/d41/f95 [0,4194304] 0 2024-02-26T10:48:04.879 INFO:tasks.workunit.client.0.smithi060.stdout:2/727: chown dd/d43/d55/d5e/d86/d97/db9/lf5 24517451 1 2024-02-26T10:48:04.879 INFO:tasks.workunit.client.0.smithi060.stdout:2/728: dread - dd/d43/d55/d6e/fd5 zero size 2024-02-26T10:48:04.879 INFO:tasks.workunit.client.0.smithi060.stdout:2/729: chown dd/d43/d60/d41/d37/d4c/f92 89949 1 2024-02-26T10:48:04.880 INFO:tasks.workunit.client.0.smithi060.stdout:2/730: creat dd/d43/d55/d6e/dd3/d6f/ff6 x:0 0 0 2024-02-26T10:48:04.896 INFO:tasks.workunit.client.0.smithi060.stdout:2/731: dread dd/d43/d60/d41/d37/d4c/d76/f84 [0,4194304] 0 2024-02-26T10:48:04.916 INFO:tasks.workunit.client.0.smithi060.stdout:2/732: dread dd/d43/d60/d41/d37/fa0 [4194304,4194304] 0 2024-02-26T10:48:04.918 INFO:tasks.workunit.client.0.smithi060.stdout:2/733: creat dd/d43/d60/d41/d37/ff7 x:0 0 0 2024-02-26T10:48:04.945 INFO:tasks.workunit.client.0.smithi060.stdout:8/735: dwrite d9/d15/d29/d2f/fe4 [0,4194304] 0 2024-02-26T10:48:04.953 INFO:tasks.workunit.client.0.smithi060.stdout:7/884: rename da/d66/dad to da/df1/dd7/d125 0 2024-02-26T10:48:04.954 INFO:tasks.workunit.client.0.smithi060.stdout:4/726: creat d0/d27/d76/dbd/d8e/fe2 x:0 0 0 2024-02-26T10:48:04.956 INFO:tasks.workunit.client.0.smithi060.stdout:8/736: creat d9/d6f/ff3 x:0 0 0 2024-02-26T10:48:04.956 INFO:tasks.workunit.client.0.smithi060.stdout:1/826: dwrite d3/db/f118 [0,4194304] 0 2024-02-26T10:48:04.959 INFO:tasks.workunit.client.0.smithi060.stdout:1/827: creat d3/db/d1e/d38/f119 x:0 0 0 2024-02-26T10:48:04.962 INFO:tasks.workunit.client.0.smithi060.stdout:4/727: dread d0/da/f4c [0,4194304] 0 2024-02-26T10:48:04.965 INFO:tasks.workunit.client.0.smithi060.stdout:7/885: symlink da/d15/d21/d2f/l126 0 2024-02-26T10:48:04.965 INFO:tasks.workunit.client.0.smithi060.stdout:4/728: symlink d0/d27/le3 0 2024-02-26T10:48:04.969 INFO:tasks.workunit.client.0.smithi060.stdout:5/720: dwrite d0/de/d20/d32/d4d/f55 [0,4194304] 0 2024-02-26T10:48:04.970 INFO:tasks.workunit.client.0.smithi060.stdout:6/782: write d6/d7/d2e/d56/f61 [1101070,17140] 0 2024-02-26T10:48:04.971 INFO:tasks.workunit.client.0.smithi060.stdout:9/799: read d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f11 [532113,115136] 0 2024-02-26T10:48:04.972 INFO:tasks.workunit.client.0.smithi060.stdout:5/721: creat d0/de/d15/fff x:0 0 0 2024-02-26T10:48:04.973 INFO:tasks.workunit.client.0.smithi060.stdout:9/800: symlink d3/d68/l112 0 2024-02-26T10:48:04.973 INFO:tasks.workunit.client.0.smithi060.stdout:9/801: chown d3/d1e/d5e/d35/d87/d9b/d8a 610718371 1 2024-02-26T10:48:04.973 INFO:tasks.workunit.client.0.smithi060.stdout:6/783: mknod d6/d7/d25/d1c/d1f/c100 0 2024-02-26T10:48:04.976 INFO:tasks.workunit.client.0.smithi060.stdout:9/802: mkdir d3/d1e/d93/da2/db7/ddd/d10e/d113 0 2024-02-26T10:48:04.986 INFO:tasks.workunit.client.0.smithi060.stdout:0/738: sync 2024-02-26T10:48:04.986 INFO:tasks.workunit.client.0.smithi060.stdout:8/737: rename d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f8a to d9/d15/ff4 0 2024-02-26T10:48:04.986 INFO:tasks.workunit.client.0.smithi060.stdout:8/738: fsync d9/d15/d29/d2f/d34/d72/d50/d6a/f71 0 2024-02-26T10:48:04.999 INFO:tasks.workunit.client.0.smithi060.stdout:4/729: write d0/da/d3b/d16/f39 [379769,17118] 0 2024-02-26T10:48:04.999 INFO:tasks.workunit.client.0.smithi060.stdout:2/734: dwrite dd/d43/d60/d41/d37/ff7 [0,4194304] 0 2024-02-26T10:48:05.003 INFO:tasks.workunit.client.0.smithi060.stdout:4/730: symlink d0/d27/d76/le4 0 2024-02-26T10:48:05.003 INFO:tasks.workunit.client.0.smithi060.stdout:4/731: write d0/da/d3b/d16/f3a [956259,7624] 0 2024-02-26T10:48:05.004 INFO:tasks.workunit.client.0.smithi060.stdout:4/732: link d0/d27/d76/dbd/d8e/fac d0/da/d57/fe5 0 2024-02-26T10:48:05.006 INFO:tasks.workunit.client.0.smithi060.stdout:4/733: getdents d0/d27 0 2024-02-26T10:48:05.015 INFO:tasks.workunit.client.0.smithi060.stdout:4/734: creat d0/da/d3b/dca/de1/fe6 x:0 0 0 2024-02-26T10:48:05.019 INFO:tasks.workunit.client.0.smithi060.stdout:1/828: dwrite d3/d35/d53/d68/dad/fb8 [8388608,4194304] 0 2024-02-26T10:48:05.020 INFO:tasks.workunit.client.0.smithi060.stdout:4/735: rmdir d0/da/dc7/db0 39 2024-02-26T10:48:05.020 INFO:tasks.workunit.client.0.smithi060.stdout:4/736: write d0/f7b [864950,44705] 0 2024-02-26T10:48:05.021 INFO:tasks.workunit.client.0.smithi060.stdout:1/829: creat d3/db/d1e/d39/f11a x:0 0 0 2024-02-26T10:48:05.021 INFO:tasks.workunit.client.0.smithi060.stdout:1/830: chown d3/f5c 1237919 1 2024-02-26T10:48:05.022 INFO:tasks.workunit.client.0.smithi060.stdout:4/737: creat d0/da/dd6/fe7 x:0 0 0 2024-02-26T10:48:05.022 INFO:tasks.workunit.client.0.smithi060.stdout:8/739: dread d9/d15/f36 [0,4194304] 0 2024-02-26T10:48:05.023 INFO:tasks.workunit.client.0.smithi060.stdout:4/738: mkdir d0/da/dd6/de8 0 2024-02-26T10:48:05.026 INFO:tasks.workunit.client.0.smithi060.stdout:4/739: mknod d0/da/d3b/d16/ce9 0 2024-02-26T10:48:05.028 INFO:tasks.workunit.client.0.smithi060.stdout:8/740: mkdir d9/d15/d29/d2f/d34/d72/df5 0 2024-02-26T10:48:05.028 INFO:tasks.workunit.client.0.smithi060.stdout:8/741: fdatasync d9/d15/d29/d2f/d34/d72/d50/f7b 0 2024-02-26T10:48:05.028 INFO:tasks.workunit.client.0.smithi060.stdout:8/742: fsync d9/d15/d3e/fac 0 2024-02-26T10:48:05.031 INFO:tasks.workunit.client.0.smithi060.stdout:5/722: dread d0/f1d [4194304,4194304] 0 2024-02-26T10:48:05.031 INFO:tasks.workunit.client.0.smithi060.stdout:8/743: write d9/d15/d29/d2f/d34/d72/d50/d6a/f61 [347475,4771] 0 2024-02-26T10:48:05.031 INFO:tasks.workunit.client.0.smithi060.stdout:7/886: dwrite da/d15/d21/d2f/d78/f10a [0,4194304] 0 2024-02-26T10:48:05.033 INFO:tasks.workunit.client.0.smithi060.stdout:8/744: symlink d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/lf6 0 2024-02-26T10:48:05.033 INFO:tasks.workunit.client.0.smithi060.stdout:7/887: readlink da/l3d 0 2024-02-26T10:48:05.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/745: creat d9/d15/d29/d2f/d34/ff7 x:0 0 0 2024-02-26T10:48:05.035 INFO:tasks.workunit.client.0.smithi060.stdout:8/746: write d9/d15/fe5 [179212,85886] 0 2024-02-26T10:48:05.036 INFO:tasks.workunit.client.0.smithi060.stdout:8/747: fdatasync d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/fbc 0 2024-02-26T10:48:05.036 INFO:tasks.workunit.client.0.smithi060.stdout:8/748: chown d9/d5a 185872265 1 2024-02-26T10:48:05.046 INFO:tasks.workunit.client.0.smithi060.stdout:1/831: dread d3/db/d1e/d38/d6c/fb9 [0,4194304] 0 2024-02-26T10:48:05.053 INFO:tasks.workunit.client.0.smithi060.stdout:6/784: dwrite d6/d7/d2e/d56/d73/fa4 [0,4194304] 0 2024-02-26T10:48:05.062 INFO:tasks.workunit.client.0.smithi060.stdout:7/888: write da/d15/d21/d29/f4a [264995,11935] 0 2024-02-26T10:48:05.064 INFO:tasks.workunit.client.0.smithi060.stdout:2/735: dwrite dd/d11/f3e [0,4194304] 0 2024-02-26T10:48:05.072 INFO:tasks.workunit.client.0.smithi060.stdout:7/889: mknod da/d15/d21/d2f/d78/dd3/c127 0 2024-02-26T10:48:05.072 INFO:tasks.workunit.client.0.smithi060.stdout:7/890: chown da/d15/d21/d2f/d78/dd3/d67/f85 92 1 2024-02-26T10:48:05.079 INFO:tasks.workunit.client.0.smithi060.stdout:0/739: dwrite d0/dd/f31 [0,4194304] 0 2024-02-26T10:48:05.089 INFO:tasks.workunit.client.0.smithi060.stdout:1/832: dread d3/db/f56 [0,4194304] 0 2024-02-26T10:48:05.106 INFO:tasks.workunit.client.0.smithi060.stdout:3/559: sync 2024-02-26T10:48:05.110 INFO:tasks.workunit.client.0.smithi060.stdout:2/736: write dd/d43/d60/d41/d37/fa0 [5302060,35986] 0 2024-02-26T10:48:05.110 INFO:tasks.workunit.client.0.smithi060.stdout:4/740: dwrite d0/d27/d76/dbd/f32 [4194304,4194304] 0 2024-02-26T10:48:05.110 INFO:tasks.workunit.client.0.smithi060.stdout:4/741: stat d0/da/d57/fc6 0 2024-02-26T10:48:05.110 INFO:tasks.workunit.client.0.smithi060.stdout:4/742: readlink d0/d27/d76/dbd/d60/l6f 0 2024-02-26T10:48:05.115 INFO:tasks.workunit.client.0.smithi060.stdout:5/723: dwrite d0/de/d20/d32/d4c/ffd [0,4194304] 0 2024-02-26T10:48:05.115 INFO:tasks.workunit.client.0.smithi060.stdout:2/737: creat dd/d43/d96/df2/ff8 x:0 0 0 2024-02-26T10:48:05.115 INFO:tasks.workunit.client.0.smithi060.stdout:2/738: write dd/d43/d60/d41/d37/d9a/db4/fcf [766421,21850] 0 2024-02-26T10:48:05.128 INFO:tasks.workunit.client.0.smithi060.stdout:6/785: dwrite d6/d7/d47/dde/ff3 [0,4194304] 0 2024-02-26T10:48:05.128 INFO:tasks.workunit.client.0.smithi060.stdout:8/749: dwrite d9/f91 [0,4194304] 0 2024-02-26T10:48:05.128 INFO:tasks.workunit.client.0.smithi060.stdout:8/750: chown d9/d5a/d58/f7a 1 1 2024-02-26T10:48:05.128 INFO:tasks.workunit.client.0.smithi060.stdout:8/751: chown d9/fc 2 1 2024-02-26T10:48:05.129 INFO:tasks.workunit.client.0.smithi060.stdout:8/752: readlink d9/d15/d29/d2f/d34/l8d 0 2024-02-26T10:48:05.131 INFO:tasks.workunit.client.0.smithi060.stdout:0/740: read d0/dd/d39/d63/fe5 [1400991,100428] 0 2024-02-26T10:48:05.131 INFO:tasks.workunit.client.0.smithi060.stdout:8/753: rmdir d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/dc1 39 2024-02-26T10:48:05.132 INFO:tasks.workunit.client.0.smithi060.stdout:8/754: dread - d9/d15/d29/d2f/d34/d72/d7e/fec zero size 2024-02-26T10:48:05.132 INFO:tasks.workunit.client.0.smithi060.stdout:7/891: dwrite da/d15/d21/d41/dc7/fef [0,4194304] 0 2024-02-26T10:48:05.132 INFO:tasks.workunit.client.0.smithi060.stdout:8/755: fsync d9/d15/d29/d2f/fa3 0 2024-02-26T10:48:05.138 INFO:tasks.workunit.client.0.smithi060.stdout:6/786: truncate d6/d7/d25/d1c/d1f/f35 558091 0 2024-02-26T10:48:05.138 INFO:tasks.workunit.client.0.smithi060.stdout:7/892: creat da/df1/dd7/dec/f128 x:0 0 0 2024-02-26T10:48:05.140 INFO:tasks.workunit.client.0.smithi060.stdout:7/893: mkdir da/d15/d21/d2f/d78/dd3/d67/d129 0 2024-02-26T10:48:05.143 INFO:tasks.workunit.client.0.smithi060.stdout:8/756: rename d9/d15/d3e/ld9 to d9/lf8 0 2024-02-26T10:48:05.143 INFO:tasks.workunit.client.0.smithi060.stdout:9/803: sync 2024-02-26T10:48:05.146 INFO:tasks.workunit.client.0.smithi060.stdout:8/757: mkdir d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/df9 0 2024-02-26T10:48:05.146 INFO:tasks.workunit.client.0.smithi060.stdout:3/560: dread d5/de/df/d4c/d68/f55 [0,4194304] 0 2024-02-26T10:48:05.151 INFO:tasks.workunit.client.0.smithi060.stdout:9/804: symlink d3/d1e/d5e/d35/deb/l114 0 2024-02-26T10:48:05.160 INFO:tasks.workunit.client.0.smithi060.stdout:5/724: write d0/de/d20/ff0 [2720630,110396] 0 2024-02-26T10:48:05.160 INFO:tasks.workunit.client.0.smithi060.stdout:5/725: chown d0/d4b/f51 8207 1 2024-02-26T10:48:05.160 INFO:tasks.workunit.client.0.smithi060.stdout:1/833: dwrite d3/db/d1e/d39/d9b/df7/f10a [0,4194304] 0 2024-02-26T10:48:05.160 INFO:tasks.workunit.client.0.smithi060.stdout:1/834: chown d3/l61 57 1 2024-02-26T10:48:05.163 INFO:tasks.workunit.client.0.smithi060.stdout:1/835: chown d3/db/d1e/d39/d9b/df7/f10a 6978263 1 2024-02-26T10:48:05.163 INFO:tasks.workunit.client.0.smithi060.stdout:1/836: read d3/d35/f65 [3407904,115516] 0 2024-02-26T10:48:05.163 INFO:tasks.workunit.client.0.smithi060.stdout:1/837: truncate d3/d35/d53/d68/f116 609333 0 2024-02-26T10:48:05.164 INFO:tasks.workunit.client.0.smithi060.stdout:9/805: symlink d3/d1e/d5e/d1b/l115 0 2024-02-26T10:48:05.166 INFO:tasks.workunit.client.0.smithi060.stdout:7/894: getdents da/d15/d21/d2f/d78/dd3 0 2024-02-26T10:48:05.166 INFO:tasks.workunit.client.0.smithi060.stdout:7/895: dread - da/d66/dbd/d111/f108 zero size 2024-02-26T10:48:05.166 INFO:tasks.workunit.client.0.smithi060.stdout:7/896: chown da/df1/dd7/d125/dd8 69 1 2024-02-26T10:48:05.166 INFO:tasks.workunit.client.0.smithi060.stdout:1/838: symlink d3/db/d1e/d39/l11b 0 2024-02-26T10:48:05.166 INFO:tasks.workunit.client.0.smithi060.stdout:1/839: stat d3/c28 0 2024-02-26T10:48:05.166 INFO:tasks.workunit.client.0.smithi060.stdout:0/741: dread d0/f55 [0,4194304] 0 2024-02-26T10:48:05.167 INFO:tasks.workunit.client.0.smithi060.stdout:9/806: mkdir d3/d1e/d57/d9c/d10c/d116 0 2024-02-26T10:48:05.169 INFO:tasks.workunit.client.0.smithi060.stdout:7/897: creat da/d66/dbd/d111/dfb/f12a x:0 0 0 2024-02-26T10:48:05.173 INFO:tasks.workunit.client.0.smithi060.stdout:2/739: dwrite dd/d43/d60/d41/d37/d9a/fb7 [0,4194304] 0 2024-02-26T10:48:05.173 INFO:tasks.workunit.client.0.smithi060.stdout:7/898: truncate da/d15/d21/d29/fe4 3157175 0 2024-02-26T10:48:05.173 INFO:tasks.workunit.client.0.smithi060.stdout:0/742: truncate d0/dd/d39/d40/d83/dc6/d58/f5b 651436 0 2024-02-26T10:48:05.173 INFO:tasks.workunit.client.0.smithi060.stdout:9/807: symlink d3/d1e/l117 0 2024-02-26T10:48:05.173 INFO:tasks.workunit.client.0.smithi060.stdout:9/808: readlink d3/d1e/d5e/ld6 0 2024-02-26T10:48:05.173 INFO:tasks.workunit.client.0.smithi060.stdout:7/899: chown da/d15/d88/dce/l40 34981254 1 2024-02-26T10:48:05.173 INFO:tasks.workunit.client.0.smithi060.stdout:7/900: stat da/d15/d21/f2c 0 2024-02-26T10:48:05.182 INFO:tasks.workunit.client.0.smithi060.stdout:5/726: read d0/d33/f63 [7690315,40861] 0 2024-02-26T10:48:05.182 INFO:tasks.workunit.client.0.smithi060.stdout:5/727: dread - d0/fc9 zero size 2024-02-26T10:48:05.183 INFO:tasks.workunit.client.0.smithi060.stdout:5/728: chown d0/de/d15/d26/c8b 13686111 1 2024-02-26T10:48:05.183 INFO:tasks.workunit.client.0.smithi060.stdout:5/729: creat d0/de/d70/d84/d92/f100 x:0 0 0 2024-02-26T10:48:05.188 INFO:tasks.workunit.client.0.smithi060.stdout:9/809: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f6d [0,4194304] 0 2024-02-26T10:48:05.189 INFO:tasks.workunit.client.0.smithi060.stdout:5/730: rename d0/de/d20/d32/l7d to d0/l101 0 2024-02-26T10:48:05.189 INFO:tasks.workunit.client.0.smithi060.stdout:9/810: creat d3/d1e/dbb/dc2/f118 x:0 0 0 2024-02-26T10:48:05.190 INFO:tasks.workunit.client.0.smithi060.stdout:9/811: mkdir d3/d68/d119 0 2024-02-26T10:48:05.191 INFO:tasks.workunit.client.0.smithi060.stdout:9/812: rmdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb 39 2024-02-26T10:48:05.192 INFO:tasks.workunit.client.0.smithi060.stdout:9/813: mkdir d3/d1e/d5e/d1b/d11a 0 2024-02-26T10:48:05.194 INFO:tasks.workunit.client.0.smithi060.stdout:7/901: dread da/d15/d88/dce/faf [0,4194304] 0 2024-02-26T10:48:05.197 INFO:tasks.workunit.client.0.smithi060.stdout:9/814: rename d3/d1e/d57/d9c/fa5 to d3/d68/d119/f11b 0 2024-02-26T10:48:05.215 INFO:tasks.workunit.client.0.smithi060.stdout:6/787: dwrite d6/d7/d25/f49 [0,4194304] 0 2024-02-26T10:48:05.215 INFO:tasks.workunit.client.0.smithi060.stdout:8/758: dread d9/d15/d29/d2f/d34/d72/d50/d6a/f45 [0,4194304] 0 2024-02-26T10:48:05.217 INFO:tasks.workunit.client.0.smithi060.stdout:8/759: creat d9/d5a/ffa x:0 0 0 2024-02-26T10:48:05.217 INFO:tasks.workunit.client.0.smithi060.stdout:8/760: fdatasync d9/d6f/ff3 0 2024-02-26T10:48:05.218 INFO:tasks.workunit.client.0.smithi060.stdout:6/788: unlink d6/d7/d25/d1c/d1f/d23/d28/f7f 0 2024-02-26T10:48:05.220 INFO:tasks.workunit.client.0.smithi060.stdout:6/789: symlink d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/da7/l101 0 2024-02-26T10:48:05.222 INFO:tasks.workunit.client.0.smithi060.stdout:6/790: getdents d6/d8d/dcc 0 2024-02-26T10:48:05.224 INFO:tasks.workunit.client.0.smithi060.stdout:9/815: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f4c [0,4194304] 0 2024-02-26T10:48:05.226 INFO:tasks.workunit.client.0.smithi060.stdout:9/816: mknod d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/c11c 0 2024-02-26T10:48:05.228 INFO:tasks.workunit.client.0.smithi060.stdout:9/817: write d3/d1e/d5e/d35/deb/d7a/fd1 [1123507,121462] 0 2024-02-26T10:48:05.233 INFO:tasks.workunit.client.0.smithi060.stdout:9/818: dread - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/f110 zero size 2024-02-26T10:48:05.235 INFO:tasks.workunit.client.0.smithi060.stdout:9/819: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/f11d x:0 0 0 2024-02-26T10:48:05.243 INFO:tasks.workunit.client.0.smithi060.stdout:1/840: dread d3/d35/d53/d68/d90/fbc [0,4194304] 0 2024-02-26T10:48:05.246 INFO:tasks.workunit.client.0.smithi060.stdout:3/561: dwrite d5/de/df/d4c/d68/f8e [8388608,4194304] 0 2024-02-26T10:48:05.256 INFO:tasks.workunit.client.0.smithi060.stdout:1/841: rename d3/db/d1e/d39/l9f to d3/db/d1e/d39/d9b/l11c 0 2024-02-26T10:48:05.274 INFO:tasks.workunit.client.0.smithi060.stdout:7/902: dwrite da/d15/d21/d2f/fbf [0,4194304] 0 2024-02-26T10:48:05.283 INFO:tasks.workunit.client.0.smithi060.stdout:8/761: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/fbc [0,4194304] 0 2024-02-26T10:48:05.286 INFO:tasks.workunit.client.0.smithi060.stdout:9/820: read d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f37 [2088101,109569] 0 2024-02-26T10:48:05.286 INFO:tasks.workunit.client.0.smithi060.stdout:9/821: readlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/l9f 0 2024-02-26T10:48:05.286 INFO:tasks.workunit.client.0.smithi060.stdout:9/822: chown d3/d1e/l7b 128153 1 2024-02-26T10:48:05.287 INFO:tasks.workunit.client.0.smithi060.stdout:9/823: dread - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/f11d zero size 2024-02-26T10:48:05.287 INFO:tasks.workunit.client.0.smithi060.stdout:0/743: dwrite d0/dd/d39/d63/ff6 [0,4194304] 0 2024-02-26T10:48:05.288 INFO:tasks.workunit.client.0.smithi060.stdout:0/744: read d0/dd/d39/d40/d83/dc6/f66 [428455,99015] 0 2024-02-26T10:48:05.290 INFO:tasks.workunit.client.0.smithi060.stdout:8/762: mkdir d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/dc1/dfb 0 2024-02-26T10:48:05.292 INFO:tasks.workunit.client.0.smithi060.stdout:0/745: link d0/f82 d0/dd/d53/d87/ff7 0 2024-02-26T10:48:05.292 INFO:tasks.workunit.client.0.smithi060.stdout:0/746: chown d0/dd/d39/d40/d83/dc6/l5e 392768 1 2024-02-26T10:48:05.292 INFO:tasks.workunit.client.0.smithi060.stdout:0/747: write d0/dd/d39/f54 [1546178,69090] 0 2024-02-26T10:48:05.293 INFO:tasks.workunit.client.0.smithi060.stdout:8/763: link d9/d15/d29/d2f/d34/d72/d50/f7b d9/d15/d29/d2f/d34/d72/d50/ffc 0 2024-02-26T10:48:05.294 INFO:tasks.workunit.client.0.smithi060.stdout:0/748: link d0/dd/d39/d40/d83/dc6/f79 d0/dd/d53/d87/d94/db7/ff8 0 2024-02-26T10:48:05.295 INFO:tasks.workunit.client.0.smithi060.stdout:3/562: dwrite d5/f24 [0,4194304] 0 2024-02-26T10:48:05.296 INFO:tasks.workunit.client.0.smithi060.stdout:6/791: dwrite d6/f11 [4194304,4194304] 0 2024-02-26T10:48:05.296 INFO:tasks.workunit.client.0.smithi060.stdout:6/792: dread - d6/d8d/dcc/fec zero size 2024-02-26T10:48:05.300 INFO:tasks.workunit.client.0.smithi060.stdout:8/764: symlink d9/d6f/lfd 0 2024-02-26T10:48:05.300 INFO:tasks.workunit.client.0.smithi060.stdout:8/765: chown d9/fd 231686 1 2024-02-26T10:48:05.300 INFO:tasks.workunit.client.0.smithi060.stdout:8/766: chown d9/d15/d29/d2f/d34/fdf 5094227 1 2024-02-26T10:48:05.300 INFO:tasks.workunit.client.0.smithi060.stdout:1/842: dread d3/db/d1e/d6d/da3/f10f [0,4194304] 0 2024-02-26T10:48:05.300 INFO:tasks.workunit.client.0.smithi060.stdout:8/767: write d9/f23 [4870879,69223] 0 2024-02-26T10:48:05.301 INFO:tasks.workunit.client.0.smithi060.stdout:3/563: rename d5/d45/d61/d80 to d5/d45/d61/d92/da1/daa 0 2024-02-26T10:48:05.302 INFO:tasks.workunit.client.0.smithi060.stdout:6/793: mknod d6/d7/d47/d69/c102 0 2024-02-26T10:48:05.302 INFO:tasks.workunit.client.0.smithi060.stdout:8/768: chown d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/fdb 15964248 1 2024-02-26T10:48:05.309 INFO:tasks.workunit.client.0.smithi060.stdout:6/794: write d6/d7/d25/d1c/d1f/d23/d28/fd4 [169684,50521] 0 2024-02-26T10:48:05.310 INFO:tasks.workunit.client.0.smithi060.stdout:5/731: dwrite d0/de/d20/d32/fed [0,4194304] 0 2024-02-26T10:48:05.310 INFO:tasks.workunit.client.0.smithi060.stdout:5/732: chown d0/de/d15/d65/d86/d9b/lfc 794472010 1 2024-02-26T10:48:05.330 INFO:tasks.workunit.client.0.smithi060.stdout:8/769: dread d9/f20 [0,4194304] 0 2024-02-26T10:48:05.347 INFO:tasks.workunit.client.0.smithi060.stdout:6/795: write d6/d7/d25/d1c/d1f/d23/d2b/d67/f8a [1621310,128135] 0 2024-02-26T10:48:05.350 INFO:tasks.workunit.client.0.smithi060.stdout:9/824: dwrite d3/d1e/d93/fce [0,4194304] 0 2024-02-26T10:48:05.350 INFO:tasks.workunit.client.0.smithi060.stdout:9/825: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/ff0 [4743957,107148] 0 2024-02-26T10:48:05.350 INFO:tasks.workunit.client.0.smithi060.stdout:9/826: stat d3/d68/l112 0 2024-02-26T10:48:05.350 INFO:tasks.workunit.client.0.smithi060.stdout:9/827: truncate d3/d1e/d5e/d35/d87/d9b/d8a/dbe/fb6 1196388 0 2024-02-26T10:48:05.351 INFO:tasks.workunit.client.0.smithi060.stdout:5/733: dwrite d0/de/d15/d65/d86/d9b/ff7 [0,4194304] 0 2024-02-26T10:48:05.353 INFO:tasks.workunit.client.0.smithi060.stdout:5/734: creat d0/d4b/d7b/f102 x:0 0 0 2024-02-26T10:48:05.353 INFO:tasks.workunit.client.0.smithi060.stdout:5/735: dread - d0/de/d15/d65/d86/ff3 zero size 2024-02-26T10:48:05.353 INFO:tasks.workunit.client.0.smithi060.stdout:5/736: chown d0/de/d15/d65/d86/dcc/fd5 8117084 1 2024-02-26T10:48:05.353 INFO:tasks.workunit.client.0.smithi060.stdout:5/737: chown d0/de/da4/fd7 9 1 2024-02-26T10:48:05.354 INFO:tasks.workunit.client.0.smithi060.stdout:2/740: sync 2024-02-26T10:48:05.354 INFO:tasks.workunit.client.0.smithi060.stdout:4/743: sync 2024-02-26T10:48:05.355 INFO:tasks.workunit.client.0.smithi060.stdout:0/749: write d0/f32 [1819259,25247] 0 2024-02-26T10:48:05.355 INFO:tasks.workunit.client.0.smithi060.stdout:7/903: dwrite da/d15/d21/d29/fe4 [0,4194304] 0 2024-02-26T10:48:05.356 INFO:tasks.workunit.client.0.smithi060.stdout:4/744: mknod d0/da/dc7/cea 0 2024-02-26T10:48:05.357 INFO:tasks.workunit.client.0.smithi060.stdout:0/750: creat d0/dd/d53/da2/ff9 x:0 0 0 2024-02-26T10:48:05.357 INFO:tasks.workunit.client.0.smithi060.stdout:0/751: chown d0/dd/d53/l84 623 1 2024-02-26T10:48:05.358 INFO:tasks.workunit.client.0.smithi060.stdout:6/796: link d6/d7/d47/c4f d6/d7/d25/c103 0 2024-02-26T10:48:05.360 INFO:tasks.workunit.client.0.smithi060.stdout:8/770: write d9/d15/d29/f2c [3287158,119168] 0 2024-02-26T10:48:05.360 INFO:tasks.workunit.client.0.smithi060.stdout:0/752: symlink d0/d37/ddd/lfa 0 2024-02-26T10:48:05.360 INFO:tasks.workunit.client.0.smithi060.stdout:6/797: creat d6/d7/d2e/d56/f104 x:0 0 0 2024-02-26T10:48:05.365 INFO:tasks.workunit.client.0.smithi060.stdout:0/753: creat d0/d42/d7a/ffb x:0 0 0 2024-02-26T10:48:05.376 INFO:tasks.workunit.client.0.smithi060.stdout:7/904: dread da/df1/f11f [0,4194304] 0 2024-02-26T10:48:05.379 INFO:tasks.workunit.client.0.smithi060.stdout:0/754: truncate d0/dd/d53/d87/d94/db7/ff8 1809615 0 2024-02-26T10:48:05.381 INFO:tasks.workunit.client.0.smithi060.stdout:0/755: stat d0/dd/c4c 0 2024-02-26T10:48:05.393 INFO:tasks.workunit.client.0.smithi060.stdout:4/745: dread d0/da/fb1 [0,4194304] 0 2024-02-26T10:48:05.404 INFO:tasks.workunit.client.0.smithi060.stdout:2/741: dwrite dd/d43/d60/d41/d37/d38/d57/fe8 [0,4194304] 0 2024-02-26T10:48:05.405 INFO:tasks.workunit.client.0.smithi060.stdout:2/742: fdatasync dd/f94 0 2024-02-26T10:48:05.406 INFO:tasks.workunit.client.0.smithi060.stdout:9/828: rename d3/d1e/d57/d9c to d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d11e 0 2024-02-26T10:48:05.408 INFO:tasks.workunit.client.0.smithi060.stdout:9/829: symlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8/dcd/l11f 0 2024-02-26T10:48:05.408 INFO:tasks.workunit.client.0.smithi060.stdout:9/830: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91 1548529 1 2024-02-26T10:48:05.409 INFO:tasks.workunit.client.0.smithi060.stdout:9/831: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c 1026162984 1 2024-02-26T10:48:05.409 INFO:tasks.workunit.client.0.smithi060.stdout:9/832: dread - d3/d1e/dbb/dc2/f118 zero size 2024-02-26T10:48:05.409 INFO:tasks.workunit.client.0.smithi060.stdout:2/743: unlink dd/d43/d55/d5e/dc6/fcb 0 2024-02-26T10:48:05.411 INFO:tasks.workunit.client.0.smithi060.stdout:9/833: getdents d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb 0 2024-02-26T10:48:05.411 INFO:tasks.workunit.client.0.smithi060.stdout:2/744: symlink dd/d43/d60/d41/d37/lf9 0 2024-02-26T10:48:05.411 INFO:tasks.workunit.client.0.smithi060.stdout:2/745: fsync dd/d43/d55/d6e/fb8 0 2024-02-26T10:48:05.411 INFO:tasks.workunit.client.0.smithi060.stdout:8/771: dwrite d9/d15/fe5 [0,4194304] 0 2024-02-26T10:48:05.411 INFO:tasks.workunit.client.0.smithi060.stdout:8/772: dread - d9/d15/d29/d2f/d34/ff7 zero size 2024-02-26T10:48:05.412 INFO:tasks.workunit.client.0.smithi060.stdout:0/756: dread d0/dd/f31 [0,4194304] 0 2024-02-26T10:48:05.433 INFO:tasks.workunit.client.0.smithi060.stdout:5/738: rename d0/de/d15/d65/d86/d9b/l80 to d0/de/d70/d84/l103 0 2024-02-26T10:48:05.435 INFO:tasks.workunit.client.0.smithi060.stdout:7/905: rename da/d15/d21/d2f/d37/c86 to da/d15/d21/d41/d6c/c12b 0 2024-02-26T10:48:05.439 INFO:tasks.workunit.client.0.smithi060.stdout:7/906: mkdir da/df1/dd7/d125/dd8/d12c 0 2024-02-26T10:48:05.439 INFO:tasks.workunit.client.0.smithi060.stdout:7/907: fdatasync da/d15/d21/ff7 0 2024-02-26T10:48:05.439 INFO:tasks.workunit.client.0.smithi060.stdout:7/908: write da/d15/d88/d3b/fde [300578,37749] 0 2024-02-26T10:48:05.439 INFO:tasks.workunit.client.0.smithi060.stdout:7/909: chown da/df1/ff8 59213461 1 2024-02-26T10:48:05.440 INFO:tasks.workunit.client.0.smithi060.stdout:7/910: mknod da/df1/c12d 0 2024-02-26T10:48:05.442 INFO:tasks.workunit.client.0.smithi060.stdout:7/911: creat da/df1/dd7/de0/f12e x:0 0 0 2024-02-26T10:48:05.442 INFO:tasks.workunit.client.0.smithi060.stdout:7/912: fsync da/d15/d21/d2f/f48 0 2024-02-26T10:48:05.443 INFO:tasks.workunit.client.0.smithi060.stdout:7/913: truncate da/d15/d21/d2f/d78/fac 1662489 0 2024-02-26T10:48:05.445 INFO:tasks.workunit.client.0.smithi060.stdout:7/914: symlink da/d15/d21/d41/l12f 0 2024-02-26T10:48:05.445 INFO:tasks.workunit.client.0.smithi060.stdout:7/915: chown da/d15/d21/d29/fc3 3 1 2024-02-26T10:48:05.446 INFO:tasks.workunit.client.0.smithi060.stdout:9/834: dwrite d3/d1e/d93/da2/fb3 [0,4194304] 0 2024-02-26T10:48:05.446 INFO:tasks.workunit.client.0.smithi060.stdout:5/739: write d0/de/d15/f44 [36051,100086] 0 2024-02-26T10:48:05.458 INFO:tasks.workunit.client.0.smithi060.stdout:5/740: dread d0/de/d20/f85 [0,4194304] 0 2024-02-26T10:48:05.460 INFO:tasks.workunit.client.0.smithi060.stdout:5/741: unlink d0/f14 0 2024-02-26T10:48:05.462 INFO:tasks.workunit.client.0.smithi060.stdout:4/746: rename d0/da/d3b/d1b/ld7 to d0/d27/d76/dbd/d60/leb 0 2024-02-26T10:48:05.467 INFO:tasks.workunit.client.0.smithi060.stdout:0/757: dwrite d0/dd/d39/d93/fb6 [0,4194304] 0 2024-02-26T10:48:05.469 INFO:tasks.workunit.client.0.smithi060.stdout:0/758: mknod d0/dd/d53/d87/d94/cfc 0 2024-02-26T10:48:05.470 INFO:tasks.workunit.client.0.smithi060.stdout:0/759: link d0/dd/d39/ccf d0/d42/d7a/cfd 0 2024-02-26T10:48:05.471 INFO:tasks.workunit.client.0.smithi060.stdout:7/916: dread da/d15/d21/d2f/f48 [0,4194304] 0 2024-02-26T10:48:05.472 INFO:tasks.workunit.client.0.smithi060.stdout:0/760: symlink d0/dd/d39/d40/dad/dd6/lfe 0 2024-02-26T10:48:05.472 INFO:tasks.workunit.client.0.smithi060.stdout:7/917: truncate da/d15/d21/d41/d6c/f6e 840185 0 2024-02-26T10:48:05.472 INFO:tasks.workunit.client.0.smithi060.stdout:0/761: truncate d0/dd/d39/d40/d83/dc0/fe6 544796 0 2024-02-26T10:48:05.488 INFO:tasks.workunit.client.0.smithi060.stdout:9/835: dread d3/d1e/d5e/d1b/f56 [0,4194304] 0 2024-02-26T10:48:05.491 INFO:tasks.workunit.client.0.smithi060.stdout:9/836: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d11e/d10c/d116/f120 x:0 0 0 2024-02-26T10:48:05.497 INFO:tasks.workunit.client.0.smithi060.stdout:9/837: creat d3/d1e/d57/f121 x:0 0 0 2024-02-26T10:48:05.501 INFO:tasks.workunit.client.0.smithi060.stdout:2/746: dwrite dd/d43/f6c [0,4194304] 0 2024-02-26T10:48:05.503 INFO:tasks.workunit.client.0.smithi060.stdout:2/747: mkdir dd/d43/d55/d5e/dfa 0 2024-02-26T10:48:05.503 INFO:tasks.workunit.client.0.smithi060.stdout:5/742: dwrite d0/f3a [4194304,4194304] 0 2024-02-26T10:48:05.504 INFO:tasks.workunit.client.0.smithi060.stdout:2/748: write dd/d43/d60/d41/d37/d4c/d76/fa3 [4950137,38156] 0 2024-02-26T10:48:05.504 INFO:tasks.workunit.client.0.smithi060.stdout:5/743: write d0/de/fd8 [366002,111264] 0 2024-02-26T10:48:05.504 INFO:tasks.workunit.client.0.smithi060.stdout:5/744: chown d0/d4b/d7b 1609 1 2024-02-26T10:48:05.504 INFO:tasks.workunit.client.0.smithi060.stdout:7/918: write da/d15/d21/d2f/f101 [5225630,100567] 0 2024-02-26T10:48:05.504 INFO:tasks.workunit.client.0.smithi060.stdout:7/919: write da/d15/d88/d3b/f54 [241328,56566] 0 2024-02-26T10:48:05.506 INFO:tasks.workunit.client.0.smithi060.stdout:2/749: unlink dd/d43/d60/d41/d37/l49 0 2024-02-26T10:48:05.508 INFO:tasks.workunit.client.0.smithi060.stdout:2/750: unlink dd/d43/d55/d5e/d86/d97/cd7 0 2024-02-26T10:48:05.508 INFO:tasks.workunit.client.0.smithi060.stdout:2/751: readlink dd/d43/d55/d5e/d86/la1 0 2024-02-26T10:48:05.508 INFO:tasks.workunit.client.0.smithi060.stdout:2/752: chown dd/d43/d96/laf 9156 1 2024-02-26T10:48:05.511 INFO:tasks.workunit.client.0.smithi060.stdout:2/753: creat dd/d43/d55/d5e/dfa/ffb x:0 0 0 2024-02-26T10:48:05.512 INFO:tasks.workunit.client.0.smithi060.stdout:2/754: rename dd/f94 to dd/dd1/ffc 0 2024-02-26T10:48:05.513 INFO:tasks.workunit.client.0.smithi060.stdout:7/920: rmdir da/d66/dbd 39 2024-02-26T10:48:05.514 INFO:tasks.workunit.client.0.smithi060.stdout:2/755: getdents dd/d43/d96 0 2024-02-26T10:48:05.514 INFO:tasks.workunit.client.0.smithi060.stdout:7/921: mkdir da/d15/d21/d41/d6c/d130 0 2024-02-26T10:48:05.514 INFO:tasks.workunit.client.0.smithi060.stdout:7/922: chown da/d15/d21/d41/dc7 0 1 2024-02-26T10:48:05.514 INFO:tasks.workunit.client.0.smithi060.stdout:7/923: stat da/d15/d21/d29/fe4 0 2024-02-26T10:48:05.515 INFO:tasks.workunit.client.0.smithi060.stdout:0/762: dwrite d0/dd/d39/d40/d83/dc6/d58/f5b [0,4194304] 0 2024-02-26T10:48:05.516 INFO:tasks.workunit.client.0.smithi060.stdout:2/756: mkdir dd/d43/d60/d41/d37/dfd 0 2024-02-26T10:48:05.516 INFO:tasks.workunit.client.0.smithi060.stdout:5/745: getdents d0/de/d15/d65/d86/d9b 0 2024-02-26T10:48:05.517 INFO:tasks.workunit.client.0.smithi060.stdout:2/757: rename dd/d43/d60/f53 to dd/d43/dd2/ffe 0 2024-02-26T10:48:05.518 INFO:tasks.workunit.client.0.smithi060.stdout:2/758: mknod dd/d43/d60/d41/d37/dfd/cff 0 2024-02-26T10:48:05.520 INFO:tasks.workunit.client.0.smithi060.stdout:0/763: creat d0/dd/d39/d40/dad/dd6/d6c/fff x:0 0 0 2024-02-26T10:48:05.523 INFO:tasks.workunit.client.0.smithi060.stdout:0/764: creat d0/dd/d39/d93/f100 x:0 0 0 2024-02-26T10:48:05.524 INFO:tasks.workunit.client.0.smithi060.stdout:0/765: symlink d0/l101 0 2024-02-26T10:48:05.526 INFO:tasks.workunit.client.0.smithi060.stdout:0/766: mkdir d0/d42/df2/d102 0 2024-02-26T10:48:05.527 INFO:tasks.workunit.client.0.smithi060.stdout:0/767: mkdir d0/dd/d39/d40/dad/dda/de4/d103 0 2024-02-26T10:48:05.529 INFO:tasks.workunit.client.0.smithi060.stdout:3/564: sync 2024-02-26T10:48:05.529 INFO:tasks.workunit.client.0.smithi060.stdout:1/843: sync 2024-02-26T10:48:05.529 INFO:tasks.workunit.client.0.smithi060.stdout:3/565: chown d5/de/c3e 11845915 1 2024-02-26T10:48:05.529 INFO:tasks.workunit.client.0.smithi060.stdout:3/566: readlink d5/d45/d61/la7 0 2024-02-26T10:48:05.529 INFO:tasks.workunit.client.0.smithi060.stdout:5/746: dread d0/d33/f62 [0,4194304] 0 2024-02-26T10:48:05.529 INFO:tasks.workunit.client.0.smithi060.stdout:5/747: dread - d0/ff4 zero size 2024-02-26T10:48:05.529 INFO:tasks.workunit.client.0.smithi060.stdout:5/748: chown d0/de/d70/d84/ca2 240465263 1 2024-02-26T10:48:05.531 INFO:tasks.workunit.client.0.smithi060.stdout:3/567: getdents d5/de/df/d11/d63/d93 0 2024-02-26T10:48:05.531 INFO:tasks.workunit.client.0.smithi060.stdout:3/568: truncate d5/de/df/d11/d63/d93/f96 122039 0 2024-02-26T10:48:05.531 INFO:tasks.workunit.client.0.smithi060.stdout:1/844: creat d3/d35/d53/f11d x:0 0 0 2024-02-26T10:48:05.531 INFO:tasks.workunit.client.0.smithi060.stdout:4/747: dwrite d0/d27/d76/dbd/d8e/fac [0,4194304] 0 2024-02-26T10:48:05.531 INFO:tasks.workunit.client.0.smithi060.stdout:2/759: rename dd/dd1/ffc to dd/d43/d60/d41/d37/d9a/db4/f100 0 2024-02-26T10:48:05.536 INFO:tasks.workunit.client.0.smithi060.stdout:2/760: truncate dd/d11/f26 2439532 0 2024-02-26T10:48:05.537 INFO:tasks.workunit.client.0.smithi060.stdout:5/749: dread d0/de/d20/f85 [0,4194304] 0 2024-02-26T10:48:05.539 INFO:tasks.workunit.client.0.smithi060.stdout:2/761: truncate dd/d43/d60/ff3 3495572 0 2024-02-26T10:48:05.539 INFO:tasks.workunit.client.0.smithi060.stdout:2/762: fdatasync dd/d43/d60/d41/d37/d4c/fc2 0 2024-02-26T10:48:05.543 INFO:tasks.workunit.client.0.smithi060.stdout:2/763: write dd/f6a [314389,71399] 0 2024-02-26T10:48:05.543 INFO:tasks.workunit.client.0.smithi060.stdout:2/764: write dd/d43/d60/d41/d37/d4c/fa2 [959714,17607] 0 2024-02-26T10:48:05.544 INFO:tasks.workunit.client.0.smithi060.stdout:6/798: sync 2024-02-26T10:48:05.546 INFO:tasks.workunit.client.0.smithi060.stdout:9/838: dwrite d3/d1e/d57/f88 [0,4194304] 0 2024-02-26T10:48:05.547 INFO:tasks.workunit.client.0.smithi060.stdout:3/569: rename d5/d37/c4b to d5/de/df/d11/d63/d93/da2/cab 0 2024-02-26T10:48:05.547 INFO:tasks.workunit.client.0.smithi060.stdout:5/750: creat d0/de/d20/d32/d4d/da3/da5/f104 x:0 0 0 2024-02-26T10:48:05.549 INFO:tasks.workunit.client.0.smithi060.stdout:7/924: dread da/d15/d21/d2f/d37/f38 [0,4194304] 0 2024-02-26T10:48:05.549 INFO:tasks.workunit.client.0.smithi060.stdout:3/570: unlink d5/de/df/d11/d63/l9c 0 2024-02-26T10:48:05.550 INFO:tasks.workunit.client.0.smithi060.stdout:7/925: mknod da/df1/dd7/de0/c131 0 2024-02-26T10:48:05.550 INFO:tasks.workunit.client.0.smithi060.stdout:7/926: read - da/d15/d21/d41/fd5 zero size 2024-02-26T10:48:05.550 INFO:tasks.workunit.client.0.smithi060.stdout:7/927: fsync da/f18 0 2024-02-26T10:48:05.551 INFO:tasks.workunit.client.0.smithi060.stdout:3/571: symlink d5/d45/d61/d92/da1/daa/lac 0 2024-02-26T10:48:05.551 INFO:tasks.workunit.client.0.smithi060.stdout:9/839: mknod d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/de9/c122 0 2024-02-26T10:48:05.552 INFO:tasks.workunit.client.0.smithi060.stdout:7/928: creat da/d66/dbd/d111/f132 x:0 0 0 2024-02-26T10:48:05.553 INFO:tasks.workunit.client.0.smithi060.stdout:3/572: mkdir d5/d45/d61/d92/da1/daa/dad 0 2024-02-26T10:48:05.555 INFO:tasks.workunit.client.0.smithi060.stdout:5/751: rename d0/fd to d0/de/d70/f105 0 2024-02-26T10:48:05.560 INFO:tasks.workunit.client.0.smithi060.stdout:7/929: mkdir da/d15/d21/d2f/d78/dd3/d133 0 2024-02-26T10:48:05.584 INFO:tasks.workunit.client.0.smithi060.stdout:5/752: read d0/de/d20/d32/d4c/ffd [2740845,118056] 0 2024-02-26T10:48:05.597 INFO:tasks.workunit.client.0.smithi060.stdout:8/773: sync 2024-02-26T10:48:05.597 INFO:tasks.workunit.client.0.smithi060.stdout:8/774: fsync d9/d15/d3e/f9b 0 2024-02-26T10:48:05.607 INFO:tasks.workunit.client.0.smithi060.stdout:4/748: dwrite d0/da/d3b/d1b/fa7 [0,4194304] 0 2024-02-26T10:48:05.607 INFO:tasks.workunit.client.0.smithi060.stdout:1/845: dread d3/d35/d53/d68/d85/fd0 [0,4194304] 0 2024-02-26T10:48:05.607 INFO:tasks.workunit.client.0.smithi060.stdout:4/749: chown d0/da/l54 919091 1 2024-02-26T10:48:05.609 INFO:tasks.workunit.client.0.smithi060.stdout:6/799: dwrite d6/d7/d25/d1c/d1f/d23/faf [0,4194304] 0 2024-02-26T10:48:05.611 INFO:tasks.workunit.client.0.smithi060.stdout:0/768: dwrite d0/dd/d39/d40/d83/dc6/f79 [0,4194304] 0 2024-02-26T10:48:05.611 INFO:tasks.workunit.client.0.smithi060.stdout:2/765: dread dd/d11/fe1 [0,4194304] 0 2024-02-26T10:48:05.616 INFO:tasks.workunit.client.0.smithi060.stdout:4/750: rename d0/d27/f5e to d0/d27/dde/fec 0 2024-02-26T10:48:05.616 INFO:tasks.workunit.client.0.smithi060.stdout:4/751: chown d0/d27/f4d 0 1 2024-02-26T10:48:05.616 INFO:tasks.workunit.client.0.smithi060.stdout:6/800: symlink d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/l105 0 2024-02-26T10:48:05.617 INFO:tasks.workunit.client.0.smithi060.stdout:8/775: creat d9/d15/d29/d2f/d34/d72/ffe x:0 0 0 2024-02-26T10:48:05.617 INFO:tasks.workunit.client.0.smithi060.stdout:8/776: readlink d9/d5a/le0 0 2024-02-26T10:48:05.618 INFO:tasks.workunit.client.0.smithi060.stdout:1/846: dread d3/d35/d53/d68/dad/fb6 [0,4194304] 0 2024-02-26T10:48:05.619 INFO:tasks.workunit.client.0.smithi060.stdout:6/801: creat d6/f106 x:0 0 0 2024-02-26T10:48:05.620 INFO:tasks.workunit.client.0.smithi060.stdout:3/573: dwrite d5/d45/f8f [0,4194304] 0 2024-02-26T10:48:05.621 INFO:tasks.workunit.client.0.smithi060.stdout:8/777: creat d9/d15/fff x:0 0 0 2024-02-26T10:48:05.622 INFO:tasks.workunit.client.0.smithi060.stdout:1/847: creat d3/db/f11e x:0 0 0 2024-02-26T10:48:05.622 INFO:tasks.workunit.client.0.smithi060.stdout:1/848: write d3/d35/f65 [3061363,102766] 0 2024-02-26T10:48:05.622 INFO:tasks.workunit.client.0.smithi060.stdout:1/849: chown d3/db/d1e/d6d/dd1/fe7 0 1 2024-02-26T10:48:05.627 INFO:tasks.workunit.client.0.smithi060.stdout:3/574: creat d5/de/df/d11/fae x:0 0 0 2024-02-26T10:48:05.627 INFO:tasks.workunit.client.0.smithi060.stdout:7/930: dwrite da/d15/d21/d2f/d78/dd3/d67/f10e [4194304,4194304] 0 2024-02-26T10:48:05.627 INFO:tasks.workunit.client.0.smithi060.stdout:7/931: readlink da/d94/lc6 0 2024-02-26T10:48:05.627 INFO:tasks.workunit.client.0.smithi060.stdout:4/752: write d0/da/d57/f75 [2918596,65646] 0 2024-02-26T10:48:05.627 INFO:tasks.workunit.client.0.smithi060.stdout:1/850: mkdir d3/db/d1e/d39/d9b/df7/d11f 0 2024-02-26T10:48:05.628 INFO:tasks.workunit.client.0.smithi060.stdout:1/851: write d3/d35/d53/f11d [350318,75205] 0 2024-02-26T10:48:05.629 INFO:tasks.workunit.client.0.smithi060.stdout:1/852: truncate d3/db/d1e/d6d/d78/f86 3257067 0 2024-02-26T10:48:05.630 INFO:tasks.workunit.client.0.smithi060.stdout:4/753: creat d0/d27/d76/d9f/fed x:0 0 0 2024-02-26T10:48:05.630 INFO:tasks.workunit.client.0.smithi060.stdout:4/754: chown d0/d27/d2f/f7d 50 1 2024-02-26T10:48:05.630 INFO:tasks.workunit.client.0.smithi060.stdout:4/755: chown d0/da/d3b/c11 50655 1 2024-02-26T10:48:05.632 INFO:tasks.workunit.client.0.smithi060.stdout:7/932: unlink da/d15/d21/d2f/d78/dd3/d67/f77 0 2024-02-26T10:48:05.632 INFO:tasks.workunit.client.0.smithi060.stdout:7/933: fsync da/d15/d21/d2f/d78/fa7 0 2024-02-26T10:48:05.633 INFO:tasks.workunit.client.0.smithi060.stdout:6/802: dread d6/d7/d25/d1c/d1f/d23/d28/fd5 [0,4194304] 0 2024-02-26T10:48:05.634 INFO:tasks.workunit.client.0.smithi060.stdout:7/934: symlink da/d15/d21/d2f/d78/dd3/d133/l134 0 2024-02-26T10:48:05.638 INFO:tasks.workunit.client.0.smithi060.stdout:8/778: rename d9/d5a/ffa to d9/d15/d29/d2f/d34/d72/f100 0 2024-02-26T10:48:05.639 INFO:tasks.workunit.client.0.smithi060.stdout:8/779: truncate d9/d15/d29/d2f/d34/d72/f100 513744 0 2024-02-26T10:48:05.641 INFO:tasks.workunit.client.0.smithi060.stdout:8/780: mkdir d9/d15/d29/d2f/d34/d72/d50/d101 0 2024-02-26T10:48:05.641 INFO:tasks.workunit.client.0.smithi060.stdout:8/781: chown d9/d15/d3e 484031 1 2024-02-26T10:48:05.644 INFO:tasks.workunit.client.0.smithi060.stdout:3/575: write d5/de/df/d4c/f70 [3281215,55274] 0 2024-02-26T10:48:05.645 INFO:tasks.workunit.client.0.smithi060.stdout:7/935: mkdir da/d15/d21/d2f/d78/dd3/d67/da8/de3/d135 0 2024-02-26T10:48:05.645 INFO:tasks.workunit.client.0.smithi060.stdout:7/936: readlink da/d94/l113 0 2024-02-26T10:48:05.647 INFO:tasks.workunit.client.0.smithi060.stdout:7/937: creat da/d15/d21/d2f/d78/dd3/d67/d129/f136 x:0 0 0 2024-02-26T10:48:05.648 INFO:tasks.workunit.client.0.smithi060.stdout:7/938: unlink da/f24 0 2024-02-26T10:48:05.649 INFO:tasks.workunit.client.0.smithi060.stdout:7/939: symlink da/d15/d88/l137 0 2024-02-26T10:48:05.651 INFO:tasks.workunit.client.0.smithi060.stdout:7/940: creat da/d15/d21/d2f/d78/dd3/d67/da8/de3/f138 x:0 0 0 2024-02-26T10:48:05.652 INFO:tasks.workunit.client.0.smithi060.stdout:7/941: stat da/df1/dd7/d125/d8f/l119 0 2024-02-26T10:48:05.653 INFO:tasks.workunit.client.0.smithi060.stdout:7/942: truncate da/df1/dd7/d125/fa4 2954141 0 2024-02-26T10:48:05.654 INFO:tasks.workunit.client.0.smithi060.stdout:7/943: creat da/d15/d21/d29/f139 x:0 0 0 2024-02-26T10:48:05.656 INFO:tasks.workunit.client.0.smithi060.stdout:7/944: creat da/d15/d21/d41/f13a x:0 0 0 2024-02-26T10:48:05.656 INFO:tasks.workunit.client.0.smithi060.stdout:7/945: truncate da/d15/d21/d2f/d78/fa1 347368 0 2024-02-26T10:48:05.673 INFO:tasks.workunit.client.0.smithi060.stdout:1/853: dwrite d3/d35/d53/d68/fdc [0,4194304] 0 2024-02-26T10:48:05.677 INFO:tasks.workunit.client.0.smithi060.stdout:1/854: read d3/db/d1e/f102 [91450,18425] 0 2024-02-26T10:48:05.679 INFO:tasks.workunit.client.0.smithi060.stdout:1/855: unlink d3/db/d1e/d6d/dd9/la8 0 2024-02-26T10:48:05.682 INFO:tasks.workunit.client.0.smithi060.stdout:1/856: mknod d3/db/d1e/d6d/dd9/c120 0 2024-02-26T10:48:05.682 INFO:tasks.workunit.client.0.smithi060.stdout:1/857: stat d3/d35/d53/d68/d85 0 2024-02-26T10:48:05.683 INFO:tasks.workunit.client.0.smithi060.stdout:1/858: mkdir d3/db/d1e/d6d/d78/dc4/d121 0 2024-02-26T10:48:05.690 INFO:tasks.workunit.client.0.smithi060.stdout:8/782: dwrite d9/d5a/fe7 [0,4194304] 0 2024-02-26T10:48:05.692 INFO:tasks.workunit.client.0.smithi060.stdout:3/576: write d5/de/df/d57/d60/f51 [490039,47550] 0 2024-02-26T10:48:05.693 INFO:tasks.workunit.client.0.smithi060.stdout:8/783: creat d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/f102 x:0 0 0 2024-02-26T10:48:05.693 INFO:tasks.workunit.client.0.smithi060.stdout:3/577: fdatasync d5/de/df/d11/f1c 0 2024-02-26T10:48:05.693 INFO:tasks.workunit.client.0.smithi060.stdout:3/578: chown d5/de/df/d4c/f6f 240496899 1 2024-02-26T10:48:05.693 INFO:tasks.workunit.client.0.smithi060.stdout:1/859: dread d3/d35/d53/f74 [0,4194304] 0 2024-02-26T10:48:05.693 INFO:tasks.workunit.client.0.smithi060.stdout:1/860: truncate d3/d35/d53/f106 381215 0 2024-02-26T10:48:05.695 INFO:tasks.workunit.client.0.smithi060.stdout:1/861: mknod d3/d35/d53/d57/c122 0 2024-02-26T10:48:05.709 INFO:tasks.workunit.client.0.smithi060.stdout:7/946: dwrite da/d15/d21/d2f/d78/fa1 [0,4194304] 0 2024-02-26T10:48:05.709 INFO:tasks.workunit.client.0.smithi060.stdout:6/803: dwrite d6/d7/d25/d1c/d1f/d23/d2b/f3b [0,4194304] 0 2024-02-26T10:48:05.712 INFO:tasks.workunit.client.0.smithi060.stdout:7/947: creat da/d15/d21/d41/f13b x:0 0 0 2024-02-26T10:48:05.719 INFO:tasks.workunit.client.0.smithi060.stdout:4/756: rename d0/da/d3b/d1b/f9c to d0/d27/fee 0 2024-02-26T10:48:05.719 INFO:tasks.workunit.client.0.smithi060.stdout:8/784: write d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f93 [2905339,3538] 0 2024-02-26T10:48:05.723 INFO:tasks.workunit.client.0.smithi060.stdout:9/840: sync 2024-02-26T10:48:05.724 INFO:tasks.workunit.client.0.smithi060.stdout:5/753: sync 2024-02-26T10:48:05.724 INFO:tasks.workunit.client.0.smithi060.stdout:5/754: chown d0/de/da4/fbd 484 1 2024-02-26T10:48:05.724 INFO:tasks.workunit.client.0.smithi060.stdout:5/755: write d0/d4b/d7b/f102 [741952,41645] 0 2024-02-26T10:48:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:3/579: rename d5/de/df/d11/fae to d5/de/df/faf 0 2024-02-26T10:48:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:1/862: rename d3 to d3/db/d1e/d6d/da3/daa/dd6/d123 22 2024-02-26T10:48:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:9/841: creat d3/d1e/d5e/d35/d87/f123 x:0 0 0 2024-02-26T10:48:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:5/756: creat d0/de/d20/d32/d4c/f106 x:0 0 0 2024-02-26T10:48:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:3/580: getdents d5/de/df/d11/d63/d93/da2 0 2024-02-26T10:48:05.736 INFO:tasks.workunit.client.0.smithi060.stdout:1/863: mkdir d3/db/d104/d124 0 2024-02-26T10:48:05.739 INFO:tasks.workunit.client.0.smithi060.stdout:8/785: read d9/d15/d3e/fb2 [646619,77273] 0 2024-02-26T10:48:05.739 INFO:tasks.workunit.client.0.smithi060.stdout:8/786: chown d9/d15/d29/d2f/d34/d72/d50/c54 31858530 1 2024-02-26T10:48:05.739 INFO:tasks.workunit.client.0.smithi060.stdout:6/804: rename d6/d8d/dcc/f88 to d6/d7/d25/d1c/d1f/d23/d28/f107 0 2024-02-26T10:48:05.739 INFO:tasks.workunit.client.0.smithi060.stdout:6/805: dread - d6/d8d/fcf zero size 2024-02-26T10:48:05.739 INFO:tasks.workunit.client.0.smithi060.stdout:6/806: chown d6/d7/d25/d80/df9/dd8 10 1 2024-02-26T10:48:05.739 INFO:tasks.workunit.client.0.smithi060.stdout:6/807: write d6/d8d/dcc/dc3/f77 [1270472,113107] 0 2024-02-26T10:48:05.740 INFO:tasks.workunit.client.0.smithi060.stdout:3/581: unlink d5/d45/d61/d92/da1/daa/lac 0 2024-02-26T10:48:05.740 INFO:tasks.workunit.client.0.smithi060.stdout:3/582: chown d5/d45/l7b 6165 1 2024-02-26T10:48:05.740 INFO:tasks.workunit.client.0.smithi060.stdout:8/787: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/dc1/dfb/f103 x:0 0 0 2024-02-26T10:48:05.740 INFO:tasks.workunit.client.0.smithi060.stdout:5/757: mknod d0/de/d15/d65/d86/d9b/c107 0 2024-02-26T10:48:05.742 INFO:tasks.workunit.client.0.smithi060.stdout:3/583: unlink d5/l8 0 2024-02-26T10:48:05.744 INFO:tasks.workunit.client.0.smithi060.stdout:3/584: creat d5/de/df/d4c/d68/fb0 x:0 0 0 2024-02-26T10:48:05.744 INFO:tasks.workunit.client.0.smithi060.stdout:3/585: chown d5/c9f 288 1 2024-02-26T10:48:05.746 INFO:tasks.workunit.client.0.smithi060.stdout:3/586: creat d5/de/df/d11/d63/d93/d97/d98/fb1 x:0 0 0 2024-02-26T10:48:05.746 INFO:tasks.workunit.client.0.smithi060.stdout:3/587: write d5/de/df/d11/d63/d93/d97/d98/fb1 [846794,7830] 0 2024-02-26T10:48:05.760 INFO:tasks.workunit.client.0.smithi060.stdout:1/864: rename d3/db/d1e/d6d/dd9 to d3/d35/d53/d68/d90/d125 0 2024-02-26T10:48:05.761 INFO:tasks.workunit.client.0.smithi060.stdout:1/865: chown d3/db/d1e/d38/c3e 670113 1 2024-02-26T10:48:05.761 INFO:tasks.workunit.client.0.smithi060.stdout:1/866: fsync d3/db/d1e/d6d/da3/feb 0 2024-02-26T10:48:05.761 INFO:tasks.workunit.client.0.smithi060.stdout:9/842: link d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/fea d3/d1e/d57/f124 0 2024-02-26T10:48:05.762 INFO:tasks.workunit.client.0.smithi060.stdout:9/843: creat d3/d1e/d5e/d35/d87/d9b/d8a/f125 x:0 0 0 2024-02-26T10:48:05.763 INFO:tasks.workunit.client.0.smithi060.stdout:9/844: write d3/d1e/d57/f79 [971838,128738] 0 2024-02-26T10:48:05.765 INFO:tasks.workunit.client.0.smithi060.stdout:9/845: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/d98/dd4/d126 0 2024-02-26T10:48:05.767 INFO:tasks.workunit.client.0.smithi060.stdout:9/846: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f6c [928866,91037] 0 2024-02-26T10:48:05.767 INFO:tasks.workunit.client.0.smithi060.stdout:9/847: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8/dcd/fda [1632076,70953] 0 2024-02-26T10:48:05.769 INFO:tasks.workunit.client.0.smithi060.stdout:7/948: dwrite da/d15/f11c [0,4194304] 0 2024-02-26T10:48:05.772 INFO:tasks.workunit.client.0.smithi060.stdout:4/757: dwrite d0/da/fa0 [0,4194304] 0 2024-02-26T10:48:05.773 INFO:tasks.workunit.client.0.smithi060.stdout:6/808: dread d6/d8d/dcc/dc3/f77 [0,4194304] 0 2024-02-26T10:48:05.773 INFO:tasks.workunit.client.0.smithi060.stdout:6/809: fsync d6/f9 0 2024-02-26T10:48:05.774 INFO:tasks.workunit.client.0.smithi060.stdout:7/949: getdents da/d15/d21/d2f/d78/dd3/d67/da8 0 2024-02-26T10:48:05.775 INFO:tasks.workunit.client.0.smithi060.stdout:4/758: mkdir d0/d27/d76/dbd/d60/def 0 2024-02-26T10:48:05.776 INFO:tasks.workunit.client.0.smithi060.stdout:4/759: mknod d0/da/cf0 0 2024-02-26T10:48:05.778 INFO:tasks.workunit.client.0.smithi060.stdout:6/810: symlink d6/d7/d2e/d56/d73/l108 0 2024-02-26T10:48:05.779 INFO:tasks.workunit.client.0.smithi060.stdout:6/811: rmdir d6/d7 39 2024-02-26T10:48:05.779 INFO:tasks.workunit.client.0.smithi060.stdout:6/812: dread - d6/d7/d2e/fab zero size 2024-02-26T10:48:05.782 INFO:tasks.workunit.client.0.smithi060.stdout:4/760: getdents d0/da 0 2024-02-26T10:48:05.782 INFO:tasks.workunit.client.0.smithi060.stdout:4/761: stat d0/da/d3b/d16/f18 0 2024-02-26T10:48:05.784 INFO:tasks.workunit.client.0.smithi060.stdout:8/788: dwrite d9/d5a/faf [0,4194304] 0 2024-02-26T10:48:05.790 INFO:tasks.workunit.client.0.smithi060.stdout:9/848: rename d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8/dcd to d3/d1e/dbb/dc2/df8/d127 0 2024-02-26T10:48:05.790 INFO:tasks.workunit.client.0.smithi060.stdout:6/813: rename d6/d8d/fcf to d6/d7/d25/d1c/f109 0 2024-02-26T10:48:05.795 INFO:tasks.workunit.client.0.smithi060.stdout:7/950: dread da/d15/d88/dce/faf [0,4194304] 0 2024-02-26T10:48:05.811 INFO:tasks.workunit.client.0.smithi060.stdout:9/849: write d3/d1e/f82 [3561648,130987] 0 2024-02-26T10:48:05.813 INFO:tasks.workunit.client.0.smithi060.stdout:9/850: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/de8/d128 0 2024-02-26T10:48:05.817 INFO:tasks.workunit.client.0.smithi060.stdout:4/762: read d0/da/d3b/f1a [365439,38872] 0 2024-02-26T10:48:05.822 INFO:tasks.workunit.client.0.smithi060.stdout:4/763: creat d0/d27/dde/ff1 x:0 0 0 2024-02-26T10:48:05.825 INFO:tasks.workunit.client.0.smithi060.stdout:8/789: write d9/d15/d29/d2f/fa3 [881062,80183] 0 2024-02-26T10:48:05.826 INFO:tasks.workunit.client.0.smithi060.stdout:5/758: dwrite d0/de/fd8 [0,4194304] 0 2024-02-26T10:48:05.828 INFO:tasks.workunit.client.0.smithi060.stdout:8/790: rename d9/d15/d29/f59 to d9/d15/f104 0 2024-02-26T10:48:05.831 INFO:tasks.workunit.client.0.smithi060.stdout:8/791: mkdir d9/d15/d29/d2f/d34/d105 0 2024-02-26T10:48:05.831 INFO:tasks.workunit.client.0.smithi060.stdout:8/792: chown d9/d15/d29/d2f/d34 2584079 1 2024-02-26T10:48:05.831 INFO:tasks.workunit.client.0.smithi060.stdout:8/793: write d9/d15/d29/d2f/fa3 [1832780,80160] 0 2024-02-26T10:48:05.833 INFO:tasks.workunit.client.0.smithi060.stdout:8/794: link d9/d15/f36 d9/da4/deb/f106 0 2024-02-26T10:48:05.834 INFO:tasks.workunit.client.0.smithi060.stdout:0/769: sync 2024-02-26T10:48:05.834 INFO:tasks.workunit.client.0.smithi060.stdout:0/770: stat d0/dd/d39/f7e 0 2024-02-26T10:48:05.838 INFO:tasks.workunit.client.0.smithi060.stdout:7/951: dwrite da/df1/dd7/d125/d8f/f11b [0,4194304] 0 2024-02-26T10:48:05.842 INFO:tasks.workunit.client.0.smithi060.stdout:7/952: symlink da/df1/dd7/d125/dd8/l13c 0 2024-02-26T10:48:05.845 INFO:tasks.workunit.client.0.smithi060.stdout:6/814: dwrite d6/d7/f12 [0,4194304] 0 2024-02-26T10:48:05.847 INFO:tasks.workunit.client.0.smithi060.stdout:9/851: dread d3/f16 [0,4194304] 0 2024-02-26T10:48:05.847 INFO:tasks.workunit.client.0.smithi060.stdout:6/815: symlink d6/d7/d2e/d56/l10a 0 2024-02-26T10:48:05.847 INFO:tasks.workunit.client.0.smithi060.stdout:6/816: fdatasync d6/d7/d2e/feb 0 2024-02-26T10:48:05.849 INFO:tasks.workunit.client.0.smithi060.stdout:4/764: dread d0/da/d3b/f95 [0,4194304] 0 2024-02-26T10:48:05.849 INFO:tasks.workunit.client.0.smithi060.stdout:4/765: write d0/d27/d76/fd3 [549657,17576] 0 2024-02-26T10:48:05.849 INFO:tasks.workunit.client.0.smithi060.stdout:6/817: rmdir d6/d6e/d6f 39 2024-02-26T10:48:05.851 INFO:tasks.workunit.client.0.smithi060.stdout:4/766: creat d0/da/dc7/db0/db5/dc0/ff2 x:0 0 0 2024-02-26T10:48:05.858 INFO:tasks.workunit.client.0.smithi060.stdout:2/766: sync 2024-02-26T10:48:05.862 INFO:tasks.workunit.client.0.smithi060.stdout:8/795: rename d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/fee to d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/f107 0 2024-02-26T10:48:05.863 INFO:tasks.workunit.client.0.smithi060.stdout:8/796: symlink d9/d15/d29/d2f/d34/l108 0 2024-02-26T10:48:05.874 INFO:tasks.workunit.client.0.smithi060.stdout:5/759: dread d0/de/d15/d26/f78 [0,4194304] 0 2024-02-26T10:48:05.876 INFO:tasks.workunit.client.0.smithi060.stdout:5/760: creat d0/de/d20/d32/d4d/da3/da5/dc8/f108 x:0 0 0 2024-02-26T10:48:05.877 INFO:tasks.workunit.client.0.smithi060.stdout:5/761: creat d0/de/d15/f109 x:0 0 0 2024-02-26T10:48:05.877 INFO:tasks.workunit.client.0.smithi060.stdout:5/762: chown d0/de/d20/c2b 3539071 1 2024-02-26T10:48:05.882 INFO:tasks.workunit.client.0.smithi060.stdout:4/767: dread d0/da/dc7/db0/db5/dc0/f25 [0,4194304] 0 2024-02-26T10:48:05.882 INFO:tasks.workunit.client.0.smithi060.stdout:7/953: dwrite da/d15/d21/d2f/d37/feb [0,4194304] 0 2024-02-26T10:48:05.883 INFO:tasks.workunit.client.0.smithi060.stdout:6/818: rename d6/d7/d25/f49 to d6/d7/d2e/f10b 0 2024-02-26T10:48:05.883 INFO:tasks.workunit.client.0.smithi060.stdout:6/819: write d6/d7/d25/d80/df9/dd8/fe3 [274545,103194] 0 2024-02-26T10:48:05.883 INFO:tasks.workunit.client.0.smithi060.stdout:4/768: symlink d0/d27/d2f/lf3 0 2024-02-26T10:48:05.883 INFO:tasks.workunit.client.0.smithi060.stdout:4/769: write d0/da/dc7/db0/fd9 [376614,102054] 0 2024-02-26T10:48:05.886 INFO:tasks.workunit.client.0.smithi060.stdout:7/954: mkdir da/d15/d21/d2f/d78/d13d 0 2024-02-26T10:48:05.886 INFO:tasks.workunit.client.0.smithi060.stdout:6/820: mkdir d6/d8d/d10c 0 2024-02-26T10:48:05.889 INFO:tasks.workunit.client.0.smithi060.stdout:6/821: rename d6/d7/d25/d1c/d1f/d23/d2b/d4d/db9 to d6/d7/d25/d1c/d1f/d23/d2b/d67/d10d 0 2024-02-26T10:48:05.904 INFO:tasks.workunit.client.0.smithi060.stdout:4/770: unlink d0/da/d3b/d1b/c70 0 2024-02-26T10:48:05.904 INFO:tasks.workunit.client.0.smithi060.stdout:4/771: chown d0/d27/d76/dbd/fbe 664060 1 2024-02-26T10:48:05.910 INFO:tasks.workunit.client.0.smithi060.stdout:0/771: dwrite d0/dd/d53/da2/ff9 [0,4194304] 0 2024-02-26T10:48:05.910 INFO:tasks.workunit.client.0.smithi060.stdout:0/772: rmdir d0/dd/d39/d40/dad/dda 39 2024-02-26T10:48:05.931 INFO:tasks.workunit.client.0.smithi060.stdout:2/767: dwrite dd/d43/f6c [4194304,4194304] 0 2024-02-26T10:48:05.938 INFO:tasks.workunit.client.0.smithi060.stdout:8/797: dwrite d9/d15/d29/d2f/d34/d72/d7e/fd2 [0,4194304] 0 2024-02-26T10:48:05.938 INFO:tasks.workunit.client.0.smithi060.stdout:5/763: dwrite d0/de/d20/d32/d4d/da3/da5/dc8/fd6 [0,4194304] 0 2024-02-26T10:48:05.947 INFO:tasks.workunit.client.0.smithi060.stdout:7/955: dwrite da/d66/dbd/d111/f132 [0,4194304] 0 2024-02-26T10:48:05.949 INFO:tasks.workunit.client.0.smithi060.stdout:2/768: dread dd/d43/d60/d41/d37/d4c/f82 [0,4194304] 0 2024-02-26T10:48:05.950 INFO:tasks.workunit.client.0.smithi060.stdout:2/769: mkdir dd/d43/d96/df2/d101 0 2024-02-26T10:48:05.951 INFO:tasks.workunit.client.0.smithi060.stdout:2/770: read - dd/d43/d55/d5e/d86/d97/fdc zero size 2024-02-26T10:48:05.951 INFO:tasks.workunit.client.0.smithi060.stdout:2/771: stat dd/d43/d55/d5e/d86 0 2024-02-26T10:48:05.951 INFO:tasks.workunit.client.0.smithi060.stdout:7/956: getdents da/d66/dbd/d111 0 2024-02-26T10:48:05.951 INFO:tasks.workunit.client.0.smithi060.stdout:7/957: fsync da/d66/fcd 0 2024-02-26T10:48:05.951 INFO:tasks.workunit.client.0.smithi060.stdout:7/958: dread - da/d15/d21/d41/d6c/f109 zero size 2024-02-26T10:48:05.956 INFO:tasks.workunit.client.0.smithi060.stdout:2/772: mknod dd/d43/d55/c102 0 2024-02-26T10:48:05.958 INFO:tasks.workunit.client.0.smithi060.stdout:2/773: creat dd/d43/dd2/f103 x:0 0 0 2024-02-26T10:48:05.961 INFO:tasks.workunit.client.0.smithi060.stdout:4/772: dwrite d0/da/d57/f96 [0,4194304] 0 2024-02-26T10:48:05.961 INFO:tasks.workunit.client.0.smithi060.stdout:2/774: unlink dd/d43/d55/d6e/dd3/d6f/d8f/fcd 0 2024-02-26T10:48:05.962 INFO:tasks.workunit.client.0.smithi060.stdout:6/822: dwrite d6/d7/d2e/f10b [0,4194304] 0 2024-02-26T10:48:05.962 INFO:tasks.workunit.client.0.smithi060.stdout:2/775: symlink dd/d43/d55/d5e/d86/d97/db9/l104 0 2024-02-26T10:48:05.964 INFO:tasks.workunit.client.0.smithi060.stdout:8/798: link d9/d5a/d9c/lb8 d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/l109 0 2024-02-26T10:48:05.964 INFO:tasks.workunit.client.0.smithi060.stdout:8/799: truncate d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f82 1460498 0 2024-02-26T10:48:05.965 INFO:tasks.workunit.client.0.smithi060.stdout:4/773: unlink d0/da/d3b/ld1 0 2024-02-26T10:48:05.965 INFO:tasks.workunit.client.0.smithi060.stdout:4/774: chown d0/da/d3b/f34 66606 1 2024-02-26T10:48:05.965 INFO:tasks.workunit.client.0.smithi060.stdout:4/775: chown d0/d27/d76/dbd/f38 171 1 2024-02-26T10:48:05.968 INFO:tasks.workunit.client.0.smithi060.stdout:7/959: rename da/d15/d88/d3b/f9d to da/d15/d21/d41/dc7/f13e 0 2024-02-26T10:48:06.008 INFO:tasks.workunit.client.0.smithi060.stdout:8/800: dwrite d9/d15/d3e/fac [0,4194304] 0 2024-02-26T10:48:06.008 INFO:tasks.workunit.client.0.smithi060.stdout:8/801: fdatasync d9/d15/d29/d2f/d34/ff7 0 2024-02-26T10:48:06.008 INFO:tasks.workunit.client.0.smithi060.stdout:5/764: dread d0/d4b/f51 [0,4194304] 0 2024-02-26T10:48:06.009 INFO:tasks.workunit.client.0.smithi060.stdout:0/773: dwrite d0/dd/d39/d40/d83/dc6/d58/f5b [4194304,4194304] 0 2024-02-26T10:48:06.012 INFO:tasks.workunit.client.0.smithi060.stdout:8/802: mknod d9/d15/d29/c10a 0 2024-02-26T10:48:06.013 INFO:tasks.workunit.client.0.smithi060.stdout:7/960: dwrite da/d15/d88/f51 [0,4194304] 0 2024-02-26T10:48:06.014 INFO:tasks.workunit.client.0.smithi060.stdout:0/774: mknod d0/dd/d39/d40/d83/dc6/c104 0 2024-02-26T10:48:06.015 INFO:tasks.workunit.client.0.smithi060.stdout:8/803: mkdir d9/d15/d3e/d10b 0 2024-02-26T10:48:06.016 INFO:tasks.workunit.client.0.smithi060.stdout:4/776: dwrite d0/da/fcc [0,4194304] 0 2024-02-26T10:48:06.016 INFO:tasks.workunit.client.0.smithi060.stdout:0/775: link d0/dd/d39/d63/f67 d0/dd/d53/d87/d94/f105 0 2024-02-26T10:48:06.016 INFO:tasks.workunit.client.0.smithi060.stdout:4/777: stat d0/da/d3b/f95 0 2024-02-26T10:48:06.018 INFO:tasks.workunit.client.0.smithi060.stdout:8/804: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/dc1/f10c x:0 0 0 2024-02-26T10:48:06.018 INFO:tasks.workunit.client.0.smithi060.stdout:8/805: stat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/fc8 0 2024-02-26T10:48:06.019 INFO:tasks.workunit.client.0.smithi060.stdout:8/806: fdatasync d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/fdb 0 2024-02-26T10:48:06.020 INFO:tasks.workunit.client.0.smithi060.stdout:8/807: symlink d9/d15/d3e/l10d 0 2024-02-26T10:48:06.020 INFO:tasks.workunit.client.0.smithi060.stdout:8/808: fdatasync d9/d15/fff 0 2024-02-26T10:48:06.021 INFO:tasks.workunit.client.0.smithi060.stdout:3/588: sync 2024-02-26T10:48:06.021 INFO:tasks.workunit.client.0.smithi060.stdout:9/852: sync 2024-02-26T10:48:06.021 INFO:tasks.workunit.client.0.smithi060.stdout:3/589: dread - d5/de/df/faf zero size 2024-02-26T10:48:06.022 INFO:tasks.workunit.client.0.smithi060.stdout:1/867: sync 2024-02-26T10:48:06.025 INFO:tasks.workunit.client.0.smithi060.stdout:9/853: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/dd2/d129 0 2024-02-26T10:48:06.025 INFO:tasks.workunit.client.0.smithi060.stdout:8/809: link d9/d15/d29/d2f/d34/d72/d50/d6a/l4f d9/d15/d29/d2f/d34/d72/d50/d6a/dc0/l10e 0 2024-02-26T10:48:06.029 INFO:tasks.workunit.client.0.smithi060.stdout:9/854: mknod d3/d1e/d93/da2/db7/ddd/c12a 0 2024-02-26T10:48:06.034 INFO:tasks.workunit.client.0.smithi060.stdout:9/855: fsync d3/d1e/d5e/f5c 0 2024-02-26T10:48:06.034 INFO:tasks.workunit.client.0.smithi060.stdout:9/856: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/f8b [918905,123753] 0 2024-02-26T10:48:06.038 INFO:tasks.workunit.client.0.smithi060.stdout:8/810: creat d9/d15/d29/d2f/d34/d105/f10f x:0 0 0 2024-02-26T10:48:06.038 INFO:tasks.workunit.client.0.smithi060.stdout:5/765: dread d0/de/d15/f2d [0,4194304] 0 2024-02-26T10:48:06.040 INFO:tasks.workunit.client.0.smithi060.stdout:9/857: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d11e/d10c/d116/f12b x:0 0 0 2024-02-26T10:48:06.043 INFO:tasks.workunit.client.0.smithi060.stdout:5/766: creat d0/de/d70/d84/f10a x:0 0 0 2024-02-26T10:48:06.044 INFO:tasks.workunit.client.0.smithi060.stdout:5/767: chown d0/de/d20/d32/cd3 4125 1 2024-02-26T10:48:06.044 INFO:tasks.workunit.client.0.smithi060.stdout:5/768: write d0/de/d20/d32/d4d/da3/fcf [306080,128152] 0 2024-02-26T10:48:06.044 INFO:tasks.workunit.client.0.smithi060.stdout:9/858: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8/d12c 0 2024-02-26T10:48:06.044 INFO:tasks.workunit.client.0.smithi060.stdout:9/859: mknod d3/d1e/d5e/d1b/d23/c12d 0 2024-02-26T10:48:06.044 INFO:tasks.workunit.client.0.smithi060.stdout:9/860: stat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/dd2/d129 0 2024-02-26T10:48:06.045 INFO:tasks.workunit.client.0.smithi060.stdout:5/769: creat d0/de/d20/d32/f10b x:0 0 0 2024-02-26T10:48:06.045 INFO:tasks.workunit.client.0.smithi060.stdout:5/770: write d0/de/d15/d65/d86/d9b/f72 [707098,21655] 0 2024-02-26T10:48:06.045 INFO:tasks.workunit.client.0.smithi060.stdout:5/771: dread - d0/de/d20/d32/d4c/dd2/fde zero size 2024-02-26T10:48:06.046 INFO:tasks.workunit.client.0.smithi060.stdout:9/861: stat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/f110 0 2024-02-26T10:48:06.047 INFO:tasks.workunit.client.0.smithi060.stdout:6/823: dwrite d6/d7/d25/d1c/d1f/d23/faf [0,4194304] 0 2024-02-26T10:48:06.047 INFO:tasks.workunit.client.0.smithi060.stdout:9/862: creat d3/d1e/f12e x:0 0 0 2024-02-26T10:48:06.047 INFO:tasks.workunit.client.0.smithi060.stdout:5/772: unlink d0/de/d15/c68 0 2024-02-26T10:48:06.048 INFO:tasks.workunit.client.0.smithi060.stdout:6/824: fsync d6/d6e/d6f/fcd 0 2024-02-26T10:48:06.048 INFO:tasks.workunit.client.0.smithi060.stdout:1/868: link d3/db/d104/d10d/fd3 d3/d35/d53/d68/d6b/f126 0 2024-02-26T10:48:06.049 INFO:tasks.workunit.client.0.smithi060.stdout:6/825: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d10e 0 2024-02-26T10:48:06.049 INFO:tasks.workunit.client.0.smithi060.stdout:6/826: write d6/d7/d47/f72 [195048,122899] 0 2024-02-26T10:48:06.051 INFO:tasks.workunit.client.0.smithi060.stdout:5/773: link d0/de/d20/d32/d4c/fb7 d0/de/f10c 0 2024-02-26T10:48:06.054 INFO:tasks.workunit.client.0.smithi060.stdout:7/961: dread da/d15/d21/d2f/d78/f96 [0,4194304] 0 2024-02-26T10:48:06.054 INFO:tasks.workunit.client.0.smithi060.stdout:7/962: chown da/d15/d88/dce/f5f 7 1 2024-02-26T10:48:06.056 INFO:tasks.workunit.client.0.smithi060.stdout:6/827: rmdir d6 39 2024-02-26T10:48:06.056 INFO:tasks.workunit.client.0.smithi060.stdout:5/774: rename d0/d33/l77 to d0/de/d15/d65/d86/l10d 0 2024-02-26T10:48:06.061 INFO:tasks.workunit.client.0.smithi060.stdout:6/828: rmdir d6/d8d/d10c 0 2024-02-26T10:48:06.061 INFO:tasks.workunit.client.0.smithi060.stdout:6/829: chown d6/d7/d2e/d56/d73/l108 14081732 1 2024-02-26T10:48:06.062 INFO:tasks.workunit.client.0.smithi060.stdout:6/830: write d6/f106 [417606,80722] 0 2024-02-26T10:48:06.062 INFO:tasks.workunit.client.0.smithi060.stdout:6/831: stat d6/d7/fd 0 2024-02-26T10:48:06.062 INFO:tasks.workunit.client.0.smithi060.stdout:4/778: dread d0/d27/d76/f93 [0,4194304] 0 2024-02-26T10:48:06.062 INFO:tasks.workunit.client.0.smithi060.stdout:4/779: truncate d0/d27/d76/dbd/d8e/fe2 460234 0 2024-02-26T10:48:06.062 INFO:tasks.workunit.client.0.smithi060.stdout:4/780: write d0/da/dc7/db0/db5/dc0/f78 [591736,36028] 0 2024-02-26T10:48:06.062 INFO:tasks.workunit.client.0.smithi060.stdout:4/781: chown d0/da/dc7/db0/db5/fd0 3 1 2024-02-26T10:48:06.063 INFO:tasks.workunit.client.0.smithi060.stdout:1/869: mkdir d3/db/d1e/d6d/da3/d127 0 2024-02-26T10:48:06.063 INFO:tasks.workunit.client.0.smithi060.stdout:4/782: creat d0/da/d57/ff4 x:0 0 0 2024-02-26T10:48:06.064 INFO:tasks.workunit.client.0.smithi060.stdout:4/783: truncate d0/da/f14 1947378 0 2024-02-26T10:48:06.065 INFO:tasks.workunit.client.0.smithi060.stdout:0/776: read d0/f78 [2827080,102786] 0 2024-02-26T10:48:06.065 INFO:tasks.workunit.client.0.smithi060.stdout:0/777: write d0/dd/d39/d63/fcc [746246,102795] 0 2024-02-26T10:48:06.066 INFO:tasks.workunit.client.0.smithi060.stdout:1/870: symlink d3/d35/d5e/l128 0 2024-02-26T10:48:06.066 INFO:tasks.workunit.client.0.smithi060.stdout:2/776: dwrite dd/d43/d60/d41/d37/f80 [0,4194304] 0 2024-02-26T10:48:06.067 INFO:tasks.workunit.client.0.smithi060.stdout:1/871: stat d3/d35/d53/d57/f83 0 2024-02-26T10:48:06.067 INFO:tasks.workunit.client.0.smithi060.stdout:4/784: creat d0/da/dc7/ff5 x:0 0 0 2024-02-26T10:48:06.068 INFO:tasks.workunit.client.0.smithi060.stdout:4/785: stat d0/d27/f4d 0 2024-02-26T10:48:06.068 INFO:tasks.workunit.client.0.smithi060.stdout:1/872: write d3/d35/d53/f112 [503957,14538] 0 2024-02-26T10:48:06.068 INFO:tasks.workunit.client.0.smithi060.stdout:1/873: read - d3/db/d1e/d6d/d78/f10c zero size 2024-02-26T10:48:06.068 INFO:tasks.workunit.client.0.smithi060.stdout:1/874: truncate d3/f69 1156384 0 2024-02-26T10:48:06.069 INFO:tasks.workunit.client.0.smithi060.stdout:0/778: creat d0/dd/d39/d40/d83/dc6/d58/f106 x:0 0 0 2024-02-26T10:48:06.076 INFO:tasks.workunit.client.0.smithi060.stdout:9/863: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/d98/fa4 [0,4194304] 0 2024-02-26T10:48:06.076 INFO:tasks.workunit.client.0.smithi060.stdout:9/864: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96/fa0 [4417271,14873] 0 2024-02-26T10:48:06.076 INFO:tasks.workunit.client.0.smithi060.stdout:9/865: dread - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/f11d zero size 2024-02-26T10:48:06.076 INFO:tasks.workunit.client.0.smithi060.stdout:9/866: write d3/d1e/d5e/d35/d87/d9b/d8a/fe0 [6429385,82222] 0 2024-02-26T10:48:06.082 INFO:tasks.workunit.client.0.smithi060.stdout:2/777: creat dd/d43/d96/df2/f105 x:0 0 0 2024-02-26T10:48:06.083 INFO:tasks.workunit.client.0.smithi060.stdout:4/786: creat d0/da/d3b/dca/ff6 x:0 0 0 2024-02-26T10:48:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/775: write d0/de/d15/d26/f78 [4168502,99452] 0 2024-02-26T10:48:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:5/776: readlink d0/de/d15/l94 0 2024-02-26T10:48:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:9/867: link d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/dd2/cd8 d3/d1e/d93/da2/db7/ddd/d10e/c12f 0 2024-02-26T10:48:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:9/868: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/f8b 293204 1 2024-02-26T10:48:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:5/777: symlink d0/de/d15/d65/d86/dcc/l10e 0 2024-02-26T10:48:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:5/778: chown d0/de/d15/f3b 131958318 1 2024-02-26T10:48:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:5/779: symlink d0/de/d15/d26/l10f 0 2024-02-26T10:48:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:7/963: write da/d15/d21/d2f/d78/fda [1492183,84829] 0 2024-02-26T10:48:06.090 INFO:tasks.workunit.client.0.smithi060.stdout:7/964: stat da/d94/faa 0 2024-02-26T10:48:06.097 INFO:tasks.workunit.client.0.smithi060.stdout:2/778: dread dd/d43/d60/d41/d37/d4c/f82 [0,4194304] 0 2024-02-26T10:48:06.097 INFO:tasks.workunit.client.0.smithi060.stdout:6/832: dread d6/f71 [0,4194304] 0 2024-02-26T10:48:06.097 INFO:tasks.workunit.client.0.smithi060.stdout:6/833: write d6/d8d/dcc/dc3/fa8 [572896,114841] 0 2024-02-26T10:48:06.098 INFO:tasks.workunit.client.0.smithi060.stdout:3/590: dwrite d5/de/df/f33 [0,4194304] 0 2024-02-26T10:48:06.103 INFO:tasks.workunit.client.0.smithi060.stdout:1/875: rename d3/db/d1e/d38/c45 to d3/db/d104/d10d/c129 0 2024-02-26T10:48:06.103 INFO:tasks.workunit.client.0.smithi060.stdout:3/591: link d5/de/df/d4c/f4f d5/de/fb2 0 2024-02-26T10:48:06.103 INFO:tasks.workunit.client.0.smithi060.stdout:6/834: mkdir d6/d7/d2e/d56/dfd/d10f 0 2024-02-26T10:48:06.107 INFO:tasks.workunit.client.0.smithi060.stdout:6/835: mknod d6/d7/d25/d1c/d1f/d23/d2b/d10e/c110 0 2024-02-26T10:48:06.111 INFO:tasks.workunit.client.0.smithi060.stdout:0/779: rename d0/d42/fd0 to d0/dd/d39/d40/dad/f107 0 2024-02-26T10:48:06.115 INFO:tasks.workunit.client.0.smithi060.stdout:6/836: creat d6/d7/d25/f111 x:0 0 0 2024-02-26T10:48:06.143 INFO:tasks.workunit.client.0.smithi060.stdout:7/965: dwrite da/d15/d21/d2f/d78/f10a [0,4194304] 0 2024-02-26T10:48:06.143 INFO:tasks.workunit.client.0.smithi060.stdout:7/966: chown da/df1/dd7/de0/f104 55089776 1 2024-02-26T10:48:06.144 INFO:tasks.workunit.client.0.smithi060.stdout:3/592: dread d5/de/f58 [0,4194304] 0 2024-02-26T10:48:06.149 INFO:tasks.workunit.client.0.smithi060.stdout:9/869: rename d3/d1e/d93/da2/db7/ddd/d10e to d3/d1e/d5e/d35/deb/d107/d130 0 2024-02-26T10:48:06.152 INFO:tasks.workunit.client.0.smithi060.stdout:9/870: creat d3/d1e/d5e/d1b/f131 x:0 0 0 2024-02-26T10:48:06.159 INFO:tasks.workunit.client.0.smithi060.stdout:0/780: dread d0/dd/d39/d40/d83/f8f [0,4194304] 0 2024-02-26T10:48:06.159 INFO:tasks.workunit.client.0.smithi060.stdout:0/781: write d0/f3 [1644645,44903] 0 2024-02-26T10:48:06.160 INFO:tasks.workunit.client.0.smithi060.stdout:0/782: write d0/dd/d39/f54 [5970046,70343] 0 2024-02-26T10:48:06.160 INFO:tasks.workunit.client.0.smithi060.stdout:1/876: dread d3/d35/d53/d68/d85/f94 [0,4194304] 0 2024-02-26T10:48:06.160 INFO:tasks.workunit.client.0.smithi060.stdout:4/787: write d0/d27/f83 [5752764,43481] 0 2024-02-26T10:48:06.168 INFO:tasks.workunit.client.0.smithi060.stdout:6/837: dwrite d6/d7/d25/d1c/d1f/d23/d2b/d4d/f58 [0,4194304] 0 2024-02-26T10:48:06.168 INFO:tasks.workunit.client.0.smithi060.stdout:6/838: write d6/d7/d25/d1c/d1f/fb7 [502405,69918] 0 2024-02-26T10:48:06.173 INFO:tasks.workunit.client.0.smithi060.stdout:2/779: dwrite dd/d43/d55/d5e/fad [0,4194304] 0 2024-02-26T10:48:06.174 INFO:tasks.workunit.client.0.smithi060.stdout:2/780: stat dd/d43/d60/fbd 0 2024-02-26T10:48:06.176 INFO:tasks.workunit.client.0.smithi060.stdout:2/781: creat dd/d43/d55/d5e/d74/f106 x:0 0 0 2024-02-26T10:48:06.184 INFO:tasks.workunit.client.0.smithi060.stdout:5/780: dwrite d0/de/d20/d32/ff9 [0,4194304] 0 2024-02-26T10:48:06.195 INFO:tasks.workunit.client.0.smithi060.stdout:3/593: write d5/de/df/d57/d60/f30 [4101655,50138] 0 2024-02-26T10:48:06.210 INFO:tasks.workunit.client.0.smithi060.stdout:6/839: write d6/d7/d25/f4b [3252899,82461] 0 2024-02-26T10:48:06.212 INFO:tasks.workunit.client.0.smithi060.stdout:3/594: dread d5/de/df/d57/d7a/f89 [0,4194304] 0 2024-02-26T10:48:06.212 INFO:tasks.workunit.client.0.smithi060.stdout:6/840: mknod d6/d8d/c112 0 2024-02-26T10:48:06.212 INFO:tasks.workunit.client.0.smithi060.stdout:6/841: readlink d6/lc7 0 2024-02-26T10:48:06.212 INFO:tasks.workunit.client.0.smithi060.stdout:3/595: chown d5/de/df/d11/l5f 25400 1 2024-02-26T10:48:06.218 INFO:tasks.workunit.client.0.smithi060.stdout:6/842: mknod d6/d7/d25/d1c/d1f/d23/d2b/d4d/c113 0 2024-02-26T10:48:06.218 INFO:tasks.workunit.client.0.smithi060.stdout:6/843: write d6/d7/d47/d69/fb1 [544531,61122] 0 2024-02-26T10:48:06.219 INFO:tasks.workunit.client.0.smithi060.stdout:6/844: creat d6/d7/d47/dde/f114 x:0 0 0 2024-02-26T10:48:06.221 INFO:tasks.workunit.client.0.smithi060.stdout:9/871: dwrite d3/d1e/f58 [0,4194304] 0 2024-02-26T10:48:06.224 INFO:tasks.workunit.client.0.smithi060.stdout:9/872: creat d3/d68/f132 x:0 0 0 2024-02-26T10:48:06.225 INFO:tasks.workunit.client.0.smithi060.stdout:9/873: chown d3/d1e/l64 31351 1 2024-02-26T10:48:06.246 INFO:tasks.workunit.client.0.smithi060.stdout:9/874: rename d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/le6 to d3/d1e/d93/l133 0 2024-02-26T10:48:06.251 INFO:tasks.workunit.client.0.smithi060.stdout:6/845: dread d6/d7/d2e/d56/d73/fa4 [0,4194304] 0 2024-02-26T10:48:06.251 INFO:tasks.workunit.client.0.smithi060.stdout:6/846: chown d6/d7/d25/d80/fae 872123299 1 2024-02-26T10:48:06.252 INFO:tasks.workunit.client.0.smithi060.stdout:6/847: rmdir d6/d7/d25/d1c/d6b 39 2024-02-26T10:48:06.253 INFO:tasks.workunit.client.0.smithi060.stdout:7/967: dwrite da/d94/faa [0,4194304] 0 2024-02-26T10:48:06.253 INFO:tasks.workunit.client.0.smithi060.stdout:1/877: dwrite d3/f9 [0,4194304] 0 2024-02-26T10:48:06.253 INFO:tasks.workunit.client.0.smithi060.stdout:7/968: read - da/d15/d21/d2f/d37/f8e zero size 2024-02-26T10:48:06.254 INFO:tasks.workunit.client.0.smithi060.stdout:1/878: truncate d3/d35/d53/f112 662826 0 2024-02-26T10:48:06.254 INFO:tasks.workunit.client.0.smithi060.stdout:1/879: chown d3/db/dc8/f9a 12673672 1 2024-02-26T10:48:06.258 INFO:tasks.workunit.client.0.smithi060.stdout:7/969: creat da/d15/d21/d29/f13f x:0 0 0 2024-02-26T10:48:06.259 INFO:tasks.workunit.client.0.smithi060.stdout:1/880: creat d3/db/d1e/d6d/da3/d127/f12a x:0 0 0 2024-02-26T10:48:06.259 INFO:tasks.workunit.client.0.smithi060.stdout:7/970: mkdir da/d15/d21/d2f/d37/d140 0 2024-02-26T10:48:06.259 INFO:tasks.workunit.client.0.smithi060.stdout:7/971: chown da/df1/dd7/d125/lc5 29044 1 2024-02-26T10:48:06.261 INFO:tasks.workunit.client.0.smithi060.stdout:7/972: creat da/d66/dbd/d111/f141 x:0 0 0 2024-02-26T10:48:06.266 INFO:tasks.workunit.client.0.smithi060.stdout:5/781: dwrite d0/d33/f3e [0,4194304] 0 2024-02-26T10:48:06.266 INFO:tasks.workunit.client.0.smithi060.stdout:7/973: dread - da/d15/d21/d2f/d78/dd3/f95 zero size 2024-02-26T10:48:06.268 INFO:tasks.workunit.client.0.smithi060.stdout:6/848: dread d6/d7/d25/d1c/d1f/d23/d28/fd5 [0,4194304] 0 2024-02-26T10:48:06.268 INFO:tasks.workunit.client.0.smithi060.stdout:6/849: stat d6/d7/d2e/d56/d73/l108 0 2024-02-26T10:48:06.273 INFO:tasks.workunit.client.0.smithi060.stdout:6/850: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f115 x:0 0 0 2024-02-26T10:48:06.274 INFO:tasks.workunit.client.0.smithi060.stdout:2/782: dwrite dd/d11/fc5 [0,4194304] 0 2024-02-26T10:48:06.276 INFO:tasks.workunit.client.0.smithi060.stdout:6/851: unlink f4 0 2024-02-26T10:48:06.276 INFO:tasks.workunit.client.0.smithi060.stdout:6/852: write d6/d7/d47/dde/ff3 [4741618,121681] 0 2024-02-26T10:48:06.278 INFO:tasks.workunit.client.0.smithi060.stdout:6/853: creat d6/d7/d47/d69/f116 x:0 0 0 2024-02-26T10:48:06.279 INFO:tasks.workunit.client.0.smithi060.stdout:6/854: creat d6/d7/d2e/d56/d73/d8c/f117 x:0 0 0 2024-02-26T10:48:06.279 INFO:tasks.workunit.client.0.smithi060.stdout:6/855: dread - d6/d7/d25/f111 zero size 2024-02-26T10:48:06.281 INFO:tasks.workunit.client.0.smithi060.stdout:6/856: link d6/d7/d25/d80/ce5 d6/d7/d25/d1c/d6b/c118 0 2024-02-26T10:48:06.281 INFO:tasks.workunit.client.0.smithi060.stdout:6/857: dread - d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac/db5/fce zero size 2024-02-26T10:48:06.283 INFO:tasks.workunit.client.0.smithi060.stdout:3/596: dwrite d5/de/df/d57/d60/f30 [0,4194304] 0 2024-02-26T10:48:06.285 INFO:tasks.workunit.client.0.smithi060.stdout:1/881: link d3/d35/d53/d68/d6b/f107 d3/db/d1e/d39/f12b 0 2024-02-26T10:48:06.285 INFO:tasks.workunit.client.0.smithi060.stdout:1/882: chown d3/db/d1e/d38/cda 483400 1 2024-02-26T10:48:06.295 INFO:tasks.workunit.client.0.smithi060.stdout:4/788: sync 2024-02-26T10:48:06.295 INFO:tasks.workunit.client.0.smithi060.stdout:7/974: write da/d15/d21/d2f/d78/dd3/d67/f10e [3872163,101954] 0 2024-02-26T10:48:06.295 INFO:tasks.workunit.client.0.smithi060.stdout:8/811: sync 2024-02-26T10:48:06.297 INFO:tasks.workunit.client.0.smithi060.stdout:3/597: truncate d5/de/df/d57/d7a/f89 133333 0 2024-02-26T10:48:06.297 INFO:tasks.workunit.client.0.smithi060.stdout:0/783: sync 2024-02-26T10:48:06.298 INFO:tasks.workunit.client.0.smithi060.stdout:4/789: truncate d0/d27/d2f/f31 5926778 0 2024-02-26T10:48:06.298 INFO:tasks.workunit.client.0.smithi060.stdout:7/975: link da/d94/faa da/df1/dd7/d125/f142 0 2024-02-26T10:48:06.299 INFO:tasks.workunit.client.0.smithi060.stdout:8/812: creat d9/d15/d3e/d10b/f110 x:0 0 0 2024-02-26T10:48:06.300 INFO:tasks.workunit.client.0.smithi060.stdout:3/598: creat d5/d45/d61/d92/da1/daa/fb3 x:0 0 0 2024-02-26T10:48:06.300 INFO:tasks.workunit.client.0.smithi060.stdout:3/599: chown d5/de/df/d4c/c5c 64391 1 2024-02-26T10:48:06.300 INFO:tasks.workunit.client.0.smithi060.stdout:8/813: fdatasync d9/fc 0 2024-02-26T10:48:06.302 INFO:tasks.workunit.client.0.smithi060.stdout:4/790: creat d0/da/dc7/db0/db5/dc0/ff7 x:0 0 0 2024-02-26T10:48:06.302 INFO:tasks.workunit.client.0.smithi060.stdout:9/875: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f80 [0,4194304] 0 2024-02-26T10:48:06.303 INFO:tasks.workunit.client.0.smithi060.stdout:7/976: mkdir da/d15/d21/d41/d6c/d130/d143 0 2024-02-26T10:48:06.304 INFO:tasks.workunit.client.0.smithi060.stdout:4/791: getdents d0/da/d3b/dca/de1 0 2024-02-26T10:48:06.304 INFO:tasks.workunit.client.0.smithi060.stdout:4/792: chown d0/d27/d76/dbd/d8e/fa1 21065109 1 2024-02-26T10:48:06.304 INFO:tasks.workunit.client.0.smithi060.stdout:4/793: dread - d0/da/d3b/dca/de1/fe6 zero size 2024-02-26T10:48:06.304 INFO:tasks.workunit.client.0.smithi060.stdout:3/600: mknod d5/d45/d61/d92/cb4 0 2024-02-26T10:48:06.305 INFO:tasks.workunit.client.0.smithi060.stdout:6/858: dread d6/d7/d25/d1c/d1f/fb7 [0,4194304] 0 2024-02-26T10:48:06.306 INFO:tasks.workunit.client.0.smithi060.stdout:9/876: symlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/l134 0 2024-02-26T10:48:06.306 INFO:tasks.workunit.client.0.smithi060.stdout:4/794: symlink d0/da/d3b/d1b/lf8 0 2024-02-26T10:48:06.307 INFO:tasks.workunit.client.0.smithi060.stdout:9/877: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/de9/d135 0 2024-02-26T10:48:06.310 INFO:tasks.workunit.client.0.smithi060.stdout:7/977: creat da/df1/dd7/d125/f144 x:0 0 0 2024-02-26T10:48:06.310 INFO:tasks.workunit.client.0.smithi060.stdout:4/795: rmdir d0/d27/d76/dbd/d60/def 0 2024-02-26T10:48:06.310 INFO:tasks.workunit.client.0.smithi060.stdout:4/796: fsync d0/da/dc7/db0/db5/dc0/f37 0 2024-02-26T10:48:06.330 INFO:tasks.workunit.client.0.smithi060.stdout:7/978: dread da/fbc [0,4194304] 0 2024-02-26T10:48:06.330 INFO:tasks.workunit.client.0.smithi060.stdout:7/979: dread - da/d15/d21/d29/f139 zero size 2024-02-26T10:48:06.330 INFO:tasks.workunit.client.0.smithi060.stdout:7/980: write da/df1/dd7/de0/f12e [799379,85757] 0 2024-02-26T10:48:06.331 INFO:tasks.workunit.client.0.smithi060.stdout:7/981: write da/d15/d21/d2f/d78/dd3/f95 [952434,7675] 0 2024-02-26T10:48:06.331 INFO:tasks.workunit.client.0.smithi060.stdout:2/783: dwrite dd/d11/f4e [0,4194304] 0 2024-02-26T10:48:06.332 INFO:tasks.workunit.client.0.smithi060.stdout:8/814: rename d9/d5a/d58/c80 to d9/d15/d29/d2f/d34/c111 0 2024-02-26T10:48:06.332 INFO:tasks.workunit.client.0.smithi060.stdout:2/784: fdatasync dd/d43/d60/d41/d37/d9a/db4/fcf 0 2024-02-26T10:48:06.332 INFO:tasks.workunit.client.0.smithi060.stdout:2/785: chown dd/d11/c16 3150151 1 2024-02-26T10:48:06.332 INFO:tasks.workunit.client.0.smithi060.stdout:2/786: fsync dd/d43/d55/d6e/fd5 0 2024-02-26T10:48:06.332 INFO:tasks.workunit.client.0.smithi060.stdout:8/815: symlink d9/d15/d29/d2f/d34/d72/d7e/l112 0 2024-02-26T10:48:06.333 INFO:tasks.workunit.client.0.smithi060.stdout:2/787: creat dd/d43/d55/f107 x:0 0 0 2024-02-26T10:48:06.345 INFO:tasks.workunit.client.0.smithi060.stdout:6/859: dread d6/d7/d25/d1c/d1f/d23/d28/f34 [0,4194304] 0 2024-02-26T10:48:06.351 INFO:tasks.workunit.client.0.smithi060.stdout:6/860: write d6/d7/d25/f46 [408773,15724] 0 2024-02-26T10:48:06.351 INFO:tasks.workunit.client.0.smithi060.stdout:6/861: dread - d6/d7/d2e/d56/d73/fdb zero size 2024-02-26T10:48:06.351 INFO:tasks.workunit.client.0.smithi060.stdout:6/862: truncate d6/d7/d25/d1c/ff6 208358 0 2024-02-26T10:48:06.352 INFO:tasks.workunit.client.0.smithi060.stdout:6/863: mkdir d6/d8d/dcc/dc3/d98/d119 0 2024-02-26T10:48:06.352 INFO:tasks.workunit.client.0.smithi060.stdout:6/864: chown d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/ff1 6 1 2024-02-26T10:48:06.352 INFO:tasks.workunit.client.0.smithi060.stdout:6/865: chown d6/d7/d25/d1c/d1f/c83 0 1 2024-02-26T10:48:06.353 INFO:tasks.workunit.client.0.smithi060.stdout:9/878: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f65 [0,4194304] 0 2024-02-26T10:48:06.359 INFO:tasks.workunit.client.0.smithi060.stdout:8/816: dread d9/d15/d3e/fac [0,4194304] 0 2024-02-26T10:48:06.360 INFO:tasks.workunit.client.0.smithi060.stdout:6/866: mknod d6/d7/d2e/c11a 0 2024-02-26T10:48:06.360 INFO:tasks.workunit.client.0.smithi060.stdout:6/867: chown d6/d7/d2e/d56/d73/lcb 328895849 1 2024-02-26T10:48:06.360 INFO:tasks.workunit.client.0.smithi060.stdout:9/879: unlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/d98/fe3 0 2024-02-26T10:48:06.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:06 smithi060 ceph-mon[31539]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 176 MiB/s rd, 202 MiB/s wr, 833 op/s 2024-02-26T10:48:06.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:06 smithi060 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:48:06.369 INFO:tasks.workunit.client.0.smithi060.stdout:2/788: dwrite dd/d43/d55/d5e/d86/d97/fce [0,4194304] 0 2024-02-26T10:48:06.370 INFO:tasks.workunit.client.0.smithi060.stdout:2/789: fsync dd/d43/d55/d6e/dd3/d6f/fa9 0 2024-02-26T10:48:06.374 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:06 smithi100 ceph-mon[36051]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 176 MiB/s rd, 202 MiB/s wr, 833 op/s 2024-02-26T10:48:06.374 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:06 smithi100 ceph-mon[36051]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:48:06.383 INFO:tasks.workunit.client.0.smithi060.stdout:7/982: write da/d15/d21/d29/f4a [199039,64593] 0 2024-02-26T10:48:06.393 INFO:tasks.workunit.client.0.smithi060.stdout:9/880: write d3/d68/fae [2332016,81392] 0 2024-02-26T10:48:06.399 INFO:tasks.workunit.client.0.smithi060.stdout:1/883: dwrite d3/f23 [0,4194304] 0 2024-02-26T10:48:06.399 INFO:tasks.workunit.client.0.smithi060.stdout:3/601: dread d5/de/df/d4c/d68/f55 [0,4194304] 0 2024-02-26T10:48:06.399 INFO:tasks.workunit.client.0.smithi060.stdout:1/884: chown d3/d35/l44 9257659 1 2024-02-26T10:48:06.399 INFO:tasks.workunit.client.0.smithi060.stdout:1/885: chown d3/db/d1e/d6d/dd1/fe7 100497173 1 2024-02-26T10:48:06.402 INFO:tasks.workunit.client.0.smithi060.stdout:3/602: fdatasync d5/de/df/d11/d63/f7c 0 2024-02-26T10:48:06.409 INFO:tasks.workunit.client.0.smithi060.stdout:1/886: write d3/db/d2e/d51/d63/f92 [502659,82684] 0 2024-02-26T10:48:06.411 INFO:tasks.workunit.client.0.smithi060.stdout:9/881: read d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f21 [2842197,52447] 0 2024-02-26T10:48:06.411 INFO:tasks.workunit.client.0.smithi060.stdout:9/882: stat d3/d1e/d5e/d1b/d23/f36 0 2024-02-26T10:48:06.414 INFO:tasks.workunit.client.0.smithi060.stdout:9/883: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fe1 [43141,116125] 0 2024-02-26T10:48:06.417 INFO:tasks.workunit.client.0.smithi060.stdout:6/868: dwrite d6/d8d/dcc/fec [0,4194304] 0 2024-02-26T10:48:06.417 INFO:tasks.workunit.client.0.smithi060.stdout:7/983: dread da/d15/d21/d2f/d78/fac [0,4194304] 0 2024-02-26T10:48:06.417 INFO:tasks.workunit.client.0.smithi060.stdout:7/984: readlink da/l3d 0 2024-02-26T10:48:06.420 INFO:tasks.workunit.client.0.smithi060.stdout:6/869: mknod d6/d7/d2e/d56/d73/c11b 0 2024-02-26T10:48:06.420 INFO:tasks.workunit.client.0.smithi060.stdout:6/870: write d6/d7/d47/d69/f116 [669588,58667] 0 2024-02-26T10:48:06.420 INFO:tasks.workunit.client.0.smithi060.stdout:7/985: getdents da/df1/dd7/d125/dd8 0 2024-02-26T10:48:06.420 INFO:tasks.workunit.client.0.smithi060.stdout:9/884: truncate d3/d1e/d5e/d35/d87/d9b/d8a/dbe/fe5 1005144 0 2024-02-26T10:48:06.420 INFO:tasks.workunit.client.0.smithi060.stdout:9/885: stat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/c13 0 2024-02-26T10:48:06.420 INFO:tasks.workunit.client.0.smithi060.stdout:7/986: creat da/d15/d88/d3b/f145 x:0 0 0 2024-02-26T10:48:06.421 INFO:tasks.workunit.client.0.smithi060.stdout:7/987: symlink da/d15/d88/d3b/l146 0 2024-02-26T10:48:06.427 INFO:tasks.workunit.client.0.smithi060.stdout:8/817: dwrite d9/d5a/d9c/fc7 [0,4194304] 0 2024-02-26T10:48:06.428 INFO:tasks.workunit.client.0.smithi060.stdout:4/797: dwrite d0/fb [0,4194304] 0 2024-02-26T10:48:06.428 INFO:tasks.workunit.client.0.smithi060.stdout:4/798: stat d0/da/f53 0 2024-02-26T10:48:06.428 INFO:tasks.workunit.client.0.smithi060.stdout:4/799: readlink d0/d27/d76/ld4 0 2024-02-26T10:48:06.429 INFO:tasks.workunit.client.0.smithi060.stdout:8/818: creat d9/d15/f113 x:0 0 0 2024-02-26T10:48:06.430 INFO:tasks.workunit.client.0.smithi060.stdout:8/819: mkdir d9/d5a/d9c/df0/d114 0 2024-02-26T10:48:06.460 INFO:tasks.workunit.client.0.smithi060.stdout:7/988: write da/d15/d21/d2f/f91 [96896,7897] 0 2024-02-26T10:48:06.463 INFO:tasks.workunit.client.0.smithi060.stdout:4/800: dread d0/da/d3b/d1b/fc1 [0,4194304] 0 2024-02-26T10:48:06.471 INFO:tasks.workunit.client.0.smithi060.stdout:5/782: sync 2024-02-26T10:48:06.471 INFO:tasks.workunit.client.0.smithi060.stdout:5/783: chown d0/de/d20/d32/f10b 252906718 1 2024-02-26T10:48:06.471 INFO:tasks.workunit.client.0.smithi060.stdout:5/784: mkdir d0/de/d20/d32/d4d/d110 0 2024-02-26T10:48:06.474 INFO:tasks.workunit.client.0.smithi060.stdout:5/785: getdents d0/de/d20/d32 0 2024-02-26T10:48:06.474 INFO:tasks.workunit.client.0.smithi060.stdout:5/786: truncate d0/de/d20/d32/d4c/fe6 640035 0 2024-02-26T10:48:06.475 INFO:tasks.workunit.client.0.smithi060.stdout:5/787: write d0/de/d15/d65/d86/d9b/fad [127441,98964] 0 2024-02-26T10:48:06.477 INFO:tasks.workunit.client.0.smithi060.stdout:5/788: rename d0/d33/cee to d0/de/d20/d32/d4c/dd2/c111 0 2024-02-26T10:48:06.477 INFO:tasks.workunit.client.0.smithi060.stdout:9/886: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/fb0 [0,4194304] 0 2024-02-26T10:48:06.477 INFO:tasks.workunit.client.0.smithi060.stdout:8/820: dread d9/d15/d29/d2f/f75 [0,4194304] 0 2024-02-26T10:48:06.477 INFO:tasks.workunit.client.0.smithi060.stdout:8/821: chown d9/d15/d29/d2f/d34/l8d 2024 1 2024-02-26T10:48:06.478 INFO:tasks.workunit.client.0.smithi060.stdout:9/887: mknod d3/d1e/d5e/d35/deb/d7a/c136 0 2024-02-26T10:48:06.480 INFO:tasks.workunit.client.0.smithi060.stdout:8/822: creat d9/d15/d29/d95/f115 x:0 0 0 2024-02-26T10:48:06.480 INFO:tasks.workunit.client.0.smithi060.stdout:8/823: readlink d9/d15/d29/d2f/d34/d72/l22 0 2024-02-26T10:48:06.480 INFO:tasks.workunit.client.0.smithi060.stdout:8/824: fdatasync d9/d15/d29/d2f/d34/d72/d50/d6a/f8e 0 2024-02-26T10:48:06.481 INFO:tasks.workunit.client.0.smithi060.stdout:5/789: unlink d0/de/d20/d32/d4d/d97/ccd 0 2024-02-26T10:48:06.481 INFO:tasks.workunit.client.0.smithi060.stdout:5/790: stat d0/de/d20/d32/d4d/d97 0 2024-02-26T10:48:06.481 INFO:tasks.workunit.client.0.smithi060.stdout:5/791: chown d0/de/d20/d32/d4c/dd2/lfb 1025831366 1 2024-02-26T10:48:06.482 INFO:tasks.workunit.client.0.smithi060.stdout:8/825: creat d9/d15/d29/d2f/d34/d72/d94/f116 x:0 0 0 2024-02-26T10:48:06.482 INFO:tasks.workunit.client.0.smithi060.stdout:9/888: symlink d3/d1e/d5e/d9e/l137 0 2024-02-26T10:48:06.488 INFO:tasks.workunit.client.0.smithi060.stdout:9/889: mknod d3/d1e/d5e/d9e/c138 0 2024-02-26T10:48:06.488 INFO:tasks.workunit.client.0.smithi060.stdout:5/792: link d0/de/d70/f105 d0/de/d70/f112 0 2024-02-26T10:48:06.488 INFO:tasks.workunit.client.0.smithi060.stdout:3/603: dwrite d5/de/df/d4c/f6b [0,4194304] 0 2024-02-26T10:48:06.488 INFO:tasks.workunit.client.0.smithi060.stdout:5/793: mkdir d0/de/d15/d65/d86/d9b/d113 0 2024-02-26T10:48:06.488 INFO:tasks.workunit.client.0.smithi060.stdout:5/794: stat d0/de/d15/cb6 0 2024-02-26T10:48:06.488 INFO:tasks.workunit.client.0.smithi060.stdout:5/795: chown d0/de/d15/d65/fe2 9755 1 2024-02-26T10:48:06.490 INFO:tasks.workunit.client.0.smithi060.stdout:3/604: mknod d5/d45/d61/d92/da1/daa/dad/cb5 0 2024-02-26T10:48:06.493 INFO:tasks.workunit.client.0.smithi060.stdout:3/605: fdatasync d5/de/df/d11/f34 0 2024-02-26T10:48:06.494 INFO:tasks.workunit.client.0.smithi060.stdout:3/606: rmdir d5/d45/d61/d92 39 2024-02-26T10:48:06.495 INFO:tasks.workunit.client.0.smithi060.stdout:4/801: dread d0/da/d3b/f34 [0,4194304] 0 2024-02-26T10:48:06.495 INFO:tasks.workunit.client.0.smithi060.stdout:7/989: dread da/d15/d88/d3b/f8c [0,4194304] 0 2024-02-26T10:48:06.495 INFO:tasks.workunit.client.0.smithi060.stdout:4/802: dread - d0/da/d3b/db9/fcb zero size 2024-02-26T10:48:06.497 INFO:tasks.workunit.client.0.smithi060.stdout:3/607: creat d5/de/df/d57/d7a/fb6 x:0 0 0 2024-02-26T10:48:06.497 INFO:tasks.workunit.client.0.smithi060.stdout:3/608: write d5/f4a [8698417,108268] 0 2024-02-26T10:48:06.497 INFO:tasks.workunit.client.0.smithi060.stdout:4/803: mkdir d0/da/d3b/d1b/df9 0 2024-02-26T10:48:06.500 INFO:tasks.workunit.client.0.smithi060.stdout:3/609: rename d5/de/df/d4c/l99 to d5/de/df/d11/lb7 0 2024-02-26T10:48:06.500 INFO:tasks.workunit.client.0.smithi060.stdout:3/610: write d5/f27 [4321039,46482] 0 2024-02-26T10:48:06.509 INFO:tasks.workunit.client.0.smithi060.stdout:1/887: dwrite d3/f23 [0,4194304] 0 2024-02-26T10:48:06.520 INFO:tasks.workunit.client.0.smithi060.stdout:1/888: write d3/db/d1e/d38/d6c/f88 [3341354,33025] 0 2024-02-26T10:48:06.524 INFO:tasks.workunit.client.0.smithi060.stdout:1/889: symlink d3/d35/d76/l12c 0 2024-02-26T10:48:06.524 INFO:tasks.workunit.client.0.smithi060.stdout:1/890: chown d3/db/d1e/d39/d9b/l11c 1 1 2024-02-26T10:48:06.524 INFO:tasks.workunit.client.0.smithi060.stdout:4/804: rename d0/d27/d2f to d0/da/d3b/ddc/dfa 0 2024-02-26T10:48:06.527 INFO:tasks.workunit.client.0.smithi060.stdout:4/805: creat d0/da/dd6/ffb x:0 0 0 2024-02-26T10:48:06.528 INFO:tasks.workunit.client.0.smithi060.stdout:1/891: creat d3/d35/d53/f12d x:0 0 0 2024-02-26T10:48:06.530 INFO:tasks.workunit.client.0.smithi060.stdout:4/806: rename d0/da/f9a to d0/d27/ffc 0 2024-02-26T10:48:06.530 INFO:tasks.workunit.client.0.smithi060.stdout:8/826: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f93 [0,4194304] 0 2024-02-26T10:48:06.547 INFO:tasks.workunit.client.0.smithi060.stdout:7/990: dwrite da/d15/d21/fb2 [0,4194304] 0 2024-02-26T10:48:06.548 INFO:tasks.workunit.client.0.smithi060.stdout:4/807: rename d0/da/d57/f75 to d0/d27/d76/dbd/d60/ffd 0 2024-02-26T10:48:06.550 INFO:tasks.workunit.client.0.smithi060.stdout:7/991: creat da/df1/dd7/d125/f147 x:0 0 0 2024-02-26T10:48:06.569 INFO:tasks.workunit.client.0.smithi060.stdout:1/892: dwrite d3/d54/de9/f108 [0,4194304] 0 2024-02-26T10:48:06.569 INFO:tasks.workunit.client.0.smithi060.stdout:1/893: fsync d3/d54/de9/f108 0 2024-02-26T10:48:06.572 INFO:tasks.workunit.client.0.smithi060.stdout:5/796: dwrite d0/de/d15/d26/f78 [4194304,4194304] 0 2024-02-26T10:48:06.574 INFO:tasks.workunit.client.0.smithi060.stdout:5/797: truncate d0/de/d15/d65/f7c 8071366 0 2024-02-26T10:48:06.574 INFO:tasks.workunit.client.0.smithi060.stdout:1/894: creat d3/f12e x:0 0 0 2024-02-26T10:48:06.574 INFO:tasks.workunit.client.0.smithi060.stdout:1/895: chown d3/db/d1e/d38/cfc 1 1 2024-02-26T10:48:06.581 INFO:tasks.workunit.client.0.smithi060.stdout:8/827: dwrite d9/d15/d29/d2f/d34/d72/f42 [0,4194304] 0 2024-02-26T10:48:06.581 INFO:tasks.workunit.client.0.smithi060.stdout:8/828: fsync d9/f91 0 2024-02-26T10:48:06.581 INFO:tasks.workunit.client.0.smithi060.stdout:8/829: write d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/f102 [1018762,5862] 0 2024-02-26T10:48:06.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:06 smithi113 ceph-mon[36126]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 176 MiB/s rd, 202 MiB/s wr, 833 op/s 2024-02-26T10:48:06.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:06 smithi113 ceph-mon[36126]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T10:48:06.594 INFO:tasks.workunit.client.0.smithi060.stdout:5/798: rename d0/de/d20/d32/d4c/dd2 to d0/de/d15/d65/d86/d9b/d113/d114 0 2024-02-26T10:48:06.594 INFO:tasks.workunit.client.0.smithi060.stdout:5/799: dread - d0/de/d20/d32/d4d/da3/da5/f104 zero size 2024-02-26T10:48:06.594 INFO:tasks.workunit.client.0.smithi060.stdout:9/890: dwrite d3/d1e/d5e/d1b/f9d [0,4194304] 0 2024-02-26T10:48:06.594 INFO:tasks.workunit.client.0.smithi060.stdout:3/611: dread d5/de/df/d11/d63/d93/f96 [0,4194304] 0 2024-02-26T10:48:06.594 INFO:tasks.workunit.client.0.smithi060.stdout:5/800: read - d0/ff4 zero size 2024-02-26T10:48:06.594 INFO:tasks.workunit.client.0.smithi060.stdout:5/801: getdents d0/de/d20/d32/d4d/d110 0 2024-02-26T10:48:06.594 INFO:tasks.workunit.client.0.smithi060.stdout:5/802: fsync d0/de/f46 0 2024-02-26T10:48:06.601 INFO:tasks.workunit.client.0.smithi060.stdout:9/891: symlink d3/d1e/d5e/d1b/d11a/l139 0 2024-02-26T10:48:06.602 INFO:tasks.workunit.client.0.smithi060.stdout:3/612: mknod d5/de/df/d11/d63/d93/d97/d98/cb8 0 2024-02-26T10:48:06.602 INFO:tasks.workunit.client.0.smithi060.stdout:3/613: dread - d5/d76/d7d/f8a zero size 2024-02-26T10:48:06.604 INFO:tasks.workunit.client.0.smithi060.stdout:4/808: dwrite d0/da/d3b/d16/f84 [0,4194304] 0 2024-02-26T10:48:06.606 INFO:tasks.workunit.client.0.smithi060.stdout:3/614: link d5/de/df/d57/d7a/la5 d5/de/df/d57/lb9 0 2024-02-26T10:48:06.611 INFO:tasks.workunit.client.0.smithi060.stdout:3/615: truncate d5/d45/d61/d92/da1/daa/f86 648823 0 2024-02-26T10:48:06.612 INFO:tasks.workunit.client.0.smithi060.stdout:3/616: dread - d5/d45/d61/d92/da1/daa/fa6 zero size 2024-02-26T10:48:06.614 INFO:tasks.workunit.client.0.smithi060.stdout:3/617: link d5/de/df/d57/c83 d5/d45/d61/d92/da1/cba 0 2024-02-26T10:48:06.615 INFO:tasks.workunit.client.0.smithi060.stdout:8/830: dread d9/d15/d29/f65 [0,4194304] 0 2024-02-26T10:48:06.616 INFO:tasks.workunit.client.0.smithi060.stdout:7/992: dwrite da/d15/d21/d2f/d78/fda [0,4194304] 0 2024-02-26T10:48:06.616 INFO:tasks.workunit.client.0.smithi060.stdout:0/784: sync 2024-02-26T10:48:06.616 INFO:tasks.workunit.client.0.smithi060.stdout:6/871: sync 2024-02-26T10:48:06.616 INFO:tasks.workunit.client.0.smithi060.stdout:2/790: sync 2024-02-26T10:48:06.616 INFO:tasks.workunit.client.0.smithi060.stdout:2/791: chown dd/d43/d55/d6e/dd3/d6f/c8a 1970810 1 2024-02-26T10:48:06.617 INFO:tasks.workunit.client.0.smithi060.stdout:3/618: mkdir d5/dbb 0 2024-02-26T10:48:06.617 INFO:tasks.workunit.client.0.smithi060.stdout:8/831: mknod d9/d5a/d58/c117 0 2024-02-26T10:48:06.619 INFO:tasks.workunit.client.0.smithi060.stdout:2/792: creat dd/d43/d55/d5e/dc6/f108 x:0 0 0 2024-02-26T10:48:06.619 INFO:tasks.workunit.client.0.smithi060.stdout:2/793: truncate dd/d43/d55/d6e/fd5 346076 0 2024-02-26T10:48:06.619 INFO:tasks.workunit.client.0.smithi060.stdout:4/809: mkdir d0/da/d3b/ddc/dfa/dfe 0 2024-02-26T10:48:06.619 INFO:tasks.workunit.client.0.smithi060.stdout:0/785: mkdir d0/d42/d7a/d108 0 2024-02-26T10:48:06.619 INFO:tasks.workunit.client.0.smithi060.stdout:0/786: fdatasync d0/f55 0 2024-02-26T10:48:06.620 INFO:tasks.workunit.client.0.smithi060.stdout:4/810: mknod d0/da/dba/cff 0 2024-02-26T10:48:06.622 INFO:tasks.workunit.client.0.smithi060.stdout:6/872: unlink d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/lb8 0 2024-02-26T10:48:06.623 INFO:tasks.workunit.client.0.smithi060.stdout:3/619: link d5/c9d d5/de/df/d57/d7a/cbc 0 2024-02-26T10:48:06.624 INFO:tasks.workunit.client.0.smithi060.stdout:6/873: truncate d6/f11 3703734 0 2024-02-26T10:48:06.626 INFO:tasks.workunit.client.0.smithi060.stdout:3/620: creat d5/de/df/d11/d63/fbd x:0 0 0 2024-02-26T10:48:06.626 INFO:tasks.workunit.client.0.smithi060.stdout:6/874: mknod d6/d7/d47/c11c 0 2024-02-26T10:48:06.626 INFO:tasks.workunit.client.0.smithi060.stdout:6/875: fsync d6/f16 0 2024-02-26T10:48:06.627 INFO:tasks.workunit.client.0.smithi060.stdout:7/993: mkdir da/d15/d21/d2f/d78/d148 0 2024-02-26T10:48:06.634 INFO:tasks.workunit.client.0.smithi060.stdout:1/896: dwrite d3/f12e [0,4194304] 0 2024-02-26T10:48:06.640 INFO:tasks.workunit.client.0.smithi060.stdout:5/803: rename d0/de/d20/d82/l91 to d0/de/d20/d32/d4d/l115 0 2024-02-26T10:48:06.642 INFO:tasks.workunit.client.0.smithi060.stdout:5/804: truncate d0/de/d20/f31 2053426 0 2024-02-26T10:48:06.644 INFO:tasks.workunit.client.0.smithi060.stdout:5/805: creat d0/dc0/f116 x:0 0 0 2024-02-26T10:48:06.644 INFO:tasks.workunit.client.0.smithi060.stdout:5/806: truncate d0/d33/f5c 992868 0 2024-02-26T10:48:06.644 INFO:tasks.workunit.client.0.smithi060.stdout:5/807: dread - d0/de/f10c zero size 2024-02-26T10:48:06.646 INFO:tasks.workunit.client.0.smithi060.stdout:5/808: mkdir d0/de/d20/d32/d117 0 2024-02-26T10:48:06.647 INFO:tasks.workunit.client.0.smithi060.stdout:5/809: chown d0/d4b/d7b/f102 1537 1 2024-02-26T10:48:06.657 INFO:tasks.workunit.client.0.smithi060.stdout:6/876: dread d6/d7/d25/d1c/d1f/d23/f9c [0,4194304] 0 2024-02-26T10:48:06.657 INFO:tasks.workunit.client.0.smithi060.stdout:4/811: write d0/d27/d76/f91 [2038702,96574] 0 2024-02-26T10:48:06.658 INFO:tasks.workunit.client.0.smithi060.stdout:3/621: dread d5/de/f46 [0,4194304] 0 2024-02-26T10:48:06.658 INFO:tasks.workunit.client.0.smithi060.stdout:3/622: readlink d5/d76/l82 0 2024-02-26T10:48:06.664 INFO:tasks.workunit.client.0.smithi060.stdout:9/892: rename d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/d96 to d3/d1e/d5e/d35/d87/d9b/d8a/d13a 0 2024-02-26T10:48:06.665 INFO:tasks.workunit.client.0.smithi060.stdout:9/893: write d3/d1e/d5e/d35/d87/d9b/d8a/d13a/f104 [250301,98183] 0 2024-02-26T10:48:06.666 INFO:tasks.workunit.client.0.smithi060.stdout:9/894: getdents d3/d1e/d5e/d35/d87 0 2024-02-26T10:48:06.668 INFO:tasks.workunit.client.0.smithi060.stdout:2/794: dwrite dd/d43/d55/d6e/dd3/d6f/d8f/dbc/fbe [0,4194304] 0 2024-02-26T10:48:06.674 INFO:tasks.workunit.client.0.smithi060.stdout:3/623: write d5/de/f3b [1453903,94801] 0 2024-02-26T10:48:06.674 INFO:tasks.workunit.client.0.smithi060.stdout:4/812: link d0/da/fcc d0/da/d3b/d1b/f100 0 2024-02-26T10:48:06.676 INFO:tasks.workunit.client.0.smithi060.stdout:4/813: mkdir d0/d27/d76/dbd/d60/d101 0 2024-02-26T10:48:06.676 INFO:tasks.workunit.client.0.smithi060.stdout:4/814: read - d0/d27/d76/d9f/fed zero size 2024-02-26T10:48:06.676 INFO:tasks.workunit.client.0.smithi060.stdout:4/815: chown d0/da/dc7/db0/db5/dc0/fda 2 1 2024-02-26T10:48:06.678 INFO:tasks.workunit.client.0.smithi060.stdout:4/816: rename d0/ff to d0/f102 0 2024-02-26T10:48:06.678 INFO:tasks.workunit.client.0.smithi060.stdout:4/817: chown d0/d27/d76/d9f 924854 1 2024-02-26T10:48:06.679 INFO:tasks.workunit.client.0.smithi060.stdout:4/818: creat d0/da/d3b/d1b/f103 x:0 0 0 2024-02-26T10:48:06.680 INFO:tasks.workunit.client.0.smithi060.stdout:7/994: rename da/d15/d21/d2f/d78/dd3/d133/l134 to da/df1/dd7/d125/dd8/l149 0 2024-02-26T10:48:06.680 INFO:tasks.workunit.client.0.smithi060.stdout:7/995: write da/d15/d21/fd2 [985412,103707] 0 2024-02-26T10:48:06.680 INFO:tasks.workunit.client.0.smithi060.stdout:7/996: fsync da/d15/d21/d2f/d78/fa7 0 2024-02-26T10:48:06.681 INFO:tasks.workunit.client.0.smithi060.stdout:7/997: symlink da/d15/d21/d41/d58/l14a 0 2024-02-26T10:48:06.688 INFO:tasks.workunit.client.0.smithi060.stdout:1/897: rename d3/db/d1e/d39/c5d to d3/db/d1e/d39/d9b/df7/d11f/c12f 0 2024-02-26T10:48:06.688 INFO:tasks.workunit.client.0.smithi060.stdout:1/898: write d3/d35/d53/d68/d85/fec [1040400,26618] 0 2024-02-26T10:48:06.688 INFO:tasks.workunit.client.0.smithi060.stdout:1/899: fsync d3/d35/d53/fd2 0 2024-02-26T10:48:06.689 INFO:tasks.workunit.client.0.smithi060.stdout:1/900: dread - d3/d35/d53/d68/d90/d125/fb4 zero size 2024-02-26T10:48:06.689 INFO:tasks.workunit.client.0.smithi060.stdout:6/877: rename d6/d7/d25/d1c/d1f/d23/d2b/la2 to d6/d7/d25/d1c/d1f/d23/l11d 0 2024-02-26T10:48:06.690 INFO:tasks.workunit.client.0.smithi060.stdout:6/878: dread - d6/d7/d25/f62 zero size 2024-02-26T10:48:06.693 INFO:tasks.workunit.client.0.smithi060.stdout:9/895: rename d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/fea to d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d8c/d91/f13b 0 2024-02-26T10:48:06.695 INFO:tasks.workunit.client.0.smithi060.stdout:3/624: rename d5/de/df/d11/d63/f64 to d5/de/df/d4c/fbe 0 2024-02-26T10:48:06.699 INFO:tasks.workunit.client.0.smithi060.stdout:9/896: mkdir d3/d13c 0 2024-02-26T10:48:06.705 INFO:tasks.workunit.client.0.smithi060.stdout:1/901: link d3/d35/d53/cdb d3/d35/c130 0 2024-02-26T10:48:06.707 INFO:tasks.workunit.client.0.smithi060.stdout:9/897: dread d3/d1e/d57/fd0 [0,4194304] 0 2024-02-26T10:48:06.711 INFO:tasks.workunit.client.0.smithi060.stdout:9/898: rmdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d11e/d10c/d116 39 2024-02-26T10:48:06.724 INFO:tasks.workunit.client.0.smithi060.stdout:3/625: dread d5/de/df/d57/d60/f30 [0,4194304] 0 2024-02-26T10:48:06.737 INFO:tasks.workunit.client.0.smithi060.stdout:4/819: rename d0/da/d3b/d1b/cae to d0/da/dc7/db0/db5/dc0/c104 0 2024-02-26T10:48:06.745 INFO:tasks.workunit.client.0.smithi060.stdout:1/902: rename d3/d35/d53/d68/dad to d3/d35/d53/d131 0 2024-02-26T10:48:06.749 INFO:tasks.workunit.client.0.smithi060.stdout:2/795: dwrite dd/d43/d55/d5e/dc6/f7a [4194304,4194304] 0 2024-02-26T10:48:06.751 INFO:tasks.workunit.client.0.smithi060.stdout:2/796: creat dd/d43/d96/f109 x:0 0 0 2024-02-26T10:48:06.752 INFO:tasks.workunit.client.0.smithi060.stdout:2/797: symlink dd/d43/d55/d6e/dd3/d6f/l10a 0 2024-02-26T10:48:06.753 INFO:tasks.workunit.client.0.smithi060.stdout:2/798: creat dd/d43/dd2/f10b x:0 0 0 2024-02-26T10:48:06.754 INFO:tasks.workunit.client.0.smithi060.stdout:9/899: dread d3/d1e/d5e/d35/d87/d9b/d8a/fb0 [0,4194304] 0 2024-02-26T10:48:06.755 INFO:tasks.workunit.client.0.smithi060.stdout:5/810: dwrite d0/de/d15/d26/f45 [0,4194304] 0 2024-02-26T10:48:06.756 INFO:tasks.workunit.client.0.smithi060.stdout:9/900: symlink d3/d1e/d5e/d35/deb/d7a/l13d 0 2024-02-26T10:48:06.756 INFO:tasks.workunit.client.0.smithi060.stdout:9/901: dread - d3/d1e/d5e/d1b/f131 zero size 2024-02-26T10:48:06.756 INFO:tasks.workunit.client.0.smithi060.stdout:6/879: dwrite d6/f18 [0,4194304] 0 2024-02-26T10:48:06.761 INFO:tasks.workunit.client.0.smithi060.stdout:9/902: unlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/lac 0 2024-02-26T10:48:06.764 INFO:tasks.workunit.client.0.smithi060.stdout:9/903: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/dd2/d129/f13e x:0 0 0 2024-02-26T10:48:06.766 INFO:tasks.workunit.client.0.smithi060.stdout:9/904: creat d3/d1e/dbb/dc2/df8/d127/f13f x:0 0 0 2024-02-26T10:48:06.767 INFO:tasks.workunit.client.0.smithi060.stdout:5/811: rmdir d0 39 2024-02-26T10:48:06.767 INFO:tasks.workunit.client.0.smithi060.stdout:5/812: stat d0/d33/l37 0 2024-02-26T10:48:06.769 INFO:tasks.workunit.client.0.smithi060.stdout:9/905: rmdir d3/d13c 0 2024-02-26T10:48:06.770 INFO:tasks.workunit.client.0.smithi060.stdout:9/906: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/f140 x:0 0 0 2024-02-26T10:48:06.772 INFO:tasks.workunit.client.0.smithi060.stdout:7/998: dwrite da/d15/d88/d3b/f145 [0,4194304] 0 2024-02-26T10:48:06.772 INFO:tasks.workunit.client.0.smithi060.stdout:9/907: write d3/d1e/d5e/d35/deb/fb4 [1621323,40843] 0 2024-02-26T10:48:06.773 INFO:tasks.workunit.client.0.smithi060.stdout:7/999: creat da/d15/d21/d2f/d37/d140/f14b x:0 0 0 2024-02-26T10:48:06.773 INFO:tasks.workunit.client.0.smithi060.stdout:4/820: dread d0/da/f80 [0,4194304] 0 2024-02-26T10:48:06.774 INFO:tasks.workunit.client.0.smithi060.stdout:4/821: stat d0/d27/d76/dbd/lcf 0 2024-02-26T10:48:06.774 INFO:tasks.workunit.client.0.smithi060.stdout:2/799: dread dd/d43/d60/d41/d37/d9a/db4/fcf [0,4194304] 0 2024-02-26T10:48:06.774 INFO:tasks.workunit.client.0.smithi060.stdout:2/800: chown dd/d43/d55/d5e/dc6/f108 127360357 1 2024-02-26T10:48:06.774 INFO:tasks.workunit.client.0.smithi060.stdout:2/801: chown dd/d43/dd2 13064 1 2024-02-26T10:48:06.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/908: rename d3/d1e/d5e/d1b/d23/f34 to d3/d1e/dbb/f141 0 2024-02-26T10:48:06.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/909: fdatasync d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/dd2/d129/f13e 0 2024-02-26T10:48:06.775 INFO:tasks.workunit.client.0.smithi060.stdout:9/910: getdents d3/d1e/d5e/d35/d87/d9b/d8a/d13a/dfb 0 2024-02-26T10:48:06.775 INFO:tasks.workunit.client.0.smithi060.stdout:2/802: symlink dd/d43/d60/l10c 0 2024-02-26T10:48:06.776 INFO:tasks.workunit.client.0.smithi060.stdout:4/822: mknod d0/da/d3b/ddc/dfa/dfe/c105 0 2024-02-26T10:48:06.778 INFO:tasks.workunit.client.0.smithi060.stdout:4/823: rename d0/da/f8b to d0/da/d3b/ddc/dfa/dfe/f106 0 2024-02-26T10:48:06.778 INFO:tasks.workunit.client.0.smithi060.stdout:4/824: stat d0/da/dc7/db0/db5/dc0/lbb 0 2024-02-26T10:48:06.781 INFO:tasks.workunit.client.0.smithi060.stdout:6/880: dread d6/d7/d2e/d56/d73/fa4 [0,4194304] 0 2024-02-26T10:48:06.782 INFO:tasks.workunit.client.0.smithi060.stdout:6/881: symlink d6/d7/d47/def/l11e 0 2024-02-26T10:48:06.782 INFO:tasks.workunit.client.0.smithi060.stdout:6/882: read - d6/d7/d2e/d56/d73/fdb zero size 2024-02-26T10:48:06.782 INFO:tasks.workunit.client.0.smithi060.stdout:6/883: chown d6/d7/d2e/feb 21773 1 2024-02-26T10:48:06.783 INFO:tasks.workunit.client.0.smithi060.stdout:6/884: creat d6/d7/d2e/d56/f11f x:0 0 0 2024-02-26T10:48:06.783 INFO:tasks.workunit.client.0.smithi060.stdout:6/885: truncate d6/d7/d2e/fab 627204 0 2024-02-26T10:48:06.800 INFO:tasks.workunit.client.0.smithi060.stdout:6/886: rename d6/d7/d2e/d56/dfd/d10f to d6/d7/d25/d120 0 2024-02-26T10:48:06.800 INFO:tasks.workunit.client.0.smithi060.stdout:6/887: truncate d6/d6e/d6f/fe9 127422 0 2024-02-26T10:48:06.801 INFO:tasks.workunit.client.0.smithi060.stdout:6/888: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac/db5/d121 0 2024-02-26T10:48:06.803 INFO:tasks.workunit.client.0.smithi060.stdout:6/889: rename d6/f40 to d6/d7/d25/d80/f122 0 2024-02-26T10:48:06.803 INFO:tasks.workunit.client.0.smithi060.stdout:6/890: chown d6/d7/d47/dde/led 33 1 2024-02-26T10:48:06.819 INFO:tasks.workunit.client.0.smithi060.stdout:9/911: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fdc [0,4194304] 0 2024-02-26T10:48:06.821 INFO:tasks.workunit.client.0.smithi060.stdout:4/825: dwrite d0/da/d3b/f23 [4194304,4194304] 0 2024-02-26T10:48:06.830 INFO:tasks.workunit.client.0.smithi060.stdout:4/826: unlink d0/da/d3b/f95 0 2024-02-26T10:48:06.840 INFO:tasks.workunit.client.0.smithi060.stdout:9/912: rename d3/d1e/d5e/d35/deb/d7a/c136 to d3/d1e/d5e/d1b/c142 0 2024-02-26T10:48:06.840 INFO:tasks.workunit.client.0.smithi060.stdout:9/913: readlink d3/d1e/d5e/d1b/d23/l102 0 2024-02-26T10:48:06.840 INFO:tasks.workunit.client.0.smithi060.stdout:9/914: truncate d3/d1e/dbb/dc2/f118 759952 0 2024-02-26T10:48:06.843 INFO:tasks.workunit.client.0.smithi060.stdout:4/827: read d0/da/dc7/db0/fd9 [9129,82645] 0 2024-02-26T10:48:06.844 INFO:tasks.workunit.client.0.smithi060.stdout:6/891: dwrite d6/d7/d25/d80/df9/fd3 [0,4194304] 0 2024-02-26T10:48:06.844 INFO:tasks.workunit.client.0.smithi060.stdout:4/828: write d0/da/d3b/ddc/dfa/f55 [4410762,44827] 0 2024-02-26T10:48:06.846 INFO:tasks.workunit.client.0.smithi060.stdout:6/892: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/da7/f123 x:0 0 0 2024-02-26T10:48:06.846 INFO:tasks.workunit.client.0.smithi060.stdout:6/893: write d6/d6e/d6f/fcd [1761987,124271] 0 2024-02-26T10:48:06.846 INFO:tasks.workunit.client.0.smithi060.stdout:6/894: write d6/d7/d47/dde/f114 [754324,65718] 0 2024-02-26T10:48:06.848 INFO:tasks.workunit.client.0.smithi060.stdout:6/895: rename d6/d7/d25/d1c/d1f/l42 to d6/d8d/dcc/dc3/l124 0 2024-02-26T10:48:06.848 INFO:tasks.workunit.client.0.smithi060.stdout:9/915: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/fe5 [531774,121823] 0 2024-02-26T10:48:06.848 INFO:tasks.workunit.client.0.smithi060.stdout:9/916: read - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f10b zero size 2024-02-26T10:48:06.851 INFO:tasks.workunit.client.0.smithi060.stdout:8/832: sync 2024-02-26T10:48:06.852 INFO:tasks.workunit.client.0.smithi060.stdout:9/917: rmdir d3/d1e/d5e/d35/d87/d9b/d8a/d13a/de8 39 2024-02-26T10:48:06.852 INFO:tasks.workunit.client.0.smithi060.stdout:0/787: sync 2024-02-26T10:48:06.852 INFO:tasks.workunit.client.0.smithi060.stdout:9/918: stat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb 0 2024-02-26T10:48:06.852 INFO:tasks.workunit.client.0.smithi060.stdout:0/788: readlink d0/dd/d39/d40/lb4 0 2024-02-26T10:48:06.853 INFO:tasks.workunit.client.0.smithi060.stdout:0/789: symlink d0/dd/d39/d40/dde/l109 0 2024-02-26T10:48:06.853 INFO:tasks.workunit.client.0.smithi060.stdout:0/790: truncate d0/dd/d39/d40/f26 1254591 0 2024-02-26T10:48:06.853 INFO:tasks.workunit.client.0.smithi060.stdout:0/791: chown d0/c11 143760197 1 2024-02-26T10:48:06.854 INFO:tasks.workunit.client.0.smithi060.stdout:0/792: chown d0/dd/l27 994 1 2024-02-26T10:48:06.855 INFO:tasks.workunit.client.0.smithi060.stdout:0/793: creat d0/d42/df2/d102/f10a x:0 0 0 2024-02-26T10:48:06.856 INFO:tasks.workunit.client.0.smithi060.stdout:0/794: chown d0/dd/d39/d40/d83/dc6/d58/f5b 48913 1 2024-02-26T10:48:06.858 INFO:tasks.workunit.client.0.smithi060.stdout:0/795: unlink d0/dd/d39/d40/f34 0 2024-02-26T10:48:06.862 INFO:tasks.workunit.client.0.smithi060.stdout:9/919: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f37 [3914844,87272] 0 2024-02-26T10:48:06.865 INFO:tasks.workunit.client.0.smithi060.stdout:9/920: truncate d3/d1e/d5e/d1b/f56 1790802 0 2024-02-26T10:48:06.866 INFO:tasks.workunit.client.0.smithi060.stdout:9/921: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/d13a/d143 0 2024-02-26T10:48:06.877 INFO:tasks.workunit.client.0.smithi060.stdout:0/796: dread d0/d42/fd7 [0,4194304] 0 2024-02-26T10:48:06.878 INFO:tasks.workunit.client.0.smithi060.stdout:4/829: write d0/d27/d76/faa [704425,9530] 0 2024-02-26T10:48:06.879 INFO:tasks.workunit.client.0.smithi060.stdout:0/797: creat d0/dd/d39/d40/dad/dda/de4/d103/f10b x:0 0 0 2024-02-26T10:48:06.883 INFO:tasks.workunit.client.0.smithi060.stdout:4/830: rename d0/d27/dde to d0/d27/d76/dbd/d8e/db3/d107 0 2024-02-26T10:48:06.883 INFO:tasks.workunit.client.0.smithi060.stdout:0/798: mkdir d0/dd/d39/d40/d83/d10c 0 2024-02-26T10:48:06.885 INFO:tasks.workunit.client.0.smithi060.stdout:0/799: mkdir d0/dd/d39/d40/dad/d10d 0 2024-02-26T10:48:06.885 INFO:tasks.workunit.client.0.smithi060.stdout:0/800: stat d0/dd/d39/d63/fe5 0 2024-02-26T10:48:06.886 INFO:tasks.workunit.client.0.smithi060.stdout:8/833: dread d9/d15/f81 [0,4194304] 0 2024-02-26T10:48:06.891 INFO:tasks.workunit.client.0.smithi060.stdout:9/922: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fd9 [2976219,1212] 0 2024-02-26T10:48:06.899 INFO:tasks.workunit.client.0.smithi060.stdout:4/831: read d0/d27/d76/f93 [1277928,99984] 0 2024-02-26T10:48:06.910 INFO:tasks.workunit.client.0.smithi060.stdout:4/832: rmdir d0/da/d3b 39 2024-02-26T10:48:06.911 INFO:tasks.workunit.client.0.smithi060.stdout:4/833: chown d0/d27/l56 25262326 1 2024-02-26T10:48:06.932 INFO:tasks.workunit.client.0.smithi060.stdout:9/923: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/ffa [0,4194304] 0 2024-02-26T10:48:06.935 INFO:tasks.workunit.client.0.smithi060.stdout:9/924: creat d3/d1e/d5e/d35/d87/d9b/d8a/d13a/de8/f144 x:0 0 0 2024-02-26T10:48:06.938 INFO:tasks.workunit.client.0.smithi060.stdout:8/834: dwrite d9/d6f/fd8 [0,4194304] 0 2024-02-26T10:48:06.939 INFO:tasks.workunit.client.0.smithi060.stdout:0/801: dwrite d0/dd/d39/d40/dad/dd6/d6c/fab [0,4194304] 0 2024-02-26T10:48:06.941 INFO:tasks.workunit.client.0.smithi060.stdout:9/925: dread d3/d1e/d57/fd0 [0,4194304] 0 2024-02-26T10:48:06.941 INFO:tasks.workunit.client.0.smithi060.stdout:9/926: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/c94 3393292 1 2024-02-26T10:48:06.945 INFO:tasks.workunit.client.0.smithi060.stdout:0/802: truncate d0/ff1 577642 0 2024-02-26T10:48:06.946 INFO:tasks.workunit.client.0.smithi060.stdout:9/927: truncate d3/d1e/d93/fab 2379863 0 2024-02-26T10:48:06.947 INFO:tasks.workunit.client.0.smithi060.stdout:9/928: symlink d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/l145 0 2024-02-26T10:48:06.950 INFO:tasks.workunit.client.0.smithi060.stdout:9/929: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/de9/f146 x:0 0 0 2024-02-26T10:48:06.950 INFO:tasks.workunit.client.0.smithi060.stdout:9/930: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32 108429 1 2024-02-26T10:48:06.964 INFO:tasks.workunit.client.0.smithi060.stdout:8/835: write d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/fbc [3026960,115613] 0 2024-02-26T10:48:06.969 INFO:tasks.workunit.client.0.smithi060.stdout:1/903: sync 2024-02-26T10:48:06.969 INFO:tasks.workunit.client.0.smithi060.stdout:2/803: sync 2024-02-26T10:48:06.969 INFO:tasks.workunit.client.0.smithi060.stdout:5/813: sync 2024-02-26T10:48:06.969 INFO:tasks.workunit.client.0.smithi060.stdout:3/626: sync 2024-02-26T10:48:06.969 INFO:tasks.workunit.client.0.smithi060.stdout:5/814: fsync d0/de/d15/f47 0 2024-02-26T10:48:06.969 INFO:tasks.workunit.client.0.smithi060.stdout:2/804: write dd/d43/d96/df2/ff8 [496106,116373] 0 2024-02-26T10:48:06.969 INFO:tasks.workunit.client.0.smithi060.stdout:5/815: write d0/de/d15/d26/fce [396742,61152] 0 2024-02-26T10:48:06.969 INFO:tasks.workunit.client.0.smithi060.stdout:2/805: chown dd/d43/d60/d41/d37/le0 280411 1 2024-02-26T10:48:06.970 INFO:tasks.workunit.client.0.smithi060.stdout:3/627: symlink d5/de/df/d11/d63/d93/d97/d98/lbf 0 2024-02-26T10:48:06.971 INFO:tasks.workunit.client.0.smithi060.stdout:3/628: truncate d5/de/f14 2529926 0 2024-02-26T10:48:06.972 INFO:tasks.workunit.client.0.smithi060.stdout:3/629: unlink d5/de/d16/l75 0 2024-02-26T10:48:06.977 INFO:tasks.workunit.client.0.smithi060.stdout:1/904: mknod d3/db/d1e/d6d/da3/daa/dd6/c132 0 2024-02-26T10:48:06.977 INFO:tasks.workunit.client.0.smithi060.stdout:1/905: chown d3/d35/d5e 2 1 2024-02-26T10:48:06.982 INFO:tasks.workunit.client.0.smithi060.stdout:0/803: dwrite d0/dd/d53/d87/fd2 [0,4194304] 0 2024-02-26T10:48:06.986 INFO:tasks.workunit.client.0.smithi060.stdout:0/804: rmdir d0/d37/ddd 39 2024-02-26T10:48:06.988 INFO:tasks.workunit.client.0.smithi060.stdout:8/836: write d9/d15/d29/f64 [1720884,128956] 0 2024-02-26T10:48:07.050 INFO:tasks.workunit.client.0.smithi060.stdout:9/931: dwrite d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f7e [0,4194304] 0 2024-02-26T10:48:07.064 INFO:tasks.workunit.client.0.smithi060.stdout:8/837: dread d9/d5a/faf [0,4194304] 0 2024-02-26T10:48:07.066 INFO:tasks.workunit.client.0.smithi060.stdout:8/838: mknod d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/c118 0 2024-02-26T10:48:07.068 INFO:tasks.workunit.client.0.smithi060.stdout:8/839: mkdir d9/d15/d29/d2f/d34/d72/d94/dba/d119 0 2024-02-26T10:48:07.085 INFO:tasks.workunit.client.0.smithi060.stdout:8/840: creat d9/f11a x:0 0 0 2024-02-26T10:48:07.085 INFO:tasks.workunit.client.0.smithi060.stdout:8/841: read - d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/f107 zero size 2024-02-26T10:48:07.085 INFO:tasks.workunit.client.0.smithi060.stdout:8/842: fdatasync d9/d15/f36 0 2024-02-26T10:48:07.085 INFO:tasks.workunit.client.0.smithi060.stdout:9/932: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/f8f [0,4194304] 0 2024-02-26T10:48:07.086 INFO:tasks.workunit.client.0.smithi060.stdout:9/933: fsync d3/d1e/d57/ff3 0 2024-02-26T10:48:07.086 INFO:tasks.workunit.client.0.smithi060.stdout:8/843: creat d9/d15/d29/d2f/d34/d72/f11b x:0 0 0 2024-02-26T10:48:07.086 INFO:tasks.workunit.client.0.smithi060.stdout:8/844: chown d9/d15/d29/f64 1 1 2024-02-26T10:48:07.087 INFO:tasks.workunit.client.0.smithi060.stdout:9/934: creat d3/d1e/d5e/d35/deb/d107/f147 x:0 0 0 2024-02-26T10:48:07.088 INFO:tasks.workunit.client.0.smithi060.stdout:8/845: stat d9/d15/d29/d2f/l87 0 2024-02-26T10:48:07.091 INFO:tasks.workunit.client.0.smithi060.stdout:9/935: link d3/d1e/d5e/d35/fd7 d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d11e/d10c/f148 0 2024-02-26T10:48:07.093 INFO:tasks.workunit.client.0.smithi060.stdout:9/936: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/dde/f149 x:0 0 0 2024-02-26T10:48:07.093 INFO:tasks.workunit.client.0.smithi060.stdout:9/937: getdents d3/d1e/d5e/d35/deb/d107/d130/d113 0 2024-02-26T10:48:07.095 INFO:tasks.workunit.client.0.smithi060.stdout:9/938: mknod d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/dc8/c14a 0 2024-02-26T10:48:07.095 INFO:tasks.workunit.client.0.smithi060.stdout:9/939: dread - d3/d68/f132 zero size 2024-02-26T10:48:07.095 INFO:tasks.workunit.client.0.smithi060.stdout:9/940: dread - d3/d1e/d5e/d1b/f131 zero size 2024-02-26T10:48:07.096 INFO:tasks.workunit.client.0.smithi060.stdout:9/941: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/ff4 [703374,67189] 0 2024-02-26T10:48:07.099 INFO:tasks.workunit.client.0.smithi060.stdout:8/846: dread d9/d5a/f68 [0,4194304] 0 2024-02-26T10:48:07.099 INFO:tasks.workunit.client.0.smithi060.stdout:8/847: readlink d9/d15/d29/d2f/d34/d72/l22 0 2024-02-26T10:48:07.127 INFO:tasks.workunit.client.0.smithi060.stdout:8/848: dread d9/d5a/d9c/fc7 [0,4194304] 0 2024-02-26T10:48:07.128 INFO:tasks.workunit.client.0.smithi060.stdout:3/630: dwrite d5/de/f46 [4194304,4194304] 0 2024-02-26T10:48:07.128 INFO:tasks.workunit.client.0.smithi060.stdout:2/806: dwrite dd/f6a [0,4194304] 0 2024-02-26T10:48:07.130 INFO:tasks.workunit.client.0.smithi060.stdout:3/631: mknod d5/de/df/d57/d60/cc0 0 2024-02-26T10:48:07.132 INFO:tasks.workunit.client.0.smithi060.stdout:3/632: mkdir d5/d45/d61/d92/da1/dc1 0 2024-02-26T10:48:07.134 INFO:tasks.workunit.client.0.smithi060.stdout:3/633: link d5/de/df/d57/d60/f3a d5/d45/d61/d92/da1/fc2 0 2024-02-26T10:48:07.135 INFO:tasks.workunit.client.0.smithi060.stdout:3/634: creat d5/d37/fc3 x:0 0 0 2024-02-26T10:48:07.147 INFO:tasks.workunit.client.0.smithi060.stdout:8/849: getdents d9/d15/d29/d2f/d34/d72/d50/d6a/dc5 0 2024-02-26T10:48:07.148 INFO:tasks.workunit.client.0.smithi060.stdout:3/635: symlink d5/de/df/d11/d63/d93/da2/lc4 0 2024-02-26T10:48:07.150 INFO:tasks.workunit.client.0.smithi060.stdout:3/636: creat d5/d45/d61/fc5 x:0 0 0 2024-02-26T10:48:07.150 INFO:tasks.workunit.client.0.smithi060.stdout:0/805: read d0/d42/fd7 [1089459,113441] 0 2024-02-26T10:48:07.151 INFO:tasks.workunit.client.0.smithi060.stdout:1/906: dwrite d3/db/d1e/d6d/d78/f86 [0,4194304] 0 2024-02-26T10:48:07.151 INFO:tasks.workunit.client.0.smithi060.stdout:1/907: write d3/db/d1e/d6d/da3/daa/fde [722501,27080] 0 2024-02-26T10:48:07.161 INFO:tasks.workunit.client.0.smithi060.stdout:9/942: rename d3/d1e/d5e/d35/d87/d9b/d8a/d13a/dfb to d3/d1e/dbb/df5/d14b 0 2024-02-26T10:48:07.163 INFO:tasks.workunit.client.0.smithi060.stdout:6/896: sync 2024-02-26T10:48:07.163 INFO:tasks.workunit.client.0.smithi060.stdout:5/816: sync 2024-02-26T10:48:07.163 INFO:tasks.workunit.client.0.smithi060.stdout:4/834: sync 2024-02-26T10:48:07.164 INFO:tasks.workunit.client.0.smithi060.stdout:2/807: rename dd/d43/d60/l4d to dd/d43/d55/d5e/d74/da5/l10d 0 2024-02-26T10:48:07.165 INFO:tasks.workunit.client.0.smithi060.stdout:6/897: symlink d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d93/l125 0 2024-02-26T10:48:07.165 INFO:tasks.workunit.client.0.smithi060.stdout:6/898: chown d6/d7/d47/dde 1639494 1 2024-02-26T10:48:07.165 INFO:tasks.workunit.client.0.smithi060.stdout:1/908: mkdir d3/d35/d53/d68/d90/d133 0 2024-02-26T10:48:07.165 INFO:tasks.workunit.client.0.smithi060.stdout:1/909: write d3/db/d1e/ff1 [1048028,22708] 0 2024-02-26T10:48:07.165 INFO:tasks.workunit.client.0.smithi060.stdout:1/910: stat d3/db/f56 0 2024-02-26T10:48:07.165 INFO:tasks.workunit.client.0.smithi060.stdout:6/899: write d6/d7/d25/d80/df9/dd8/fe3 [506551,81164] 0 2024-02-26T10:48:07.168 INFO:tasks.workunit.client.0.smithi060.stdout:5/817: creat d0/de/d15/d65/d86/d9b/f118 x:0 0 0 2024-02-26T10:48:07.168 INFO:tasks.workunit.client.0.smithi060.stdout:2/808: link dd/d43/d55/d6e/dd3/l50 dd/d43/d55/d5e/d86/d97/db9/dde/l10e 0 2024-02-26T10:48:07.170 INFO:tasks.workunit.client.0.smithi060.stdout:0/806: rename d0/dd/d39/d40/lba to d0/dd/d39/d40/d83/dc0/l10e 0 2024-02-26T10:48:07.172 INFO:tasks.workunit.client.0.smithi060.stdout:2/809: link dd/d43/fb0 dd/d43/f10f 0 2024-02-26T10:48:07.172 INFO:tasks.workunit.client.0.smithi060.stdout:2/810: chown dd/d43/d60/d41/d37/d38/d57/f79 243281114 1 2024-02-26T10:48:07.172 INFO:tasks.workunit.client.0.smithi060.stdout:1/911: rename d3/l17 to d3/d35/dc2/l134 0 2024-02-26T10:48:07.176 INFO:tasks.workunit.client.0.smithi060.stdout:1/912: chown d3/d35/d53/d68/d90/fbc 66315304 1 2024-02-26T10:48:07.176 INFO:tasks.workunit.client.0.smithi060.stdout:2/811: getdents dd/d43/d60/d41/d37/d4c 0 2024-02-26T10:48:07.209 INFO:tasks.workunit.client.0.smithi060.stdout:4/835: dwrite d0/da/d57/fc6 [4194304,4194304] 0 2024-02-26T10:48:07.211 INFO:tasks.workunit.client.0.smithi060.stdout:4/836: creat d0/da/dba/f108 x:0 0 0 2024-02-26T10:48:07.213 INFO:tasks.workunit.client.0.smithi060.stdout:8/850: dwrite d9/d6f/ff3 [0,4194304] 0 2024-02-26T10:48:07.213 INFO:tasks.workunit.client.0.smithi060.stdout:8/851: chown d9/d15/d29/d2f/d34/d72/d50/ffc 1375310554 1 2024-02-26T10:48:07.223 INFO:tasks.workunit.client.0.smithi060.stdout:4/837: creat d0/da/d3b/d16/f109 x:0 0 0 2024-02-26T10:48:07.224 INFO:tasks.workunit.client.0.smithi060.stdout:4/838: truncate d0/d27/fee 2505 0 2024-02-26T10:48:07.224 INFO:tasks.workunit.client.0.smithi060.stdout:4/839: dread - d0/da/dba/fcd zero size 2024-02-26T10:48:07.228 INFO:tasks.workunit.client.0.smithi060.stdout:3/637: dwrite d5/de/df/d11/d63/f7c [0,4194304] 0 2024-02-26T10:48:07.228 INFO:tasks.workunit.client.0.smithi060.stdout:3/638: write d5/d37/fc3 [229400,33633] 0 2024-02-26T10:48:07.228 INFO:tasks.workunit.client.0.smithi060.stdout:8/852: link d9/d15/d29/d2f/d34/d72/d7e/fde d9/d15/d29/d2f/d34/d72/d94/dba/d119/f11c 0 2024-02-26T10:48:07.231 INFO:tasks.workunit.client.0.smithi060.stdout:9/943: dwrite d3/d1e/d5e/d1b/f9d [0,4194304] 0 2024-02-26T10:48:07.231 INFO:tasks.workunit.client.0.smithi060.stdout:0/807: dread d0/dd/d33/f88 [0,4194304] 0 2024-02-26T10:48:07.234 INFO:tasks.workunit.client.0.smithi060.stdout:1/913: dwrite d3/d35/d53/f106 [0,4194304] 0 2024-02-26T10:48:07.234 INFO:tasks.workunit.client.0.smithi060.stdout:9/944: truncate d3/f16 928567 0 2024-02-26T10:48:07.235 INFO:tasks.workunit.client.0.smithi060.stdout:9/945: write d3/d68/fae [3594217,32078] 0 2024-02-26T10:48:07.236 INFO:tasks.workunit.client.0.smithi060.stdout:0/808: symlink d0/dd/d39/d40/dad/dd6/l10f 0 2024-02-26T10:48:07.242 INFO:tasks.workunit.client.0.smithi060.stdout:9/946: rmdir d3/d1e/dbb/df5 39 2024-02-26T10:48:07.244 INFO:tasks.workunit.client.0.smithi060.stdout:0/809: truncate d0/dd/d39/d40/dad/fcb 2647059 0 2024-02-26T10:48:07.245 INFO:tasks.workunit.client.0.smithi060.stdout:9/947: mkdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d11e/d14c 0 2024-02-26T10:48:07.248 INFO:tasks.workunit.client.0.smithi060.stdout:9/948: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/dd2/f14d x:0 0 0 2024-02-26T10:48:07.254 INFO:tasks.workunit.client.0.smithi060.stdout:4/840: dread d0/f74 [0,4194304] 0 2024-02-26T10:48:07.254 INFO:tasks.workunit.client.0.smithi060.stdout:6/900: dwrite d6/d7/d2e/f10b [0,4194304] 0 2024-02-26T10:48:07.254 INFO:tasks.workunit.client.0.smithi060.stdout:6/901: dread - d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f74 zero size 2024-02-26T10:48:07.254 INFO:tasks.workunit.client.0.smithi060.stdout:9/949: creat d3/d1e/d5e/d1b/d23/f14e x:0 0 0 2024-02-26T10:48:07.254 INFO:tasks.workunit.client.0.smithi060.stdout:9/950: write d3/d1e/d5e/d1b/d23/f14e [1020423,37716] 0 2024-02-26T10:48:07.256 INFO:tasks.workunit.client.0.smithi060.stdout:9/951: mknod d3/d1e/dbb/c14f 0 2024-02-26T10:48:07.257 INFO:tasks.workunit.client.0.smithi060.stdout:9/952: dread - d3/d68/d119/f11b zero size 2024-02-26T10:48:07.257 INFO:tasks.workunit.client.0.smithi060.stdout:8/853: dread d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/fbc [0,4194304] 0 2024-02-26T10:48:07.262 INFO:tasks.workunit.client.0.smithi060.stdout:2/812: dwrite dd/d11/fe1 [0,4194304] 0 2024-02-26T10:48:07.264 INFO:tasks.workunit.client.0.smithi060.stdout:2/813: mkdir dd/d43/dd2/d110 0 2024-02-26T10:48:07.266 INFO:tasks.workunit.client.0.smithi060.stdout:2/814: symlink dd/d43/d55/d6e/l111 0 2024-02-26T10:48:07.266 INFO:tasks.workunit.client.0.smithi060.stdout:3/639: rename d5/de/df/d11/d63/d93/da2/cab to d5/d45/cc6 0 2024-02-26T10:48:07.267 INFO:tasks.workunit.client.0.smithi060.stdout:2/815: mknod dd/d43/d55/d5e/d86/d97/db9/c112 0 2024-02-26T10:48:07.268 INFO:tasks.workunit.client.0.smithi060.stdout:3/640: creat d5/fc7 x:0 0 0 2024-02-26T10:48:07.269 INFO:tasks.workunit.client.0.smithi060.stdout:3/641: chown d5/de/f3b 49 1 2024-02-26T10:48:07.269 INFO:tasks.workunit.client.0.smithi060.stdout:3/642: fsync d5/d45/f49 0 2024-02-26T10:48:07.274 INFO:tasks.workunit.client.0.smithi060.stdout:2/816: link dd/d43/d55/d5e/d86/d97/fce dd/d43/d60/f113 0 2024-02-26T10:48:07.274 INFO:tasks.workunit.client.0.smithi060.stdout:2/817: readlink dd/d43/d60/l9b 0 2024-02-26T10:48:07.275 INFO:tasks.workunit.client.0.smithi060.stdout:2/818: stat dd/d43/d55/d5e/d74/da5 0 2024-02-26T10:48:07.275 INFO:tasks.workunit.client.0.smithi060.stdout:2/819: chown dd/d43/d60/d41/d37/d38/d57/l71 33342909 1 2024-02-26T10:48:07.275 INFO:tasks.workunit.client.0.smithi060.stdout:1/914: write d3/d35/d5e/f97 [2962229,108748] 0 2024-02-26T10:48:07.275 INFO:tasks.workunit.client.0.smithi060.stdout:1/915: dread - d3/d35/d53/ff8 zero size 2024-02-26T10:48:07.276 INFO:tasks.workunit.client.0.smithi060.stdout:9/953: dread d3/d1e/d5e/d35/deb/d7a/fd1 [0,4194304] 0 2024-02-26T10:48:07.276 INFO:tasks.workunit.client.0.smithi060.stdout:3/643: mkdir d5/d45/d61/d92/da1/dc8 0 2024-02-26T10:48:07.276 INFO:tasks.workunit.client.0.smithi060.stdout:9/954: rmdir d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d11e/d10c 39 2024-02-26T10:48:07.277 INFO:tasks.workunit.client.0.smithi060.stdout:9/955: chown d3/d1e/dbb/cf7 252 1 2024-02-26T10:48:07.277 INFO:tasks.workunit.client.0.smithi060.stdout:3/644: mknod d5/de/cc9 0 2024-02-26T10:48:07.277 INFO:tasks.workunit.client.0.smithi060.stdout:3/645: fdatasync d5/de/df/d57/f74 0 2024-02-26T10:48:07.278 INFO:tasks.workunit.client.0.smithi060.stdout:9/956: creat d3/d1e/d5e/d35/d87/d9b/d8a/d13a/f150 x:0 0 0 2024-02-26T10:48:07.278 INFO:tasks.workunit.client.0.smithi060.stdout:9/957: chown d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d11e/d10c/f148 60921 1 2024-02-26T10:48:07.278 INFO:tasks.workunit.client.0.smithi060.stdout:3/646: symlink d5/de/d16/lca 0 2024-02-26T10:48:07.278 INFO:tasks.workunit.client.0.smithi060.stdout:3/647: chown d5/de/df/d4c/l71 9924 1 2024-02-26T10:48:07.278 INFO:tasks.workunit.client.0.smithi060.stdout:3/648: truncate d5/de/d16/f6d 731136 0 2024-02-26T10:48:07.278 INFO:tasks.workunit.client.0.smithi060.stdout:3/649: chown d5/de/df/d11/f34 24311432 1 2024-02-26T10:48:07.279 INFO:tasks.workunit.client.0.smithi060.stdout:3/650: mkdir d5/de/df/d4c/dcb 0 2024-02-26T10:48:07.280 INFO:tasks.workunit.client.0.smithi060.stdout:9/958: unlink d3/lb 0 2024-02-26T10:48:07.284 INFO:tasks.workunit.client.0.smithi060.stdout:3/651: mknod d5/d45/d61/d92/da1/daa/dad/ccc 0 2024-02-26T10:48:07.284 INFO:tasks.workunit.client.0.smithi060.stdout:3/652: write d5/de/df/d57/d7a/fb6 [230483,25842] 0 2024-02-26T10:48:07.284 INFO:tasks.workunit.client.0.smithi060.stdout:3/653: fsync d5/de/df/d11/d63/f7f 0 2024-02-26T10:48:07.286 INFO:tasks.workunit.client.0.smithi060.stdout:3/654: unlink d5/d45/d61/d92/da1/fc2 0 2024-02-26T10:48:07.287 INFO:tasks.workunit.client.0.smithi060.stdout:6/902: dread d6/d7/d47/f55 [0,4194304] 0 2024-02-26T10:48:07.287 INFO:tasks.workunit.client.0.smithi060.stdout:6/903: readlink d6/d7/d47/dde/led 0 2024-02-26T10:48:07.287 INFO:tasks.workunit.client.0.smithi060.stdout:3/655: creat d5/d45/d61/d92/da1/daa/fcd x:0 0 0 2024-02-26T10:48:07.287 INFO:tasks.workunit.client.0.smithi060.stdout:3/656: chown d5/de/df/faf 4502 1 2024-02-26T10:48:07.288 INFO:tasks.workunit.client.0.smithi060.stdout:6/904: stat d6/d8d/dcc/da0/cca 0 2024-02-26T10:48:07.288 INFO:tasks.workunit.client.0.smithi060.stdout:6/905: truncate d6/d8d/dcc/dc3/fc6 398875 0 2024-02-26T10:48:07.288 INFO:tasks.workunit.client.0.smithi060.stdout:1/916: link d3/db/d1e/d39/cea d3/d35/d53/d68/c135 0 2024-02-26T10:48:07.289 INFO:tasks.workunit.client.0.smithi060.stdout:8/854: rename d9/d15/d29/d2f/d34/d72/d50/d6a/d63/cda to d9/d5a/d9c/df0/d114/c11d 0 2024-02-26T10:48:07.291 INFO:tasks.workunit.client.0.smithi060.stdout:3/657: mknod d5/d76/cce 0 2024-02-26T10:48:07.292 INFO:tasks.workunit.client.0.smithi060.stdout:6/906: symlink d6/d8d/dcc/dc3/d98/l126 0 2024-02-26T10:48:07.293 INFO:tasks.workunit.client.0.smithi060.stdout:3/658: mkdir d5/dbb/dcf 0 2024-02-26T10:48:07.295 INFO:tasks.workunit.client.0.smithi060.stdout:6/907: mknod d6/d7/d25/c127 0 2024-02-26T10:48:07.295 INFO:tasks.workunit.client.0.smithi060.stdout:6/908: chown d6/d7/d25/d1c/d1f/d23/d2b/lf2 6 1 2024-02-26T10:48:07.299 INFO:tasks.workunit.client.0.smithi060.stdout:9/959: rename d3/d68/l77 to d3/d1e/d5e/d35/d87/l151 0 2024-02-26T10:48:07.302 INFO:tasks.workunit.client.0.smithi060.stdout:2/820: dread dd/d43/d60/d41/d37/d4c/f92 [0,4194304] 0 2024-02-26T10:48:07.302 INFO:tasks.workunit.client.0.smithi060.stdout:2/821: stat dd/l3a 0 2024-02-26T10:48:07.305 INFO:tasks.workunit.client.0.smithi060.stdout:4/841: dwrite d0/d27/d76/dbd/d8e/fe2 [0,4194304] 0 2024-02-26T10:48:07.307 INFO:tasks.workunit.client.0.smithi060.stdout:3/659: dread d5/de/f58 [0,4194304] 0 2024-02-26T10:48:07.312 INFO:tasks.workunit.client.0.smithi060.stdout:6/909: write d6/d7/f53 [116161,16516] 0 2024-02-26T10:48:07.315 INFO:tasks.workunit.client.0.smithi060.stdout:3/660: mkdir d5/dbb/dcf/dd0 0 2024-02-26T10:48:07.317 INFO:tasks.workunit.client.0.smithi060.stdout:8/855: dread d9/d6f/ff3 [0,4194304] 0 2024-02-26T10:48:07.318 INFO:tasks.workunit.client.0.smithi060.stdout:6/910: unlink d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/dac/db5/fce 0 2024-02-26T10:48:07.320 INFO:tasks.workunit.client.0.smithi060.stdout:1/917: rmdir d3/d35/d53 39 2024-02-26T10:48:07.321 INFO:tasks.workunit.client.0.smithi060.stdout:1/918: mkdir d3/db/d1e/d6d/dd1/d136 0 2024-02-26T10:48:07.323 INFO:tasks.workunit.client.0.smithi060.stdout:3/661: dread d5/f27 [0,4194304] 0 2024-02-26T10:48:07.325 INFO:tasks.workunit.client.0.smithi060.stdout:2/822: link dd/d43/d55/d5e/d86/d97/d9c/fea dd/d43/f114 0 2024-02-26T10:48:07.325 INFO:tasks.workunit.client.0.smithi060.stdout:2/823: stat dd/d43/d55/d5e/dc6/c8c 0 2024-02-26T10:48:07.326 INFO:tasks.workunit.client.0.smithi060.stdout:1/919: creat d3/d35/d53/d68/d6b/dc9/f137 x:0 0 0 2024-02-26T10:48:07.327 INFO:tasks.workunit.client.0.smithi060.stdout:2/824: unlink dd/d43/d55/d5e/d74/da5/fa7 0 2024-02-26T10:48:07.327 INFO:tasks.workunit.client.0.smithi060.stdout:2/825: write dd/f68 [2281604,36144] 0 2024-02-26T10:48:07.327 INFO:tasks.workunit.client.0.smithi060.stdout:8/856: link d9/d15/d29/d2f/d34/d72/d50/d6a/d63/fc8 d9/d5a/d9c/df0/f11e 0 2024-02-26T10:48:07.329 INFO:tasks.workunit.client.0.smithi060.stdout:2/826: symlink dd/d43/d55/d6e/l115 0 2024-02-26T10:48:07.330 INFO:tasks.workunit.client.0.smithi060.stdout:8/857: creat d9/d15/d29/d2f/d34/d72/d7e/f11f x:0 0 0 2024-02-26T10:48:07.330 INFO:tasks.workunit.client.0.smithi060.stdout:8/858: rmdir d9/da4/deb 39 2024-02-26T10:48:07.331 INFO:tasks.workunit.client.0.smithi060.stdout:1/920: getdents d3/d35/d53/d68/d6b/dc9 0 2024-02-26T10:48:07.332 INFO:tasks.workunit.client.0.smithi060.stdout:2/827: symlink dd/d43/dd2/d110/l116 0 2024-02-26T10:48:07.334 INFO:tasks.workunit.client.0.smithi060.stdout:2/828: mkdir dd/d43/d60/d41/d37/d69/d117 0 2024-02-26T10:48:07.334 INFO:tasks.workunit.client.0.smithi060.stdout:2/829: chown dd/d43/d55/d5e/d74/fb6 6519490 1 2024-02-26T10:48:07.334 INFO:tasks.workunit.client.0.smithi060.stdout:8/859: unlink d9/d5a/faf 0 2024-02-26T10:48:07.334 INFO:tasks.workunit.client.0.smithi060.stdout:2/830: readlink dd/d43/dd2/d110/l116 0 2024-02-26T10:48:07.335 INFO:tasks.workunit.client.0.smithi060.stdout:3/662: dread d5/d76/f78 [0,4194304] 0 2024-02-26T10:48:07.335 INFO:tasks.workunit.client.0.smithi060.stdout:4/842: rename d0/da/d3b/d1b/c2e to d0/d27/d76/dbd/d8e/db3/d107/c10a 0 2024-02-26T10:48:07.336 INFO:tasks.workunit.client.0.smithi060.stdout:2/831: mkdir dd/d43/d60/d41/d37/dfd/d118 0 2024-02-26T10:48:07.337 INFO:tasks.workunit.client.0.smithi060.stdout:3/663: creat d5/de/df/d11/d63/d93/fd1 x:0 0 0 2024-02-26T10:48:07.337 INFO:tasks.workunit.client.0.smithi060.stdout:1/921: rmdir d3/db 39 2024-02-26T10:48:07.337 INFO:tasks.workunit.client.0.smithi060.stdout:1/922: read - d3/db/d1e/d38/d6c/f113 zero size 2024-02-26T10:48:07.338 INFO:tasks.workunit.client.0.smithi060.stdout:2/832: creat dd/d43/d60/d41/d37/d38/f119 x:0 0 0 2024-02-26T10:48:07.338 INFO:tasks.workunit.client.0.smithi060.stdout:2/833: chown dd/d43/d55/d6e/dd3/l50 1404334969 1 2024-02-26T10:48:07.338 INFO:tasks.workunit.client.0.smithi060.stdout:2/834: chown dd/d43/d55/d6e/dd3/d6f/d8f/dbc 164 1 2024-02-26T10:48:07.340 INFO:tasks.workunit.client.0.smithi060.stdout:6/911: rename d6/d7/d25/d80/df9/dd8/fe3 to d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f128 0 2024-02-26T10:48:07.340 INFO:tasks.workunit.client.0.smithi060.stdout:8/860: symlink d9/d15/d29/d2f/d34/d72/d50/d6a/l120 0 2024-02-26T10:48:07.341 INFO:tasks.workunit.client.0.smithi060.stdout:2/835: creat dd/d43/d96/f11a x:0 0 0 2024-02-26T10:48:07.341 INFO:tasks.workunit.client.0.smithi060.stdout:2/836: chown dd/d43/d55/d6e/dd3/d6f/d8f 4 1 2024-02-26T10:48:07.341 INFO:tasks.workunit.client.0.smithi060.stdout:2/837: fsync dd/d43/d55/d5e/d74/fc3 0 2024-02-26T10:48:07.341 INFO:tasks.workunit.client.0.smithi060.stdout:2/838: dread - dd/d43/d55/d5e/d74/fc3 zero size 2024-02-26T10:48:07.342 INFO:tasks.workunit.client.0.smithi060.stdout:3/664: rename d5/de/c67 to d5/d45/d61/d92/da1/dc1/cd2 0 2024-02-26T10:48:07.342 INFO:tasks.workunit.client.0.smithi060.stdout:3/665: readlink d5/d76/d7d/l9e 0 2024-02-26T10:48:07.342 INFO:tasks.workunit.client.0.smithi060.stdout:3/666: write d5/d37/fc3 [1193929,27893] 0 2024-02-26T10:48:07.342 INFO:tasks.workunit.client.0.smithi060.stdout:2/839: truncate dd/d43/d60/fbd 1142205 0 2024-02-26T10:48:07.342 INFO:tasks.workunit.client.0.smithi060.stdout:4/843: rmdir d0/da/dc7/db0 39 2024-02-26T10:48:07.343 INFO:tasks.workunit.client.0.smithi060.stdout:2/840: mknod dd/d43/d96/df2/d101/c11b 0 2024-02-26T10:48:07.343 INFO:tasks.workunit.client.0.smithi060.stdout:6/912: unlink d6/d7/d47/dde/f114 0 2024-02-26T10:48:07.344 INFO:tasks.workunit.client.0.smithi060.stdout:2/841: creat dd/d43/d96/f11c x:0 0 0 2024-02-26T10:48:07.345 INFO:tasks.workunit.client.0.smithi060.stdout:6/913: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/da7/f129 x:0 0 0 2024-02-26T10:48:07.345 INFO:tasks.workunit.client.0.smithi060.stdout:2/842: unlink dd/d43/d60/c6b 0 2024-02-26T10:48:07.348 INFO:tasks.workunit.client.0.smithi060.stdout:1/923: unlink d3/db/d1e/fb7 0 2024-02-26T10:48:07.349 INFO:tasks.workunit.client.0.smithi060.stdout:2/843: truncate dd/d43/d60/d41/d37/fa0 2869462 0 2024-02-26T10:48:07.349 INFO:tasks.workunit.client.0.smithi060.stdout:4/844: rmdir d0/d27/d76/dbd/d8e/db3 39 2024-02-26T10:48:07.349 INFO:tasks.workunit.client.0.smithi060.stdout:6/914: mknod d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/c12a 0 2024-02-26T10:48:07.351 INFO:tasks.workunit.client.0.smithi060.stdout:4/845: getdents d0/d27/d76/dbd 0 2024-02-26T10:48:07.351 INFO:tasks.workunit.client.0.smithi060.stdout:4/846: read d0/da/d3b/f34 [2519594,71002] 0 2024-02-26T10:48:07.351 INFO:tasks.workunit.client.0.smithi060.stdout:1/924: rmdir d3/db/d1e/d39/d9b 39 2024-02-26T10:48:07.351 INFO:tasks.workunit.client.0.smithi060.stdout:4/847: chown d0/d27/d76/dbd/d60/fb6 3 1 2024-02-26T10:48:07.351 INFO:tasks.workunit.client.0.smithi060.stdout:4/848: truncate d0/d27/d76/dbd/d8e/db3/d107/ff1 806253 0 2024-02-26T10:48:07.353 INFO:tasks.workunit.client.0.smithi060.stdout:6/915: rename d6/f106 to d6/d7/d25/d80/df9/dd8/f12b 0 2024-02-26T10:48:07.355 INFO:tasks.workunit.client.0.smithi060.stdout:9/960: dwrite d3/d1e/d5e/d1b/f41 [0,4194304] 0 2024-02-26T10:48:07.357 INFO:tasks.workunit.client.0.smithi060.stdout:1/925: mknod d3/d35/c138 0 2024-02-26T10:48:07.358 INFO:tasks.workunit.client.0.smithi060.stdout:4/849: symlink d0/d27/d76/dbd/d8e/l10b 0 2024-02-26T10:48:07.358 INFO:tasks.workunit.client.0.smithi060.stdout:4/850: write d0/da/fe0 [613136,5654] 0 2024-02-26T10:48:07.360 INFO:tasks.workunit.client.0.smithi060.stdout:6/916: rename d6/d6e/d6f/lc8 to d6/d7/d47/def/l12c 0 2024-02-26T10:48:07.360 INFO:tasks.workunit.client.0.smithi060.stdout:6/917: fsync d6/f7c 0 2024-02-26T10:48:07.361 INFO:tasks.workunit.client.0.smithi060.stdout:1/926: creat d3/d54/dc0/f139 x:0 0 0 2024-02-26T10:48:07.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/851: mknod d0/c10c 0 2024-02-26T10:48:07.361 INFO:tasks.workunit.client.0.smithi060.stdout:4/852: stat d0/da/d3b/d1b/lf8 0 2024-02-26T10:48:07.362 INFO:tasks.workunit.client.0.smithi060.stdout:9/961: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/f152 x:0 0 0 2024-02-26T10:48:07.364 INFO:tasks.workunit.client.0.smithi060.stdout:4/853: rename d0/da/dc7/db0/db5/dc0/f3e to d0/d27/d76/d9f/f10d 0 2024-02-26T10:48:07.364 INFO:tasks.workunit.client.0.smithi060.stdout:4/854: stat d0/da/d3b/db9 0 2024-02-26T10:48:07.364 INFO:tasks.workunit.client.0.smithi060.stdout:9/962: creat d3/d1e/dbb/df5/f153 x:0 0 0 2024-02-26T10:48:07.376 INFO:tasks.workunit.client.0.smithi060.stdout:9/963: creat d3/d1e/d5e/d1b/f154 x:0 0 0 2024-02-26T10:48:07.376 INFO:tasks.workunit.client.0.smithi060.stdout:2/844: write dd/d43/d60/d41/f36 [1929359,25409] 0 2024-02-26T10:48:07.376 INFO:tasks.workunit.client.0.smithi060.stdout:2/845: write f4 [8863839,74821] 0 2024-02-26T10:48:07.377 INFO:tasks.workunit.client.0.smithi060.stdout:9/964: mknod d3/d1e/dbb/dc2/df8/c155 0 2024-02-26T10:48:07.378 INFO:tasks.workunit.client.0.smithi060.stdout:2/846: symlink dd/d43/d55/d5e/d74/da5/def/l11d 0 2024-02-26T10:48:07.378 INFO:tasks.workunit.client.0.smithi060.stdout:9/965: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/f156 x:0 0 0 2024-02-26T10:48:07.390 INFO:tasks.workunit.client.0.smithi060.stdout:1/927: dread d3/d35/d53/f74 [0,4194304] 0 2024-02-26T10:48:07.390 INFO:tasks.workunit.client.0.smithi060.stdout:9/966: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/dcb/fcc [0,4194304] 0 2024-02-26T10:48:07.390 INFO:tasks.workunit.client.0.smithi060.stdout:3/667: dread d5/de/df/d4c/d68/f55 [0,4194304] 0 2024-02-26T10:48:07.390 INFO:tasks.workunit.client.0.smithi060.stdout:3/668: stat d5 0 2024-02-26T10:48:07.390 INFO:tasks.workunit.client.0.smithi060.stdout:3/669: chown d5/de/d16/l40 58 1 2024-02-26T10:48:07.392 INFO:tasks.workunit.client.0.smithi060.stdout:8/861: dwrite d9/d15/d29/d2f/d34/d72/d7e/f11f [0,4194304] 0 2024-02-26T10:48:07.394 INFO:tasks.workunit.client.0.smithi060.stdout:3/670: symlink d5/d45/d61/d92/da1/dc1/ld3 0 2024-02-26T10:48:07.394 INFO:tasks.workunit.client.0.smithi060.stdout:3/671: write d5/de/f3b [1556114,110926] 0 2024-02-26T10:48:07.396 INFO:tasks.workunit.client.0.smithi060.stdout:3/672: mkdir d5/de/df/d11/d63/d93/d97/d98/dd4 0 2024-02-26T10:48:07.409 INFO:tasks.workunit.client.0.smithi060.stdout:6/918: dread d6/d7/d25/d1c/d1f/d23/d2b/d67/f8a [0,4194304] 0 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:8/862: symlink d9/d15/d29/d2f/d34/d72/d94/dba/d119/l121 0 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:8/863: dread - d9/d6f/fe9 zero size 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:3/673: dread d5/de/df/d11/d63/f7f [0,4194304] 0 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:0/810: sync 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:5/818: sync 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:5/819: link d0/de/d15/d65/d86/cc1 d0/de/d15/d65/d86/c119 0 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:2/847: dread dd/d43/d60/f1e [0,4194304] 0 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:2/848: chown dd/d43/d55/f8e 73 1 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:2/849: truncate dd/d43/d55/d5e/dc6/f108 572058 0 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:2/850: chown dd/d43/d60/d41/d37/d4c/f92 1547 1 2024-02-26T10:48:07.410 INFO:tasks.workunit.client.0.smithi060.stdout:3/674: mknod d5/de/df/d11/d63/d93/d97/cd5 0 2024-02-26T10:48:07.411 INFO:tasks.workunit.client.0.smithi060.stdout:4/855: write d0/da/d57/fe5 [838146,77121] 0 2024-02-26T10:48:07.412 INFO:tasks.workunit.client.0.smithi060.stdout:2/851: rename dd/d43/d55/d5e/dc6/c8c to dd/c11e 0 2024-02-26T10:48:07.413 INFO:tasks.workunit.client.0.smithi060.stdout:3/675: unlink d5/d37/fc3 0 2024-02-26T10:48:07.414 INFO:tasks.workunit.client.0.smithi060.stdout:2/852: truncate dd/d43/d60/d41/f6d 2295806 0 2024-02-26T10:48:07.414 INFO:tasks.workunit.client.0.smithi060.stdout:3/676: creat d5/d45/fd6 x:0 0 0 2024-02-26T10:48:07.420 INFO:tasks.workunit.client.0.smithi060.stdout:9/967: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3a/fdc [0,4194304] 0 2024-02-26T10:48:07.420 INFO:tasks.workunit.client.0.smithi060.stdout:9/968: dread - d3/d1e/d5e/d35/d87/f123 zero size 2024-02-26T10:48:07.421 INFO:tasks.workunit.client.0.smithi060.stdout:5/820: link d0/de/d15/d26/ddd/cea d0/de/d70/d84/d92/c11a 0 2024-02-26T10:48:07.422 INFO:tasks.workunit.client.0.smithi060.stdout:9/969: read - d3/d1e/d5e/d35/fd7 zero size 2024-02-26T10:48:07.423 INFO:tasks.workunit.client.0.smithi060.stdout:9/970: symlink d3/d1e/d5e/d1b/d11a/l157 0 2024-02-26T10:48:07.424 INFO:tasks.workunit.client.0.smithi060.stdout:9/971: readlink d3/d1e/d5e/d1b/d11a/l139 0 2024-02-26T10:48:07.424 INFO:tasks.workunit.client.0.smithi060.stdout:5/821: truncate d0/de/d15/d65/f7c 2786873 0 2024-02-26T10:48:07.426 INFO:tasks.workunit.client.0.smithi060.stdout:8/864: dread d9/d6f/fd8 [0,4194304] 0 2024-02-26T10:48:07.440 INFO:tasks.workunit.client.0.smithi060.stdout:4/856: rename d0/da/d3b/dca to d0/d27/d76/dbd/d10e 0 2024-02-26T10:48:07.443 INFO:tasks.workunit.client.0.smithi060.stdout:4/857: truncate d0/d27/d76/dbd/d8e/fac 3651055 0 2024-02-26T10:48:07.445 INFO:tasks.workunit.client.0.smithi060.stdout:4/858: read d0/d27/d76/dbd/f38 [65698,80231] 0 2024-02-26T10:48:07.445 INFO:tasks.workunit.client.0.smithi060.stdout:4/859: readlink d0/d27/d76/dbd/lcf 0 2024-02-26T10:48:07.449 INFO:tasks.workunit.client.0.smithi060.stdout:3/677: dread d5/de/df/d57/f94 [0,4194304] 0 2024-02-26T10:48:07.449 INFO:tasks.workunit.client.0.smithi060.stdout:3/678: stat d5/d76/d7d/l9e 0 2024-02-26T10:48:07.453 INFO:tasks.workunit.client.0.smithi060.stdout:8/865: dread d9/d5a/f69 [0,4194304] 0 2024-02-26T10:48:07.455 INFO:tasks.workunit.client.0.smithi060.stdout:0/811: write d0/dd/d53/f64 [5101538,64249] 0 2024-02-26T10:48:07.456 INFO:tasks.workunit.client.0.smithi060.stdout:3/679: fsync d5/de/df/d57/d7a/f8b 0 2024-02-26T10:48:07.461 INFO:tasks.workunit.client.0.smithi060.stdout:8/866: creat d9/da4/f122 x:0 0 0 2024-02-26T10:48:07.461 INFO:tasks.workunit.client.0.smithi060.stdout:8/867: stat d9/d5a/f7f 0 2024-02-26T10:48:07.461 INFO:tasks.workunit.client.0.smithi060.stdout:0/812: creat d0/dd/d39/d40/dad/d10d/f110 x:0 0 0 2024-02-26T10:48:07.467 INFO:tasks.workunit.client.0.smithi060.stdout:6/919: dwrite d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f128 [0,4194304] 0 2024-02-26T10:48:07.467 INFO:tasks.workunit.client.0.smithi060.stdout:6/920: fsync d6/d7/d2e/d56/f11f 0 2024-02-26T10:48:07.469 INFO:tasks.workunit.client.0.smithi060.stdout:5/822: rename d0/de/d20/d32/d4d/da3/da5/dc8/ffa to d0/de/d20/d82/f11b 0 2024-02-26T10:48:07.469 INFO:tasks.workunit.client.0.smithi060.stdout:5/823: dread - d0/de/d20/d82/fe8 zero size 2024-02-26T10:48:07.470 INFO:tasks.workunit.client.0.smithi060.stdout:6/921: chown d6/d7/d25/d1c/d1f/d23/d28/l66 3944830 1 2024-02-26T10:48:07.471 INFO:tasks.workunit.client.0.smithi060.stdout:6/922: creat d6/d7/d2e/d56/d73/d8c/f12d x:0 0 0 2024-02-26T10:48:07.471 INFO:tasks.workunit.client.0.smithi060.stdout:6/923: truncate d6/d8d/dcc/dc3/d98/fe1 1924535 0 2024-02-26T10:48:07.471 INFO:tasks.workunit.client.0.smithi060.stdout:6/924: fsync d6/d7/d47/f9b 0 2024-02-26T10:48:07.473 INFO:tasks.workunit.client.0.smithi060.stdout:6/925: creat d6/d7/d25/f12e x:0 0 0 2024-02-26T10:48:07.477 INFO:tasks.workunit.client.0.smithi060.stdout:6/926: creat d6/d8d/dcc/dc3/d98/d119/f12f x:0 0 0 2024-02-26T10:48:07.478 INFO:tasks.workunit.client.0.smithi060.stdout:6/927: dread - d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/da7/f129 zero size 2024-02-26T10:48:07.478 INFO:tasks.workunit.client.0.smithi060.stdout:6/928: mkdir d6/d7/d25/d130 0 2024-02-26T10:48:07.495 INFO:tasks.workunit.client.0.smithi060.stdout:0/813: dread d0/dd/d39/d40/f19 [0,4194304] 0 2024-02-26T10:48:07.496 INFO:tasks.workunit.client.0.smithi060.stdout:8/868: dwrite d9/d15/d3e/d10b/f110 [0,4194304] 0 2024-02-26T10:48:07.496 INFO:tasks.workunit.client.0.smithi060.stdout:8/869: dread - d9/d15/d29/d2f/d34/ff7 zero size 2024-02-26T10:48:07.497 INFO:tasks.workunit.client.0.smithi060.stdout:8/870: creat d9/da4/f123 x:0 0 0 2024-02-26T10:48:07.498 INFO:tasks.workunit.client.0.smithi060.stdout:8/871: rmdir d9/d15/d29/d2f/d34/d72/d94/dba 39 2024-02-26T10:48:07.499 INFO:tasks.workunit.client.0.smithi060.stdout:4/860: dwrite d0/f98 [0,4194304] 0 2024-02-26T10:48:07.500 INFO:tasks.workunit.client.0.smithi060.stdout:8/872: getdents d9/d15/d3e/d10b 0 2024-02-26T10:48:07.503 INFO:tasks.workunit.client.0.smithi060.stdout:8/873: unlink d9/d15/d3e/fb6 0 2024-02-26T10:48:07.504 INFO:tasks.workunit.client.0.smithi060.stdout:4/861: creat d0/da/d3b/ddc/dfa/dfe/f10f x:0 0 0 2024-02-26T10:48:07.507 INFO:tasks.workunit.client.0.smithi060.stdout:4/862: truncate d0/da/d3b/d1b/f86 2341399 0 2024-02-26T10:48:07.512 INFO:tasks.workunit.client.0.smithi060.stdout:4/863: rename d0/da/d3b/ddc/dfa/f55 to d0/da/d3b/d1b/f110 0 2024-02-26T10:48:07.513 INFO:tasks.workunit.client.0.smithi060.stdout:4/864: readlink d0/d27/d76/le4 0 2024-02-26T10:48:07.514 INFO:tasks.workunit.client.0.smithi060.stdout:4/865: mkdir d0/da/dc7/db0/d111 0 2024-02-26T10:48:07.516 INFO:tasks.workunit.client.0.smithi060.stdout:4/866: creat d0/d27/d76/dbd/d8e/f112 x:0 0 0 2024-02-26T10:48:07.516 INFO:tasks.workunit.client.0.smithi060.stdout:5/824: dwrite d0/de/d15/d65/d86/dcc/fd5 [0,4194304] 0 2024-02-26T10:48:07.518 INFO:tasks.workunit.client.0.smithi060.stdout:4/867: creat d0/d27/d76/dbd/d8e/db3/f113 x:0 0 0 2024-02-26T10:48:07.518 INFO:tasks.workunit.client.0.smithi060.stdout:6/929: dread d6/d7/d2e/d56/f61 [0,4194304] 0 2024-02-26T10:48:07.522 INFO:tasks.workunit.client.0.smithi060.stdout:5/825: symlink d0/de/d20/d32/d4d/d110/l11c 0 2024-02-26T10:48:07.522 INFO:tasks.workunit.client.0.smithi060.stdout:5/826: dread - d0/de/d20/d32/d4c/fb7 zero size 2024-02-26T10:48:07.522 INFO:tasks.workunit.client.0.smithi060.stdout:4/868: mkdir d0/d27/d76/d9f/d114 0 2024-02-26T10:48:07.522 INFO:tasks.workunit.client.0.smithi060.stdout:6/930: creat d6/d7/d25/d1c/d1f/f131 x:0 0 0 2024-02-26T10:48:07.522 INFO:tasks.workunit.client.0.smithi060.stdout:6/931: stat d6/d7/d25/f12e 0 2024-02-26T10:48:07.523 INFO:tasks.workunit.client.0.smithi060.stdout:6/932: chown d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/fd1 575465 1 2024-02-26T10:48:07.523 INFO:tasks.workunit.client.0.smithi060.stdout:4/869: unlink d0/d27/d76/dbd/d8e/db3/f113 0 2024-02-26T10:48:07.523 INFO:tasks.workunit.client.0.smithi060.stdout:6/933: creat d6/d7/f132 x:0 0 0 2024-02-26T10:48:07.523 INFO:tasks.workunit.client.0.smithi060.stdout:4/870: mkdir d0/da/dc7/d115 0 2024-02-26T10:48:07.523 INFO:tasks.workunit.client.0.smithi060.stdout:6/934: symlink d6/d7/d47/d69/l133 0 2024-02-26T10:48:07.529 INFO:tasks.workunit.client.0.smithi060.stdout:3/680: dwrite d5/de/df/f21 [0,4194304] 0 2024-02-26T10:48:07.540 INFO:tasks.workunit.client.0.smithi060.stdout:0/814: dwrite d0/dd/d53/d87/d94/f105 [0,4194304] 0 2024-02-26T10:48:07.540 INFO:tasks.workunit.client.0.smithi060.stdout:0/815: chown d0/dd/d39/d40/d83/dc6/f79 138 1 2024-02-26T10:48:07.544 INFO:tasks.workunit.client.0.smithi060.stdout:8/874: dwrite d9/d15/f113 [0,4194304] 0 2024-02-26T10:48:07.544 INFO:tasks.workunit.client.0.smithi060.stdout:8/875: chown d9/d15/d29/d2f/d34/d72/d50/d6a/l120 66957473 1 2024-02-26T10:48:07.547 INFO:tasks.workunit.client.0.smithi060.stdout:3/681: rmdir d5/de/df/d11/d63/d93/da2 39 2024-02-26T10:48:07.547 INFO:tasks.workunit.client.0.smithi060.stdout:2/853: sync 2024-02-26T10:48:07.548 INFO:tasks.workunit.client.0.smithi060.stdout:1/928: sync 2024-02-26T10:48:07.548 INFO:tasks.workunit.client.0.smithi060.stdout:9/972: sync 2024-02-26T10:48:07.548 INFO:tasks.workunit.client.0.smithi060.stdout:9/973: stat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/d98/dd4 0 2024-02-26T10:48:07.548 INFO:tasks.workunit.client.0.smithi060.stdout:9/974: read - d3/d68/f100 zero size 2024-02-26T10:48:07.548 INFO:tasks.workunit.client.0.smithi060.stdout:9/975: fdatasync d3/d1e/d5e/f5c 0 2024-02-26T10:48:07.548 INFO:tasks.workunit.client.0.smithi060.stdout:9/976: stat d3/d1e/d5e/d1b/f9d 0 2024-02-26T10:48:07.549 INFO:tasks.workunit.client.0.smithi060.stdout:3/682: unlink d5/d45/l69 0 2024-02-26T10:48:07.549 INFO:tasks.workunit.client.0.smithi060.stdout:3/683: fdatasync d5/d45/d61/d92/da1/daa/fb3 0 2024-02-26T10:48:07.549 INFO:tasks.workunit.client.0.smithi060.stdout:1/929: creat d3/d35/f13a x:0 0 0 2024-02-26T10:48:07.549 INFO:tasks.workunit.client.0.smithi060.stdout:1/930: dread - d3/d35/d53/ff8 zero size 2024-02-26T10:48:07.551 INFO:tasks.workunit.client.0.smithi060.stdout:3/684: mkdir d5/de/dd7 0 2024-02-26T10:48:07.552 INFO:tasks.workunit.client.0.smithi060.stdout:5/827: dread d0/de/f29 [0,4194304] 0 2024-02-26T10:48:07.552 INFO:tasks.workunit.client.0.smithi060.stdout:5/828: readlink d0/de/d70/lfe 0 2024-02-26T10:48:07.553 INFO:tasks.workunit.client.0.smithi060.stdout:3/685: mknod d5/de/df/d11/d63/d93/cd8 0 2024-02-26T10:48:07.554 INFO:tasks.workunit.client.0.smithi060.stdout:3/686: mknod d5/de/df/d4c/dcb/cd9 0 2024-02-26T10:48:07.556 INFO:tasks.workunit.client.0.smithi060.stdout:9/977: link d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/f65 d3/d68/d101/f158 0 2024-02-26T10:48:07.557 INFO:tasks.workunit.client.0.smithi060.stdout:3/687: symlink d5/de/df/d11/d63/d93/d97/lda 0 2024-02-26T10:48:07.566 INFO:tasks.workunit.client.0.smithi060.stdout:9/978: dread d3/d1e/d5e/d35/d87/d9b/d8a/dbe/fb6 [0,4194304] 0 2024-02-26T10:48:07.566 INFO:tasks.workunit.client.0.smithi060.stdout:9/979: stat d3/d1e/d5e/d1b/d23/fd5 0 2024-02-26T10:48:07.566 INFO:tasks.workunit.client.0.smithi060.stdout:9/980: write d3/d68/f132 [790173,121413] 0 2024-02-26T10:48:07.568 INFO:tasks.workunit.client.0.smithi060.stdout:9/981: creat d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d74/f159 x:0 0 0 2024-02-26T10:48:07.568 INFO:tasks.workunit.client.0.smithi060.stdout:9/982: chown d3/d1e/d5e/d1b/d23/c12d 5380350 1 2024-02-26T10:48:07.570 INFO:tasks.workunit.client.0.smithi060.stdout:9/983: mknod d3/d1e/dbb/df5/c15a 0 2024-02-26T10:48:07.572 INFO:tasks.workunit.client.0.smithi060.stdout:9/984: creat d3/d1e/d5e/dc9/f15b x:0 0 0 2024-02-26T10:48:07.572 INFO:tasks.workunit.client.0.smithi060.stdout:9/985: truncate d3/d1e/dbb/df5/f153 970340 0 2024-02-26T10:48:07.572 INFO:tasks.workunit.client.0.smithi060.stdout:9/986: write d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d32/d3c/de9/f146 [827760,69266] 0 2024-02-26T10:48:07.572 INFO:tasks.workunit.client.0.smithi060.stdout:9/987: write d3/f16 [1404781,39381] 0 2024-02-26T10:48:07.574 INFO:tasks.workunit.client.0.smithi060.stdout:9/988: mknod d3/d1e/d57/c15c 0 2024-02-26T10:48:07.574 INFO:tasks.workunit.client.0.smithi060.stdout:9/989: chown d3/d1e/d5e/d35/fd7 2143 1 2024-02-26T10:48:07.575 INFO:tasks.workunit.client.0.smithi060.stdout:9/990: chown d3/d1e/d93/da2/db7/cd3 8608 1 2024-02-26T10:48:07.575 INFO:tasks.workunit.client.0.smithi060.stdout:9/991: chown d3/d1e/d5e/d35/d87/d9b/d8a 5 1 2024-02-26T10:48:07.575 INFO:tasks.workunit.client.0.smithi060.stdout:1/931: rename d3/d35/d53/fd2 to d3/db/d2e/f13b 0 2024-02-26T10:48:07.575 INFO:tasks.workunit.client.0.smithi060.stdout:1/932: chown d3/d35/d53/d68/d90/cf5 17319220 1 2024-02-26T10:48:07.576 INFO:tasks.workunit.client.0.smithi060.stdout:1/933: fsync d3/d35/d53/d57/f81 0 2024-02-26T10:48:07.577 INFO:tasks.workunit.client.0.smithi060.stdout:3/688: rename d5/c9f to d5/d76/d7d/cdb 0 2024-02-26T10:48:07.579 INFO:tasks.workunit.client.0.smithi060.stdout:8/876: dread d9/d15/d29/d2f/d34/d72/d94/f9e [0,4194304] 0 2024-02-26T10:48:07.579 INFO:tasks.workunit.client.0.smithi060.stdout:8/877: fsync d9/d15/d29/d2f/d34/d72/d50/d6a/f45 0 2024-02-26T10:48:07.581 INFO:tasks.workunit.client.0.smithi060.stdout:8/878: mkdir d9/d5a/d9c/df0/d124 0 2024-02-26T10:48:07.581 INFO:tasks.workunit.client.0.smithi060.stdout:5/829: write d0/f81 [197977,126603] 0 2024-02-26T10:48:07.581 INFO:tasks.workunit.client.0.smithi060.stdout:5/830: dread - d0/de/fca zero size 2024-02-26T10:48:07.582 INFO:tasks.workunit.client.0.smithi060.stdout:8/879: creat d9/d5a/d9c/df0/d124/f125 x:0 0 0 2024-02-26T10:48:07.582 INFO:tasks.workunit.client.0.smithi060.stdout:3/689: creat d5/d45/d61/d92/fdc x:0 0 0 2024-02-26T10:48:07.582 INFO:tasks.workunit.client.0.smithi060.stdout:3/690: dread - d5/de/df/d11/d63/fbd zero size 2024-02-26T10:48:07.585 INFO:tasks.workunit.client.0.smithi060.stdout:5/831: mknod d0/de/d15/d26/ddd/c11d 0 2024-02-26T10:48:07.585 INFO:tasks.workunit.client.0.smithi060.stdout:8/880: creat d9/da4/deb/f126 x:0 0 0 2024-02-26T10:48:07.585 INFO:tasks.workunit.client.0.smithi060.stdout:3/691: creat d5/de/df/d4c/d68/fdd x:0 0 0 2024-02-26T10:48:07.585 INFO:tasks.workunit.client.0.smithi060.stdout:8/881: chown d9/d15/d29/d2f/d34/d72/d50/d6a 12753006 1 2024-02-26T10:48:07.586 INFO:tasks.workunit.client.0.smithi060.stdout:5/832: chown d0/de/d70/d84/d92/cc5 1460 1 2024-02-26T10:48:07.586 INFO:tasks.workunit.client.0.smithi060.stdout:3/692: mknod d5/de/cde 0 2024-02-26T10:48:07.589 INFO:tasks.workunit.client.0.smithi060.stdout:0/816: dwrite d0/dd/fbb [0,4194304] 0 2024-02-26T10:48:07.589 INFO:tasks.workunit.client.0.smithi060.stdout:0/817: read d0/dd/d39/f54 [3986873,7275] 0 2024-02-26T10:48:07.589 INFO:tasks.workunit.client.0.smithi060.stdout:0/818: chown d0/cc 749834 1 2024-02-26T10:48:07.589 INFO:tasks.workunit.client.0.smithi060.stdout:3/693: rmdir d5/d76 39 2024-02-26T10:48:07.601 INFO:tasks.workunit.client.0.smithi060.stdout:9/992: write d3/d1e/d93/da2/fb3 [4090267,94628] 0 2024-02-26T10:48:07.605 INFO:tasks.workunit.client.0.smithi060.stdout:9/993: creat d3/d1e/d5e/d35/d87/f15d x:0 0 0 2024-02-26T10:48:07.607 INFO:tasks.workunit.client.0.smithi060.stdout:2/854: dwrite dd/d43/d60/d41/d37/d69/f7e [0,4194304] 0 2024-02-26T10:48:07.607 INFO:tasks.workunit.client.0.smithi060.stdout:2/855: readlink dd/d43/d96/laf 0 2024-02-26T10:48:07.607 INFO:tasks.workunit.client.0.smithi060.stdout:2/856: chown dd/d43/d96/df2/lf4 14694 1 2024-02-26T10:48:07.607 INFO:tasks.workunit.client.0.smithi060.stdout:1/934: dread d3/d35/d53/d131/fb8 [0,4194304] 0 2024-02-26T10:48:07.608 INFO:tasks.workunit.client.0.smithi060.stdout:5/833: link d0/de/d15/d65/d86/cc1 d0/de/d20/d32/d4d/c11e 0 2024-02-26T10:48:07.608 INFO:tasks.workunit.client.0.smithi060.stdout:5/834: write d0/d33/f62 [55965,47063] 0 2024-02-26T10:48:07.611 INFO:tasks.workunit.client.0.smithi060.stdout:5/835: rename d0/de/d20/d32/d4d/da3/da5/dc8/fd6 to d0/de/d20/d32/d4d/da3/da5/dc8/f11f 0 2024-02-26T10:48:07.613 INFO:tasks.workunit.client.0.smithi060.stdout:5/836: mknod d0/de/d20/d32/d4d/da3/c120 0 2024-02-26T10:48:07.613 INFO:tasks.workunit.client.0.smithi060.stdout:5/837: truncate d0/de/d20/d82/f11b 29812 0 2024-02-26T10:48:07.615 INFO:tasks.workunit.client.0.smithi060.stdout:5/838: creat d0/de/d20/d32/d4d/da3/da5/dc8/f121 x:0 0 0 2024-02-26T10:48:07.619 INFO:tasks.workunit.client.0.smithi060.stdout:5/839: symlink d0/de/d20/d32/d4d/da3/da5/l122 0 2024-02-26T10:48:07.621 INFO:tasks.workunit.client.0.smithi060.stdout:1/935: write d3/d35/d5e/f66 [7234474,5576] 0 2024-02-26T10:48:07.623 INFO:tasks.workunit.client.0.smithi060.stdout:1/936: mknod d3/c13c 0 2024-02-26T10:48:07.627 INFO:tasks.workunit.client.0.smithi060.stdout:1/937: mkdir d3/db/d2e/d51/dbb/d13d 0 2024-02-26T10:48:07.627 INFO:tasks.workunit.client.0.smithi060.stdout:1/938: write d3/d35/d53/f12d [535245,1272] 0 2024-02-26T10:48:07.627 INFO:tasks.workunit.client.0.smithi060.stdout:1/939: fdatasync d3/d35/d53/d57/f87 0 2024-02-26T10:48:07.633 INFO:tasks.workunit.client.0.smithi060.stdout:9/994: link d3/d1e/d5e/d35/d87/d9b/d8a/dbe/d10/f7e d3/d1e/d93/f15e 0 2024-02-26T10:48:07.635 INFO:tasks.workunit.client.0.smithi060.stdout:8/882: dwrite d9/d15/d29/f7d [0,4194304] 0 2024-02-26T10:48:07.635 INFO:tasks.workunit.client.0.smithi060.stdout:8/883: rmdir d9/d15/d3e/d10b 39 2024-02-26T10:48:07.638 INFO:tasks.workunit.client.0.smithi060.stdout:3/694: dwrite d5/d45/f8f [0,4194304] 0 2024-02-26T10:48:07.638 INFO:tasks.workunit.client.0.smithi060.stdout:8/884: mkdir d9/d15/d3e/d127 0 2024-02-26T10:48:07.639 INFO:tasks.workunit.client.0.smithi060.stdout:3/695: truncate d5/d45/d61/d92/da1/daa/f86 1637822 0 2024-02-26T10:48:07.640 INFO:tasks.workunit.client.0.smithi060.stdout:9/995: unlink d3/fdf 0 2024-02-26T10:48:07.641 INFO:tasks.workunit.client.0.smithi060.stdout:3/696: link d5/de/df/d11/d63/d93/d97/lda d5/d45/d61/d92/da1/daa/ldf 0 2024-02-26T10:48:07.642 INFO:tasks.workunit.client.0.smithi060.stdout:9/996: creat d3/d1e/d5e/d9e/f15f x:0 0 0 2024-02-26T10:48:07.642 INFO:tasks.workunit.client.0.smithi060.stdout:5/840: read d0/d33/f5f [3437180,2903] 0 2024-02-26T10:48:07.643 INFO:tasks.workunit.client.0.smithi060.stdout:9/997: mknod d3/d1e/dbb/c160 0 2024-02-26T10:48:07.643 INFO:tasks.workunit.client.0.smithi060.stdout:9/998: dread - d3/d1e/d5e/d35/d87/d9b/d8a/dbe/dc/d11e/d10c/d116/f120 zero size 2024-02-26T10:48:07.644 INFO:tasks.workunit.client.0.smithi060.stdout:9/999: fsync d3/d1e/d5e/d35/d87/d9b/d8a/dbe/f2f 0 2024-02-26T10:48:07.644 INFO:tasks.workunit.client.0.smithi060.stdout:8/885: symlink d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/l128 0 2024-02-26T10:48:07.644 INFO:tasks.workunit.client.0.smithi060.stdout:8/886: chown d9/fc 109434 1 2024-02-26T10:48:07.644 INFO:tasks.workunit.client.0.smithi060.stdout:3/697: mkdir d5/de/df/d11/de0 0 2024-02-26T10:48:07.645 INFO:tasks.workunit.client.0.smithi060.stdout:8/887: creat d9/d15/d29/d2f/d34/d72/d50/d6a/dc0/f129 x:0 0 0 2024-02-26T10:48:07.645 INFO:tasks.workunit.client.0.smithi060.stdout:8/888: dread - d9/d5a/f7f zero size 2024-02-26T10:48:07.645 INFO:tasks.workunit.client.0.smithi060.stdout:5/841: rmdir d0/de/d15/d65/d86/d9b/d113 39 2024-02-26T10:48:07.645 INFO:tasks.workunit.client.0.smithi060.stdout:3/698: creat d5/d45/d61/d92/da1/dc8/fe1 x:0 0 0 2024-02-26T10:48:07.645 INFO:tasks.workunit.client.0.smithi060.stdout:8/889: truncate d9/fd 1367237 0 2024-02-26T10:48:07.645 INFO:tasks.workunit.client.0.smithi060.stdout:3/699: chown d5/de/df/l2f 33605 1 2024-02-26T10:48:07.646 INFO:tasks.workunit.client.0.smithi060.stdout:8/890: creat d9/da4/deb/f12a x:0 0 0 2024-02-26T10:48:07.647 INFO:tasks.workunit.client.0.smithi060.stdout:3/700: creat d5/d37/fe2 x:0 0 0 2024-02-26T10:48:07.648 INFO:tasks.workunit.client.0.smithi060.stdout:8/891: getdents d9/d15/d29/d2f/d34/d72/d94/dd4 0 2024-02-26T10:48:07.648 INFO:tasks.workunit.client.0.smithi060.stdout:8/892: fsync d9/d15/f60 0 2024-02-26T10:48:07.653 INFO:tasks.workunit.client.0.smithi060.stdout:3/701: link d5/de/df/d11/d63/d93/f96 d5/de/df/fe3 0 2024-02-26T10:48:07.654 INFO:tasks.workunit.client.0.smithi060.stdout:3/702: mknod d5/de/ce4 0 2024-02-26T10:48:07.655 INFO:tasks.workunit.client.0.smithi060.stdout:3/703: rename f3 to d5/de/df/d57/fe5 0 2024-02-26T10:48:07.657 INFO:tasks.workunit.client.0.smithi060.stdout:3/704: creat d5/d37/fe6 x:0 0 0 2024-02-26T10:48:07.658 INFO:tasks.workunit.client.0.smithi060.stdout:3/705: mknod d5/de/df/d4c/dcb/ce7 0 2024-02-26T10:48:07.658 INFO:tasks.workunit.client.0.smithi060.stdout:3/706: write d5/f27 [1765671,86143] 0 2024-02-26T10:48:07.658 INFO:tasks.workunit.client.0.smithi060.stdout:3/707: creat d5/de/fe8 x:0 0 0 2024-02-26T10:48:07.658 INFO:tasks.workunit.client.0.smithi060.stdout:3/708: write d5/d45/f48 [356654,28466] 0 2024-02-26T10:48:07.683 INFO:tasks.workunit.client.0.smithi060.stdout:8/893: dwrite f8 [0,4194304] 0 2024-02-26T10:48:07.685 INFO:tasks.workunit.client.0.smithi060.stdout:8/894: creat d9/d15/d29/d2f/d34/d72/d94/dba/f12b x:0 0 0 2024-02-26T10:48:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:1/940: dwrite d3/db/d1e/d38/fe2 [0,4194304] 0 2024-02-26T10:48:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:5/842: dread d0/d4b/f51 [0,4194304] 0 2024-02-26T10:48:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:5/843: creat d0/de/d20/d32/d4d/d110/f123 x:0 0 0 2024-02-26T10:48:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:3/709: dwrite d5/de/df/f33 [0,4194304] 0 2024-02-26T10:48:07.698 INFO:tasks.workunit.client.0.smithi060.stdout:3/710: symlink d5/de/df/d4c/le9 0 2024-02-26T10:48:07.700 INFO:tasks.workunit.client.0.smithi060.stdout:3/711: mkdir d5/de/df/d57/dea 0 2024-02-26T10:48:07.701 INFO:tasks.workunit.client.0.smithi060.stdout:5/844: dread d0/de/d20/d32/d4c/fe6 [0,4194304] 0 2024-02-26T10:48:07.701 INFO:tasks.workunit.client.0.smithi060.stdout:5/845: dread - d0/de/d20/d32/d4d/da3/da5/dc8/f108 zero size 2024-02-26T10:48:07.702 INFO:tasks.workunit.client.0.smithi060.stdout:5/846: symlink d0/de/l124 0 2024-02-26T10:48:07.702 INFO:tasks.workunit.client.0.smithi060.stdout:3/712: link d5/d76/d7d/f8a d5/de/df/feb 0 2024-02-26T10:48:07.704 INFO:tasks.workunit.client.0.smithi060.stdout:3/713: unlink d5/d45/d61/d92/da1/daa/f86 0 2024-02-26T10:48:07.707 INFO:tasks.workunit.client.0.smithi060.stdout:3/714: mkdir d5/d45/d61/d92/dec 0 2024-02-26T10:48:07.714 INFO:tasks.workunit.client.0.smithi060.stdout:8/895: rename d9/d15/d29/d2f/d34/d72/d50/d6a/dc5/l109 to d9/d5a/d9c/df0/d114/l12c 0 2024-02-26T10:48:07.715 INFO:tasks.workunit.client.0.smithi060.stdout:5/847: dread d0/de/da4/fd7 [0,4194304] 0 2024-02-26T10:48:07.715 INFO:tasks.workunit.client.0.smithi060.stdout:5/848: write d0/de/d15/d65/fd0 [17928,72185] 0 2024-02-26T10:48:07.716 INFO:tasks.workunit.client.0.smithi060.stdout:6/935: sync 2024-02-26T10:48:07.716 INFO:tasks.workunit.client.0.smithi060.stdout:4/871: sync 2024-02-26T10:48:07.716 INFO:tasks.workunit.client.0.smithi060.stdout:0/819: sync 2024-02-26T10:48:07.716 INFO:tasks.workunit.client.0.smithi060.stdout:2/857: sync 2024-02-26T10:48:07.716 INFO:tasks.workunit.client.0.smithi060.stdout:2/858: fdatasync dd/d43/d55/d5e/d86/d97/fdc 0 2024-02-26T10:48:07.716 INFO:tasks.workunit.client.0.smithi060.stdout:2/859: write dd/d43/dd2/ffe [1958352,6687] 0 2024-02-26T10:48:07.717 INFO:tasks.workunit.client.0.smithi060.stdout:1/941: rename d3/db/d1e/d6d/da3/daa/dcd/c115 to d3/db/d1e/d39/d9b/df7/c13e 0 2024-02-26T10:48:07.717 INFO:tasks.workunit.client.0.smithi060.stdout:1/942: chown d3/d35/d53/cd7 521239 1 2024-02-26T10:48:07.717 INFO:tasks.workunit.client.0.smithi060.stdout:2/860: read dd/d43/d60/d41/d37/d69/f72 [826086,2188] 0 2024-02-26T10:48:07.717 INFO:tasks.workunit.client.0.smithi060.stdout:2/861: write dd/d43/d55/f107 [426479,90543] 0 2024-02-26T10:48:07.717 INFO:tasks.workunit.client.0.smithi060.stdout:2/862: write dd/d43/d55/f8e [786235,62461] 0 2024-02-26T10:48:07.718 INFO:tasks.workunit.client.0.smithi060.stdout:8/896: getdents d9/d15/d29/d2f/d34/d72/d7e 0 2024-02-26T10:48:07.722 INFO:tasks.workunit.client.0.smithi060.stdout:8/897: mknod d9/d15/d29/c12d 0 2024-02-26T10:48:07.722 INFO:tasks.workunit.client.0.smithi060.stdout:1/943: unlink d3/db/d1e/d6d/da3/daa/dd6/c132 0 2024-02-26T10:48:07.722 INFO:tasks.workunit.client.0.smithi060.stdout:2/863: truncate dd/d43/d60/d41/d37/d38/d57/fe8 109703 0 2024-02-26T10:48:07.722 INFO:tasks.workunit.client.0.smithi060.stdout:1/944: stat d3/d35/d53/d68/d85/fd0 0 2024-02-26T10:48:07.722 INFO:tasks.workunit.client.0.smithi060.stdout:2/864: creat dd/d43/d55/d5e/d86/d97/d9c/f11f x:0 0 0 2024-02-26T10:48:07.722 INFO:tasks.workunit.client.0.smithi060.stdout:2/865: fsync dd/d43/d55/f5a 0 2024-02-26T10:48:07.724 INFO:tasks.workunit.client.0.smithi060.stdout:4/872: stat d0/da/d3b/d1b/f86 0 2024-02-26T10:48:07.724 INFO:tasks.workunit.client.0.smithi060.stdout:4/873: chown d0/da/dc7/db0/db5/dc0/lbb 20115739 1 2024-02-26T10:48:07.739 INFO:tasks.workunit.client.0.smithi060.stdout:4/874: dread d0/d27/d76/fd3 [0,4194304] 0 2024-02-26T10:48:07.750 INFO:tasks.workunit.client.0.smithi060.stdout:4/875: dread d0/da/dc7/db0/fd9 [0,4194304] 0 2024-02-26T10:48:07.759 INFO:tasks.workunit.client.0.smithi060.stdout:8/898: dwrite d9/d6f/fd8 [4194304,4194304] 0 2024-02-26T10:48:07.759 INFO:tasks.workunit.client.0.smithi060.stdout:8/899: truncate d9/da4/deb/f106 4627941 0 2024-02-26T10:48:07.759 INFO:tasks.workunit.client.0.smithi060.stdout:8/900: getdents d9/d15/d3e/d10b 0 2024-02-26T10:48:07.761 INFO:tasks.workunit.client.0.smithi060.stdout:8/901: mkdir d9/d15/d29/d2f/d34/d72/d50/d12e 0 2024-02-26T10:48:07.764 INFO:tasks.workunit.client.0.smithi060.stdout:8/902: mknod d9/d15/d29/d2f/c12f 0 2024-02-26T10:48:07.764 INFO:tasks.workunit.client.0.smithi060.stdout:0/820: dwrite d0/dd/d39/d40/f19 [4194304,4194304] 0 2024-02-26T10:48:07.766 INFO:tasks.workunit.client.0.smithi060.stdout:8/903: creat d9/d15/d29/d2f/d34/d72/d50/d6a/f130 x:0 0 0 2024-02-26T10:48:07.766 INFO:tasks.workunit.client.0.smithi060.stdout:0/821: rename d0/dd/d39/d40/d83/d10c to d0/d42/d7a/d108/d111 0 2024-02-26T10:48:07.768 INFO:tasks.workunit.client.0.smithi060.stdout:8/904: getdents d9/d5a/d58 0 2024-02-26T10:48:07.769 INFO:tasks.workunit.client.0.smithi060.stdout:0/822: symlink d0/dd/d39/l112 0 2024-02-26T10:48:07.771 INFO:tasks.workunit.client.0.smithi060.stdout:8/905: mkdir d9/d15/d29/d2f/d131 0 2024-02-26T10:48:07.771 INFO:tasks.workunit.client.0.smithi060.stdout:8/906: dread - d9/d15/d29/d2f/d34/fdf zero size 2024-02-26T10:48:07.772 INFO:tasks.workunit.client.0.smithi060.stdout:0/823: symlink d0/dd/d39/d93/l113 0 2024-02-26T10:48:07.775 INFO:tasks.workunit.client.0.smithi060.stdout:8/907: link d9/d5a/d9c/df0/f11e d9/d15/d29/d2f/d34/d72/d94/dba/f132 0 2024-02-26T10:48:07.779 INFO:tasks.workunit.client.0.smithi060.stdout:3/715: dwrite d5/de/df/d11/d63/f7c [0,4194304] 0 2024-02-26T10:48:07.779 INFO:tasks.workunit.client.0.smithi060.stdout:5/849: dread d0/de/d15/d26/f45 [0,4194304] 0 2024-02-26T10:48:07.779 INFO:tasks.workunit.client.0.smithi060.stdout:8/908: truncate d9/fd 177516 0 2024-02-26T10:48:07.779 INFO:tasks.workunit.client.0.smithi060.stdout:8/909: read - d9/d5a/fcc zero size 2024-02-26T10:48:07.781 INFO:tasks.workunit.client.0.smithi060.stdout:5/850: mknod d0/de/d20/d82/c125 0 2024-02-26T10:48:07.791 INFO:tasks.workunit.client.0.smithi060.stdout:4/876: write d0/da/d3b/d1b/fab [735288,63400] 0 2024-02-26T10:48:07.799 INFO:tasks.workunit.client.0.smithi060.stdout:0/824: write d0/dd/d39/d93/fb6 [159750,35668] 0 2024-02-26T10:48:07.808 INFO:tasks.workunit.client.0.smithi060.stdout:3/716: dread d5/de/df/d11/f20 [0,4194304] 0 2024-02-26T10:48:07.808 INFO:tasks.workunit.client.0.smithi060.stdout:3/717: write d5/de/df/d11/d63/d93/d97/d98/fb1 [1765963,44038] 0 2024-02-26T10:48:07.808 INFO:tasks.workunit.client.0.smithi060.stdout:3/718: read - d5/de/df/d11/d63/d93/d97/fa0 zero size 2024-02-26T10:48:07.809 INFO:tasks.workunit.client.0.smithi060.stdout:3/719: mknod d5/de/df/d11/d63/d93/ced 0 2024-02-26T10:48:07.818 INFO:tasks.workunit.client.0.smithi060.stdout:2/866: write dd/d43/d60/d41/d37/ff7 [652379,72401] 0 2024-02-26T10:48:07.828 INFO:tasks.workunit.client.0.smithi060.stdout:2/867: symlink dd/d43/d60/l120 0 2024-02-26T10:48:07.828 INFO:tasks.workunit.client.0.smithi060.stdout:8/910: write d9/d5a/d9c/fc7 [313620,104769] 0 2024-02-26T10:48:07.831 INFO:tasks.workunit.client.0.smithi060.stdout:1/945: dwrite d3/d35/fa5 [0,4194304] 0 2024-02-26T10:48:07.837 INFO:tasks.workunit.client.0.smithi060.stdout:5/851: dwrite d0/ff4 [0,4194304] 0 2024-02-26T10:48:07.837 INFO:tasks.workunit.client.0.smithi060.stdout:5/852: truncate d0/de/d20/d82/fe8 271969 0 2024-02-26T10:48:07.837 INFO:tasks.workunit.client.0.smithi060.stdout:1/946: mkdir d3/d35/d53/d68/d6b/dc9/d13f 0 2024-02-26T10:48:07.837 INFO:tasks.workunit.client.0.smithi060.stdout:1/947: fdatasync d3/db/d1e/d39/f11a 0 2024-02-26T10:48:07.838 INFO:tasks.workunit.client.0.smithi060.stdout:1/948: mkdir d3/db/dc8/d140 0 2024-02-26T10:48:07.839 INFO:tasks.workunit.client.0.smithi060.stdout:4/877: dwrite d0/d27/d76/dbd/f87 [0,4194304] 0 2024-02-26T10:48:07.841 INFO:tasks.workunit.client.0.smithi060.stdout:4/878: mkdir d0/da/dba/d116 0 2024-02-26T10:48:07.850 INFO:tasks.workunit.client.0.smithi060.stdout:4/879: truncate d0/da/d3b/f23 6238741 0 2024-02-26T10:48:07.853 INFO:tasks.workunit.client.0.smithi060.stdout:4/880: creat d0/d27/d76/dbd/d60/d101/f117 x:0 0 0 2024-02-26T10:48:07.853 INFO:tasks.workunit.client.0.smithi060.stdout:4/881: fdatasync d0/da/d3b/d16/f19 0 2024-02-26T10:48:07.854 INFO:tasks.workunit.client.0.smithi060.stdout:4/882: rmdir d0/d27/d52 39 2024-02-26T10:48:07.855 INFO:tasks.workunit.client.0.smithi060.stdout:4/883: readlink d0/l2c 0 2024-02-26T10:48:07.855 INFO:tasks.workunit.client.0.smithi060.stdout:8/911: rename d9/d5a/d58/c117 to d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/dc1/dfb/c133 0 2024-02-26T10:48:07.856 INFO:tasks.workunit.client.0.smithi060.stdout:4/884: symlink d0/da/dba/l118 0 2024-02-26T10:48:07.856 INFO:tasks.workunit.client.0.smithi060.stdout:4/885: readlink d0/da/dc7/l9d 0 2024-02-26T10:48:07.856 INFO:tasks.workunit.client.0.smithi060.stdout:8/912: truncate d9/d15/d29/d2f/d34/d72/f42 70911 0 2024-02-26T10:48:07.858 INFO:tasks.workunit.client.0.smithi060.stdout:8/913: mknod d9/d15/d3e/d10b/c134 0 2024-02-26T10:48:07.863 INFO:tasks.workunit.client.0.smithi060.stdout:3/720: dwrite d5/d45/d61/d92/da1/daa/fcd [0,4194304] 0 2024-02-26T10:48:07.871 INFO:tasks.workunit.client.0.smithi060.stdout:3/721: creat d5/dbb/fee x:0 0 0 2024-02-26T10:48:07.874 INFO:tasks.workunit.client.0.smithi060.stdout:0/825: dwrite d0/dd/d53/da2/ff9 [4194304,4194304] 0 2024-02-26T10:48:07.875 INFO:tasks.workunit.client.0.smithi060.stdout:2/868: dwrite dd/d43/d96/f109 [0,4194304] 0 2024-02-26T10:48:07.876 INFO:tasks.workunit.client.0.smithi060.stdout:6/936: sync 2024-02-26T10:48:07.876 INFO:tasks.workunit.client.0.smithi060.stdout:4/886: rename d0/da/d3b/db9 to d0/d27/d76/dbd/d119 0 2024-02-26T10:48:07.877 INFO:tasks.workunit.client.0.smithi060.stdout:6/937: truncate d6/d7/f132 865428 0 2024-02-26T10:48:07.878 INFO:tasks.workunit.client.0.smithi060.stdout:2/869: link dd/c90 dd/d43/d96/df2/d101/c121 0 2024-02-26T10:48:07.878 INFO:tasks.workunit.client.0.smithi060.stdout:2/870: readlink dd/d43/d60/d41/d37/d4c/lab 0 2024-02-26T10:48:07.879 INFO:tasks.workunit.client.0.smithi060.stdout:4/887: creat d0/da/d3b/ddc/dfa/f11a x:0 0 0 2024-02-26T10:48:07.879 INFO:tasks.workunit.client.0.smithi060.stdout:4/888: readlink d0/d27/d76/le4 0 2024-02-26T10:48:07.879 INFO:tasks.workunit.client.0.smithi060.stdout:3/722: rename d5/de/df/d57/d60/c59 to d5/cef 0 2024-02-26T10:48:07.882 INFO:tasks.workunit.client.0.smithi060.stdout:3/723: creat d5/d45/d61/d92/da1/ff0 x:0 0 0 2024-02-26T10:48:07.882 INFO:tasks.workunit.client.0.smithi060.stdout:3/724: readlink l1 0 2024-02-26T10:48:07.885 INFO:tasks.workunit.client.0.smithi060.stdout:4/889: rename d0/da/d57 to d0/d27/d76/dbd/d60/d11b 0 2024-02-26T10:48:07.887 INFO:tasks.workunit.client.0.smithi060.stdout:4/890: symlink d0/da/dba/l11c 0 2024-02-26T10:48:07.887 INFO:tasks.workunit.client.0.smithi060.stdout:4/891: fdatasync d0/da/f7c 0 2024-02-26T10:48:07.887 INFO:tasks.workunit.client.0.smithi060.stdout:4/892: chown d0/da/d3b/d1b/fa7 161 1 2024-02-26T10:48:07.893 INFO:tasks.workunit.client.0.smithi060.stdout:0/826: dread d0/dd/d39/d40/d83/f8f [0,4194304] 0 2024-02-26T10:48:07.893 INFO:tasks.workunit.client.0.smithi060.stdout:0/827: readlink d0/dd/d53/d87/d94/db7/ld5 0 2024-02-26T10:48:07.897 INFO:tasks.workunit.client.0.smithi060.stdout:5/853: dwrite d0/de/d15/d65/d86/d9b/f118 [0,4194304] 0 2024-02-26T10:48:07.900 INFO:tasks.workunit.client.0.smithi060.stdout:8/914: dread d9/d15/d29/f7d [0,4194304] 0 2024-02-26T10:48:07.900 INFO:tasks.workunit.client.0.smithi060.stdout:8/915: stat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/c118 0 2024-02-26T10:48:07.903 INFO:tasks.workunit.client.0.smithi060.stdout:8/916: getdents d9/d15/d29/d2f/d34/d72/d94 0 2024-02-26T10:48:07.903 INFO:tasks.workunit.client.0.smithi060.stdout:8/917: chown d9/d6f/lfd 3623518 1 2024-02-26T10:48:07.909 INFO:tasks.workunit.client.0.smithi060.stdout:6/938: dread d6/d7/d2e/d56/f61 [0,4194304] 0 2024-02-26T10:48:07.909 INFO:tasks.workunit.client.0.smithi060.stdout:6/939: chown d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/da7/le7 4444 1 2024-02-26T10:48:07.915 INFO:tasks.workunit.client.0.smithi060.stdout:2/871: dwrite dd/d43/d96/df2/ff8 [0,4194304] 0 2024-02-26T10:48:07.916 INFO:tasks.workunit.client.0.smithi060.stdout:2/872: chown dd/d43/d55/d5e/dc6 27 1 2024-02-26T10:48:07.917 INFO:tasks.workunit.client.0.smithi060.stdout:0/828: link d0/l56 d0/dd/d39/d40/dad/d10d/l114 0 2024-02-26T10:48:07.917 INFO:tasks.workunit.client.0.smithi060.stdout:8/918: creat d9/d15/d3e/d10b/f135 x:0 0 0 2024-02-26T10:48:07.917 INFO:tasks.workunit.client.0.smithi060.stdout:2/873: getdents dd/d43 0 2024-02-26T10:48:07.918 INFO:tasks.workunit.client.0.smithi060.stdout:3/725: dread d5/de/df/d4c/f4f [0,4194304] 0 2024-02-26T10:48:07.918 INFO:tasks.workunit.client.0.smithi060.stdout:8/919: getdents d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9 0 2024-02-26T10:48:07.919 INFO:tasks.workunit.client.0.smithi060.stdout:6/940: creat d6/d7/d25/d1c/d1f/d23/d2b/f134 x:0 0 0 2024-02-26T10:48:07.919 INFO:tasks.workunit.client.0.smithi060.stdout:0/829: creat d0/d42/d7a/f115 x:0 0 0 2024-02-26T10:48:07.921 INFO:tasks.workunit.client.0.smithi060.stdout:3/726: rmdir d5/de/df/d57/dea 0 2024-02-26T10:48:07.922 INFO:tasks.workunit.client.0.smithi060.stdout:6/941: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d10d/de6/d135 0 2024-02-26T10:48:07.922 INFO:tasks.workunit.client.0.smithi060.stdout:2/874: symlink dd/d43/d96/l122 0 2024-02-26T10:48:07.928 INFO:tasks.workunit.client.0.smithi060.stdout:6/942: mkdir d6/d7/d25/d120/d136 0 2024-02-26T10:48:07.928 INFO:tasks.workunit.client.0.smithi060.stdout:5/854: dread d0/de/d15/d65/f7c [0,4194304] 0 2024-02-26T10:48:07.940 INFO:tasks.workunit.client.0.smithi060.stdout:3/727: rename d5/de/df/d11/l5f to d5/d45/d61/d92/lf1 0 2024-02-26T10:48:07.940 INFO:tasks.workunit.client.0.smithi060.stdout:6/943: creat d6/d7/d25/d1c/d1f/d23/d2b/d67/f137 x:0 0 0 2024-02-26T10:48:07.940 INFO:tasks.workunit.client.0.smithi060.stdout:4/893: dwrite d0/da/dba/f108 [0,4194304] 0 2024-02-26T10:48:07.940 INFO:tasks.workunit.client.0.smithi060.stdout:4/894: truncate d0/da/d3b/d16/f24 967484 0 2024-02-26T10:48:07.940 INFO:tasks.workunit.client.0.smithi060.stdout:0/830: write d0/dd/d53/d87/d94/da6/f9d [3908744,25047] 0 2024-02-26T10:48:07.940 INFO:tasks.workunit.client.0.smithi060.stdout:0/831: write d0/dd/d39/d40/dde/fea [633127,94932] 0 2024-02-26T10:48:07.940 INFO:tasks.workunit.client.0.smithi060.stdout:0/832: read - d0/dd/d39/d40/dad/dda/de4/d103/f10b zero size 2024-02-26T10:48:07.942 INFO:tasks.workunit.client.0.smithi060.stdout:5/855: rename d0/de/f10c to d0/de/d70/d84/d92/f126 0 2024-02-26T10:48:07.942 INFO:tasks.workunit.client.0.smithi060.stdout:6/944: creat d6/d6e/f138 x:0 0 0 2024-02-26T10:48:07.943 INFO:tasks.workunit.client.0.smithi060.stdout:4/895: unlink d0/d27/ffc 0 2024-02-26T10:48:07.943 INFO:tasks.workunit.client.0.smithi060.stdout:8/920: read d9/d15/d29/fe6 [2502952,103715] 0 2024-02-26T10:48:07.943 INFO:tasks.workunit.client.0.smithi060.stdout:8/921: chown d9/f24 220056648 1 2024-02-26T10:48:07.946 INFO:tasks.workunit.client.0.smithi060.stdout:0/833: creat d0/d42/d7a/d108/d111/f116 x:0 0 0 2024-02-26T10:48:07.947 INFO:tasks.workunit.client.0.smithi060.stdout:5/856: rmdir d0/de/d20/d32 39 2024-02-26T10:48:07.947 INFO:tasks.workunit.client.0.smithi060.stdout:5/857: chown d0/f30 1 1 2024-02-26T10:48:07.947 INFO:tasks.workunit.client.0.smithi060.stdout:5/858: chown d0/de/f12 10 1 2024-02-26T10:48:07.948 INFO:tasks.workunit.client.0.smithi060.stdout:8/922: creat d9/d5a/d9c/df0/f136 x:0 0 0 2024-02-26T10:48:07.952 INFO:tasks.workunit.client.0.smithi060.stdout:0/834: unlink d0/dd/d39/d40/d83/dc6/d58/f106 0 2024-02-26T10:48:07.954 INFO:tasks.workunit.client.0.smithi060.stdout:8/923: symlink d9/d15/l137 0 2024-02-26T10:48:07.956 INFO:tasks.workunit.client.0.smithi060.stdout:8/924: symlink d9/d15/d3e/d127/l138 0 2024-02-26T10:48:07.958 INFO:tasks.workunit.client.0.smithi060.stdout:5/859: unlink d0/de/d15/d65/d86/lcb 0 2024-02-26T10:48:07.958 INFO:tasks.workunit.client.0.smithi060.stdout:1/949: sync 2024-02-26T10:48:07.959 INFO:tasks.workunit.client.0.smithi060.stdout:5/860: mknod d0/de/d20/d32/d4d/da3/da5/c127 0 2024-02-26T10:48:07.959 INFO:tasks.workunit.client.0.smithi060.stdout:5/861: truncate d0/de/d20/d82/fe8 402541 0 2024-02-26T10:48:07.960 INFO:tasks.workunit.client.0.smithi060.stdout:5/862: write d0/d33/f43 [3964516,127079] 0 2024-02-26T10:48:07.961 INFO:tasks.workunit.client.0.smithi060.stdout:1/950: unlink d3/d35/d5e/f97 0 2024-02-26T10:48:07.993 INFO:tasks.workunit.client.0.smithi060.stdout:5/863: dread d0/de/d15/d26/f45 [0,4194304] 0 2024-02-26T10:48:07.995 INFO:tasks.workunit.client.0.smithi060.stdout:5/864: creat d0/de/d15/d26/ddd/f128 x:0 0 0 2024-02-26T10:48:07.997 INFO:tasks.workunit.client.0.smithi060.stdout:3/728: dwrite d5/f6 [4194304,4194304] 0 2024-02-26T10:48:08.005 INFO:tasks.workunit.client.0.smithi060.stdout:6/945: dwrite d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/ff1 [4194304,4194304] 0 2024-02-26T10:48:08.008 INFO:tasks.workunit.client.0.smithi060.stdout:6/946: creat d6/d7/d25/d1c/d1f/d23/d2b/d10e/f139 x:0 0 0 2024-02-26T10:48:08.008 INFO:tasks.workunit.client.0.smithi060.stdout:6/947: fdatasync d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/fd1 0 2024-02-26T10:48:08.028 INFO:tasks.workunit.client.0.smithi060.stdout:2/875: dwrite dd/d11/fc5 [4194304,4194304] 0 2024-02-26T10:48:08.029 INFO:tasks.workunit.client.0.smithi060.stdout:2/876: fdatasync dd/d43/d60/d41/d37/d4c/fe5 0 2024-02-26T10:48:08.035 INFO:tasks.workunit.client.0.smithi060.stdout:2/877: creat dd/d43/d60/d41/d37/d69/d117/f123 x:0 0 0 2024-02-26T10:48:08.038 INFO:tasks.workunit.client.0.smithi060.stdout:1/951: dwrite d3/db/ff3 [0,4194304] 0 2024-02-26T10:48:08.039 INFO:tasks.workunit.client.0.smithi060.stdout:0/835: dwrite d0/dd/d39/d93/f100 [0,4194304] 0 2024-02-26T10:48:08.039 INFO:tasks.workunit.client.0.smithi060.stdout:0/836: chown d0/dd/d39/d40/f8d 5993295 1 2024-02-26T10:48:08.039 INFO:tasks.workunit.client.0.smithi060.stdout:0/837: chown d0/dd/d39/d40/dde/fea 1041 1 2024-02-26T10:48:08.039 INFO:tasks.workunit.client.0.smithi060.stdout:0/838: truncate d0/f5f 47554 0 2024-02-26T10:48:08.039 INFO:tasks.workunit.client.0.smithi060.stdout:3/729: dwrite d5/d45/d61/d92/da1/ff0 [0,4194304] 0 2024-02-26T10:48:08.046 INFO:tasks.workunit.client.0.smithi060.stdout:3/730: dread d5/de/df/fe3 [0,4194304] 0 2024-02-26T10:48:08.046 INFO:tasks.workunit.client.0.smithi060.stdout:2/878: dread dd/d43/d55/f5a [0,4194304] 0 2024-02-26T10:48:08.048 INFO:tasks.workunit.client.0.smithi060.stdout:2/879: mknod dd/d43/d60/d41/d37/dfd/c124 0 2024-02-26T10:48:08.049 INFO:tasks.workunit.client.0.smithi060.stdout:6/948: dwrite d6/d7/d25/d80/fae [0,4194304] 0 2024-02-26T10:48:08.051 INFO:tasks.workunit.client.0.smithi060.stdout:2/880: mkdir dd/d43/d55/d5e/d86/d97/db9/d125 0 2024-02-26T10:48:08.052 INFO:tasks.workunit.client.0.smithi060.stdout:2/881: truncate dd/d43/d55/d5e/d74/fb6 1467325 0 2024-02-26T10:48:08.052 INFO:tasks.workunit.client.0.smithi060.stdout:2/882: chown dd/d43/d60/d41/d37/d69/fe4 36273 1 2024-02-26T10:48:08.059 INFO:tasks.workunit.client.0.smithi060.stdout:3/731: getdents d5/d45/d61/d92/da1/dc8 0 2024-02-26T10:48:08.060 INFO:tasks.workunit.client.0.smithi060.stdout:6/949: creat d6/d7/d25/d1c/d1f/d23/d2b/d4d/f13a x:0 0 0 2024-02-26T10:48:08.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/732: chown d5/d45/d61/d92/cb4 184295198 1 2024-02-26T10:48:08.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/733: dread - d5/de/df/d4c/d68/fdd zero size 2024-02-26T10:48:08.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/734: chown d5/d45/cc6 51 1 2024-02-26T10:48:08.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/735: chown d5/de/df/f81 361505092 1 2024-02-26T10:48:08.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/736: chown d5/d76 262190586 1 2024-02-26T10:48:08.061 INFO:tasks.workunit.client.0.smithi060.stdout:3/737: chown d5/de/df/d4c/c5c 5383501 1 2024-02-26T10:48:08.062 INFO:tasks.workunit.client.0.smithi060.stdout:3/738: mknod d5/de/df/d57/cf2 0 2024-02-26T10:48:08.062 INFO:tasks.workunit.client.0.smithi060.stdout:0/839: dread d0/dd/fbb [0,4194304] 0 2024-02-26T10:48:08.064 INFO:tasks.workunit.client.0.smithi060.stdout:0/840: unlink d0/dd/c46 0 2024-02-26T10:48:08.064 INFO:tasks.workunit.client.0.smithi060.stdout:0/841: truncate d0/dd/d39/f61 4535389 0 2024-02-26T10:48:08.066 INFO:tasks.workunit.client.0.smithi060.stdout:4/896: dwrite d0/f5 [0,4194304] 0 2024-02-26T10:48:08.067 INFO:tasks.workunit.client.0.smithi060.stdout:8/925: write d9/d15/d29/f64 [4964047,51489] 0 2024-02-26T10:48:08.067 INFO:tasks.workunit.client.0.smithi060.stdout:8/926: readlink d9/d6f/lfd 0 2024-02-26T10:48:08.067 INFO:tasks.workunit.client.0.smithi060.stdout:4/897: symlink d0/da/dc7/l11d 0 2024-02-26T10:48:08.067 INFO:tasks.workunit.client.0.smithi060.stdout:4/898: fsync d0/da/d3b/d16/f109 0 2024-02-26T10:48:08.072 INFO:tasks.workunit.client.0.smithi060.stdout:8/927: unlink d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f93 0 2024-02-26T10:48:08.072 INFO:tasks.workunit.client.0.smithi060.stdout:4/899: symlink d0/da/d3b/d1b/df9/l11e 0 2024-02-26T10:48:08.073 INFO:tasks.workunit.client.0.smithi060.stdout:2/883: write dd/d43/d55/d5e/d74/fb6 [682558,8465] 0 2024-02-26T10:48:08.073 INFO:tasks.workunit.client.0.smithi060.stdout:4/900: mkdir d0/da/dba/d11f 0 2024-02-26T10:48:08.076 INFO:tasks.workunit.client.0.smithi060.stdout:4/901: mknod d0/d27/d76/d9f/c120 0 2024-02-26T10:48:08.077 INFO:tasks.workunit.client.0.smithi060.stdout:4/902: creat d0/da/dba/d116/f121 x:0 0 0 2024-02-26T10:48:08.078 INFO:tasks.workunit.client.0.smithi060.stdout:4/903: truncate d0/da/dc7/db0/db5/dc0/f37 4408870 0 2024-02-26T10:48:08.078 INFO:tasks.workunit.client.0.smithi060.stdout:4/904: dread - d0/da/f7c zero size 2024-02-26T10:48:08.082 INFO:tasks.workunit.client.0.smithi060.stdout:5/865: dwrite d0/de/d20/d32/d4d/f55 [4194304,4194304] 0 2024-02-26T10:48:08.083 INFO:tasks.workunit.client.0.smithi060.stdout:4/905: unlink d0/da/d3b/ddc/dfa/f11a 0 2024-02-26T10:48:08.083 INFO:tasks.workunit.client.0.smithi060.stdout:4/906: write d0/d27/d76/dbd/d60/d11b/f77 [1964029,46022] 0 2024-02-26T10:48:08.083 INFO:tasks.workunit.client.0.smithi060.stdout:4/907: truncate d0/d27/d76/dbd/d119/fdd 876856 0 2024-02-26T10:48:08.085 INFO:tasks.workunit.client.0.smithi060.stdout:3/739: read d5/de/df/d57/d60/f30 [4087202,73924] 0 2024-02-26T10:48:08.085 INFO:tasks.workunit.client.0.smithi060.stdout:4/908: mkdir d0/da/dd6/d122 0 2024-02-26T10:48:08.089 INFO:tasks.workunit.client.0.smithi060.stdout:4/909: mkdir d0/da/dd6/d122/d123 0 2024-02-26T10:48:08.089 INFO:tasks.workunit.client.0.smithi060.stdout:4/910: read - d0/da/dd6/fe7 zero size 2024-02-26T10:48:08.090 INFO:tasks.workunit.client.0.smithi060.stdout:4/911: mknod d0/d27/d76/dbd/d60/d11b/c124 0 2024-02-26T10:48:08.093 INFO:tasks.workunit.client.0.smithi060.stdout:5/866: rmdir d0 39 2024-02-26T10:48:08.093 INFO:tasks.workunit.client.0.smithi060.stdout:4/912: creat d0/d27/d76/d9f/d114/f125 x:0 0 0 2024-02-26T10:48:08.093 INFO:tasks.workunit.client.0.smithi060.stdout:5/867: truncate d0/de/d20/d32/d4d/da3/da5/dc8/f121 547612 0 2024-02-26T10:48:08.093 INFO:tasks.workunit.client.0.smithi060.stdout:4/913: truncate d0/da/dc7/db0/db5/dc0/ff7 214114 0 2024-02-26T10:48:08.095 INFO:tasks.workunit.client.0.smithi060.stdout:5/868: creat d0/de/d20/d32/d117/f129 x:0 0 0 2024-02-26T10:48:08.095 INFO:tasks.workunit.client.0.smithi060.stdout:6/950: dread d6/d7/f12 [0,4194304] 0 2024-02-26T10:48:08.097 INFO:tasks.workunit.client.0.smithi060.stdout:6/951: mknod d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/c13b 0 2024-02-26T10:48:08.102 INFO:tasks.workunit.client.0.smithi060.stdout:6/952: mkdir d6/d7/d25/d80/df9/d13c 0 2024-02-26T10:48:08.103 INFO:tasks.workunit.client.0.smithi060.stdout:6/953: mknod d6/d8d/c13d 0 2024-02-26T10:48:08.114 INFO:tasks.workunit.client.0.smithi060.stdout:1/952: dwrite d3/db/f11e [0,4194304] 0 2024-02-26T10:48:08.116 INFO:tasks.workunit.client.0.smithi060.stdout:5/869: dread d0/d33/f62 [0,4194304] 0 2024-02-26T10:48:08.116 INFO:tasks.workunit.client.0.smithi060.stdout:1/953: creat d3/d35/f141 x:0 0 0 2024-02-26T10:48:08.117 INFO:tasks.workunit.client.0.smithi060.stdout:5/870: getdents d0/de/d20/d32/d4d/da3 0 2024-02-26T10:48:08.117 INFO:tasks.workunit.client.0.smithi060.stdout:5/871: chown d0/de 0 1 2024-02-26T10:48:08.126 INFO:tasks.workunit.client.0.smithi060.stdout:4/914: write d0/da/d3b/d16/f92 [1573541,24854] 0 2024-02-26T10:48:08.126 INFO:tasks.workunit.client.0.smithi060.stdout:8/928: dwrite d9/da4/f123 [0,4194304] 0 2024-02-26T10:48:08.127 INFO:tasks.workunit.client.0.smithi060.stdout:8/929: chown d9/d15/d29/d2f/d34/ff7 126907315 1 2024-02-26T10:48:08.127 INFO:tasks.workunit.client.0.smithi060.stdout:8/930: write d9/d15/d29/d2f/d34/d72/ffe [800536,14982] 0 2024-02-26T10:48:08.127 INFO:tasks.workunit.client.0.smithi060.stdout:4/915: symlink d0/da/dd6/d122/l126 0 2024-02-26T10:48:08.127 INFO:tasks.workunit.client.0.smithi060.stdout:4/916: fsync d0/d27/d76/dbd/d60/d11b/f65 0 2024-02-26T10:48:08.132 INFO:tasks.workunit.client.0.smithi060.stdout:2/884: dread dd/d43/d55/f5a [0,4194304] 0 2024-02-26T10:48:08.132 INFO:tasks.workunit.client.0.smithi060.stdout:0/842: dwrite d0/dd/d39/d40/d83/dc0/fe6 [0,4194304] 0 2024-02-26T10:48:08.132 INFO:tasks.workunit.client.0.smithi060.stdout:0/843: truncate d0/dd/d39/d40/dad/f107 869127 0 2024-02-26T10:48:08.132 INFO:tasks.workunit.client.0.smithi060.stdout:2/885: mkdir dd/d43/d60/d41/d37/d9a/d126 0 2024-02-26T10:48:08.132 INFO:tasks.workunit.client.0.smithi060.stdout:2/886: chown dd/d43/d60/d41/l93 937 1 2024-02-26T10:48:08.135 INFO:tasks.workunit.client.0.smithi060.stdout:4/917: creat d0/f127 x:0 0 0 2024-02-26T10:48:08.155 INFO:tasks.workunit.client.0.smithi060.stdout:6/954: write d6/f71 [1106442,17885] 0 2024-02-26T10:48:08.155 INFO:tasks.workunit.client.0.smithi060.stdout:3/740: dwrite d5/de/df/faf [0,4194304] 0 2024-02-26T10:48:08.159 INFO:tasks.workunit.client.0.smithi060.stdout:8/931: rename d9/d15/d29/d2f/d34/d72/d50/d6a/dc5 to d9/d15/d29/d2f/d34/d105/d139 0 2024-02-26T10:48:08.162 INFO:tasks.workunit.client.0.smithi060.stdout:2/887: rename dd/d43/d96/df2/d101 to dd/d43/d55/d6e/dd3/d6f/d8f/dbc/d127 0 2024-02-26T10:48:08.171 INFO:tasks.workunit.client.0.smithi060.stdout:1/954: dwrite d3/db/d2e/d51/d63/f92 [0,4194304] 0 2024-02-26T10:48:08.173 INFO:tasks.workunit.client.0.smithi060.stdout:4/918: write d0/da/d3b/d1b/f100 [352308,127457] 0 2024-02-26T10:48:08.181 INFO:tasks.workunit.client.0.smithi060.stdout:0/844: dwrite d0/dd/d39/d40/dad/dd6/fcd [0,4194304] 0 2024-02-26T10:48:08.181 INFO:tasks.workunit.client.0.smithi060.stdout:1/955: mknod d3/d35/d76/c142 0 2024-02-26T10:48:08.185 INFO:tasks.workunit.client.0.smithi060.stdout:5/872: dwrite d0/de/d15/f44 [4194304,4194304] 0 2024-02-26T10:48:08.186 INFO:tasks.workunit.client.0.smithi060.stdout:0/845: symlink d0/d42/df2/d102/l117 0 2024-02-26T10:48:08.186 INFO:tasks.workunit.client.0.smithi060.stdout:3/741: write d5/de/df/d4c/f4f [6352874,71804] 0 2024-02-26T10:48:08.188 INFO:tasks.workunit.client.0.smithi060.stdout:0/846: creat d0/dd/d39/d40/dad/dd6/f118 x:0 0 0 2024-02-26T10:48:08.188 INFO:tasks.workunit.client.0.smithi060.stdout:5/873: link d0/de/d20/d32/d4c/f106 d0/de/d15/f12a 0 2024-02-26T10:48:08.188 INFO:tasks.workunit.client.0.smithi060.stdout:3/742: rmdir d5/d45/d61/d92/da1/dc1 39 2024-02-26T10:48:08.189 INFO:tasks.workunit.client.0.smithi060.stdout:0/847: mkdir d0/dd/d53/d87/d119 0 2024-02-26T10:48:08.194 INFO:tasks.workunit.client.0.smithi060.stdout:4/919: rmdir d0/d27/d76/dbd/d119 39 2024-02-26T10:48:08.194 INFO:tasks.workunit.client.0.smithi060.stdout:0/848: symlink d0/dd/d53/d87/d119/l11a 0 2024-02-26T10:48:08.194 INFO:tasks.workunit.client.0.smithi060.stdout:3/743: symlink d5/d45/d61/d92/dec/lf3 0 2024-02-26T10:48:08.198 INFO:tasks.workunit.client.0.smithi060.stdout:3/744: creat d5/d45/d61/d92/da1/daa/dad/ff4 x:0 0 0 2024-02-26T10:48:08.202 INFO:tasks.workunit.client.0.smithi060.stdout:3/745: creat d5/de/df/d57/ff5 x:0 0 0 2024-02-26T10:48:08.206 INFO:tasks.workunit.client.0.smithi060.stdout:3/746: creat d5/de/df/d57/ff6 x:0 0 0 2024-02-26T10:48:08.228 INFO:tasks.workunit.client.0.smithi060.stdout:1/956: dwrite d3/d35/d53/d68/d85/f89 [0,4194304] 0 2024-02-26T10:48:08.228 INFO:tasks.workunit.client.0.smithi060.stdout:0/849: write d0/dd/d39/d40/d83/dc0/fe6 [3268074,2346] 0 2024-02-26T10:48:08.230 INFO:tasks.workunit.client.0.smithi060.stdout:0/850: mkdir d0/dd/d39/d40/d83/d11b 0 2024-02-26T10:48:08.231 INFO:tasks.workunit.client.0.smithi060.stdout:8/932: dwrite d9/d15/d3e/f88 [0,4194304] 0 2024-02-26T10:48:08.231 INFO:tasks.workunit.client.0.smithi060.stdout:8/933: truncate d9/d15/d29/d2f/d34/ff7 59667 0 2024-02-26T10:48:08.234 INFO:tasks.workunit.client.0.smithi060.stdout:2/888: dread dd/f1d [4194304,4194304] 0 2024-02-26T10:48:08.237 INFO:tasks.workunit.client.0.smithi060.stdout:2/889: rename dd/d43/d55/l64 to dd/d43/d55/d5e/d86/d97/l128 0 2024-02-26T10:48:08.238 INFO:tasks.workunit.client.0.smithi060.stdout:2/890: rmdir dd/d43/d55/d5e/d86/d97/db9/dde 39 2024-02-26T10:48:08.242 INFO:tasks.workunit.client.0.smithi060.stdout:4/920: dread d0/d27/d76/dbd/d8e/fe2 [0,4194304] 0 2024-02-26T10:48:08.243 INFO:tasks.workunit.client.0.smithi060.stdout:6/955: dwrite d6/f71 [0,4194304] 0 2024-02-26T10:48:08.263 INFO:tasks.workunit.client.0.smithi060.stdout:5/874: dwrite d0/d33/f43 [0,4194304] 0 2024-02-26T10:48:08.266 INFO:tasks.workunit.client.0.smithi060.stdout:5/875: symlink d0/de/d15/d65/d86/l12b 0 2024-02-26T10:48:08.266 INFO:tasks.workunit.client.0.smithi060.stdout:3/747: dwrite d5/de/df/d57/d60/fa8 [0,4194304] 0 2024-02-26T10:48:08.268 INFO:tasks.workunit.client.0.smithi060.stdout:3/748: creat d5/dbb/ff7 x:0 0 0 2024-02-26T10:48:08.271 INFO:tasks.workunit.client.0.smithi060.stdout:3/749: rename d5/d37 to d5/de/df/d4c/dcb/df8 0 2024-02-26T10:48:08.273 INFO:tasks.workunit.client.0.smithi060.stdout:3/750: write d5/de/df/d11/f62 [3382617,25951] 0 2024-02-26T10:48:08.274 INFO:tasks.workunit.client.0.smithi060.stdout:8/934: read d9/d15/d29/d2f/d34/d72/d7e/fd2 [2869513,26289] 0 2024-02-26T10:48:08.274 INFO:tasks.workunit.client.0.smithi060.stdout:3/751: symlink d5/de/df/d4c/dcb/lf9 0 2024-02-26T10:48:08.278 INFO:tasks.workunit.client.0.smithi060.stdout:3/752: symlink d5/d76/lfa 0 2024-02-26T10:48:08.279 INFO:tasks.workunit.client.0.smithi060.stdout:6/956: dread d6/d7/d47/f9b [0,4194304] 0 2024-02-26T10:48:08.279 INFO:tasks.workunit.client.0.smithi060.stdout:6/957: readlink d6/d8d/dcc/dc3/l124 0 2024-02-26T10:48:08.281 INFO:tasks.workunit.client.0.smithi060.stdout:6/958: creat d6/d8d/dcc/dc3/d98/d119/f13e x:0 0 0 2024-02-26T10:48:08.281 INFO:tasks.workunit.client.0.smithi060.stdout:6/959: write d6/d7/d2e/d56/f61 [3931887,40330] 0 2024-02-26T10:48:08.288 INFO:tasks.workunit.client.0.smithi060.stdout:8/935: dread d9/d5a/f68 [0,4194304] 0 2024-02-26T10:48:08.290 INFO:tasks.workunit.client.0.smithi060.stdout:8/936: creat d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/dc1/f13a x:0 0 0 2024-02-26T10:48:08.292 INFO:tasks.workunit.client.0.smithi060.stdout:8/937: mknod d9/d15/d29/d2f/d34/d72/d7e/c13b 0 2024-02-26T10:48:08.296 INFO:tasks.workunit.client.0.smithi060.stdout:0/851: dwrite d0/dd/d39/f61 [0,4194304] 0 2024-02-26T10:48:08.296 INFO:tasks.workunit.client.0.smithi060.stdout:5/876: dread d0/de/d15/f3b [0,4194304] 0 2024-02-26T10:48:08.296 INFO:tasks.workunit.client.0.smithi060.stdout:5/877: stat d0/de/d20/d32/d4d/da3/da5/f104 0 2024-02-26T10:48:08.297 INFO:tasks.workunit.client.0.smithi060.stdout:5/878: creat d0/de/d20/d32/d4c/f12c x:0 0 0 2024-02-26T10:48:08.298 INFO:tasks.workunit.client.0.smithi060.stdout:1/957: dwrite d3/f2b [0,4194304] 0 2024-02-26T10:48:08.306 INFO:tasks.workunit.client.0.smithi060.stdout:3/753: write d5/d45/f8f [2876516,51451] 0 2024-02-26T10:48:08.307 INFO:tasks.workunit.client.0.smithi060.stdout:3/754: mkdir d5/de/df/d11/d63/d93/d97/d98/dd4/dfb 0 2024-02-26T10:48:08.315 INFO:tasks.workunit.client.0.smithi060.stdout:6/960: rename d6/d7/d25/f5c to d6/d7/d2e/d56/d73/f13f 0 2024-02-26T10:48:08.315 INFO:tasks.workunit.client.0.smithi060.stdout:6/961: write d6/d7/d2e/fb6 [5774408,27685] 0 2024-02-26T10:48:08.316 INFO:tasks.workunit.client.0.smithi060.stdout:2/891: dwrite dd/d43/dd2/f10b [0,4194304] 0 2024-02-26T10:48:08.317 INFO:tasks.workunit.client.0.smithi060.stdout:6/962: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d10e/d140 0 2024-02-26T10:48:08.318 INFO:tasks.workunit.client.0.smithi060.stdout:6/963: creat d6/d7/d25/d1c/d1f/d23/d2b/d10e/f141 x:0 0 0 2024-02-26T10:48:08.319 INFO:tasks.workunit.client.0.smithi060.stdout:6/964: truncate d6/d7/d25/d1c/d1f/fbb 698818 0 2024-02-26T10:48:08.322 INFO:tasks.workunit.client.0.smithi060.stdout:2/892: creat dd/d43/d96/df2/f129 x:0 0 0 2024-02-26T10:48:08.322 INFO:tasks.workunit.client.0.smithi060.stdout:6/965: mknod d6/d8d/dcc/dc3/d98/c142 0 2024-02-26T10:48:08.326 INFO:tasks.workunit.client.0.smithi060.stdout:3/755: rmdir d5 39 2024-02-26T10:48:08.326 INFO:tasks.workunit.client.0.smithi060.stdout:3/756: fdatasync d5/de/df/d4c/dcb/df8/f56 0 2024-02-26T10:48:08.326 INFO:tasks.workunit.client.0.smithi060.stdout:4/921: dwrite d0/da/d3b/ddc/dfa/f7d [4194304,4194304] 0 2024-02-26T10:48:08.326 INFO:tasks.workunit.client.0.smithi060.stdout:3/757: getdents d5/de/df/d4c/d68 0 2024-02-26T10:48:08.333 INFO:tasks.workunit.client.0.smithi060.stdout:4/922: write d0/f7b [762778,24793] 0 2024-02-26T10:48:08.333 INFO:tasks.workunit.client.0.smithi060.stdout:4/923: read d0/f7b [791060,125160] 0 2024-02-26T10:48:08.333 INFO:tasks.workunit.client.0.smithi060.stdout:4/924: chown d0/d27/d76 59 1 2024-02-26T10:48:08.333 INFO:tasks.workunit.client.0.smithi060.stdout:4/925: write d0/da/d3b/f34 [522150,21827] 0 2024-02-26T10:48:08.333 INFO:tasks.workunit.client.0.smithi060.stdout:8/938: dread d9/d15/d29/d2f/d34/f8c [0,4194304] 0 2024-02-26T10:48:08.339 INFO:tasks.workunit.client.0.smithi060.stdout:8/939: write d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f82 [1649106,49419] 0 2024-02-26T10:48:08.343 INFO:tasks.workunit.client.0.smithi060.stdout:8/940: link d9/d5a/d9c/lb8 d9/d15/d3e/d127/l13c 0 2024-02-26T10:48:08.351 INFO:tasks.workunit.client.0.smithi060.stdout:1/958: dread d3/db/d1e/d39/fff [0,4194304] 0 2024-02-26T10:48:08.351 INFO:tasks.workunit.client.0.smithi060.stdout:1/959: fdatasync d3/db/d1e/d39/fe3 0 2024-02-26T10:48:08.355 INFO:tasks.workunit.client.0.smithi060.stdout:0/852: dwrite d0/dd/d39/d40/d83/f8f [0,4194304] 0 2024-02-26T10:48:08.355 INFO:tasks.workunit.client.0.smithi060.stdout:0/853: write d0/dd/d39/d40/dad/d10d/f110 [41915,85307] 0 2024-02-26T10:48:08.357 INFO:tasks.workunit.client.0.smithi060.stdout:5/879: dwrite d0/de/d15/f5a [0,4194304] 0 2024-02-26T10:48:08.369 INFO:tasks.workunit.client.0.smithi060.stdout:2/893: dread dd/d11/fc5 [0,4194304] 0 2024-02-26T10:48:08.371 INFO:tasks.workunit.client.0.smithi060.stdout:2/894: getdents dd/d43/dd2/d110 0 2024-02-26T10:48:08.372 INFO:tasks.workunit.client.0.smithi060.stdout:2/895: symlink dd/d43/d60/d41/d37/d9a/d126/l12a 0 2024-02-26T10:48:08.373 INFO:tasks.workunit.client.0.smithi060.stdout:5/880: mkdir d0/de/d20/d32/d4c/d12d 0 2024-02-26T10:48:08.373 INFO:tasks.workunit.client.0.smithi060.stdout:2/896: mknod dd/d43/d55/d5e/c12b 0 2024-02-26T10:48:08.374 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:08 smithi100 ceph-mon[36051]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s rd, 170 MiB/s wr, 779 op/s 2024-02-26T10:48:08.375 INFO:tasks.workunit.client.0.smithi060.stdout:2/897: mkdir dd/d43/d55/d6e/dd3/d6f/d8f/dbc/d12c 0 2024-02-26T10:48:08.377 INFO:tasks.workunit.client.0.smithi060.stdout:2/898: mkdir dd/d43/d55/d5e/d12d 0 2024-02-26T10:48:08.377 INFO:tasks.workunit.client.0.smithi060.stdout:2/899: stat dd/d43/dd2/f10b 0 2024-02-26T10:48:08.386 INFO:tasks.workunit.client.0.smithi060.stdout:5/881: dread d0/de/d20/d32/d4d/da3/fcf [0,4194304] 0 2024-02-26T10:48:08.386 INFO:tasks.workunit.client.0.smithi060.stdout:0/854: write d0/dd/d39/d40/dad/dd6/d6c/fab [3923655,115146] 0 2024-02-26T10:48:08.402 INFO:tasks.workunit.client.0.smithi060.stdout:6/966: write d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/fd1 [2277842,3865] 0 2024-02-26T10:48:08.405 INFO:tasks.workunit.client.0.smithi060.stdout:4/926: dwrite d0/d27/d76/d9f/f10d [0,4194304] 0 2024-02-26T10:48:08.412 INFO:tasks.workunit.client.0.smithi060.stdout:3/758: dread d5/fd [0,4194304] 0 2024-02-26T10:48:08.412 INFO:tasks.workunit.client.0.smithi060.stdout:4/927: mknod d0/da/d3b/ddc/dfa/dfe/c128 0 2024-02-26T10:48:08.414 INFO:tasks.workunit.client.0.smithi060.stdout:4/928: unlink d0/da/f80 0 2024-02-26T10:48:08.415 INFO:tasks.workunit.client.0.smithi060.stdout:3/759: mkdir d5/d45/d61/d92/da1/daa/dad/dfc 0 2024-02-26T10:48:08.415 INFO:tasks.workunit.client.0.smithi060.stdout:4/929: dread - d0/da/d3b/ddc/dfa/dfe/f106 zero size 2024-02-26T10:48:08.417 INFO:tasks.workunit.client.0.smithi060.stdout:3/760: creat d5/d45/d61/d92/ffd x:0 0 0 2024-02-26T10:48:08.417 INFO:tasks.workunit.client.0.smithi060.stdout:8/941: dwrite d9/d15/d29/d2f/d34/d72/d94/f116 [0,4194304] 0 2024-02-26T10:48:08.427 INFO:tasks.workunit.client.0.smithi060.stdout:4/930: link d0/da/dc7/db0/db5/dc0/f37 d0/d27/d76/d9f/f129 0 2024-02-26T10:48:08.427 INFO:tasks.workunit.client.0.smithi060.stdout:4/931: chown d0/d27/d76/dbd/d119 98 1 2024-02-26T10:48:08.427 INFO:tasks.workunit.client.0.smithi060.stdout:1/960: dwrite d3/d35/d53/f11d [0,4194304] 0 2024-02-26T10:48:08.427 INFO:tasks.workunit.client.0.smithi060.stdout:1/961: stat d3/db/d2e/f95 0 2024-02-26T10:48:08.427 INFO:tasks.workunit.client.0.smithi060.stdout:2/900: dwrite dd/d43/d60/d41/d37/d69/fe4 [0,4194304] 0 2024-02-26T10:48:08.429 INFO:tasks.workunit.client.0.smithi060.stdout:3/761: link d5/de/df/d4c/f70 d5/dbb/dcf/ffe 0 2024-02-26T10:48:08.438 INFO:tasks.workunit.client.0.smithi060.stdout:5/882: dwrite d0/de/d20/d32/d4c/ff1 [0,4194304] 0 2024-02-26T10:48:08.439 INFO:tasks.workunit.client.0.smithi060.stdout:5/883: creat d0/d4b/d7b/f12e x:0 0 0 2024-02-26T10:48:08.439 INFO:tasks.workunit.client.0.smithi060.stdout:5/884: dread - d0/de/d20/d32/d4d/da3/da5/f104 zero size 2024-02-26T10:48:08.455 INFO:tasks.workunit.client.0.smithi060.stdout:8/942: dread d9/d15/fe5 [0,4194304] 0 2024-02-26T10:48:08.463 INFO:tasks.workunit.client.0.smithi060.stdout:8/943: rmdir d9/d15 39 2024-02-26T10:48:08.472 INFO:tasks.workunit.client.0.smithi060.stdout:0/855: dwrite d0/dd/d33/f88 [0,4194304] 0 2024-02-26T10:48:08.474 INFO:tasks.workunit.client.0.smithi060.stdout:3/762: dread d5/dbb/dcf/ffe [0,4194304] 0 2024-02-26T10:48:08.474 INFO:tasks.workunit.client.0.smithi060.stdout:3/763: fdatasync d5/f27 0 2024-02-26T10:48:08.477 INFO:tasks.workunit.client.0.smithi060.stdout:3/764: creat d5/de/df/d11/d63/d93/d97/d98/dd4/fff x:0 0 0 2024-02-26T10:48:08.477 INFO:tasks.workunit.client.0.smithi060.stdout:3/765: chown d5/de/df/d11/d63/d93 60 1 2024-02-26T10:48:08.478 INFO:tasks.workunit.client.0.smithi060.stdout:3/766: getdents d5/de/df/d11/d63/d93/d97/d98 0 2024-02-26T10:48:08.478 INFO:tasks.workunit.client.0.smithi060.stdout:3/767: readlink d5/l19 0 2024-02-26T10:48:08.480 INFO:tasks.workunit.client.0.smithi060.stdout:3/768: truncate d5/de/df/d11/f1c 485068 0 2024-02-26T10:48:08.487 INFO:tasks.workunit.client.0.smithi060.stdout:3/769: mknod d5/de/df/c100 0 2024-02-26T10:48:08.487 INFO:tasks.workunit.client.0.smithi060.stdout:6/967: dwrite d6/d7/d47/f55 [0,4194304] 0 2024-02-26T10:48:08.487 INFO:tasks.workunit.client.0.smithi060.stdout:6/968: fsync d6/d7/d25/d80/df9/dd8/f12b 0 2024-02-26T10:48:08.487 INFO:tasks.workunit.client.0.smithi060.stdout:6/969: truncate d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/f90 998072 0 2024-02-26T10:48:08.487 INFO:tasks.workunit.client.0.smithi060.stdout:6/970: getdents d6/d8d/dcc/dc3/d98/d119 0 2024-02-26T10:48:08.492 INFO:tasks.workunit.client.0.smithi060.stdout:6/971: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d4d/d143 0 2024-02-26T10:48:08.500 INFO:tasks.workunit.client.0.smithi060.stdout:2/901: dwrite dd/f68 [0,4194304] 0 2024-02-26T10:48:08.506 INFO:tasks.workunit.client.0.smithi060.stdout:5/885: dread d0/de/d15/f2f [0,4194304] 0 2024-02-26T10:48:08.518 INFO:tasks.workunit.client.0.smithi060.stdout:6/972: dread d6/f7c [0,4194304] 0 2024-02-26T10:48:08.518 INFO:tasks.workunit.client.0.smithi060.stdout:4/932: dwrite d0/da/d3b/d1b/f100 [0,4194304] 0 2024-02-26T10:48:08.518 INFO:tasks.workunit.client.0.smithi060.stdout:4/933: read d0/da/d3b/d1b/fab [264337,128083] 0 2024-02-26T10:48:08.519 INFO:tasks.workunit.client.0.smithi060.stdout:4/934: chown d0/da/cc9 27543956 1 2024-02-26T10:48:08.520 INFO:tasks.workunit.client.0.smithi060.stdout:3/770: write d5/de/df/d11/f20 [2869608,100472] 0 2024-02-26T10:48:08.521 INFO:tasks.workunit.client.0.smithi060.stdout:4/935: getdents d0/da/dd6/d122 0 2024-02-26T10:48:08.523 INFO:tasks.workunit.client.0.smithi060.stdout:3/771: creat d5/de/df/d11/d63/d93/d97/f101 x:0 0 0 2024-02-26T10:48:08.523 INFO:tasks.workunit.client.0.smithi060.stdout:3/772: write d5/f6 [3181774,101278] 0 2024-02-26T10:48:08.523 INFO:tasks.workunit.client.0.smithi060.stdout:3/773: write d5/de/df/d57/ff6 [919159,76799] 0 2024-02-26T10:48:08.524 INFO:tasks.workunit.client.0.smithi060.stdout:4/936: mkdir d0/d27/d76/dbd/d119/dc5/d12a 0 2024-02-26T10:48:08.524 INFO:tasks.workunit.client.0.smithi060.stdout:8/944: dwrite d9/d15/d29/d2f/d34/d72/d94/dba/f12b [0,4194304] 0 2024-02-26T10:48:08.525 INFO:tasks.workunit.client.0.smithi060.stdout:6/973: dread d6/d7/d2e/fab [0,4194304] 0 2024-02-26T10:48:08.535 INFO:tasks.workunit.client.0.smithi060.stdout:3/774: mkdir d5/de/df/d11/d63/d93/d97/d98/d102 0 2024-02-26T10:48:08.535 INFO:tasks.workunit.client.0.smithi060.stdout:4/937: creat d0/da/dba/d116/f12b x:0 0 0 2024-02-26T10:48:08.536 INFO:tasks.workunit.client.0.smithi060.stdout:4/938: mknod d0/da/dd6/c12c 0 2024-02-26T10:48:08.537 INFO:tasks.workunit.client.0.smithi060.stdout:3/775: symlink d5/d45/d61/d92/l103 0 2024-02-26T10:48:08.540 INFO:tasks.workunit.client.0.smithi060.stdout:0/856: dwrite d0/dd/f4e [0,4194304] 0 2024-02-26T10:48:08.540 INFO:tasks.workunit.client.0.smithi060.stdout:4/939: symlink d0/da/l12d 0 2024-02-26T10:48:08.540 INFO:tasks.workunit.client.0.smithi060.stdout:4/940: write d0/d27/d76/dbd/fbe [504454,52304] 0 2024-02-26T10:48:08.540 INFO:tasks.workunit.client.0.smithi060.stdout:3/776: creat d5/d45/d61/d92/da1/dc8/f104 x:0 0 0 2024-02-26T10:48:08.540 INFO:tasks.workunit.client.0.smithi060.stdout:3/777: chown d5/de/fe8 1 1 2024-02-26T10:48:08.543 INFO:tasks.workunit.client.0.smithi060.stdout:4/941: rename d0/l2c to d0/d27/d76/dbd/d60/d101/l12e 0 2024-02-26T10:48:08.543 INFO:tasks.workunit.client.0.smithi060.stdout:4/942: write d0/d27/d76/d9f/fed [540753,55032] 0 2024-02-26T10:48:08.544 INFO:tasks.workunit.client.0.smithi060.stdout:4/943: readlink d0/da/dc7/db0/db5/dc0/ld5 0 2024-02-26T10:48:08.545 INFO:tasks.workunit.client.0.smithi060.stdout:4/944: creat d0/d27/d76/dbd/d119/dc5/f12f x:0 0 0 2024-02-26T10:48:08.548 INFO:tasks.workunit.client.0.smithi060.stdout:2/902: dwrite dd/d43/d55/d5e/d86/d97/d9c/f11f [0,4194304] 0 2024-02-26T10:48:08.558 INFO:tasks.workunit.client.0.smithi060.stdout:2/903: rmdir dd/d43/d55/d5e/d74 39 2024-02-26T10:48:08.560 INFO:tasks.workunit.client.0.smithi060.stdout:2/904: truncate dd/d11/fe1 3209409 0 2024-02-26T10:48:08.561 INFO:tasks.workunit.client.0.smithi060.stdout:8/945: dread d9/d15/d29/d2f/d34/d72/d7e/f11f [0,4194304] 0 2024-02-26T10:48:08.562 INFO:tasks.workunit.client.0.smithi060.stdout:3/778: dread d5/de/df/f81 [0,4194304] 0 2024-02-26T10:48:08.564 INFO:tasks.workunit.client.0.smithi060.stdout:8/946: symlink d9/d5a/d9c/l13d 0 2024-02-26T10:48:08.564 INFO:tasks.workunit.client.0.smithi060.stdout:8/947: chown d9/d15/d29/d2f/d34/d72/d7e/fec 340287 1 2024-02-26T10:48:08.564 INFO:tasks.workunit.client.0.smithi060.stdout:3/779: mkdir d5/d45/d61/d92/d105 0 2024-02-26T10:48:08.571 INFO:tasks.workunit.client.0.smithi060.stdout:5/886: dwrite d0/de/d15/d65/d86/d9b/f72 [0,4194304] 0 2024-02-26T10:48:08.571 INFO:tasks.workunit.client.0.smithi060.stdout:4/945: dread d0/da/d3b/d1b/fc1 [0,4194304] 0 2024-02-26T10:48:08.571 INFO:tasks.workunit.client.0.smithi060.stdout:4/946: readlink d0/d27/d52/l88 0 2024-02-26T10:48:08.571 INFO:tasks.workunit.client.0.smithi060.stdout:4/947: chown d0/d27/d76/dbd/d8e 240 1 2024-02-26T10:48:08.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:08 smithi113 ceph-mon[36126]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s rd, 170 MiB/s wr, 779 op/s 2024-02-26T10:48:08.582 INFO:tasks.workunit.client.0.smithi060.stdout:8/948: rename d9/d5a/d9c/df0/f136 to d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f13e 0 2024-02-26T10:48:08.582 INFO:tasks.workunit.client.0.smithi060.stdout:8/949: getdents d9/d15/d29/d2f/d34/d72/df5 0 2024-02-26T10:48:08.582 INFO:tasks.workunit.client.0.smithi060.stdout:0/857: dwrite d0/dd/d39/f7c [0,4194304] 0 2024-02-26T10:48:08.589 INFO:tasks.workunit.client.0.smithi060.stdout:0/858: mknod d0/dd/d39/d40/d83/d11b/c11c 0 2024-02-26T10:48:08.590 INFO:tasks.workunit.client.0.smithi060.stdout:3/780: rename d5/de/df/d57/d7a/fb6 to d5/de/df/d11/d63/d93/d97/f106 0 2024-02-26T10:48:08.591 INFO:tasks.workunit.client.0.smithi060.stdout:3/781: readlink d5/de/df/l3f 0 2024-02-26T10:48:08.591 INFO:tasks.workunit.client.0.smithi060.stdout:3/782: write d5/de/df/d11/d63/d93/d97/fa0 [757846,91729] 0 2024-02-26T10:48:08.591 INFO:tasks.workunit.client.0.smithi060.stdout:3/783: getdents d5/de/df/d57/d60 0 2024-02-26T10:48:08.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:08 smithi060 ceph-mon[31539]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s rd, 170 MiB/s wr, 779 op/s 2024-02-26T10:48:08.614 INFO:tasks.workunit.client.0.smithi060.stdout:4/948: rename d0/da/d3b/f34 to d0/f130 0 2024-02-26T10:48:08.616 INFO:tasks.workunit.client.0.smithi060.stdout:5/887: dwrite d0/d33/fef [0,4194304] 0 2024-02-26T10:48:08.616 INFO:tasks.workunit.client.0.smithi060.stdout:4/949: mknod d0/d27/d52/c131 0 2024-02-26T10:48:08.617 INFO:tasks.workunit.client.0.smithi060.stdout:5/888: creat d0/de/d15/d65/d86/dcc/f12f x:0 0 0 2024-02-26T10:48:08.620 INFO:tasks.workunit.client.0.smithi060.stdout:5/889: creat d0/de/d20/d32/d4d/da3/da5/f130 x:0 0 0 2024-02-26T10:48:08.634 INFO:tasks.workunit.client.0.smithi060.stdout:3/784: dwrite d5/de/df/d4c/f4f [0,4194304] 0 2024-02-26T10:48:08.642 INFO:tasks.workunit.client.0.smithi060.stdout:0/859: dwrite d0/d42/d7a/ffb [0,4194304] 0 2024-02-26T10:48:08.645 INFO:tasks.workunit.client.0.smithi060.stdout:0/860: mkdir d0/dd/d53/d87/d94/d11d 0 2024-02-26T10:48:08.645 INFO:tasks.workunit.client.0.smithi060.stdout:0/861: read d0/d42/fd7 [461459,124247] 0 2024-02-26T10:48:08.645 INFO:tasks.workunit.client.0.smithi060.stdout:0/862: write d0/dd/d53/da2/faa [1020678,80439] 0 2024-02-26T10:48:08.645 INFO:tasks.workunit.client.0.smithi060.stdout:0/863: dread - d0/dd/d39/d40/dad/dd6/d6c/f9a zero size 2024-02-26T10:48:08.646 INFO:tasks.workunit.client.0.smithi060.stdout:0/864: creat d0/d42/df2/f11e x:0 0 0 2024-02-26T10:48:08.648 INFO:tasks.workunit.client.0.smithi060.stdout:0/865: mkdir d0/dd/d53/d11f 0 2024-02-26T10:48:08.669 INFO:tasks.workunit.client.0.smithi060.stdout:4/950: dwrite d0/d27/d76/d9f/f129 [0,4194304] 0 2024-02-26T10:48:08.671 INFO:tasks.workunit.client.0.smithi060.stdout:3/785: write d5/f6 [7517987,85139] 0 2024-02-26T10:48:08.671 INFO:tasks.workunit.client.0.smithi060.stdout:8/950: dwrite d9/d15/d29/d2f/fc9 [0,4194304] 0 2024-02-26T10:48:08.672 INFO:tasks.workunit.client.0.smithi060.stdout:4/951: creat d0/d27/d76/dbd/d60/d11b/f132 x:0 0 0 2024-02-26T10:48:08.672 INFO:tasks.workunit.client.0.smithi060.stdout:5/890: dwrite d0/de/f5d [0,4194304] 0 2024-02-26T10:48:08.673 INFO:tasks.workunit.client.0.smithi060.stdout:8/951: mkdir d9/d15/d29/d2f/d34/d72/d94/dba/d13f 0 2024-02-26T10:48:08.674 INFO:tasks.workunit.client.0.smithi060.stdout:0/866: read d0/dd/d39/f7c [2062723,129115] 0 2024-02-26T10:48:08.674 INFO:tasks.workunit.client.0.smithi060.stdout:0/867: chown d0/d42/c86 23 1 2024-02-26T10:48:08.674 INFO:tasks.workunit.client.0.smithi060.stdout:0/868: write d0/dd/d39/d40/dad/dd6/d6c/fa0 [2351100,91815] 0 2024-02-26T10:48:08.675 INFO:tasks.workunit.client.0.smithi060.stdout:4/952: creat d0/d27/d76/d9f/f133 x:0 0 0 2024-02-26T10:48:08.676 INFO:tasks.workunit.client.0.smithi060.stdout:3/786: symlink d5/de/df/d11/d63/d93/d97/d98/dd4/dfb/l107 0 2024-02-26T10:48:08.676 INFO:tasks.workunit.client.0.smithi060.stdout:3/787: write d5/de/df/d11/f20 [5273386,72138] 0 2024-02-26T10:48:08.679 INFO:tasks.workunit.client.0.smithi060.stdout:0/869: rename d0/f82 to d0/dd/d39/d93/f120 0 2024-02-26T10:48:08.679 INFO:tasks.workunit.client.0.smithi060.stdout:0/870: fdatasync d0/f5f 0 2024-02-26T10:48:08.680 INFO:tasks.workunit.client.0.smithi060.stdout:4/953: creat d0/da/dba/d116/f134 x:0 0 0 2024-02-26T10:48:08.680 INFO:tasks.workunit.client.0.smithi060.stdout:4/954: stat d0/da/d3b/d1b/f86 0 2024-02-26T10:48:08.682 INFO:tasks.workunit.client.0.smithi060.stdout:3/788: creat d5/de/df/d4c/dcb/df8/f108 x:0 0 0 2024-02-26T10:48:08.684 INFO:tasks.workunit.client.0.smithi060.stdout:0/871: fdatasync d0/dd/faf 0 2024-02-26T10:48:08.684 INFO:tasks.workunit.client.0.smithi060.stdout:0/872: readlink d0/d37/l49 0 2024-02-26T10:48:08.687 INFO:tasks.workunit.client.0.smithi060.stdout:4/955: rename d0/da/dc7/db0/db5/dc0/f25 to d0/d27/d76/d9f/d114/f135 0 2024-02-26T10:48:08.691 INFO:tasks.workunit.client.0.smithi060.stdout:3/789: creat d5/de/df/d11/de0/f109 x:0 0 0 2024-02-26T10:48:08.692 INFO:tasks.workunit.client.0.smithi060.stdout:3/790: creat d5/d45/d61/f10a x:0 0 0 2024-02-26T10:48:08.724 INFO:tasks.workunit.client.0.smithi060.stdout:5/891: dwrite d0/de/d20/d32/fd1 [0,4194304] 0 2024-02-26T10:48:08.724 INFO:tasks.workunit.client.0.smithi060.stdout:5/892: fdatasync d0/de/d20/d32/d4d/f55 0 2024-02-26T10:48:08.725 INFO:tasks.workunit.client.0.smithi060.stdout:5/893: write d0/de/d20/f85 [766833,81331] 0 2024-02-26T10:48:08.725 INFO:tasks.workunit.client.0.smithi060.stdout:5/894: truncate d0/de/d15/d65/d86/dcc/f12f 335048 0 2024-02-26T10:48:08.728 INFO:tasks.workunit.client.0.smithi060.stdout:5/895: getdents d0/d33 0 2024-02-26T10:48:08.731 INFO:tasks.workunit.client.0.smithi060.stdout:5/896: mkdir d0/de/d70/d84/d131 0 2024-02-26T10:48:08.733 INFO:tasks.workunit.client.0.smithi060.stdout:5/897: dread - d0/de/d15/d65/d86/d9b/d113/d114/fde zero size 2024-02-26T10:48:08.735 INFO:tasks.workunit.client.0.smithi060.stdout:1/962: sync 2024-02-26T10:48:08.738 INFO:tasks.workunit.client.0.smithi060.stdout:4/956: dwrite d0/f4a [0,4194304] 0 2024-02-26T10:48:08.739 INFO:tasks.workunit.client.0.smithi060.stdout:1/963: read d3/db/d1e/d39/f12b [761689,39291] 0 2024-02-26T10:48:08.741 INFO:tasks.workunit.client.0.smithi060.stdout:1/964: creat d3/db/dc8/f143 x:0 0 0 2024-02-26T10:48:08.741 INFO:tasks.workunit.client.0.smithi060.stdout:1/965: chown d3/d35/d53/d68/d90/d125/c101 3252851 1 2024-02-26T10:48:08.742 INFO:tasks.workunit.client.0.smithi060.stdout:0/873: dwrite d0/dd/d39/d93/f120 [4194304,4194304] 0 2024-02-26T10:48:08.753 INFO:tasks.workunit.client.0.smithi060.stdout:3/791: dread d5/de/df/d11/f62 [0,4194304] 0 2024-02-26T10:48:08.754 INFO:tasks.workunit.client.0.smithi060.stdout:8/952: dwrite d9/d15/d29/d2f/d34/d72/d7e/fec [0,4194304] 0 2024-02-26T10:48:08.757 INFO:tasks.workunit.client.0.smithi060.stdout:8/953: mknod d9/d15/d29/d2f/d34/d72/d94/dd4/c140 0 2024-02-26T10:48:08.759 INFO:tasks.workunit.client.0.smithi060.stdout:8/954: creat d9/d15/d29/d2f/d34/d72/d94/f141 x:0 0 0 2024-02-26T10:48:08.760 INFO:tasks.workunit.client.0.smithi060.stdout:6/974: sync 2024-02-26T10:48:08.760 INFO:tasks.workunit.client.0.smithi060.stdout:2/905: sync 2024-02-26T10:48:08.760 INFO:tasks.workunit.client.0.smithi060.stdout:1/966: rename d3/d35/d53/d68/d85/f94 to d3/d35/d5e/f144 0 2024-02-26T10:48:08.762 INFO:tasks.workunit.client.0.smithi060.stdout:1/967: mkdir d3/db/d1e/d6d/d78/dc4/d121/d145 0 2024-02-26T10:48:08.762 INFO:tasks.workunit.client.0.smithi060.stdout:3/792: rename d5/d45/d61/d92/da1/dc8 to d5/de/d16/d10b 0 2024-02-26T10:48:08.779 INFO:tasks.workunit.client.0.smithi060.stdout:4/957: dread d0/da/d3b/ddc/dfa/f6e [0,4194304] 0 2024-02-26T10:48:08.793 INFO:tasks.workunit.client.0.smithi060.stdout:0/874: dwrite d0/dd/d39/d40/f1f [4194304,4194304] 0 2024-02-26T10:48:08.793 INFO:tasks.workunit.client.0.smithi060.stdout:0/875: fsync d0/dd/d39/d40/dad/d10d/f110 0 2024-02-26T10:48:08.793 INFO:tasks.workunit.client.0.smithi060.stdout:1/968: read d3/d35/d53/d68/d6b/f126 [3820825,107024] 0 2024-02-26T10:48:08.794 INFO:tasks.workunit.client.0.smithi060.stdout:0/876: rename d0/dd/d39/d40/f1f to d0/dd/d33/f121 0 2024-02-26T10:48:08.795 INFO:tasks.workunit.client.0.smithi060.stdout:1/969: link d3/f2b d3/db/dc8/d140/f146 0 2024-02-26T10:48:08.798 INFO:tasks.workunit.client.0.smithi060.stdout:0/877: mkdir d0/dd/d39/d40/dde/d122 0 2024-02-26T10:48:08.799 INFO:tasks.workunit.client.0.smithi060.stdout:1/970: symlink d3/db/d1e/d6d/d78/dc4/l147 0 2024-02-26T10:48:08.799 INFO:tasks.workunit.client.0.smithi060.stdout:2/906: write dd/f68 [1418486,78239] 0 2024-02-26T10:48:08.799 INFO:tasks.workunit.client.0.smithi060.stdout:3/793: dwrite d5/de/df/d57/d60/fa8 [0,4194304] 0 2024-02-26T10:48:08.800 INFO:tasks.workunit.client.0.smithi060.stdout:0/878: rmdir d0/dd/d39/d40/dad 39 2024-02-26T10:48:08.800 INFO:tasks.workunit.client.0.smithi060.stdout:0/879: truncate d0/dd/d39/d40/dde/fb1 530407 0 2024-02-26T10:48:08.802 INFO:tasks.workunit.client.0.smithi060.stdout:0/880: mkdir d0/dd/d39/d40/dad/dda/de4/d103/d123 0 2024-02-26T10:48:08.818 INFO:tasks.workunit.client.0.smithi060.stdout:3/794: dread d5/de/df/d11/f1c [0,4194304] 0 2024-02-26T10:48:08.820 INFO:tasks.workunit.client.0.smithi060.stdout:3/795: rename d5/dbb/ff7 to d5/d45/d61/d92/da1/f10c 0 2024-02-26T10:48:08.820 INFO:tasks.workunit.client.0.smithi060.stdout:3/796: fdatasync d5/de/df/d57/f74 0 2024-02-26T10:48:08.822 INFO:tasks.workunit.client.0.smithi060.stdout:3/797: rename d5/de/df/d4c/f6f to d5/d45/d61/d92/da1/daa/f10d 0 2024-02-26T10:48:08.825 INFO:tasks.workunit.client.0.smithi060.stdout:4/958: dwrite d0/da/d3b/d1b/f100 [0,4194304] 0 2024-02-26T10:48:08.825 INFO:tasks.workunit.client.0.smithi060.stdout:3/798: creat d5/de/df/d11/d63/d93/d97/d98/dd4/dfb/f10e x:0 0 0 2024-02-26T10:48:08.826 INFO:tasks.workunit.client.0.smithi060.stdout:1/971: dread d3/d35/d53/d57/f73 [0,4194304] 0 2024-02-26T10:48:08.827 INFO:tasks.workunit.client.0.smithi060.stdout:3/799: creat d5/d45/f10f x:0 0 0 2024-02-26T10:48:08.828 INFO:tasks.workunit.client.0.smithi060.stdout:3/800: creat d5/d45/f110 x:0 0 0 2024-02-26T10:48:08.829 INFO:tasks.workunit.client.0.smithi060.stdout:4/959: fsync d0/da/d3b/ddc/dfa/f31 0 2024-02-26T10:48:08.830 INFO:tasks.workunit.client.0.smithi060.stdout:4/960: readlink d0/da/d3b/d1b/lf8 0 2024-02-26T10:48:08.832 INFO:tasks.workunit.client.0.smithi060.stdout:4/961: truncate d0/da/d3b/d1b/fa7 852486 0 2024-02-26T10:48:08.832 INFO:tasks.workunit.client.0.smithi060.stdout:4/962: truncate d0/da/d3b/d16/f18 5079164 0 2024-02-26T10:48:08.833 INFO:tasks.workunit.client.0.smithi060.stdout:4/963: creat d0/da/dc7/db0/db5/dc0/f136 x:0 0 0 2024-02-26T10:48:08.838 INFO:tasks.workunit.client.0.smithi060.stdout:1/972: write d3/db/d1e/d38/d6c/f88 [724907,93361] 0 2024-02-26T10:48:08.851 INFO:tasks.workunit.client.0.smithi060.stdout:2/907: dwrite dd/d43/d60/fbd [0,4194304] 0 2024-02-26T10:48:08.852 INFO:tasks.workunit.client.0.smithi060.stdout:2/908: dread - dd/d43/d55/d5e/d86/d97/d9c/fea zero size 2024-02-26T10:48:08.852 INFO:tasks.workunit.client.0.smithi060.stdout:1/973: readlink d3/d35/dc2/l134 0 2024-02-26T10:48:08.852 INFO:tasks.workunit.client.0.smithi060.stdout:1/974: fdatasync d3/db/d2e/f33 0 2024-02-26T10:48:08.852 INFO:tasks.workunit.client.0.smithi060.stdout:1/975: write d3/db/d1e/d6d/da3/daa/dd6/fdf [314838,94173] 0 2024-02-26T10:48:08.854 INFO:tasks.workunit.client.0.smithi060.stdout:2/909: symlink dd/d11/l12e 0 2024-02-26T10:48:08.855 INFO:tasks.workunit.client.0.smithi060.stdout:1/976: mknod d3/db/d104/d10d/c148 0 2024-02-26T10:48:08.858 INFO:tasks.workunit.client.0.smithi060.stdout:0/881: dwrite d0/dd/d53/d87/ff7 [4194304,4194304] 0 2024-02-26T10:48:08.859 INFO:tasks.workunit.client.0.smithi060.stdout:2/910: creat dd/d43/d60/d41/d37/d9a/d126/f12f x:0 0 0 2024-02-26T10:48:08.860 INFO:tasks.workunit.client.0.smithi060.stdout:0/882: creat d0/dd/f124 x:0 0 0 2024-02-26T10:48:08.861 INFO:tasks.workunit.client.0.smithi060.stdout:2/911: mkdir dd/d43/d60/d130 0 2024-02-26T10:48:08.866 INFO:tasks.workunit.client.0.smithi060.stdout:2/912: unlink dd/d43/fb0 0 2024-02-26T10:48:08.866 INFO:tasks.workunit.client.0.smithi060.stdout:2/913: stat dd/d43/dd2 0 2024-02-26T10:48:08.867 INFO:tasks.workunit.client.0.smithi060.stdout:2/914: rmdir dd/d43/d55/d5e/d74 39 2024-02-26T10:48:08.869 INFO:tasks.workunit.client.0.smithi060.stdout:2/915: mknod dd/d43/d60/d41/d37/dfd/c131 0 2024-02-26T10:48:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:2/916: creat dd/d43/dd2/f132 x:0 0 0 2024-02-26T10:48:08.884 INFO:tasks.workunit.client.0.smithi060.stdout:4/964: dwrite d0/da/f53 [0,4194304] 0 2024-02-26T10:48:08.884 INFO:tasks.workunit.client.0.smithi060.stdout:4/965: truncate d0/f4a 4622921 0 2024-02-26T10:48:08.886 INFO:tasks.workunit.client.0.smithi060.stdout:4/966: mkdir d0/da/d3b/d1b/df9/d137 0 2024-02-26T10:48:08.895 INFO:tasks.workunit.client.0.smithi060.stdout:3/801: dwrite d5/de/df/f81 [0,4194304] 0 2024-02-26T10:48:08.897 INFO:tasks.workunit.client.0.smithi060.stdout:3/802: mknod d5/de/df/d11/d63/d93/c111 0 2024-02-26T10:48:08.897 INFO:tasks.workunit.client.0.smithi060.stdout:3/803: chown d5/d45/d61/d92/da1/daa/dad/dfc 2306 1 2024-02-26T10:48:08.897 INFO:tasks.workunit.client.0.smithi060.stdout:3/804: write d5/de/df/d11/f20 [1193664,31904] 0 2024-02-26T10:48:08.897 INFO:tasks.workunit.client.0.smithi060.stdout:3/805: chown d5/de/df/d4c/f70 1007 1 2024-02-26T10:48:08.898 INFO:tasks.workunit.client.0.smithi060.stdout:4/967: dread d0/da/d3b/d16/f3a [0,4194304] 0 2024-02-26T10:48:08.898 INFO:tasks.workunit.client.0.smithi060.stdout:4/968: truncate d0/d27/d76/dbd/d8e/f112 480193 0 2024-02-26T10:48:08.898 INFO:tasks.workunit.client.0.smithi060.stdout:3/806: symlink d5/de/df/d11/de0/l112 0 2024-02-26T10:48:08.900 INFO:tasks.workunit.client.0.smithi060.stdout:4/969: truncate d0/d27/d76/dbd/d60/d11b/fe5 533583 0 2024-02-26T10:48:08.900 INFO:tasks.workunit.client.0.smithi060.stdout:4/970: chown d0/d27/d76/dbd/c63 125 1 2024-02-26T10:48:08.902 INFO:tasks.workunit.client.0.smithi060.stdout:4/971: symlink d0/da/dc7/db0/db5/dc0/l138 0 2024-02-26T10:48:08.903 INFO:tasks.workunit.client.0.smithi060.stdout:4/972: creat d0/d27/d76/d9f/d114/f139 x:0 0 0 2024-02-26T10:48:08.907 INFO:tasks.workunit.client.0.smithi060.stdout:4/973: symlink d0/d27/d76/dbd/d10e/de1/l13a 0 2024-02-26T10:48:08.907 INFO:tasks.workunit.client.0.smithi060.stdout:4/974: read - d0/da/dba/d116/f134 zero size 2024-02-26T10:48:08.909 INFO:tasks.workunit.client.0.smithi060.stdout:0/883: dwrite d0/dd/d39/d40/dad/d10d/f110 [0,4194304] 0 2024-02-26T10:48:08.909 INFO:tasks.workunit.client.0.smithi060.stdout:0/884: chown d0/dd/d39/d40/d83/dc6/d58/f5b 19 1 2024-02-26T10:48:08.912 INFO:tasks.workunit.client.0.smithi060.stdout:3/807: mknod d5/d45/d61/d92/c113 0 2024-02-26T10:48:08.912 INFO:tasks.workunit.client.0.smithi060.stdout:4/975: symlink d0/da/d3b/ddc/dfa/dfe/l13b 0 2024-02-26T10:48:08.914 INFO:tasks.workunit.client.0.smithi060.stdout:3/808: creat d5/dbb/dcf/dd0/f114 x:0 0 0 2024-02-26T10:48:08.914 INFO:tasks.workunit.client.0.smithi060.stdout:3/809: chown d5/d45/d61 770 1 2024-02-26T10:48:08.914 INFO:tasks.workunit.client.0.smithi060.stdout:4/976: mknod d0/da/d3b/c13c 0 2024-02-26T10:48:08.914 INFO:tasks.workunit.client.0.smithi060.stdout:3/810: truncate d5/de/df/d57/f74 102378 0 2024-02-26T10:48:08.915 INFO:tasks.workunit.client.0.smithi060.stdout:4/977: symlink d0/d27/d52/l13d 0 2024-02-26T10:48:08.917 INFO:tasks.workunit.client.0.smithi060.stdout:3/811: fdatasync d5/de/df/d11/f20 0 2024-02-26T10:48:08.917 INFO:tasks.workunit.client.0.smithi060.stdout:3/812: truncate d5/de/df/d57/d60/f3a 4543143 0 2024-02-26T10:48:08.918 INFO:tasks.workunit.client.0.smithi060.stdout:3/813: fsync d5/de/df/d4c/dcb/df8/f41 0 2024-02-26T10:48:08.918 INFO:tasks.workunit.client.0.smithi060.stdout:4/978: rmdir d0/da/d3b/d16 39 2024-02-26T10:48:08.926 INFO:tasks.workunit.client.0.smithi060.stdout:1/977: dwrite d3/d35/d53/f112 [0,4194304] 0 2024-02-26T10:48:08.927 INFO:tasks.workunit.client.0.smithi060.stdout:4/979: creat d0/da/f13e x:0 0 0 2024-02-26T10:48:08.928 INFO:tasks.workunit.client.0.smithi060.stdout:4/980: chown d0/da/d3b/d16/f3f 15 1 2024-02-26T10:48:08.928 INFO:tasks.workunit.client.0.smithi060.stdout:4/981: write d0/d27/d76/f8a [2238621,126478] 0 2024-02-26T10:48:08.928 INFO:tasks.workunit.client.0.smithi060.stdout:4/982: readlink d0/d27/le3 0 2024-02-26T10:48:08.929 INFO:tasks.workunit.client.0.smithi060.stdout:4/983: creat d0/da/d3b/d1b/f13f x:0 0 0 2024-02-26T10:48:08.930 INFO:tasks.workunit.client.0.smithi060.stdout:4/984: chown d0/c61 7 1 2024-02-26T10:48:08.930 INFO:tasks.workunit.client.0.smithi060.stdout:4/985: creat d0/da/dba/d116/f140 x:0 0 0 2024-02-26T10:48:08.932 INFO:tasks.workunit.client.0.smithi060.stdout:4/986: getdents d0/d27/d76/dbd/d119/dc5 0 2024-02-26T10:48:08.939 INFO:tasks.workunit.client.0.smithi060.stdout:1/978: write d3/db/d1e/f102 [3439687,57074] 0 2024-02-26T10:48:08.939 INFO:tasks.workunit.client.0.smithi060.stdout:1/979: truncate d3/db/d1e/d38/d6c/fb9 4934763 0 2024-02-26T10:48:08.941 INFO:tasks.workunit.client.0.smithi060.stdout:3/814: dread d5/de/f3b [0,4194304] 0 2024-02-26T10:48:08.941 INFO:tasks.workunit.client.0.smithi060.stdout:2/917: dwrite dd/d43/f10f [0,4194304] 0 2024-02-26T10:48:08.945 INFO:tasks.workunit.client.0.smithi060.stdout:1/980: mkdir d3/db/d2e/d149 0 2024-02-26T10:48:08.949 INFO:tasks.workunit.client.0.smithi060.stdout:0/885: dwrite d0/dd/d53/da2/ff9 [8388608,4194304] 0 2024-02-26T10:48:08.949 INFO:tasks.workunit.client.0.smithi060.stdout:0/886: chown d0/dd/d39/d40/dad/dd6/d6c/fff 30660 1 2024-02-26T10:48:08.950 INFO:tasks.workunit.client.0.smithi060.stdout:3/815: dread d5/de/df/d11/d63/d93/d97/f106 [0,4194304] 0 2024-02-26T10:48:08.950 INFO:tasks.workunit.client.0.smithi060.stdout:3/816: chown d5/de/cde 0 1 2024-02-26T10:48:08.952 INFO:tasks.workunit.client.0.smithi060.stdout:4/987: rename d0/da/d3b/ddc/dfa/f31 to d0/d27/d76/dbd/d8e/f141 0 2024-02-26T10:48:08.953 INFO:tasks.workunit.client.0.smithi060.stdout:4/988: creat d0/da/d3b/d1b/df9/d137/f142 x:0 0 0 2024-02-26T10:48:08.953 INFO:tasks.workunit.client.0.smithi060.stdout:4/989: truncate d0/f127 86809 0 2024-02-26T10:48:08.955 INFO:tasks.workunit.client.0.smithi060.stdout:5/898: sync 2024-02-26T10:48:08.955 INFO:tasks.workunit.client.0.smithi060.stdout:6/975: sync 2024-02-26T10:48:08.956 INFO:tasks.workunit.client.0.smithi060.stdout:8/955: sync 2024-02-26T10:48:08.957 INFO:tasks.workunit.client.0.smithi060.stdout:6/976: mkdir d6/d7/d25/d80/df9/d144 0 2024-02-26T10:48:08.957 INFO:tasks.workunit.client.0.smithi060.stdout:8/956: truncate d9/d15/d29/d2f/d34/d72/f4c 4192713 0 2024-02-26T10:48:08.957 INFO:tasks.workunit.client.0.smithi060.stdout:5/899: mknod d0/de/d20/c132 0 2024-02-26T10:48:08.958 INFO:tasks.workunit.client.0.smithi060.stdout:6/977: mknod d6/d7/d25/d130/c145 0 2024-02-26T10:48:08.958 INFO:tasks.workunit.client.0.smithi060.stdout:5/900: getdents d0/d4b/d7b 0 2024-02-26T10:48:08.960 INFO:tasks.workunit.client.0.smithi060.stdout:6/978: mknod d6/d7/d25/d1c/d1f/d23/d2b/d67/d10d/c146 0 2024-02-26T10:48:08.960 INFO:tasks.workunit.client.0.smithi060.stdout:6/979: write d6/d7/d2e/d56/f104 [610868,10010] 0 2024-02-26T10:48:08.960 INFO:tasks.workunit.client.0.smithi060.stdout:5/901: link d0/de/d20/d32/cd3 d0/d4b/d7b/c133 0 2024-02-26T10:48:08.967 INFO:tasks.workunit.client.0.smithi060.stdout:2/918: read dd/d43/d60/f113 [1238821,55130] 0 2024-02-26T10:48:08.972 INFO:tasks.workunit.client.0.smithi060.stdout:0/887: write d0/dd/d53/da2/ff9 [6978407,33084] 0 2024-02-26T10:48:08.972 INFO:tasks.workunit.client.0.smithi060.stdout:6/980: symlink d6/d7/d25/d80/df9/dd8/l147 0 2024-02-26T10:48:08.972 INFO:tasks.workunit.client.0.smithi060.stdout:5/902: mkdir d0/de/d20/d82/d134 0 2024-02-26T10:48:08.972 INFO:tasks.workunit.client.0.smithi060.stdout:2/919: symlink dd/d43/d55/d5e/d86/l133 0 2024-02-26T10:48:08.975 INFO:tasks.workunit.client.0.smithi060.stdout:0/888: rmdir d0/dd/d39/d93 39 2024-02-26T10:48:08.977 INFO:tasks.workunit.client.0.smithi060.stdout:5/903: mknod d0/d4b/c135 0 2024-02-26T10:48:08.977 INFO:tasks.workunit.client.0.smithi060.stdout:2/920: mknod dd/d43/d60/d41/c134 0 2024-02-26T10:48:08.979 INFO:tasks.workunit.client.0.smithi060.stdout:5/904: rename d0/de/d15/cb6 to d0/de/d15/d65/d86/dcc/c136 0 2024-02-26T10:48:08.980 INFO:tasks.workunit.client.0.smithi060.stdout:1/981: dread d3/d35/d53/d68/d85/f89 [0,4194304] 0 2024-02-26T10:48:08.985 INFO:tasks.workunit.client.0.smithi060.stdout:5/905: mknod d0/de/d20/d32/d4d/c137 0 2024-02-26T10:48:08.985 INFO:tasks.workunit.client.0.smithi060.stdout:5/906: chown d0/de/d70 49708 1 2024-02-26T10:48:08.985 INFO:tasks.workunit.client.0.smithi060.stdout:5/907: truncate d0/de/d15/d65/d86/d9b/d113/d114/fde 80083 0 2024-02-26T10:48:08.985 INFO:tasks.workunit.client.0.smithi060.stdout:0/889: stat d0/lee 0 2024-02-26T10:48:08.989 INFO:tasks.workunit.client.0.smithi060.stdout:0/890: mkdir d0/dd/d53/d125 0 2024-02-26T10:48:08.989 INFO:tasks.workunit.client.0.smithi060.stdout:0/891: dread - d0/dd/d53/d87/d94/da6/dc7/fe7 zero size 2024-02-26T10:48:08.989 INFO:tasks.workunit.client.0.smithi060.stdout:3/817: dread d5/d45/d61/d92/da1/ff0 [0,4194304] 0 2024-02-26T10:48:08.990 INFO:tasks.workunit.client.0.smithi060.stdout:3/818: write d5/de/df/d57/ff5 [590651,604] 0 2024-02-26T10:48:08.991 INFO:tasks.workunit.client.0.smithi060.stdout:3/819: truncate d5/de/d16/f6d 1080593 0 2024-02-26T10:48:08.991 INFO:tasks.workunit.client.0.smithi060.stdout:3/820: truncate d5/de/df/d4c/f6b 4820661 0 2024-02-26T10:48:08.992 INFO:tasks.workunit.client.0.smithi060.stdout:3/821: unlink d5/de/c73 0 2024-02-26T10:48:08.993 INFO:tasks.workunit.client.0.smithi060.stdout:3/822: truncate d5/de/df/d11/d63/d93/d97/d98/dd4/fff 423487 0 2024-02-26T10:48:08.999 INFO:tasks.workunit.client.0.smithi060.stdout:2/921: rename dd/d43/d60/d41/d37/d38/d57/fe8 to dd/d43/d55/d5e/d86/d97/d9c/f135 0 2024-02-26T10:48:09.012 INFO:tasks.workunit.client.0.smithi060.stdout:1/982: dread d3/d35/d5e/fe4 [0,4194304] 0 2024-02-26T10:48:09.015 INFO:tasks.workunit.client.0.smithi060.stdout:1/983: creat d3/db/d1e/d6d/d78/f14a x:0 0 0 2024-02-26T10:48:09.016 INFO:tasks.workunit.client.0.smithi060.stdout:1/984: chown d3/db/d1e/d38/d6c/cca 571 1 2024-02-26T10:48:09.016 INFO:tasks.workunit.client.0.smithi060.stdout:1/985: symlink d3/db/d1e/d6d/da3/daa/l14b 0 2024-02-26T10:48:09.026 INFO:tasks.workunit.client.0.smithi060.stdout:0/892: write d0/dd/d39/f61 [3628026,59522] 0 2024-02-26T10:48:09.026 INFO:tasks.workunit.client.0.smithi060.stdout:0/893: stat d0/dd/d53/d125 0 2024-02-26T10:48:09.026 INFO:tasks.workunit.client.0.smithi060.stdout:0/894: chown d0/dd/d39/d40/f26 7868 1 2024-02-26T10:48:09.034 INFO:tasks.workunit.client.0.smithi060.stdout:8/957: dwrite d9/d15/f36 [0,4194304] 0 2024-02-26T10:48:09.038 INFO:tasks.workunit.client.0.smithi060.stdout:8/958: mknod d9/d15/d29/d2f/d34/d72/d94/dba/d13f/c142 0 2024-02-26T10:48:09.043 INFO:tasks.workunit.client.0.smithi060.stdout:3/823: dread d5/de/df/faf [0,4194304] 0 2024-02-26T10:48:09.051 INFO:tasks.workunit.client.0.smithi060.stdout:1/986: read d3/db/ff3 [2230089,65144] 0 2024-02-26T10:48:09.052 INFO:tasks.workunit.client.0.smithi060.stdout:2/922: dwrite dd/d43/d96/df2/f129 [0,4194304] 0 2024-02-26T10:48:09.053 INFO:tasks.workunit.client.0.smithi060.stdout:1/987: rmdir d3/db/d1e/d39/d9b/df7 39 2024-02-26T10:48:09.055 INFO:tasks.workunit.client.0.smithi060.stdout:0/895: dread d0/f3 [0,4194304] 0 2024-02-26T10:48:09.055 INFO:tasks.workunit.client.0.smithi060.stdout:3/824: creat d5/dbb/dcf/dd0/f115 x:0 0 0 2024-02-26T10:48:09.057 INFO:tasks.workunit.client.0.smithi060.stdout:3/825: rename d5/d45/d61/la7 to d5/d45/d61/d92/dec/l116 0 2024-02-26T10:48:09.058 INFO:tasks.workunit.client.0.smithi060.stdout:0/896: symlink d0/d42/d7a/l126 0 2024-02-26T10:48:09.060 INFO:tasks.workunit.client.0.smithi060.stdout:0/897: getdents d0/dd/d39/d40/d83/dc0 0 2024-02-26T10:48:09.065 INFO:tasks.workunit.client.0.smithi060.stdout:8/959: dread d9/d15/ff4 [0,4194304] 0 2024-02-26T10:48:09.066 INFO:tasks.workunit.client.0.smithi060.stdout:5/908: dwrite d0/de/d15/d65/f9d [0,4194304] 0 2024-02-26T10:48:09.070 INFO:tasks.workunit.client.0.smithi060.stdout:1/988: link d3/db/d1e/d39/l11b d3/d35/d53/l14c 0 2024-02-26T10:48:09.071 INFO:tasks.workunit.client.0.smithi060.stdout:5/909: symlink d0/de/d20/d32/l138 0 2024-02-26T10:48:09.071 INFO:tasks.workunit.client.0.smithi060.stdout:5/910: write d0/de/d15/d65/fe2 [380511,18706] 0 2024-02-26T10:48:09.072 INFO:tasks.workunit.client.0.smithi060.stdout:5/911: getdents d0/de/d20/d32/d4d/da3/da5 0 2024-02-26T10:48:09.073 INFO:tasks.workunit.client.0.smithi060.stdout:5/912: fdatasync d0/de/d20/d32/d4d/da3/fcf 0 2024-02-26T10:48:09.075 INFO:tasks.workunit.client.0.smithi060.stdout:1/989: write d3/d35/d53/d68/d90/d125/fb4 [792828,22579] 0 2024-02-26T10:48:09.079 INFO:tasks.workunit.client.0.smithi060.stdout:1/990: creat d3/db/d1e/d6d/da3/daa/dcd/f14d x:0 0 0 2024-02-26T10:48:09.079 INFO:tasks.workunit.client.0.smithi060.stdout:1/991: dread - d3/db/d1e/d6d/d78/f14a zero size 2024-02-26T10:48:09.079 INFO:tasks.workunit.client.0.smithi060.stdout:1/992: write d3/db/d2e/d51/fac [4814606,89313] 0 2024-02-26T10:48:09.079 INFO:tasks.workunit.client.0.smithi060.stdout:1/993: truncate d3/db/d2e/f13b 126844 0 2024-02-26T10:48:09.081 INFO:tasks.workunit.client.0.smithi060.stdout:0/898: dread d0/d42/fd7 [0,4194304] 0 2024-02-26T10:48:09.086 INFO:tasks.workunit.client.0.smithi060.stdout:1/994: write d3/d35/d53/f74 [162027,35456] 0 2024-02-26T10:48:09.087 INFO:tasks.workunit.client.0.smithi060.stdout:1/995: readlink d3/db/d1e/d6d/lcf 0 2024-02-26T10:48:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.095+0000 7f0047200640 1 -- 172.21.15.60:0/685181521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0040075c20 msgr2=0x7f0040076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.095+0000 7f0047200640 1 --2- 172.21.15.60:0/685181521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0040075c20 0x7f0040076040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f002c01a910 tx=0x7f002c03cc70 comp rx=0 tx=0).stop 2024-02-26T10:48:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.095+0000 7f0047200640 1 -- 172.21.15.60:0/685181521 shutdown_connections 2024-02-26T10:48:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.095+0000 7f0047200640 1 --2- 172.21.15.60:0/685181521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0040076e60 0x7f00400772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.095+0000 7f0047200640 1 --2- 172.21.15.60:0/685181521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0040077800 0x7f00400780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.095+0000 7f0047200640 1 --2- 172.21.15.60:0/685181521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0040075c20 0x7f0040076040 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.095+0000 7f0047200640 1 -- 172.21.15.60:0/685181521 >> 172.21.15.60:0/685181521 conn(0x7f00400fea50 msgr2=0x7f0040100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:09.090 INFO:tasks.workunit.client.0.smithi060.stdout:3/826: dread d5/de/df/d4c/f6b [0,4194304] 0 2024-02-26T10:48:09.090 INFO:tasks.workunit.client.0.smithi060.stdout:0/899: getdents d0/d42/df2 0 2024-02-26T10:48:09.090 INFO:tasks.workunit.client.0.smithi060.stdout:0/900: write d0/dd/d39/d63/fe5 [544975,24347] 0 2024-02-26T10:48:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.095+0000 7f0047200640 1 -- 172.21.15.60:0/685181521 shutdown_connections 2024-02-26T10:48:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.095+0000 7f0047200640 1 -- 172.21.15.60:0/685181521 wait complete. 2024-02-26T10:48:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.096+0000 7f0047200640 1 Processor -- start 2024-02-26T10:48:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.097+0000 7f0047200640 1 -- start start 2024-02-26T10:48:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.097+0000 7f0047200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0040075c20 0x7f0040195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.091 INFO:tasks.workunit.client.0.smithi060.stdout:3/827: creat d5/dbb/f117 x:0 0 0 2024-02-26T10:48:09.091 INFO:tasks.workunit.client.0.smithi060.stdout:3/828: readlink d5/de/d16/l40 0 2024-02-26T10:48:09.091 INFO:tasks.workunit.client.0.smithi060.stdout:3/829: write d5/d45/d61/d92/ffd [187881,31712] 0 2024-02-26T10:48:09.091 INFO:tasks.workunit.client.0.smithi060.stdout:3/830: fdatasync d5/de/df/d4c/f70 0 2024-02-26T10:48:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.097+0000 7f0047200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0040076e60 0x7f00401957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.097+0000 7f0047200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0040077800 0x7f004019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.097+0000 7f0047200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040104d80 con 0x7f0040075c20 2024-02-26T10:48:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.097+0000 7f0047200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040104c00 con 0x7f0040077800 2024-02-26T10:48:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.097+0000 7f0047200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040104f00 con 0x7f0040076e60 2024-02-26T10:48:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.098+0000 7f003f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0040076e60 0x7f00401957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.098+0000 7f0045200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0040077800 0x7f004019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.098+0000 7f003f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0040076e60 0x7f00401957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:36438/0 (socket says 172.21.15.60:36438) 2024-02-26T10:48:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.098+0000 7f003f400640 1 -- 172.21.15.60:0/914993998 learned_addr learned my addr 172.21.15.60:0/914993998 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.098+0000 7f003f400640 1 -- 172.21.15.60:0/914993998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0040077800 msgr2=0x7f004019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.098+0000 7f003f400640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0040077800 0x7f004019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.098+0000 7f003f400640 1 -- 172.21.15.60:0/914993998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0040075c20 msgr2=0x7f0040195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.098+0000 7f003f400640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0040075c20 0x7f0040195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.098+0000 7f003f400640 1 -- 172.21.15.60:0/914993998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004019cea0 con 0x7f0040076e60 2024-02-26T10:48:09.093 INFO:tasks.workunit.client.0.smithi060.stdout:3/831: creat d5/de/df/d11/d63/d93/d97/d98/dd4/f118 x:0 0 0 2024-02-26T10:48:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.099+0000 7f003f400640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0040076e60 0x7f00401957c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f003000ea60 tx=0x7f003000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.099+0000 7f003cc00640 1 -- 172.21.15.60:0/914993998 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003000cde0 con 0x7f0040076e60 2024-02-26T10:48:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.099+0000 7f003cc00640 1 -- 172.21.15.60:0/914993998 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00300035f0 con 0x7f0040076e60 2024-02-26T10:48:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.099+0000 7f0047200640 1 -- 172.21.15.60:0/914993998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004019d190 con 0x7f0040076e60 2024-02-26T10:48:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.099+0000 7f0047200640 1 -- 172.21.15.60:0/914993998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004019d730 con 0x7f0040076e60 2024-02-26T10:48:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.099+0000 7f003cc00640 1 -- 172.21.15.60:0/914993998 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0030010670 con 0x7f0040076e60 2024-02-26T10:48:09.095 INFO:tasks.workunit.client.0.smithi060.stdout:0/901: rmdir d0/d42/df2 39 2024-02-26T10:48:09.096 INFO:tasks.workunit.client.0.smithi060.stdout:0/902: fsync d0/dd/f2b 0 2024-02-26T10:48:09.096 INFO:tasks.workunit.client.0.smithi060.stdout:0/903: write d0/dd/d39/f61 [4920421,33983] 0 2024-02-26T10:48:09.096 INFO:tasks.workunit.client.0.smithi060.stdout:0/904: write d0/dd/f124 [613973,80593] 0 2024-02-26T10:48:09.096 INFO:tasks.workunit.client.0.smithi060.stdout:0/905: write d0/d42/df2/f11e [16727,59022] 0 2024-02-26T10:48:09.096 INFO:tasks.workunit.client.0.smithi060.stdout:8/960: dread d9/d15/d29/d2f/d34/d72/d94/dba/f132 [0,4194304] 0 2024-02-26T10:48:09.096 INFO:tasks.workunit.client.0.smithi060.stdout:8/961: write d9/d15/d29/d2f/d34/d72/d50/d6a/d63/d89/ff2 [707593,16430] 0 2024-02-26T10:48:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.103+0000 7f003cc00640 1 -- 172.21.15.60:0/914993998 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0030004000 con 0x7f0040076e60 2024-02-26T10:48:09.099 INFO:tasks.workunit.client.0.smithi060.stdout:8/962: unlink d9/d15/d29/d2f/d34/d72/f4c 0 2024-02-26T10:48:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.107+0000 7f003cc00640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f00240a1ef0 0x7f00240a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.107+0000 7f003cc00640 1 -- 172.21.15.60:0/914993998 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f00300ce340 con 0x7f0040076e60 2024-02-26T10:48:09.102 INFO:tasks.workunit.client.0.smithi060.stdout:8/963: symlink d9/d15/d29/d2f/d131/l143 0 2024-02-26T10:48:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.109+0000 7f004a1b8740 1 Processor -- start 2024-02-26T10:48:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.109+0000 7f004a1b8740 1 -- start start 2024-02-26T10:48:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.109+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82515c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.109+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82516210 0x55eb8251ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.109+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb8251b380 0x55eb8251d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.109+0000 7f003fe00640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f00240a1ef0 0x7f00240a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.110+0000 7f003fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82515c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.110+0000 7f003fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82515c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41484/0 (socket says 172.21.15.60:41484) 2024-02-26T10:48:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.110+0000 7f003fe00640 1 -- 172.21.15.60:0/3627934770 learned_addr learned my addr 172.21.15.60:0/3627934770 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.110+0000 7f004a1b8740 1 -- 172.21.15.60:0/3627934770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8247b340 con 0x55eb82515820 2024-02-26T10:48:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.110+0000 7f004a1b8740 1 -- 172.21.15.60:0/3627934770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb82424520 con 0x55eb82516210 2024-02-26T10:48:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.110+0000 7f004a1b8740 1 -- 172.21.15.60:0/3627934770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb824773e0 con 0x55eb8251b380 2024-02-26T10:48:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.110+0000 7f003f400640 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82516210 0x55eb8251ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.111+0000 7f0045200640 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb8251b380 0x55eb8251d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.111+0000 7f0045200640 1 -- 172.21.15.60:0/3627934770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82516210 msgr2=0x55eb8251ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.111+0000 7f0045200640 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82516210 0x55eb8251ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.111+0000 7f0045200640 1 -- 172.21.15.60:0/3627934770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 msgr2=0x55eb82515c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.111+0000 7f0045200640 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82515c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.111+0000 7f0045200640 1 -- 172.21.15.60:0/3627934770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb82438390 con 0x55eb8251b380 2024-02-26T10:48:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.112+0000 7f0045200640 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb8251b380 0x55eb8251d810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0038007e40 tx=0x7f003800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c05963215793e054 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.112+0000 7f0035600640 1 -- 172.21.15.60:0/3627934770 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038003b70 con 0x55eb8251b380 2024-02-26T10:48:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.112+0000 7f0035600640 1 -- 172.21.15.60:0/3627934770 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0038003d10 con 0x55eb8251b380 2024-02-26T10:48:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.112+0000 7f0035600640 1 -- 172.21.15.60:0/3627934770 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038004520 con 0x55eb8251b380 2024-02-26T10:48:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.112+0000 7f003fe00640 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82515c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:48:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.112+0000 7f003f400640 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82516210 0x55eb8251ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:48:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.113+0000 7f004a1b8740 1 -- 172.21.15.60:0/3627934770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb8251b380 msgr2=0x55eb8251d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.113+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb8251b380 0x55eb8251d810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0038007e40 tx=0x7f003800f790 comp rx=0 tx=0).stop 2024-02-26T10:48:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.114+0000 7f004a1b8740 1 -- 172.21.15.60:0/3627934770 shutdown_connections 2024-02-26T10:48:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.114+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82516210 0x55eb8251ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.114+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb8251b380 0x55eb8251d810 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.114+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3627934770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82515c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.114+0000 7f003fe00640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f00240a1ef0 0x7f00240a43b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f002c005c20 tx=0x7f002c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.114+0000 7f004a1b8740 1 -- 172.21.15.60:0/3627934770 >> 172.21.15.60:0/3627934770 conn(0x55eb82510d90 msgr2=0x55eb82511190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.114+0000 7f004a1b8740 1 -- 172.21.15.60:0/3627934770 shutdown_connections 2024-02-26T10:48:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.114+0000 7f004a1b8740 1 -- 172.21.15.60:0/3627934770 wait complete. 2024-02-26T10:48:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.115+0000 7f004a1b8740 1 -- start start 2024-02-26T10:48:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.116+0000 7f004a1b8740 1 Processor -- start 2024-02-26T10:48:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.117+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82515820 0x55eb8251fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.117+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb8251ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.117+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb8251b380 0x55eb82526f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.117+0000 7f004a1b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8251e330 con 0x55eb8251b380 2024-02-26T10:48:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.117+0000 7f004a1b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb82437980 con 0x55eb82515820 2024-02-26T10:48:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.117+0000 7f004a1b8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8251e4b0 con 0x55eb82516210 2024-02-26T10:48:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.118+0000 7f0045200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb8251b380 0x55eb82526f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.118+0000 7f003f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb8251ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.118+0000 7f003f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb8251ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:36456/0 (socket says 172.21.15.60:36456) 2024-02-26T10:48:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.118+0000 7f003f400640 1 -- 172.21.15.60:0/975674698 learned_addr learned my addr 172.21.15.60:0/975674698 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.118+0000 7f004a1b8740 10 client.0 _ll_register_callbacks cb 0x55eb82473310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f003fe00640 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82515820 0x55eb8251fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f003f400640 1 -- 172.21.15.60:0/975674698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82515820 msgr2=0x55eb8251fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f003f400640 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82515820 0x55eb8251fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f003f400640 1 -- 172.21.15.60:0/975674698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb8251b380 msgr2=0x55eb82526f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f003f400640 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb8251b380 0x55eb82526f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f003f400640 1 -- 172.21.15.60:0/975674698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb82190210 con 0x55eb82516210 2024-02-26T10:48:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f003fe00640 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82515820 0x55eb8251fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:48:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f0045200640 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb8251b380 0x55eb82526f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:48:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f003f400640 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb8251ff40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0030097120 tx=0x7f003009ad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b60d9fe599899e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.119+0000 7f002a000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-02-26T10:48:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.120+0000 7f002a000640 1 -- 172.21.15.60:0/975674698 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00300b2020 con 0x55eb82516210 2024-02-26T10:48:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.120+0000 7f002a000640 1 -- 172.21.15.60:0/975674698 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f0030097330 con 0x55eb82516210 2024-02-26T10:48:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.120+0000 7f004a1b8740 1 -- 172.21.15.60:0/975674698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0038003d10 con 0x55eb82516210 2024-02-26T10:48:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.120+0000 7f002a000640 1 -- 172.21.15.60:0/975674698 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00300ce160 con 0x55eb82516210 2024-02-26T10:48:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.120+0000 7f004a1b8740 10 client.26110 fetch_fsmap learned FSMap version 30 2024-02-26T10:48:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.120+0000 7f004a1b8740 1 -- 172.21.15.60:0/975674698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55eb8225c620 con 0x55eb82516210 2024-02-26T10:48:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.124+0000 7f002a000640 1 -- 172.21.15.60:0/975674698 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0038003d10 con 0x55eb82516210 2024-02-26T10:48:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.124+0000 7f002a000640 1 -- 172.21.15.60:0/975674698 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f003009a220 con 0x55eb82516210 2024-02-26T10:48:09.118 INFO:tasks.workunit.client.0.smithi060.stdout:1/996: rename d3/d35/d53/d68/c135 to d3/d35/c14e 0 2024-02-26T10:48:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.124+0000 7f004a1b8740 10 client.26110 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:48:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.125+0000 7f004a1b8740 10 client.26110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.125+0000 7f004a1b8740 20 client.26110 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.125+0000 7f004a1b8740 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55eb8252b8a0 0x55eb8252dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.125+0000 7f004a1b8740 4 client.26110 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:48:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.125+0000 7f004a1b8740 1 -- 172.21.15.60:0/975674698 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb82488880 con 0x55eb8252b8a0 2024-02-26T10:48:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.126+0000 7f003fe00640 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55eb8252b8a0 0x55eb8252dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.127+0000 7f003fe00640 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55eb8252b8a0 0x55eb8252dce0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b816b392d476eb83 server_cookie=cc5b6199555f24fb in_seq=0 out_seq=0 2024-02-26T10:48:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.127+0000 7f002a000640 10 client.26110 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.128+0000 7f002a000640 1 -- 172.21.15.60:0/975674698 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb82488880 con 0x55eb8252b8a0 2024-02-26T10:48:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.128+0000 7f002a000640 10 client.26110 handle_command_reply: tid=0 2024-02-26T10:48:09.123 INFO:tasks.workunit.client.0.smithi060.stdout:2/923: dwrite dd/d43/d55/d6e/dd3/fb2 [0,4194304] 0 2024-02-26T10:48:09.123 INFO:tasks.workunit.client.0.smithi060.stdout:2/924: readlink dd/d43/d55/d6e/l111 0 2024-02-26T10:48:09.123 INFO:tasks.workunit.client.0.smithi060.stdout:2/925: stat dd/d43/d60/d41/d37/d69/f7e 0 2024-02-26T10:48:09.124 INFO:tasks.workunit.client.0.smithi060.stdout:1/997: truncate d3/db/d1e/f102 3973769 0 2024-02-26T10:48:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.129+0000 7f004a1b8740 1 client.26110 shutdown 2024-02-26T10:48:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.132+0000 7f004a1b8740 1 -- 172.21.15.60:0/975674698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 msgr2=0x55eb8251ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.132+0000 7f004a1b8740 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb8251ff40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0030097120 tx=0x7f003009ad80 comp rx=0 tx=0).stop 2024-02-26T10:48:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.132+0000 7f004a1b8740 1 -- 172.21.15.60:0/975674698 shutdown_connections 2024-02-26T10:48:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.133+0000 7f004a1b8740 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55eb8252b8a0 0x55eb8252dce0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.133+0000 7f004a1b8740 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb8251b380 0x55eb82526f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.133+0000 7f004a1b8740 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb8251ff40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.133+0000 7f004a1b8740 1 --2- 172.21.15.60:0/975674698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb82515820 0x55eb8251fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.133+0000 7f004a1b8740 1 -- 172.21.15.60:0/975674698 >> 172.21.15.60:0/975674698 conn(0x55eb82510d90 msgr2=0x55eb8250ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.133+0000 7f002a000640 0 client.26110 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.133+0000 7f004a1b8740 1 -- 172.21.15.60:0/975674698 shutdown_connections 2024-02-26T10:48:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.133+0000 7f004a1b8740 1 -- 172.21.15.60:0/975674698 wait complete. 2024-02-26T10:48:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.134+0000 7f004a1b8740 20 client.26110 trim_cache size 0 max 16384 2024-02-26T10:48:09.140 INFO:tasks.workunit.client.0.smithi060.stdout:1/998: link d3/db/f56 d3/db/d1e/d6d/d78/dc4/d121/d145/f14f 0 2024-02-26T10:48:09.140 INFO:tasks.workunit.client.0.smithi060.stdout:1/999: truncate d3/db/d1e/d6d/fc3 537120 0 2024-02-26T10:48:09.141 INFO:tasks.workunit.client.0.smithi060.stdout:5/913: dwrite d0/d4b/f57 [0,4194304] 0 2024-02-26T10:48:09.141 INFO:tasks.workunit.client.0.smithi060.stdout:5/914: write d0/de/d20/d82/f11b [971774,62544] 0 2024-02-26T10:48:09.147 INFO:tasks.workunit.client.0.smithi060.stdout:3/832: dwrite d5/de/df/f33 [0,4194304] 0 2024-02-26T10:48:09.151 INFO:tasks.workunit.client.0.smithi060.stdout:5/915: symlink d0/de/d15/d65/d86/dcc/l139 0 2024-02-26T10:48:09.151 INFO:tasks.workunit.client.0.smithi060.stdout:5/916: stat d0/de/d20/d32/d4c/d12d 0 2024-02-26T10:48:09.151 INFO:tasks.workunit.client.0.smithi060.stdout:2/926: dread dd/d43/d55/f107 [0,4194304] 0 2024-02-26T10:48:09.152 INFO:tasks.workunit.client.0.smithi060.stdout:3/833: rename d5/de/df/d11/de0/l112 to d5/dbb/dcf/dd0/l119 0 2024-02-26T10:48:09.153 INFO:tasks.workunit.client.0.smithi060.stdout:6/981: sync 2024-02-26T10:48:09.155 INFO:tasks.workunit.client.0.smithi060.stdout:3/834: mknod d5/dbb/dcf/c11a 0 2024-02-26T10:48:09.155 INFO:tasks.workunit.client.0.smithi060.stdout:4/990: sync 2024-02-26T10:48:09.155 INFO:tasks.workunit.client.0.smithi060.stdout:4/991: fsync d0/da/dc7/db0/db5/dc0/f8 0 2024-02-26T10:48:09.156 INFO:tasks.workunit.client.0.smithi060.stdout:2/927: rename dd/d43/d55/d5e/d86/d97/d9c/fea to dd/d43/d55/d5e/d86/d97/d9c/f136 0 2024-02-26T10:48:09.161 INFO:tasks.workunit.client.0.smithi060.stdout:8/964: dwrite d9/da4/f123 [0,4194304] 0 2024-02-26T10:48:09.162 INFO:tasks.workunit.client.0.smithi060.stdout:8/965: creat d9/d15/d3e/d127/f144 x:0 0 0 2024-02-26T10:48:09.164 INFO:tasks.workunit.client.0.smithi060.stdout:6/982: write d6/d6e/d6f/fcd [1375461,99946] 0 2024-02-26T10:48:09.164 INFO:tasks.workunit.client.0.smithi060.stdout:6/983: write d6/d7/d25/d1c/d1f/d23/d28/f34 [2128243,89680] 0 2024-02-26T10:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.174+0000 7f004a1b8740 1 Processor -- start 2024-02-26T10:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.174+0000 7f004a1b8740 1 -- start start 2024-02-26T10:48:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.174+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82515820 0x55eb8251d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.174+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82516210 0x55eb82526f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.174+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb8251fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.175+0000 7f004a1b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0020012060 con 0x55eb82516210 2024-02-26T10:48:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.175+0000 7f004a1b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002000fb40 con 0x55eb8251b380 2024-02-26T10:48:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.175+0000 7f004a1b8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002000dbe0 con 0x55eb82515820 2024-02-26T10:48:09.169 INFO:tasks.workunit.client.0.smithi060.stdout:6/984: mkdir d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/d148 0 2024-02-26T10:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.175+0000 7f003fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82515820 0x55eb8251d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.175+0000 7f003fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82515820 0x55eb8251d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:36462/0 (socket says 172.21.15.60:36462) 2024-02-26T10:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.175+0000 7f003fe00640 1 -- 172.21.15.60:0/2726894252 learned_addr learned my addr 172.21.15.60:0/2726894252 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.175+0000 7f0045200640 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb8251fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.176+0000 7f003f400640 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82516210 0x55eb82526f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.171 INFO:tasks.workunit.client.0.smithi060.stdout:2/928: unlink dd/d11/f3e 0 2024-02-26T10:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.176+0000 7f0045200640 1 -- 172.21.15.60:0/2726894252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82515820 msgr2=0x55eb8251d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.176+0000 7f0045200640 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82515820 0x55eb8251d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.176+0000 7f0045200640 1 -- 172.21.15.60:0/2726894252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82516210 msgr2=0x55eb82526f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.176+0000 7f0045200640 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82516210 0x55eb82526f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.176+0000 7f0045200640 1 -- 172.21.15.60:0/2726894252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb82520730 con 0x55eb8251b380 2024-02-26T10:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.176+0000 7f003fe00640 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82515820 0x55eb8251d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.178+0000 7f0045200640 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb8251fa00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f003800fc90 tx=0x7f0038007b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945ac55d553dc60 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.178+0000 7f0034c00640 1 -- 172.21.15.60:0/2726894252 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038026070 con 0x55eb8251b380 2024-02-26T10:48:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.178+0000 7f0034c00640 1 -- 172.21.15.60:0/2726894252 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0038003db0 con 0x55eb8251b380 2024-02-26T10:48:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.178+0000 7f0034c00640 1 -- 172.21.15.60:0/2726894252 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003801d040 con 0x55eb8251b380 2024-02-26T10:48:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.179+0000 7f004a1b8740 1 -- 172.21.15.60:0/2726894252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 msgr2=0x55eb8251fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.179+0000 7f004a1b8740 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb8251fa00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f003800fc90 tx=0x7f0038007b40 comp rx=0 tx=0).stop 2024-02-26T10:48:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.180+0000 7f004a1b8740 1 -- 172.21.15.60:0/2726894252 shutdown_connections 2024-02-26T10:48:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.181+0000 7f004a1b8740 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb8251fa00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.181+0000 7f004a1b8740 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82516210 0x55eb82526f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.181+0000 7f004a1b8740 1 --2- 172.21.15.60:0/2726894252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82515820 0x55eb8251d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.181+0000 7f004a1b8740 1 -- 172.21.15.60:0/2726894252 >> 172.21.15.60:0/2726894252 conn(0x55eb82510d90 msgr2=0x55eb82532640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:09.175 INFO:tasks.workunit.client.0.smithi060.stdout:0/906: dwrite d0/dd/f77 [0,4194304] 0 2024-02-26T10:48:09.176 INFO:tasks.workunit.client.0.smithi060.stdout:0/907: truncate d0/dd/fbb 101051 0 2024-02-26T10:48:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.183+0000 7f004a1b8740 1 -- 172.21.15.60:0/2726894252 shutdown_connections 2024-02-26T10:48:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.184+0000 7f004a1b8740 1 -- 172.21.15.60:0/2726894252 wait complete. 2024-02-26T10:48:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.184+0000 7f004a1b8740 1 -- start start 2024-02-26T10:48:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.185+0000 7f004a1b8740 1 Processor -- start 2024-02-26T10:48:09.179 INFO:tasks.workunit.client.0.smithi060.stdout:0/908: mkdir d0/d42/d7a/d108/d111/d127 0 2024-02-26T10:48:09.179 INFO:tasks.workunit.client.0.smithi060.stdout:0/909: fsync d0/dd/d39/d40/f26 0 2024-02-26T10:48:09.179 INFO:tasks.workunit.client.0.smithi060.stdout:0/910: fsync d0/dd/d39/fb8 0 2024-02-26T10:48:09.179 INFO:tasks.workunit.client.0.smithi060.stdout:4/992: dread d0/da/d3b/d16/f19 [0,4194304] 0 2024-02-26T10:48:09.179 INFO:tasks.workunit.client.0.smithi060.stdout:4/993: write d0/d27/d76/dbd/d8e/f99 [790215,35933] 0 2024-02-26T10:48:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.186+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82524b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.186+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb82525080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.186+0000 7f004a1b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb82526090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.187+0000 7f004a1b8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002000e1e0 con 0x55eb82515820 2024-02-26T10:48:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.187+0000 7f004a1b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002000f4e0 con 0x55eb8251b380 2024-02-26T10:48:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.187+0000 7f004a1b8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002000e820 con 0x55eb82516210 2024-02-26T10:48:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.187+0000 7f003fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82524b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.187+0000 7f003fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82524b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41516/0 (socket says 172.21.15.60:41516) 2024-02-26T10:48:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.187+0000 7f003fe00640 1 -- 172.21.15.60:0/3307477648 learned_addr learned my addr 172.21.15.60:0/3307477648 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.187+0000 7f0045200640 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb82526090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.188+0000 7f003f400640 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb82525080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.188+0000 7f0045200640 1 -- 172.21.15.60:0/3307477648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 msgr2=0x55eb82525080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.188+0000 7f0045200640 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb82525080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.188+0000 7f0045200640 1 -- 172.21.15.60:0/3307477648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 msgr2=0x55eb82524b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.188+0000 7f0045200640 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82524b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.188+0000 7f0045200640 1 -- 172.21.15.60:0/3307477648 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb82526680 con 0x55eb8251b380 2024-02-26T10:48:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.189+0000 7f004a1b8740 10 client.0 _ll_register_callbacks cb 0x7f0038007940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.189+0000 7f004a1b8740 1 -- 172.21.15.60:0/3307477648 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0038003db0 con 0x55eb8251b380 2024-02-26T10:48:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.192+0000 7f003f400640 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb82525080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:48:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.192+0000 7f0045200640 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb82526090 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0038007bd0 tx=0x7f0038003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9696a7809f6fa10 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.192+0000 7f002a000640 10 client.26117 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.193+0000 7f002a000640 1 -- 172.21.15.60:0/3307477648 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038026040 con 0x55eb8251b380 2024-02-26T10:48:09.187 INFO:tasks.workunit.client.0.smithi060.stdout:6/985: dread d6/d7/d25/d1c/d1f/d23/f3a [0,4194304] 0 2024-02-26T10:48:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.193+0000 7f002a000640 1 -- 172.21.15.60:0/3307477648 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f0038005900 con 0x55eb8251b380 2024-02-26T10:48:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.193+0000 7f002a000640 1 -- 172.21.15.60:0/3307477648 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0038010640 con 0x55eb8251b380 2024-02-26T10:48:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.193+0000 7f004a1b8740 10 client.26117 fetch_fsmap learned FSMap version 30 2024-02-26T10:48:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.194+0000 7f004a1b8740 1 -- 172.21.15.60:0/3307477648 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x55eb825820b0 con 0x55eb8251b380 2024-02-26T10:48:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.194+0000 7f002a000640 1 -- 172.21.15.60:0/3307477648 <== mon.2 v2:172.21.15.100:3300/0 4 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f00380107e0 con 0x55eb8251b380 2024-02-26T10:48:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.195+0000 7f004a1b8740 10 client.26117 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:48:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.195+0000 7f004a1b8740 10 client.26117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.195+0000 7f004a1b8740 20 client.26117 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.195+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55eb825826e0 0x55eb82584b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.195+0000 7f004a1b8740 4 client.26117 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:48:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.195+0000 7f004a1b8740 1 -- 172.21.15.60:0/3307477648 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eb82482bd0 con 0x55eb825826e0 2024-02-26T10:48:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.195+0000 7f003fe00640 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55eb825826e0 0x55eb82584b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.196+0000 7f003fe00640 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55eb825826e0 0x55eb82584b00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484bb93ea5afd852 server_cookie=8f664bcacc5d2851 in_seq=0 out_seq=0 2024-02-26T10:48:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.196+0000 7f002a000640 10 client.26117 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:09.195 INFO:tasks.workunit.client.0.smithi060.stdout:4/994: rename d0/da/dc7/db0/db5/dc0/f37 to d0/d27/d76/dbd/d60/f143 0 2024-02-26T10:48:09.197 INFO:tasks.workunit.client.0.smithi060.stdout:6/986: mknod d6/c149 0 2024-02-26T10:48:09.197 INFO:tasks.workunit.client.0.smithi060.stdout:0/911: symlink d0/dd/d53/da2/l128 0 2024-02-26T10:48:09.198 INFO:tasks.workunit.client.0.smithi060.stdout:0/912: mknod d0/dd/d39/d93/c129 0 2024-02-26T10:48:09.200 INFO:tasks.workunit.client.0.smithi060.stdout:4/995: rename d0/f130 to d0/da/dc7/db0/d111/f144 0 2024-02-26T10:48:09.206 INFO:tasks.workunit.client.0.smithi060.stdout:5/917: dwrite d0/de/d15/d65/fa0 [0,4194304] 0 2024-02-26T10:48:09.219 INFO:tasks.workunit.client.0.smithi060.stdout:8/966: dwrite d9/d5a/d58/f8f [0,4194304] 0 2024-02-26T10:48:09.219 INFO:tasks.workunit.client.0.smithi060.stdout:8/967: chown d9/d15/d29/d2f/f31 1 1 2024-02-26T10:48:09.219 INFO:tasks.workunit.client.0.smithi060.stdout:8/968: fdatasync d9/d15/d29/d2f/d34/f8c 0 2024-02-26T10:48:09.221 INFO:tasks.workunit.client.0.smithi060.stdout:8/969: unlink d9/d15/c99 0 2024-02-26T10:48:09.224 INFO:tasks.workunit.client.0.smithi060.stdout:8/970: mknod d9/d15/d29/d2f/d131/c145 0 2024-02-26T10:48:09.224 INFO:tasks.workunit.client.0.smithi060.stdout:2/929: dwrite dd/d43/d60/d41/d37/d4c/f92 [4194304,4194304] 0 2024-02-26T10:48:09.227 INFO:tasks.workunit.client.0.smithi060.stdout:2/930: link dd/d11/l51 dd/d43/d55/d6e/l137 0 2024-02-26T10:48:09.228 INFO:tasks.workunit.client.0.smithi060.stdout:2/931: creat dd/d43/d96/df2/f138 x:0 0 0 2024-02-26T10:48:09.229 INFO:tasks.workunit.client.0.smithi060.stdout:3/835: dwrite d5/de/df/d4c/d68/f8e [4194304,4194304] 0 2024-02-26T10:48:09.232 INFO:tasks.workunit.client.0.smithi060.stdout:3/836: truncate d5/de/df/d4c/f70 1552810 0 2024-02-26T10:48:09.234 INFO:tasks.workunit.client.0.smithi060.stdout:3/837: creat d5/d45/d61/d92/da1/daa/f11b x:0 0 0 2024-02-26T10:48:09.234 INFO:tasks.workunit.client.0.smithi060.stdout:3/838: readlink d5/d45/l7b 0 2024-02-26T10:48:09.234 INFO:tasks.workunit.client.0.smithi060.stdout:3/839: stat d5/d45/d61/d92/da1 0 2024-02-26T10:48:09.236 INFO:tasks.workunit.client.0.smithi060.stdout:3/840: creat d5/de/f11c x:0 0 0 2024-02-26T10:48:09.236 INFO:tasks.workunit.client.0.smithi060.stdout:3/841: read - d5/de/d16/d10b/fe1 zero size 2024-02-26T10:48:09.240 INFO:tasks.workunit.client.0.smithi060.stdout:4/996: creat d0/da/d3b/d1b/f145 x:0 0 0 2024-02-26T10:48:09.241 INFO:tasks.workunit.client.0.smithi060.stdout:2/932: truncate dd/d43/dd2/f10b 888135 0 2024-02-26T10:48:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:2/933: link dd/d43/d55/d6e/c8b dd/d43/d55/d5e/d86/d97/c139 0 2024-02-26T10:48:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:0/913: write d0/dd/f77 [1860382,11768] 0 2024-02-26T10:48:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:3/842: creat d5/f11d x:0 0 0 2024-02-26T10:48:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:3/843: fsync d5/d45/fd6 0 2024-02-26T10:48:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:6/987: dwrite d6/d7/fd [0,4194304] 0 2024-02-26T10:48:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:6/988: dread - d6/d7/d25/d1c/f109 zero size 2024-02-26T10:48:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:3/844: write d5/de/f58 [1251863,111636] 0 2024-02-26T10:48:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:3/845: chown d5/d45/d61/d92/da1/daa/dad/ff4 3228427 1 2024-02-26T10:48:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:2/934: creat dd/d43/d55/d5e/d74/f13a x:0 0 0 2024-02-26T10:48:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:4/997: symlink d0/da/d3b/l146 0 2024-02-26T10:48:09.255 INFO:tasks.workunit.client.0.smithi060.stdout:0/914: link d0/dd/d39/d40/dad/d10d/l114 d0/dd/d39/d40/d83/dc6/l12a 0 2024-02-26T10:48:09.266 INFO:tasks.workunit.client.0.smithi060.stdout:6/989: unlink d6/d7/d25/d1c/d1f/d23/d2b/d10e/f139 0 2024-02-26T10:48:09.266 INFO:tasks.workunit.client.0.smithi060.stdout:4/998: unlink d0/da/dc7/cea 0 2024-02-26T10:48:09.266 INFO:tasks.workunit.client.0.smithi060.stdout:4/999: chown d0/d27/d76/dbd/d60/d11b 34 1 2024-02-26T10:48:09.266 INFO:tasks.workunit.client.0.smithi060.stdout:3/846: mknod d5/de/df/d11/d63/d93/d97/d98/dd4/dfb/c11e 0 2024-02-26T10:48:09.266 INFO:tasks.workunit.client.0.smithi060.stdout:3/847: read - d5/d45/d61/d92/da1/daa/f11b zero size 2024-02-26T10:48:09.318 INFO:tasks.workunit.client.0.smithi060.stdout:0/915: read d0/dd/d39/d40/f17 [9050744,108640] 0 2024-02-26T10:48:09.320 INFO:tasks.workunit.client.0.smithi060.stdout:0/916: truncate d0/dd/d33/f121 3421496 0 2024-02-26T10:48:09.324 INFO:tasks.workunit.client.0.smithi060.stdout:0/917: creat d0/dd/d39/d40/d83/dc6/d58/f12b x:0 0 0 2024-02-26T10:48:09.324 INFO:tasks.workunit.client.0.smithi060.stdout:3/848: fsync d5/de/f11c 0 2024-02-26T10:48:09.328 INFO:tasks.workunit.client.0.smithi060.stdout:0/918: truncate d0/dd/d39/d40/d83/dc6/d58/fd3 1328993 0 2024-02-26T10:48:09.329 INFO:tasks.workunit.client.0.smithi060.stdout:3/849: mknod d5/de/df/d11/d63/d93/da2/c11f 0 2024-02-26T10:48:09.332 INFO:tasks.workunit.client.0.smithi060.stdout:3/850: readlink d5/de/df/d11/l50 0 2024-02-26T10:48:09.335 INFO:tasks.workunit.client.0.smithi060.stdout:3/851: rename d5/de/df/d4c/d68/f55 to d5/de/df/d4c/d68/f120 0 2024-02-26T10:48:09.339 INFO:tasks.workunit.client.0.smithi060.stdout:3/852: unlink d5/l44 0 2024-02-26T10:48:09.339 INFO:tasks.workunit.client.0.smithi060.stdout:3/853: write d5/dbb/fee [738248,127888] 0 2024-02-26T10:48:09.344 INFO:tasks.workunit.client.0.smithi060.stdout:3/854: symlink d5/de/df/d11/d63/d93/d97/d98/l121 0 2024-02-26T10:48:09.385 INFO:tasks.workunit.client.0.smithi060.stdout:2/935: dread dd/d43/d60/d41/d37/d4c/f92 [0,4194304] 0 2024-02-26T10:48:09.385 INFO:tasks.workunit.client.0.smithi060.stdout:2/936: rmdir dd/d43/d60/d41/d37/d4c 39 2024-02-26T10:48:09.396 INFO:tasks.workunit.client.0.smithi060.stdout:0/919: dread d0/dd/d39/d40/d83/dc0/fe6 [0,4194304] 0 2024-02-26T10:48:09.396 INFO:tasks.workunit.client.0.smithi060.stdout:0/920: dread - d0/d42/d7a/d108/d111/f116 zero size 2024-02-26T10:48:09.403 INFO:tasks.workunit.client.0.smithi060.stdout:3/855: dread d5/de/df/d57/f94 [4194304,4194304] 0 2024-02-26T10:48:09.408 INFO:tasks.workunit.client.0.smithi060.stdout:5/918: creat d0/de/d15/d65/d86/f13a x:0 0 0 2024-02-26T10:48:09.421 INFO:tasks.workunit.client.0.smithi060.stdout:8/971: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/f46 [0,4194304] 0 2024-02-26T10:48:09.422 INFO:tasks.workunit.client.0.smithi060.stdout:6/990: dwrite d6/d7/d47/f55 [0,4194304] 0 2024-02-26T10:48:09.441 INFO:tasks.workunit.client.0.smithi060.stdout:0/921: rename d0/dd/d39/d40/c7d to d0/d42/df2/d102/c12c 0 2024-02-26T10:48:09.442 INFO:tasks.workunit.client.0.smithi060.stdout:0/922: creat d0/dd/d39/d40/dde/d122/f12d x:0 0 0 2024-02-26T10:48:09.444 INFO:tasks.workunit.client.0.smithi060.stdout:0/923: symlink d0/dd/d39/d40/d83/dc6/ddc/l12e 0 2024-02-26T10:48:09.444 INFO:tasks.workunit.client.0.smithi060.stdout:0/924: dread - d0/dd/d53/d87/d94/da6/dc7/fe3 zero size 2024-02-26T10:48:09.447 INFO:tasks.workunit.client.0.smithi060.stdout:3/856: dread d5/de/f58 [0,4194304] 0 2024-02-26T10:48:09.452 INFO:tasks.workunit.client.0.smithi060.stdout:3/857: rmdir d5/de/df/d4c/dcb 39 2024-02-26T10:48:09.458 INFO:tasks.workunit.client.0.smithi060.stdout:5/919: dread d0/d33/fa1 [0,4194304] 0 2024-02-26T10:48:09.458 INFO:tasks.workunit.client.0.smithi060.stdout:5/920: write d0/de/d15/f12a [623019,97636] 0 2024-02-26T10:48:09.458 INFO:tasks.workunit.client.0.smithi060.stdout:5/921: dread - d0/de/d20/d32/d4c/fb7 zero size 2024-02-26T10:48:09.463 INFO:tasks.workunit.client.0.smithi060.stdout:2/937: dwrite dd/d43/d55/f8e [0,4194304] 0 2024-02-26T10:48:09.471 INFO:tasks.workunit.client.0.smithi060.stdout:8/972: dwrite d9/f24 [0,4194304] 0 2024-02-26T10:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.478+0000 7f002a000640 1 -- 172.21.15.60:0/3307477648 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eb82482bd0 con 0x55eb825826e0 2024-02-26T10:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.478+0000 7f002a000640 10 client.26117 handle_command_reply: tid=0 2024-02-26T10:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.479+0000 7f004a1b8740 1 client.26117 shutdown 2024-02-26T10:48:09.476 INFO:tasks.workunit.client.0.smithi060.stdout:5/922: dread d0/de/d20/f58 [0,4194304] 0 2024-02-26T10:48:09.476 INFO:tasks.workunit.client.0.smithi060.stdout:8/973: mknod d9/da4/c146 0 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.482+0000 7f004a1b8740 1 -- 172.21.15.60:0/3307477648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 msgr2=0x55eb82526090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.482+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb82526090 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0038007bd0 tx=0x7f0038003c40 comp rx=0 tx=0).stop 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f004a1b8740 1 -- 172.21.15.60:0/3307477648 shutdown_connections 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55eb825826e0 0x55eb82584b00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f002a000640 0 client.26117 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb8251b380 0x55eb82526090 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb82516210 0x55eb82525080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f004a1b8740 1 --2- 172.21.15.60:0/3307477648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eb82515820 0x55eb82524b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f004a1b8740 1 -- 172.21.15.60:0/3307477648 >> 172.21.15.60:0/3307477648 conn(0x55eb82510d90 msgr2=0x55eb8251fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f004a1b8740 1 -- 172.21.15.60:0/3307477648 shutdown_connections 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f004a1b8740 1 -- 172.21.15.60:0/3307477648 wait complete. 2024-02-26T10:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.483+0000 7f004a1b8740 20 client.26117 trim_cache size 0 max 16384 2024-02-26T10:48:09.478 INFO:tasks.workunit.client.0.smithi060.stdout:6/991: dwrite d6/d7/d25/d1c/d1f/d23/f50 [0,4194304] 0 2024-02-26T10:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:48:09.479 INFO:tasks.workunit.client.0.smithi060.stdout:5/923: rename d0/de/d15/d65/d86/dcc/fd9 to d0/de/d70/f13b 0 2024-02-26T10:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.485+0000 7f0047200640 1 -- 172.21.15.60:0/914993998 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f00240a1ef0 msgr2=0x7f00240a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.485+0000 7f0047200640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f00240a1ef0 0x7f00240a43b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f002c005c20 tx=0x7f002c005fd0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.485+0000 7f0047200640 1 -- 172.21.15.60:0/914993998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0040076e60 msgr2=0x7f00401957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.485+0000 7f0047200640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0040076e60 0x7f00401957c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f003000ea60 tx=0x7f003000ef20 comp rx=0 tx=0).stop 2024-02-26T10:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.487+0000 7f0047200640 1 -- 172.21.15.60:0/914993998 shutdown_connections 2024-02-26T10:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.487+0000 7f0047200640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f00240a1ef0 0x7f00240a43b0 secure :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f002c005c20 tx=0x7f002c005fd0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.487+0000 7f0047200640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0040077800 0x7f004019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.487+0000 7f0047200640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0040076e60 0x7f00401957c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.487+0000 7f0047200640 1 --2- 172.21.15.60:0/914993998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0040075c20 0x7f0040195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.487+0000 7f0047200640 1 -- 172.21.15.60:0/914993998 >> 172.21.15.60:0/914993998 conn(0x7f00400fea50 msgr2=0x7f0040100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:09.481 INFO:tasks.workunit.client.0.smithi060.stdout:8/974: creat d9/da4/f147 x:0 0 0 2024-02-26T10:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.488+0000 7f0047200640 1 -- 172.21.15.60:0/914993998 shutdown_connections 2024-02-26T10:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:09.488+0000 7f0047200640 1 -- 172.21.15.60:0/914993998 wait complete. 2024-02-26T10:48:09.483 INFO:tasks.workunit.client.0.smithi060.stdout:8/975: symlink d9/d15/d29/d95/l148 0 2024-02-26T10:48:09.485 INFO:tasks.workunit.client.0.smithi060.stdout:8/976: getdents d9/d15/d29/d2f/d34/d105/d139 0 2024-02-26T10:48:09.488 INFO:tasks.workunit.client.0.smithi060.stdout:0/925: read d0/d42/f60 [1717587,73036] 0 2024-02-26T10:48:09.490 INFO:tasks.workunit.client.0.smithi060.stdout:3/858: dwrite d5/de/df/d11/d63/d93/d97/d98/fb1 [0,4194304] 0 2024-02-26T10:48:09.491 INFO:tasks.workunit.client.0.smithi060.stdout:0/926: getdents d0/dd/d53 0 2024-02-26T10:48:09.497 INFO:tasks.workunit.client.0.smithi060.stdout:3/859: link d5/d45/f110 d5/de/df/d11/de0/f122 0 2024-02-26T10:48:09.497 INFO:tasks.workunit.client.0.smithi060.stdout:3/860: dread - d5/de/df/d4c/d68/fb0 zero size 2024-02-26T10:48:09.498 INFO:tasks.workunit.client.0.smithi060.stdout:3/861: dread - d5/de/df/d11/d63/d93/d97/d98/dd4/dfb/f10e zero size 2024-02-26T10:48:09.498 INFO:tasks.workunit.client.0.smithi060.stdout:3/862: mkdir d5/de/d16/d10b/d123 0 2024-02-26T10:48:09.499 INFO:tasks.workunit.client.0.smithi060.stdout:0/927: truncate d0/f78 3478135 0 2024-02-26T10:48:09.499 INFO:tasks.workunit.client.0.smithi060.stdout:0/928: write d0/dd/d53/d87/d94/da6/dc7/fe3 [688314,108329] 0 2024-02-26T10:48:09.500 INFO:tasks.workunit.client.0.smithi060.stdout:0/929: dread - d0/dd/d39/d40/dde/d122/f12d zero size 2024-02-26T10:48:09.500 INFO:tasks.workunit.client.0.smithi060.stdout:0/930: write d0/d42/d7a/d108/d111/f116 [669003,16022] 0 2024-02-26T10:48:09.500 INFO:tasks.workunit.client.0.smithi060.stdout:0/931: chown d0/dd/d39/d40/d83/dc6/l12a 106225 1 2024-02-26T10:48:09.500 INFO:tasks.workunit.client.0.smithi060.stdout:3/863: creat d5/de/df/d4c/dcb/f124 x:0 0 0 2024-02-26T10:48:09.502 INFO:tasks.workunit.client.0.smithi060.stdout:3/864: creat d5/de/df/f125 x:0 0 0 2024-02-26T10:48:09.502 INFO:tasks.workunit.client.0.smithi060.stdout:6/992: dread d6/d7/d25/d1c/d1f/d23/d28/fd5 [0,4194304] 0 2024-02-26T10:48:09.502 INFO:tasks.workunit.client.0.smithi060.stdout:6/993: dread - d6/d6e/fc9 zero size 2024-02-26T10:48:09.503 INFO:tasks.workunit.client.0.smithi060.stdout:8/977: link d9/d15/f60 d9/d15/d29/d2f/d34/d72/d94/dd4/f149 0 2024-02-26T10:48:09.503 INFO:tasks.workunit.client.0.smithi060.stdout:8/978: readlink l7 0 2024-02-26T10:48:09.504 INFO:tasks.workunit.client.0.smithi060.stdout:3/865: unlink d5/d76/cce 0 2024-02-26T10:48:09.504 INFO:tasks.workunit.client.0.smithi060.stdout:2/938: read dd/d43/d60/d41/d37/d4c/f92 [4669583,26998] 0 2024-02-26T10:48:09.507 INFO:tasks.workunit.client.0.smithi060.stdout:0/932: symlink d0/d42/df2/l12f 0 2024-02-26T10:48:09.508 INFO:tasks.workunit.client.0.smithi060.stdout:0/933: readlink d0/dd/d53/d87/d119/l11a 0 2024-02-26T10:48:09.508 INFO:tasks.workunit.client.0.smithi060.stdout:2/939: getdents dd/d43/d60/d41/d37/d9a/db4 0 2024-02-26T10:48:09.508 INFO:tasks.workunit.client.0.smithi060.stdout:2/940: dread - dd/d43/d60/d41/d37/d4c/fc2 zero size 2024-02-26T10:48:09.511 INFO:tasks.workunit.client.0.smithi060.stdout:2/941: getdents dd/d43/d96 0 2024-02-26T10:48:09.514 INFO:tasks.workunit.client.0.smithi060.stdout:0/934: chown d0/dd/c3a 14133 1 2024-02-26T10:48:09.515 INFO:tasks.workunit.client.0.smithi060.stdout:2/942: rename dd/d43/d55/d5e/fad to dd/d43/d55/d6e/dd3/d6f/d8f/dbc/d12c/f13b 0 2024-02-26T10:48:09.517 INFO:tasks.workunit.client.0.smithi060.stdout:0/935: rename d0/dd/d53/d87/d94/da6/dc7/fe7 to d0/dd/d39/d40/d83/dc0/f130 0 2024-02-26T10:48:09.517 INFO:tasks.workunit.client.0.smithi060.stdout:2/943: creat dd/d43/d96/f13c x:0 0 0 2024-02-26T10:48:09.523 INFO:tasks.workunit.client.0.smithi060.stdout:2/944: creat dd/d43/d55/d5e/d86/d97/db9/d125/f13d x:0 0 0 2024-02-26T10:48:09.540 INFO:tasks.workunit.client.0.smithi060.stdout:0/936: write d0/dd/d39/d40/dde/fb1 [605645,11231] 0 2024-02-26T10:48:09.540 INFO:tasks.workunit.client.0.smithi060.stdout:0/937: write d0/dd/fd1 [805414,11068] 0 2024-02-26T10:48:09.541 INFO:tasks.workunit.client.0.smithi060.stdout:5/924: dread d0/de/d20/d32/d4d/da3/da5/dc8/f11f [0,4194304] 0 2024-02-26T10:48:09.542 INFO:tasks.workunit.client.0.smithi060.stdout:5/925: getdents d0/d4b/d7b 0 2024-02-26T10:48:09.543 INFO:tasks.workunit.client.0.smithi060.stdout:5/926: creat d0/de/d20/d82/d134/f13c x:0 0 0 2024-02-26T10:48:09.550 INFO:tasks.workunit.client.0.smithi060.stdout:6/994: write d6/d7/ff [3102560,107803] 0 2024-02-26T10:48:09.551 INFO:tasks.workunit.client.0.smithi060.stdout:6/995: mknod d6/d8d/dcc/dc3/d98/c14a 0 2024-02-26T10:48:09.559 INFO:tasks.workunit.client.0.smithi060.stdout:6/996: creat d6/d8d/dcc/dc3/d98/f14b x:0 0 0 2024-02-26T10:48:09.560 INFO:tasks.workunit.client.0.smithi060.stdout:2/945: dread dd/d43/d55/d6e/dd3/fb2 [0,4194304] 0 2024-02-26T10:48:09.563 INFO:tasks.workunit.client.0.smithi060.stdout:0/938: dread d0/dd/d33/f8c [0,4194304] 0 2024-02-26T10:48:09.565 INFO:tasks.workunit.client.0.smithi060.stdout:6/997: truncate d6/d7/d25/d80/fae 2713290 0 2024-02-26T10:48:09.565 INFO:tasks.workunit.client.0.smithi060.stdout:6/998: write d6/d7/d25/d1c/d1f/d23/d2b/d67/d78/d84/fb2 [4640037,36914] 0 2024-02-26T10:48:09.565 INFO:tasks.workunit.client.0.smithi060.stdout:6/999: dread - d6/d7/d25/f62 zero size 2024-02-26T10:48:09.568 INFO:tasks.workunit.client.0.smithi060.stdout:0/939: rename d0/dd/d39/d40/dad/dd6/c9e to d0/dd/d53/d87/d94/da6/c131 0 2024-02-26T10:48:09.573 INFO:tasks.workunit.client.0.smithi060.stdout:3/866: dwrite d5/de/f46 [0,4194304] 0 2024-02-26T10:48:09.574 INFO:tasks.workunit.client.0.smithi060.stdout:3/867: creat d5/de/df/d11/d63/d93/d97/d98/f126 x:0 0 0 2024-02-26T10:48:09.590 INFO:tasks.workunit.client.0.smithi060.stdout:3/868: dread d5/de/df/d4c/f70 [0,4194304] 0 2024-02-26T10:48:09.590 INFO:tasks.workunit.client.0.smithi060.stdout:3/869: fdatasync d5/de/df/d57/d60/f1e 0 2024-02-26T10:48:09.593 INFO:tasks.workunit.client.0.smithi060.stdout:0/940: dread d0/dd/d39/f7c [0,4194304] 0 2024-02-26T10:48:09.594 INFO:tasks.workunit.client.0.smithi060.stdout:0/941: fdatasync d0/dd/d39/d63/fcc 0 2024-02-26T10:48:09.612 INFO:tasks.workunit.client.0.smithi060.stdout:5/927: dwrite d0/f17 [0,4194304] 0 2024-02-26T10:48:09.627 INFO:tasks.workunit.client.0.smithi060.stdout:3/870: dwrite d5/de/df/feb [0,4194304] 0 2024-02-26T10:48:09.627 INFO:tasks.workunit.client.0.smithi060.stdout:3/871: write d5/d45/fd6 [194060,101723] 0 2024-02-26T10:48:09.635 INFO:tasks.workunit.client.0.smithi060.stdout:0/942: dwrite d0/dd/f31 [0,4194304] 0 2024-02-26T10:48:09.636 INFO:tasks.workunit.client.0.smithi060.stdout:0/943: creat d0/dd/d39/d63/f132 x:0 0 0 2024-02-26T10:48:09.636 INFO:tasks.workunit.client.0.smithi060.stdout:3/872: dread d5/de/df/d11/d63/d93/f96 [0,4194304] 0 2024-02-26T10:48:09.637 INFO:tasks.workunit.client.0.smithi060.stdout:3/873: fdatasync d5/de/df/d11/d63/d93/d97/f106 0 2024-02-26T10:48:09.637 INFO:tasks.workunit.client.0.smithi060.stdout:3/874: chown d5/d45/f10f 4348512 1 2024-02-26T10:48:09.637 INFO:tasks.workunit.client.0.smithi060.stdout:3/875: chown d5/de/df/d4c/dcb/df8 0 1 2024-02-26T10:48:09.637 INFO:tasks.workunit.client.0.smithi060.stdout:3/876: truncate d5/dbb/dcf/dd0/f114 694245 0 2024-02-26T10:48:09.637 INFO:tasks.workunit.client.0.smithi060.stdout:3/877: chown d5/de/df/d57/c83 3054 1 2024-02-26T10:48:09.638 INFO:tasks.workunit.client.0.smithi060.stdout:0/944: mknod d0/d42/d7a/dc8/c133 0 2024-02-26T10:48:09.638 INFO:tasks.workunit.client.0.smithi060.stdout:3/878: rename d5/l77 to d5/de/df/d11/d63/d93/d97/l127 0 2024-02-26T10:48:09.657 INFO:tasks.workunit.client.0.smithi060.stdout:0/945: dread d0/dd/d39/f54 [0,4194304] 0 2024-02-26T10:48:09.658 INFO:tasks.workunit.client.0.smithi060.stdout:0/946: creat d0/dd/d53/d87/d94/d11d/f134 x:0 0 0 2024-02-26T10:48:09.658 INFO:tasks.workunit.client.0.smithi060.stdout:0/947: fsync d0/dd/d39/fb8 0 2024-02-26T10:48:09.660 INFO:tasks.workunit.client.0.smithi060.stdout:0/948: creat d0/f135 x:0 0 0 2024-02-26T10:48:09.661 INFO:tasks.workunit.client.0.smithi060.stdout:3/879: dread d5/de/df/d57/d60/f1e [0,4194304] 0 2024-02-26T10:48:09.661 INFO:tasks.workunit.client.0.smithi060.stdout:3/880: write d5/de/df/d4c/dcb/df8/fe6 [584376,10180] 0 2024-02-26T10:48:09.663 INFO:tasks.workunit.client.0.smithi060.stdout:3/881: mkdir d5/d45/d61/d92/da1/daa/dad/dfc/d128 0 2024-02-26T10:48:09.664 INFO:tasks.workunit.client.0.smithi060.stdout:3/882: write d5/de/df/d57/d60/f30 [2011625,101458] 0 2024-02-26T10:48:09.665 INFO:tasks.workunit.client.0.smithi060.stdout:3/883: link d5/d76/lfa d5/de/df/d4c/dcb/df8/l129 0 2024-02-26T10:48:09.665 INFO:tasks.workunit.client.0.smithi060.stdout:3/884: chown d5/d45/d61/d92/d105 1060761489 1 2024-02-26T10:48:09.666 INFO:tasks.workunit.client.0.smithi060.stdout:3/885: write d5/d45/fd6 [278251,15796] 0 2024-02-26T10:48:09.666 INFO:tasks.workunit.client.0.smithi060.stdout:5/928: dwrite d0/de/d15/f2d [0,4194304] 0 2024-02-26T10:48:09.668 INFO:tasks.workunit.client.0.smithi060.stdout:3/886: rename d5/d45/d61/d92/da1/dc1/ld3 to d5/d76/d7d/l12a 0 2024-02-26T10:48:09.668 INFO:tasks.workunit.client.0.smithi060.stdout:3/887: dread - d5/de/df/d11/de0/f109 zero size 2024-02-26T10:48:09.668 INFO:tasks.workunit.client.0.smithi060.stdout:5/929: mknod d0/de/d15/d65/c13d 0 2024-02-26T10:48:09.668 INFO:tasks.workunit.client.0.smithi060.stdout:5/930: chown d0/de/d20/d32/d4d/da3/da5/dc8/f121 227888727 1 2024-02-26T10:48:09.668 INFO:tasks.workunit.client.0.smithi060.stdout:5/931: fdatasync d0/de/d20/ff0 0 2024-02-26T10:48:09.668 INFO:tasks.workunit.client.0.smithi060.stdout:3/888: creat d5/de/d16/d10b/d123/f12b x:0 0 0 2024-02-26T10:48:09.669 INFO:tasks.workunit.client.0.smithi060.stdout:5/932: mknod d0/de/d15/d65/d86/dcc/c13e 0 2024-02-26T10:48:09.669 INFO:tasks.workunit.client.0.smithi060.stdout:5/933: chown d0/de/d20/d32/d4d/da3/da5 3 1 2024-02-26T10:48:09.673 INFO:tasks.workunit.client.0.smithi060.stdout:3/889: symlink d5/d76/d7d/l12c 0 2024-02-26T10:48:09.674 INFO:tasks.workunit.client.0.smithi060.stdout:3/890: mknod d5/de/df/d11/d63/d93/d97/d98/dd4/dfb/c12d 0 2024-02-26T10:48:09.674 INFO:tasks.workunit.client.0.smithi060.stdout:3/891: truncate d5/d45/f110 572597 0 2024-02-26T10:48:09.675 INFO:tasks.workunit.client.0.smithi060.stdout:3/892: creat d5/de/df/d11/d63/f12e x:0 0 0 2024-02-26T10:48:09.676 INFO:tasks.workunit.client.0.smithi060.stdout:3/893: mknod d5/de/df/d11/d63/c12f 0 2024-02-26T10:48:09.677 INFO:tasks.workunit.client.0.smithi060.stdout:3/894: creat d5/d45/d61/d92/da1/daa/dad/dfc/d128/f130 x:0 0 0 2024-02-26T10:48:09.710 INFO:tasks.workunit.client.0.smithi060.stdout:5/934: dwrite d0/d33/f5c [0,4194304] 0 2024-02-26T10:48:09.712 INFO:tasks.workunit.client.0.smithi060.stdout:3/895: dwrite d5/dbb/dcf/dd0/f115 [0,4194304] 0 2024-02-26T10:48:09.713 INFO:tasks.workunit.client.0.smithi060.stdout:5/935: creat d0/de/d20/d82/d134/f13f x:0 0 0 2024-02-26T10:48:09.714 INFO:tasks.workunit.client.0.smithi060.stdout:5/936: mknod d0/de/d15/d65/d86/dcc/c140 0 2024-02-26T10:48:09.715 INFO:tasks.workunit.client.0.smithi060.stdout:5/937: symlink d0/de/d15/d65/l141 0 2024-02-26T10:48:09.734 INFO:tasks.workunit.client.0.smithi060.stdout:3/896: dread d5/f24 [0,4194304] 0 2024-02-26T10:48:09.736 INFO:tasks.workunit.client.0.smithi060.stdout:3/897: mkdir d5/de/d16/d131 0 2024-02-26T10:48:09.754 INFO:tasks.workunit.client.0.smithi060.stdout:3/898: dread d5/de/df/feb [0,4194304] 0 2024-02-26T10:48:09.754 INFO:tasks.workunit.client.0.smithi060.stdout:3/899: stat d5/dbb/dcf/c11a 0 2024-02-26T10:48:09.792 INFO:tasks.workunit.client.0.smithi060.stdout:3/900: dwrite d5/de/df/d4c/dcb/df8/f56 [0,4194304] 0 2024-02-26T10:48:09.795 INFO:tasks.workunit.client.0.smithi060.stdout:3/901: rename d5/de/c31 to d5/d45/c132 0 2024-02-26T10:48:09.827 INFO:tasks.workunit.client.0.smithi060.stdout:3/902: dread d5/de/df/d4c/dcb/df8/f41 [0,4194304] 0 2024-02-26T10:48:09.828 INFO:tasks.workunit.client.0.smithi060.stdout:3/903: creat d5/de/df/d4c/dcb/f133 x:0 0 0 2024-02-26T10:48:09.829 INFO:tasks.workunit.client.0.smithi060.stdout:3/904: truncate d5/de/d16/d10b/fe1 921498 0 2024-02-26T10:48:09.873 INFO:tasks.workunit.client.0.smithi060.stdout:3/905: dwrite d5/de/df/d57/d60/f1e [0,4194304] 0 2024-02-26T10:48:09.881 INFO:tasks.workunit.client.0.smithi060.stdout:8/979: sync 2024-02-26T10:48:09.881 INFO:tasks.workunit.client.0.smithi060.stdout:2/946: sync 2024-02-26T10:48:09.881 INFO:tasks.workunit.client.0.smithi060.stdout:5/938: sync 2024-02-26T10:48:09.881 INFO:tasks.workunit.client.0.smithi060.stdout:0/949: sync 2024-02-26T10:48:09.881 INFO:tasks.workunit.client.0.smithi060.stdout:0/950: stat d0/dd/d53/d87/ff7 0 2024-02-26T10:48:09.881 INFO:tasks.workunit.client.0.smithi060.stdout:8/980: truncate d9/d15/d29/d2f/d34/d105/d139/f102 1870748 0 2024-02-26T10:48:09.881 INFO:tasks.workunit.client.0.smithi060.stdout:8/981: write d9/fc [601744,17545] 0 2024-02-26T10:48:09.884 INFO:tasks.workunit.client.0.smithi060.stdout:8/982: mknod d9/d15/d29/d2f/d34/d72/d50/d12e/c14a 0 2024-02-26T10:48:09.886 INFO:tasks.workunit.client.0.smithi060.stdout:8/983: creat d9/d15/d29/d2f/d34/d72/d50/f14b x:0 0 0 2024-02-26T10:48:09.888 INFO:tasks.workunit.client.0.smithi060.stdout:5/939: mknod d0/de/d20/d82/d134/c142 0 2024-02-26T10:48:09.892 INFO:tasks.workunit.client.0.smithi060.stdout:0/951: rmdir d0/dd/d39 39 2024-02-26T10:48:09.893 INFO:tasks.workunit.client.0.smithi060.stdout:0/952: creat d0/dd/d39/d63/f136 x:0 0 0 2024-02-26T10:48:09.895 INFO:tasks.workunit.client.0.smithi060.stdout:0/953: mknod d0/d42/df2/d102/c137 0 2024-02-26T10:48:09.926 INFO:tasks.workunit.client.0.smithi060.stdout:3/906: dwrite d5/dbb/f117 [0,4194304] 0 2024-02-26T10:48:09.926 INFO:tasks.workunit.client.0.smithi060.stdout:3/907: write d5/d45/d61/d92/da1/daa/f11b [1029243,28873] 0 2024-02-26T10:48:09.929 INFO:tasks.workunit.client.0.smithi060.stdout:3/908: symlink d5/dbb/dcf/l134 0 2024-02-26T10:48:09.930 INFO:tasks.workunit.client.0.smithi060.stdout:3/909: creat d5/de/d16/d10b/f135 x:0 0 0 2024-02-26T10:48:09.933 INFO:tasks.workunit.client.0.smithi060.stdout:2/947: read dd/d43/d96/df2/ff8 [105464,56856] 0 2024-02-26T10:48:09.936 INFO:tasks.workunit.client.0.smithi060.stdout:2/948: rmdir dd/d43/d55/d5e/dc6 39 2024-02-26T10:48:09.936 INFO:tasks.workunit.client.0.smithi060.stdout:2/949: fdatasync dd/d11/f4e 0 2024-02-26T10:48:09.968 INFO:tasks.workunit.client.0.smithi060.stdout:3/910: read d5/de/df/d4c/dcb/df8/f41 [78828,5241] 0 2024-02-26T10:48:09.974 INFO:tasks.workunit.client.0.smithi060.stdout:8/984: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/dc0/f129 [0,4194304] 0 2024-02-26T10:48:09.985 INFO:tasks.workunit.client.0.smithi060.stdout:3/911: link d5/dbb/dcf/dd0/f115 d5/d45/d61/d92/da1/daa/dad/dfc/f136 0 2024-02-26T10:48:09.991 INFO:tasks.workunit.client.0.smithi060.stdout:0/954: dwrite d0/f78 [0,4194304] 0 2024-02-26T10:48:09.991 INFO:tasks.workunit.client.0.smithi060.stdout:0/955: chown d0/dd/d39/d40/dad/dd6 6850416 1 2024-02-26T10:48:09.996 INFO:tasks.workunit.client.0.smithi060.stdout:0/956: rmdir d0/d42/d7a/d108/d111/d127 0 2024-02-26T10:48:09.997 INFO:tasks.workunit.client.0.smithi060.stdout:0/957: getdents d0/dd/d39/d40/dad/dda 0 2024-02-26T10:48:09.997 INFO:tasks.workunit.client.0.smithi060.stdout:2/950: dread dd/d43/d60/d41/d37/d4c/f92 [0,4194304] 0 2024-02-26T10:48:09.998 INFO:tasks.workunit.client.0.smithi060.stdout:0/958: mknod d0/dd/d39/d40/dad/dd6/d6c/c138 0 2024-02-26T10:48:09.998 INFO:tasks.workunit.client.0.smithi060.stdout:0/959: chown d0/dd/d39/d93 1096524 1 2024-02-26T10:48:09.998 INFO:tasks.workunit.client.0.smithi060.stdout:2/951: symlink dd/d43/d55/d6e/dd3/l13e 0 2024-02-26T10:48:09.999 INFO:tasks.workunit.client.0.smithi060.stdout:0/960: truncate d0/dd/d39/d40/d83/dc6/d58/f5b 9179 0 2024-02-26T10:48:10.000 INFO:tasks.workunit.client.0.smithi060.stdout:0/961: mkdir d0/dd/d53/d87/d94/da6/d139 0 2024-02-26T10:48:10.001 INFO:tasks.workunit.client.0.smithi060.stdout:2/952: getdents dd/d43/d96/df2 0 2024-02-26T10:48:10.001 INFO:tasks.workunit.client.0.smithi060.stdout:2/953: chown dd/d43/d55/d6e/l111 437455220 1 2024-02-26T10:48:10.015 INFO:tasks.workunit.client.0.smithi060.stdout:5/940: dwrite d0/de/d15/d26/fce [0,4194304] 0 2024-02-26T10:48:10.021 INFO:tasks.workunit.client.0.smithi060.stdout:2/954: dread dd/d43/d60/d41/d37/d4c/f82 [0,4194304] 0 2024-02-26T10:48:10.025 INFO:tasks.workunit.client.0.smithi060.stdout:5/941: link d0/de/d15/d65/d86/d9b/ff7 d0/d4b/f143 0 2024-02-26T10:48:10.025 INFO:tasks.workunit.client.0.smithi060.stdout:5/942: write d0/de/d20/d32/d4d/da3/da5/f104 [913331,68928] 0 2024-02-26T10:48:10.026 INFO:tasks.workunit.client.0.smithi060.stdout:5/943: rmdir d0/de/d20/d32/d4d/d110 39 2024-02-26T10:48:10.027 INFO:tasks.workunit.client.0.smithi060.stdout:5/944: creat d0/de/d20/d32/d4d/f144 x:0 0 0 2024-02-26T10:48:10.031 INFO:tasks.workunit.client.0.smithi060.stdout:8/985: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/d63/da9/fbc [0,4194304] 0 2024-02-26T10:48:10.031 INFO:tasks.workunit.client.0.smithi060.stdout:8/986: chown d9/d5a/d58/ld0 621 1 2024-02-26T10:48:10.031 INFO:tasks.workunit.client.0.smithi060.stdout:8/987: fsync d9/d5a/d9c/df0/d124/f125 0 2024-02-26T10:48:10.037 INFO:tasks.workunit.client.0.smithi060.stdout:8/988: creat d9/d15/d29/d2f/d34/d72/d94/f14c x:0 0 0 2024-02-26T10:48:10.038 INFO:tasks.workunit.client.0.smithi060.stdout:8/989: chown d9/da4/f122 7413 1 2024-02-26T10:48:10.038 INFO:tasks.workunit.client.0.smithi060.stdout:3/912: dwrite d5/de/df/d11/d63/d93/d97/d98/fb1 [0,4194304] 0 2024-02-26T10:48:10.042 INFO:tasks.workunit.client.0.smithi060.stdout:5/945: rename d0/d4b/d7b/laf to d0/de/d20/d32/d4d/da3/da5/l145 0 2024-02-26T10:48:10.043 INFO:tasks.workunit.client.0.smithi060.stdout:5/946: getdents d0/de/d70/d84/d92 0 2024-02-26T10:48:10.043 INFO:tasks.workunit.client.0.smithi060.stdout:5/947: stat d0/de/d15/d65/f7c 0 2024-02-26T10:48:10.056 INFO:tasks.workunit.client.0.smithi060.stdout:2/955: dwrite dd/d43/d60/d41/d37/d38/f119 [0,4194304] 0 2024-02-26T10:48:10.064 INFO:tasks.workunit.client.0.smithi060.stdout:8/990: getdents d9/d5a/d9c 0 2024-02-26T10:48:10.073 INFO:tasks.workunit.client.0.smithi060.stdout:3/913: rename d5/d45/cc6 to d5/de/d16/d10b/c137 0 2024-02-26T10:48:10.079 INFO:tasks.workunit.client.0.smithi060.stdout:2/956: write dd/d43/f6c [4304508,116087] 0 2024-02-26T10:48:10.083 INFO:tasks.workunit.client.0.smithi060.stdout:5/948: dwrite d0/de/d70/d84/d92/f126 [0,4194304] 0 2024-02-26T10:48:10.088 INFO:tasks.workunit.client.0.smithi060.stdout:8/991: dread d9/d15/d29/d2f/fe4 [0,4194304] 0 2024-02-26T10:48:10.088 INFO:tasks.workunit.client.0.smithi060.stdout:3/914: rename d5/de/df/d11/d63 to d5/d45/d61/d92/d105/d138 0 2024-02-26T10:48:10.088 INFO:tasks.workunit.client.0.smithi060.stdout:3/915: dread - d5/d45/d61/d92/da1/daa/fa6 zero size 2024-02-26T10:48:10.090 INFO:tasks.workunit.client.0.smithi060.stdout:8/992: symlink d9/d15/d29/d2f/d34/d72/d50/l14d 0 2024-02-26T10:48:10.090 INFO:tasks.workunit.client.0.smithi060.stdout:8/993: truncate d9/d15/d29/d2f/f98 649120 0 2024-02-26T10:48:10.091 INFO:tasks.workunit.client.0.smithi060.stdout:5/949: getdents d0/d4b/d7b 0 2024-02-26T10:48:10.091 INFO:tasks.workunit.client.0.smithi060.stdout:5/950: truncate d0/de/d20/d32/d4c/f106 932884 0 2024-02-26T10:48:10.091 INFO:tasks.workunit.client.0.smithi060.stdout:3/916: creat d5/de/df/d57/d7a/f139 x:0 0 0 2024-02-26T10:48:10.094 INFO:tasks.workunit.client.0.smithi060.stdout:5/951: read d0/de/d20/f58 [166885,5093] 0 2024-02-26T10:48:10.094 INFO:tasks.workunit.client.0.smithi060.stdout:5/952: truncate d0/dc0/fe4 192192 0 2024-02-26T10:48:10.097 INFO:tasks.workunit.client.0.smithi060.stdout:5/953: link d0/de/d70/d84/ldf d0/dc0/l146 0 2024-02-26T10:48:10.098 INFO:tasks.workunit.client.0.smithi060.stdout:5/954: creat d0/dc0/f147 x:0 0 0 2024-02-26T10:48:10.098 INFO:tasks.workunit.client.0.smithi060.stdout:5/955: write d0/de/d20/f85 [370313,109546] 0 2024-02-26T10:48:10.100 INFO:tasks.workunit.client.0.smithi060.stdout:5/956: symlink d0/de/d15/d65/d86/d9b/d113/l148 0 2024-02-26T10:48:10.100 INFO:tasks.workunit.client.0.smithi060.stdout:5/957: write d0/de/f29 [902210,93776] 0 2024-02-26T10:48:10.103 INFO:tasks.workunit.client.0.smithi060.stdout:3/917: read d5/de/f14 [2140807,51211] 0 2024-02-26T10:48:10.105 INFO:tasks.workunit.client.0.smithi060.stdout:2/957: dread dd/d43/d60/d41/d37/d4c/d76/f84 [0,4194304] 0 2024-02-26T10:48:10.106 INFO:tasks.workunit.client.0.smithi060.stdout:2/958: creat dd/dd1/f13f x:0 0 0 2024-02-26T10:48:10.106 INFO:tasks.workunit.client.0.smithi060.stdout:2/959: chown dd/d43/d60/d41/f31 248963 1 2024-02-26T10:48:10.107 INFO:tasks.workunit.client.0.smithi060.stdout:5/958: fdatasync d0/de/d20/d32/d4d/f53 0 2024-02-26T10:48:10.109 INFO:tasks.workunit.client.0.smithi060.stdout:3/918: unlink d5/d45/d61/d92/d105/d138/d93/d97/fa0 0 2024-02-26T10:48:10.109 INFO:tasks.workunit.client.0.smithi060.stdout:3/919: getdents d5/de/dd7 0 2024-02-26T10:48:10.109 INFO:tasks.workunit.client.0.smithi060.stdout:5/959: rename d0/de/d15/d65/lae to d0/de/d70/d84/d92/l149 0 2024-02-26T10:48:10.111 INFO:tasks.workunit.client.0.smithi060.stdout:5/960: mknod d0/de/d20/d82/c14a 0 2024-02-26T10:48:10.111 INFO:tasks.workunit.client.0.smithi060.stdout:5/961: dread - d0/de/d20/d82/d134/f13f zero size 2024-02-26T10:48:10.112 INFO:tasks.workunit.client.0.smithi060.stdout:5/962: symlink d0/de/d20/d82/l14b 0 2024-02-26T10:48:10.116 INFO:tasks.workunit.client.0.smithi060.stdout:3/920: dread d5/de/df/d11/f1c [0,4194304] 0 2024-02-26T10:48:10.118 INFO:tasks.workunit.client.0.smithi060.stdout:3/921: creat d5/d45/d61/d92/d105/d138/d93/f13a x:0 0 0 2024-02-26T10:48:10.124 INFO:tasks.workunit.client.0.smithi060.stdout:2/960: dread dd/d11/fe1 [0,4194304] 0 2024-02-26T10:48:10.124 INFO:tasks.workunit.client.0.smithi060.stdout:2/961: write dd/d43/d55/d5e/d86/d97/fce [2221369,45013] 0 2024-02-26T10:48:10.124 INFO:tasks.workunit.client.0.smithi060.stdout:2/962: truncate dd/d43/d55/fe2 248231 0 2024-02-26T10:48:10.124 INFO:tasks.workunit.client.0.smithi060.stdout:2/963: truncate dd/d43/d55/d5e/d74/f13a 807280 0 2024-02-26T10:48:10.129 INFO:tasks.workunit.client.0.smithi060.stdout:2/964: rename dd/d43/d60/d41/d37/d69/cae to dd/dd1/c140 0 2024-02-26T10:48:10.131 INFO:tasks.workunit.client.0.smithi060.stdout:2/965: creat dd/d43/d55/d5e/d74/da5/def/f141 x:0 0 0 2024-02-26T10:48:10.131 INFO:tasks.workunit.client.0.smithi060.stdout:2/966: truncate dd/d43/d60/d41/d37/d4c/fc2 671790 0 2024-02-26T10:48:10.132 INFO:tasks.workunit.client.0.smithi060.stdout:2/967: creat dd/d43/d55/d6e/dd3/d6f/d8f/dbc/d127/f142 x:0 0 0 2024-02-26T10:48:10.132 INFO:tasks.workunit.client.0.smithi060.stdout:2/968: fdatasync dd/d43/d60/f3f 0 2024-02-26T10:48:10.145 INFO:tasks.workunit.client.0.smithi060.stdout:3/922: write d5/de/df/d11/f34 [2332386,12952] 0 2024-02-26T10:48:10.145 INFO:tasks.workunit.client.0.smithi060.stdout:3/923: chown d5/d76/d7d/l9e 61561 1 2024-02-26T10:48:10.145 INFO:tasks.workunit.client.0.smithi060.stdout:3/924: chown d5/fd 13 1 2024-02-26T10:48:10.150 INFO:tasks.workunit.client.0.smithi060.stdout:2/969: write dd/d43/d55/d6e/dd3/fb2 [3550292,57547] 0 2024-02-26T10:48:10.151 INFO:tasks.workunit.client.0.smithi060.stdout:8/994: dwrite d9/d15/d29/d2f/d34/d72/d50/d6a/d63/f82 [0,4194304] 0 2024-02-26T10:48:10.151 INFO:tasks.workunit.client.0.smithi060.stdout:8/995: dread - d9/d15/d29/d2f/d34/d72/d94/dba/d119/f11c zero size 2024-02-26T10:48:10.151 INFO:tasks.workunit.client.0.smithi060.stdout:8/996: read - d9/d15/d29/d2f/d34/fdf zero size 2024-02-26T10:48:10.151 INFO:tasks.workunit.client.0.smithi060.stdout:5/963: dread d0/de/d15/d65/f6c [0,4194304] 0 2024-02-26T10:48:10.151 INFO:tasks.workunit.client.0.smithi060.stdout:8/997: write d9/da4/f122 [502930,11011] 0 2024-02-26T10:48:10.194 INFO:tasks.workunit.client.0.smithi060.stdout:8/998: write d9/d15/d29/d2f/d34/d72/d94/f9e [4747909,38861] 0 2024-02-26T10:48:10.203 INFO:tasks.workunit.client.0.smithi060.stdout:3/925: dread d5/d45/d61/d92/da1/daa/f11b [0,4194304] 0 2024-02-26T10:48:10.203 INFO:tasks.workunit.client.0.smithi060.stdout:5/964: dwrite d0/de/d15/d65/d86/dcc/fd5 [0,4194304] 0 2024-02-26T10:48:10.205 INFO:tasks.workunit.client.0.smithi060.stdout:8/999: dread d9/d5a/d58/fab [0,4194304] 0 2024-02-26T10:48:10.221 INFO:tasks.workunit.client.0.smithi060.stdout:5/965: write d0/de/d15/d26/fce [2856901,57907] 0 2024-02-26T10:48:10.222 INFO:tasks.workunit.client.0.smithi060.stdout:5/966: readlink d0/de/d20/d32/d4d/d97/l99 0 2024-02-26T10:48:10.227 INFO:tasks.workunit.client.0.smithi060.stdout:5/967: stat d0/de/d15/d65/d86/cc1 0 2024-02-26T10:48:10.227 INFO:tasks.workunit.client.0.smithi060.stdout:5/968: fdatasync d0/de/da4/fbd 0 2024-02-26T10:48:10.234 INFO:tasks.workunit.client.0.smithi060.stdout:0/962: sync 2024-02-26T10:48:10.235 INFO:tasks.workunit.client.0.smithi060.stdout:0/963: getdents d0/dd/d39/d40/dad/dda/de4/d103 0 2024-02-26T10:48:10.236 INFO:tasks.workunit.client.0.smithi060.stdout:0/964: write d0/dd/d53/d87/fa9 [2471476,59497] 0 2024-02-26T10:48:10.245 INFO:tasks.workunit.client.0.smithi060.stdout:5/969: dread d0/de/d20/d32/fed [0,4194304] 0 2024-02-26T10:48:10.246 INFO:tasks.workunit.client.0.smithi060.stdout:5/970: creat d0/de/d15/d65/d86/d9b/d113/f14c x:0 0 0 2024-02-26T10:48:10.253 INFO:tasks.workunit.client.0.smithi060.stdout:0/965: dread d0/dd/d53/f64 [4194304,4194304] 0 2024-02-26T10:48:10.254 INFO:tasks.workunit.client.0.smithi060.stdout:5/971: symlink d0/de/d20/d32/d4d/d110/l14d 0 2024-02-26T10:48:10.254 INFO:tasks.workunit.client.0.smithi060.stdout:5/972: write d0/de/d20/d32/d4d/d110/f123 [433366,44067] 0 2024-02-26T10:48:10.255 INFO:tasks.workunit.client.0.smithi060.stdout:5/973: mkdir d0/de/d20/d32/d4c/d14e 0 2024-02-26T10:48:10.255 INFO:tasks.workunit.client.0.smithi060.stdout:5/974: dread - d0/de/d20/d32/f10b zero size 2024-02-26T10:48:10.255 INFO:tasks.workunit.client.0.smithi060.stdout:0/966: rename d0/dd/d33 to d0/dd/d39/d40/dad/dda/de4/d13a 0 2024-02-26T10:48:10.256 INFO:tasks.workunit.client.0.smithi060.stdout:5/975: creat d0/de/d15/d65/d86/dcc/f14f x:0 0 0 2024-02-26T10:48:10.256 INFO:tasks.workunit.client.0.smithi060.stdout:5/976: write d0/de/d20/ff0 [2789318,55788] 0 2024-02-26T10:48:10.256 INFO:tasks.workunit.client.0.smithi060.stdout:5/977: readlink d0/de/le9 0 2024-02-26T10:48:10.257 INFO:tasks.workunit.client.0.smithi060.stdout:5/978: symlink d0/de/d20/l150 0 2024-02-26T10:48:10.257 INFO:tasks.workunit.client.0.smithi060.stdout:5/979: stat d0/d33/fef 0 2024-02-26T10:48:10.257 INFO:tasks.workunit.client.0.smithi060.stdout:5/980: chown d0/de/d15/d65/d86/dcc/c136 128059694 1 2024-02-26T10:48:10.283 INFO:tasks.workunit.client.0.smithi060.stdout:5/981: write d0/de/d20/d32/f89 [817414,128758] 0 2024-02-26T10:48:10.285 INFO:tasks.workunit.client.0.smithi060.stdout:5/982: link d0/de/d70/d84/d92/f126 d0/de/d20/f151 0 2024-02-26T10:48:10.320 INFO:tasks.workunit.client.0.smithi060.stdout:5/983: dread d0/de/d20/f151 [0,4194304] 0 2024-02-26T10:48:10.320 INFO:tasks.workunit.client.0.smithi060.stdout:5/984: fsync d0/de/d15/d65/fb1 0 2024-02-26T10:48:10.320 INFO:tasks.workunit.client.0.smithi060.stdout:5/985: write d0/de/d20/f56 [121246,106825] 0 2024-02-26T10:48:10.369 INFO:tasks.workunit.client.0.smithi060.stdout:5/986: dwrite d0/de/d20/ff0 [0,4194304] 0 2024-02-26T10:48:10.370 INFO:tasks.workunit.client.0.smithi060.stdout:3/926: sync 2024-02-26T10:48:10.370 INFO:tasks.workunit.client.0.smithi060.stdout:2/970: sync 2024-02-26T10:48:10.371 INFO:tasks.workunit.client.0.smithi060.stdout:2/971: creat dd/d43/d55/d5e/d74/da5/def/f143 x:0 0 0 2024-02-26T10:48:10.373 INFO:tasks.workunit.client.0.smithi060.stdout:3/927: rename d5/de/df/d4c/d68/fb0 to d5/de/d16/d131/f13b 0 2024-02-26T10:48:10.373 INFO:tasks.workunit.client.0.smithi060.stdout:3/928: truncate d5/d45/d61/d92/d105/d138/f12e 113310 0 2024-02-26T10:48:10.373 INFO:tasks.workunit.client.0.smithi060.stdout:2/972: mkdir dd/d43/d96/d144 0 2024-02-26T10:48:10.373 INFO:tasks.workunit.client.0.smithi060.stdout:2/973: rmdir dd/d43/d55/d6e/dd3/d6f/d8f/dbc 39 2024-02-26T10:48:10.374 INFO:tasks.workunit.client.0.smithi060.stdout:3/929: symlink d5/de/df/d4c/d68/l13c 0 2024-02-26T10:48:10.374 INFO:tasks.workunit.client.0.smithi060.stdout:2/974: write f5 [1872884,41143] 0 2024-02-26T10:48:10.375 INFO:tasks.workunit.client.0.smithi060.stdout:2/975: write dd/d43/d55/d6e/dd3/fb2 [5216488,39268] 0 2024-02-26T10:48:10.376 INFO:tasks.workunit.client.0.smithi060.stdout:2/976: write dd/d43/d60/d41/d37/d4c/f82 [991793,123555] 0 2024-02-26T10:48:10.376 INFO:tasks.workunit.client.0.smithi060.stdout:2/977: readlink dd/d43/d55/d5e/d86/d97/db9/dde/l10e 0 2024-02-26T10:48:10.378 INFO:tasks.workunit.client.0.smithi060.stdout:3/930: symlink d5/d45/d61/d92/d105/l13d 0 2024-02-26T10:48:10.378 INFO:tasks.workunit.client.0.smithi060.stdout:3/931: truncate d5/de/df/fe3 215631 0 2024-02-26T10:48:10.417 INFO:tasks.workunit.client.0.smithi060.stdout:5/987: dwrite d0/de/d15/d65/d86/d9b/d113/f14c [0,4194304] 0 2024-02-26T10:48:10.425 INFO:tasks.workunit.client.0.smithi060.stdout:0/967: sync 2024-02-26T10:48:10.425 INFO:tasks.workunit.client.0.smithi060.stdout:0/968: write d0/dd/d39/d40/dad/dd6/d6c/fdf [1059198,29026] 0 2024-02-26T10:48:10.426 INFO:tasks.workunit.client.0.smithi060.stdout:0/969: write d0/dd/f31 [5190660,88901] 0 2024-02-26T10:48:10.426 INFO:tasks.workunit.client.0.smithi060.stdout:3/932: dread d5/f24 [0,4194304] 0 2024-02-26T10:48:10.428 INFO:tasks.workunit.client.0.smithi060.stdout:3/933: link d5/d45/d61/d92/d105/d138/d93/d97/d98/dd4/dfb/c12d d5/d45/d61/c13e 0 2024-02-26T10:48:10.461 INFO:tasks.workunit.client.0.smithi060.stdout:0/970: dread d0/dd/d39/d40/dad/d10d/f110 [0,4194304] 0 2024-02-26T10:48:10.465 INFO:tasks.workunit.client.0.smithi060.stdout:0/971: rename d0/d37/cf5 to d0/dd/c13b 0 2024-02-26T10:48:10.466 INFO:tasks.workunit.client.0.smithi060.stdout:0/972: dread - d0/dd/d53/d87/d94/d11d/f134 zero size 2024-02-26T10:48:10.468 INFO:tasks.workunit.client.0.smithi060.stdout:0/973: mknod d0/dd/d39/d40/d83/dc0/c13c 0 2024-02-26T10:48:10.469 INFO:tasks.workunit.client.0.smithi060.stdout:0/974: write d0/dd/d39/d40/dad/dd6/d6c/fab [4924927,119130] 0 2024-02-26T10:48:10.469 INFO:tasks.workunit.client.0.smithi060.stdout:0/975: getdents d0/dd/d53 0 2024-02-26T10:48:10.471 INFO:tasks.workunit.client.0.smithi060.stdout:0/976: creat d0/dd/d53/da2/f13d x:0 0 0 2024-02-26T10:48:10.473 INFO:tasks.workunit.client.0.smithi060.stdout:0/977: link d0/dd/f2b d0/dd/d39/d40/dde/d122/f13e 0 2024-02-26T10:48:10.475 INFO:tasks.workunit.client.0.smithi060.stdout:0/978: unlink d0/dd/d53/d87/fd2 0 2024-02-26T10:48:10.477 INFO:tasks.workunit.client.0.smithi060.stdout:0/979: mkdir d0/dd/d39/d40/d83/d11b/d13f 0 2024-02-26T10:48:10.490 INFO:tasks.workunit.client.0.smithi060.stdout:2/978: dwrite dd/d43/d96/f109 [4194304,4194304] 0 2024-02-26T10:48:10.491 INFO:tasks.workunit.client.0.smithi060.stdout:2/979: chown dd/d43/d55/d5e/d74/da5 64700089 1 2024-02-26T10:48:10.497 INFO:tasks.workunit.client.0.smithi060.stdout:0/980: dread d0/ff1 [0,4194304] 0 2024-02-26T10:48:10.497 INFO:tasks.workunit.client.0.smithi060.stdout:2/980: mkdir dd/d43/d55/d5e/d86/d97/d9c/d145 0 2024-02-26T10:48:10.498 INFO:tasks.workunit.client.0.smithi060.stdout:2/981: truncate dd/d43/dd2/f10b 1301850 0 2024-02-26T10:48:10.499 INFO:tasks.workunit.client.0.smithi060.stdout:2/982: mkdir dd/d43/d60/d41/d37/dfd/d146 0 2024-02-26T10:48:10.499 INFO:tasks.workunit.client.0.smithi060.stdout:0/981: creat d0/dd/d39/d40/d83/f140 x:0 0 0 2024-02-26T10:48:10.501 INFO:tasks.workunit.client.0.smithi060.stdout:2/983: unlink dd/d11/le6 0 2024-02-26T10:48:10.506 INFO:tasks.workunit.client.0.smithi060.stdout:2/984: mkdir dd/d147 0 2024-02-26T10:48:10.527 INFO:tasks.workunit.client.0.smithi060.stdout:5/988: dwrite d0/fc9 [0,4194304] 0 2024-02-26T10:48:10.527 INFO:tasks.workunit.client.0.smithi060.stdout:5/989: chown d0/de/d15/f3b 1281881 1 2024-02-26T10:48:10.529 INFO:tasks.workunit.client.0.smithi060.stdout:2/985: dread dd/d43/d60/d41/d37/d9a/db4/fd6 [0,4194304] 0 2024-02-26T10:48:10.531 INFO:tasks.workunit.client.0.smithi060.stdout:5/990: creat d0/de/d20/d32/d117/f152 x:0 0 0 2024-02-26T10:48:10.535 INFO:tasks.workunit.client.0.smithi060.stdout:5/991: getdents d0/d4b 0 2024-02-26T10:48:10.535 INFO:tasks.workunit.client.0.smithi060.stdout:2/986: mknod dd/d43/d60/d41/d37/d4c/d76/c148 0 2024-02-26T10:48:10.536 INFO:tasks.workunit.client.0.smithi060.stdout:2/987: symlink dd/d43/d60/d41/d37/d9a/d126/l149 0 2024-02-26T10:48:10.537 INFO:tasks.workunit.client.0.smithi060.stdout:2/988: creat dd/d147/f14a x:0 0 0 2024-02-26T10:48:10.538 INFO:tasks.workunit.client.0.smithi060.stdout:2/989: unlink dd/d11/cc4 0 2024-02-26T10:48:10.540 INFO:tasks.workunit.client.0.smithi060.stdout:2/990: mkdir dd/dd1/d14b 0 2024-02-26T10:48:10.541 INFO:tasks.workunit.client.0.smithi060.stdout:2/991: mknod dd/d43/d60/d41/d37/d69/c14c 0 2024-02-26T10:48:10.542 INFO:tasks.workunit.client.0.smithi060.stdout:2/992: symlink dd/d43/l14d 0 2024-02-26T10:48:10.551 INFO:tasks.workunit.client.0.smithi060.stdout:3/934: dwrite d5/de/df/d4c/f70 [0,4194304] 0 2024-02-26T10:48:10.563 INFO:tasks.workunit.client.0.smithi060.stdout:3/935: read d5/d45/d61/d92/da1/daa/f11b [637277,103853] 0 2024-02-26T10:48:10.564 INFO:tasks.workunit.client.0.smithi060.stdout:3/936: mkdir d5/d76/d13f 0 2024-02-26T10:48:10.564 INFO:tasks.workunit.client.0.smithi060.stdout:3/937: write d5/de/df/d4c/dcb/f124 [4033,72835] 0 2024-02-26T10:48:10.566 INFO:tasks.workunit.client.0.smithi060.stdout:5/992: write d0/d33/f3e [618555,54821] 0 2024-02-26T10:48:10.566 INFO:tasks.workunit.client.0.smithi060.stdout:5/993: write d0/de/d70/d84/f10a [16374,106051] 0 2024-02-26T10:48:10.566 INFO:tasks.workunit.client.0.smithi060.stdout:5/994: dread - d0/de/d20/d32/d4c/f12c zero size 2024-02-26T10:48:10.566 INFO:tasks.workunit.client.0.smithi060.stdout:5/995: write d0/de/d15/d26/ddd/f128 [953008,122267] 0 2024-02-26T10:48:10.567 INFO:tasks.workunit.client.0.smithi060.stdout:3/938: symlink d5/d45/d61/d92/da1/daa/l140 0 2024-02-26T10:48:10.568 INFO:tasks.workunit.client.0.smithi060.stdout:3/939: creat d5/f141 x:0 0 0 2024-02-26T10:48:10.573 INFO:tasks.workunit.client.0.smithi060.stdout:3/940: write d5/d45/fd6 [76931,50281] 0 2024-02-26T10:48:10.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:10 smithi113 ceph-mon[36126]: pgmap v491: 129 pgs: 129 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 173 MiB/s rd, 206 MiB/s wr, 775 op/s 2024-02-26T10:48:10.602 INFO:tasks.workunit.client.0.smithi060.stdout:2/993: dwrite dd/dd1/f13f [0,4194304] 0 2024-02-26T10:48:10.604 INFO:tasks.workunit.client.0.smithi060.stdout:3/941: write d5/de/df/d57/d60/f1e [752734,27887] 0 2024-02-26T10:48:10.605 INFO:tasks.workunit.client.0.smithi060.stdout:2/994: truncate dd/d43/d60/d41/d37/d38/f119 109774 0 2024-02-26T10:48:10.606 INFO:tasks.workunit.client.0.smithi060.stdout:3/942: creat d5/de/df/d11/f142 x:0 0 0 2024-02-26T10:48:10.606 INFO:tasks.workunit.client.0.smithi060.stdout:3/943: write d5/de/df/d57/f74 [145239,116869] 0 2024-02-26T10:48:10.608 INFO:tasks.workunit.client.0.smithi060.stdout:3/944: symlink d5/d45/d61/d92/da1/daa/l143 0 2024-02-26T10:48:10.608 INFO:tasks.workunit.client.0.smithi060.stdout:2/995: rename dd/d43/d60/d41/d37/d4c/fc2 to dd/d11/f14e 0 2024-02-26T10:48:10.608 INFO:tasks.workunit.client.0.smithi060.stdout:3/945: write d5/d45/d61/d92/d105/d138/d93/f96 [256536,45561] 0 2024-02-26T10:48:10.608 INFO:tasks.workunit.client.0.smithi060.stdout:3/946: readlink d5/de/d16/l40 0 2024-02-26T10:48:10.608 INFO:tasks.workunit.client.0.smithi060.stdout:3/947: chown d5/d45/d61/d92/da1 0 1 2024-02-26T10:48:10.610 INFO:tasks.workunit.client.0.smithi060.stdout:0/982: sync 2024-02-26T10:48:10.610 INFO:tasks.workunit.client.0.smithi060.stdout:2/996: rename dd/d43/d60/l120 to dd/d43/d55/d5e/l14f 0 2024-02-26T10:48:10.611 INFO:tasks.workunit.client.0.smithi060.stdout:5/996: dread d0/de/d70/d84/d92/f126 [0,4194304] 0 2024-02-26T10:48:10.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:10 smithi060 ceph-mon[31539]: pgmap v491: 129 pgs: 129 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 173 MiB/s rd, 206 MiB/s wr, 775 op/s 2024-02-26T10:48:10.613 INFO:tasks.workunit.client.0.smithi060.stdout:5/997: rmdir d0/de/d70/d84/d92 39 2024-02-26T10:48:10.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:10 smithi100 ceph-mon[36051]: pgmap v491: 129 pgs: 129 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 173 MiB/s rd, 206 MiB/s wr, 775 op/s 2024-02-26T10:48:10.641 INFO:tasks.workunit.client.0.smithi060.stdout:2/997: dread dd/d43/d60/d41/d37/d4c/d76/fa3 [0,4194304] 0 2024-02-26T10:48:10.644 INFO:tasks.workunit.client.0.smithi060.stdout:5/998: rename d0/de/d20/d32 to d0/de/d15/d65/d86/d9b/d113/d153 0 2024-02-26T10:48:10.646 INFO:tasks.workunit.client.0.smithi060.stdout:5/999: truncate d0/ff4 2654963 0 2024-02-26T10:48:10.646 INFO:tasks.workunit.client.0.smithi060.stdout:2/998: symlink dd/d43/d60/d41/d37/d4c/d76/da6/ddd/l150 0 2024-02-26T10:48:10.646 INFO:tasks.workunit.client.0.smithi060.stdout:2/999: fdatasync dd/dd1/f13f 0 2024-02-26T10:48:10.658 INFO:tasks.workunit.client.0.smithi060.stdout:3/948: read d5/de/df/f33 [1711792,21942] 0 2024-02-26T10:48:10.659 INFO:tasks.workunit.client.0.smithi060.stdout:3/949: truncate d5/d45/d61/d92/da1/ff0 2532751 0 2024-02-26T10:48:10.661 INFO:tasks.workunit.client.0.smithi060.stdout:3/950: creat d5/de/df/d11/f144 x:0 0 0 2024-02-26T10:48:10.663 INFO:tasks.workunit.client.0.smithi060.stdout:0/983: dwrite d0/dd/d53/d87/d94/db7/fc4 [0,4194304] 0 2024-02-26T10:48:10.666 INFO:tasks.workunit.client.0.smithi060.stdout:0/984: mknod d0/dd/d39/d40/dad/dda/de4/d13a/c141 0 2024-02-26T10:48:10.668 INFO:tasks.workunit.client.0.smithi060.stdout:0/985: creat d0/dd/d53/d87/d119/f142 x:0 0 0 2024-02-26T10:48:10.668 INFO:tasks.workunit.client.0.smithi060.stdout:0/986: fsync d0/dd/d39/f7e 0 2024-02-26T10:48:10.670 INFO:tasks.workunit.client.0.smithi060.stdout:0/987: creat d0/dd/d39/d40/dad/dda/de4/d13a/f143 x:0 0 0 2024-02-26T10:48:10.671 INFO:tasks.workunit.client.0.smithi060.stdout:0/988: mkdir d0/dd/d53/d87/d94/db7/d144 0 2024-02-26T10:48:10.674 INFO:tasks.workunit.client.0.smithi060.stdout:0/989: rename d0/dd/d39/d63 to d0/d37/d145 0 2024-02-26T10:48:10.675 INFO:tasks.workunit.client.0.smithi060.stdout:0/990: mkdir d0/dd/d53/d87/d119/d146 0 2024-02-26T10:48:10.702 INFO:tasks.workunit.client.0.smithi060.stdout:0/991: dread d0/dd/d39/f54 [0,4194304] 0 2024-02-26T10:48:10.715 INFO:tasks.workunit.client.0.smithi060.stdout:3/951: dwrite d5/d45/d61/d92/da1/daa/fb3 [0,4194304] 0 2024-02-26T10:48:10.718 INFO:tasks.workunit.client.0.smithi060.stdout:3/952: rename d5/d45/f110 to d5/de/d16/d10b/f145 0 2024-02-26T10:48:10.720 INFO:tasks.workunit.client.0.smithi060.stdout:3/953: unlink d5/de/df/d57/d60/f51 0 2024-02-26T10:48:10.720 INFO:tasks.workunit.client.0.smithi060.stdout:3/954: chown d5/de/df/d4c/dcb/cd9 1821183 1 2024-02-26T10:48:10.721 INFO:tasks.workunit.client.0.smithi060.stdout:3/955: fdatasync d5/de/df/d57/ff5 0 2024-02-26T10:48:10.722 INFO:tasks.workunit.client.0.smithi060.stdout:3/956: link d5/f6 d5/de/df/d4c/f146 0 2024-02-26T10:48:10.723 INFO:tasks.workunit.client.0.smithi060.stdout:3/957: symlink d5/de/d16/d131/l147 0 2024-02-26T10:48:10.724 INFO:tasks.workunit.client.0.smithi060.stdout:3/958: write d5/d45/d61/d92/d105/d138/d93/d97/f106 [570275,72645] 0 2024-02-26T10:48:10.728 INFO:tasks.workunit.client.0.smithi060.stdout:0/992: dread d0/dd/d39/f7c [0,4194304] 0 2024-02-26T10:48:10.762 INFO:tasks.workunit.client.0.smithi060.stdout:0/993: dread d0/d37/d145/fe5 [0,4194304] 0 2024-02-26T10:48:10.763 INFO:tasks.workunit.client.0.smithi060.stdout:0/994: mknod d0/d37/c147 0 2024-02-26T10:48:10.765 INFO:tasks.workunit.client.0.smithi060.stdout:0/995: mkdir d0/d42/d7a/d148 0 2024-02-26T10:48:10.765 INFO:tasks.workunit.client.0.smithi060.stdout:0/996: write d0/d37/d145/fcc [1729004,26457] 0 2024-02-26T10:48:10.806 INFO:tasks.workunit.client.0.smithi060.stdout:0/997: dwrite d0/dd/d39/d40/dad/dd6/d6c/f9a [0,4194304] 0 2024-02-26T10:48:10.807 INFO:tasks.workunit.client.0.smithi060.stdout:0/998: fsync d0/d37/d145/f132 0 2024-02-26T10:48:10.808 INFO:tasks.workunit.client.0.smithi060.stdout:0/999: creat d0/dd/d53/d87/d94/da6/d139/f149 x:0 0 0 2024-02-26T10:48:10.813 INFO:tasks.workunit.client.0.smithi060.stdout:3/959: sync 2024-02-26T10:48:10.854 INFO:tasks.workunit.client.0.smithi060.stdout:3/960: write d5/de/df/d4c/f6b [3692182,93896] 0 2024-02-26T10:48:10.856 INFO:tasks.workunit.client.0.smithi060.stdout:3/961: getdents d5/dbb 0 2024-02-26T10:48:10.857 INFO:tasks.workunit.client.0.smithi060.stdout:3/962: rmdir d5/d45/d61/d92/dec 39 2024-02-26T10:48:10.858 INFO:tasks.workunit.client.0.smithi060.stdout:3/963: mknod d5/d76/d13f/c148 0 2024-02-26T10:48:10.858 INFO:tasks.workunit.client.0.smithi060.stdout:3/964: write d5/de/df/d11/f20 [3719453,78024] 0 2024-02-26T10:48:10.860 INFO:tasks.workunit.client.0.smithi060.stdout:3/965: getdents d5/d45/d61/d92/d105/d138/d93/d97/d98 0 2024-02-26T10:48:10.860 INFO:tasks.workunit.client.0.smithi060.stdout:3/966: dread - d5/d45/d61/d92/da1/daa/dad/dfc/d128/f130 zero size 2024-02-26T10:48:10.942 INFO:tasks.workunit.client.0.smithi060.stdout:3/967: dwrite d5/de/df/d4c/f91 [0,4194304] 0 2024-02-26T10:48:11.017 INFO:tasks.workunit.client.0.smithi060.stdout:3/968: dwrite d5/d45/d61/d92/da1/daa/fa6 [0,4194304] 0 2024-02-26T10:48:11.020 INFO:tasks.workunit.client.0.smithi060.stdout:3/969: rename d5/de/f84 to d5/de/df/d4c/dcb/f149 0 2024-02-26T10:48:11.059 INFO:tasks.workunit.client.0.smithi060.stdout:3/970: dread d5/d45/d61/d92/da1/daa/fa6 [0,4194304] 0 2024-02-26T10:48:11.065 INFO:tasks.workunit.client.0.smithi060.stdout:3/971: write d5/d45/d61/d92/d105/d138/d93/d97/f106 [144651,79561] 0 2024-02-26T10:48:11.161 INFO:tasks.workunit.client.0.smithi060.stdout:3/972: sync 2024-02-26T10:48:11.198 INFO:tasks.workunit.client.0.smithi060.stdout:3/973: write d5/dbb/dcf/ffe [1584243,20837] 0 2024-02-26T10:48:11.232 INFO:tasks.workunit.client.0.smithi060.stdout:3/974: write d5/de/df/f81 [1945425,123112] 0 2024-02-26T10:48:11.284 INFO:tasks.workunit.client.0.smithi060.stdout:3/975: dwrite d5/de/df/d4c/f91 [0,4194304] 0 2024-02-26T10:48:11.284 INFO:tasks.workunit.client.0.smithi060.stdout:3/976: chown d5/dbb 39 1 2024-02-26T10:48:11.284 INFO:tasks.workunit.client.0.smithi060.stdout:3/977: truncate d5/d45/d61/fc5 54293 0 2024-02-26T10:48:11.285 INFO:tasks.workunit.client.0.smithi060.stdout:3/978: truncate d5/f4a 7215209 0 2024-02-26T10:48:11.287 INFO:tasks.workunit.client.0.smithi060.stdout:3/979: mknod d5/d76/c14a 0 2024-02-26T10:48:11.288 INFO:tasks.workunit.client.0.smithi060.stdout:3/980: truncate d5/de/df/d57/d60/f3a 243854 0 2024-02-26T10:48:11.360 INFO:tasks.workunit.client.0.smithi060.stdout:3/981: dwrite d5/de/df/d57/d7a/f89 [0,4194304] 0 2024-02-26T10:48:11.362 INFO:tasks.workunit.client.0.smithi060.stdout:3/982: mknod d5/d45/d61/d92/d105/d138/d93/d97/d98/c14b 0 2024-02-26T10:48:11.364 INFO:tasks.workunit.client.0.smithi060.stdout:3/983: creat d5/d45/d61/d92/da1/f14c x:0 0 0 2024-02-26T10:48:11.371 INFO:tasks.workunit.client.0.smithi060.stdout:3/984: rmdir d5/d45/d61/d92/da1 39 2024-02-26T10:48:11.373 INFO:tasks.workunit.client.0.smithi060.stdout:3/985: unlink d5/de/df/d4c/dcb/df8/fe6 0 2024-02-26T10:48:11.458 INFO:tasks.workunit.client.0.smithi060.stdout:3/986: dwrite d5/f6 [12582912,4194304] 0 2024-02-26T10:48:11.477 INFO:tasks.workunit.client.0.smithi060.stdout:3/987: dread d5/d45/d61/d92/da1/daa/f11b [0,4194304] 0 2024-02-26T10:48:11.548 INFO:tasks.workunit.client.0.smithi060.stdout:3/988: dwrite d5/de/df/d57/d7a/f8b [0,4194304] 0 2024-02-26T10:48:11.548 INFO:tasks.workunit.client.0.smithi060.stdout:3/989: write d5/dbb/dcf/ffe [4281647,60696] 0 2024-02-26T10:48:11.552 INFO:tasks.workunit.client.0.smithi060.stdout:3/990: mknod d5/de/c14d 0 2024-02-26T10:48:11.552 INFO:tasks.workunit.client.0.smithi060.stdout:3/991: dread - d5/d45/d61/d92/fdc zero size 2024-02-26T10:48:11.553 INFO:tasks.workunit.client.0.smithi060.stdout:3/992: creat d5/d45/d61/d92/da1/daa/dad/f14e x:0 0 0 2024-02-26T10:48:11.555 INFO:tasks.workunit.client.0.smithi060.stdout:3/993: symlink d5/d76/d13f/l14f 0 2024-02-26T10:48:11.558 INFO:tasks.workunit.client.0.smithi060.stdout:3/994: rename d5/d76/d7d/l9e to d5/dbb/dcf/dd0/l150 0 2024-02-26T10:48:11.558 INFO:tasks.workunit.client.0.smithi060.stdout:3/995: truncate d5/de/df/d11/de0/f109 145875 0 2024-02-26T10:48:11.628 INFO:tasks.workunit.client.0.smithi060.stdout:3/996: dwrite d5/de/df/d11/de0/f109 [0,4194304] 0 2024-02-26T10:48:11.629 INFO:tasks.workunit.client.0.smithi060.stdout:3/997: getdents d5/d45/d61/d92/d105/d138/d93/d97/d98/d102 0 2024-02-26T10:48:11.629 INFO:tasks.workunit.client.0.smithi060.stdout:3/998: fdatasync d5/fd 0 2024-02-26T10:48:11.629 INFO:tasks.workunit.client.0.smithi060.stdout:3/999: stat d5/de/f11c 0 2024-02-26T10:48:11.639 INFO:tasks.workunit.client.0.smithi060.stderr:+ rm -rf -- ./tmp.mIOfQxL0FK 2024-02-26T10:48:12.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:12 smithi113 ceph-mon[36126]: pgmap v492: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 MiB/s rd, 204 MiB/s wr, 634 op/s 2024-02-26T10:48:12.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:12 smithi060 ceph-mon[31539]: pgmap v492: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 MiB/s rd, 204 MiB/s wr, 634 op/s 2024-02-26T10:48:12.646 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:12 smithi100 ceph-mon[36051]: pgmap v492: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 MiB/s rd, 204 MiB/s wr, 634 op/s 2024-02-26T10:48:14.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:14 smithi113 ceph-mon[36126]: pgmap v493: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 143 MiB/s rd, 180 MiB/s wr, 404 op/s 2024-02-26T10:48:14.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:14 smithi060 ceph-mon[31539]: pgmap v493: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 143 MiB/s rd, 180 MiB/s wr, 404 op/s 2024-02-26T10:48:14.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:14 smithi100 ceph-mon[36051]: pgmap v493: 129 pgs: 129 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 143 MiB/s rd, 180 MiB/s wr, 404 op/s 2024-02-26T10:48:16.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:16 smithi113 ceph-mon[36126]: pgmap v494: 129 pgs: 129 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s rd, 180 MiB/s wr, 468 op/s 2024-02-26T10:48:16.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:16 smithi060 ceph-mon[31539]: pgmap v494: 129 pgs: 129 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s rd, 180 MiB/s wr, 468 op/s 2024-02-26T10:48:16.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:16 smithi100 ceph-mon[36051]: pgmap v494: 129 pgs: 129 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s rd, 180 MiB/s wr, 468 op/s 2024-02-26T10:48:18.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:18 smithi113 ceph-mon[36126]: pgmap v495: 129 pgs: 129 active+clean; 1.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 MiB/s rd, 118 MiB/s wr, 389 op/s 2024-02-26T10:48:18.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:18 smithi060 ceph-mon[31539]: pgmap v495: 129 pgs: 129 active+clean; 1.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 MiB/s rd, 118 MiB/s wr, 389 op/s 2024-02-26T10:48:18.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:18 smithi100 ceph-mon[36051]: pgmap v495: 129 pgs: 129 active+clean; 1.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 MiB/s rd, 118 MiB/s wr, 389 op/s 2024-02-26T10:48:20.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:20 smithi113 ceph-mon[36126]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 102 MiB/s wr, 427 op/s 2024-02-26T10:48:20.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:20 smithi060 ceph-mon[31539]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 102 MiB/s wr, 427 op/s 2024-02-26T10:48:20.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:20 smithi100 ceph-mon[36051]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 102 MiB/s wr, 427 op/s 2024-02-26T10:48:22.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:22 smithi113 ceph-mon[36126]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 54 MiB/s wr, 387 op/s 2024-02-26T10:48:22.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:22 smithi060 ceph-mon[31539]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 54 MiB/s wr, 387 op/s 2024-02-26T10:48:22.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:22 smithi100 ceph-mon[36051]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 54 MiB/s wr, 387 op/s 2024-02-26T10:48:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.715+0000 7fe515a00640 1 -- 172.21.15.60:0/2337710542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101046a0 msgr2=0x7fe510104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.715+0000 7fe515a00640 1 --2- 172.21.15.60:0/2337710542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101046a0 0x7fe510104b20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009900 tx=0x7fe4fc02bc30 comp rx=0 tx=0).stop 2024-02-26T10:48:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.715+0000 7fe515a00640 1 -- 172.21.15.60:0/2337710542 shutdown_connections 2024-02-26T10:48:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.715+0000 7fe515a00640 1 --2- 172.21.15.60:0/2337710542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe510105060 0x7fe51010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.715+0000 7fe515a00640 1 --2- 172.21.15.60:0/2337710542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101046a0 0x7fe510104b20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.715+0000 7fe515a00640 1 --2- 172.21.15.60:0/2337710542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5101034b0 0x7fe5101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.715+0000 7fe515a00640 1 -- 172.21.15.60:0/2337710542 >> 172.21.15.60:0/2337710542 conn(0x7fe5100fea20 msgr2=0x7fe510100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.715+0000 7fe515a00640 1 -- 172.21.15.60:0/2337710542 shutdown_connections 2024-02-26T10:48:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.715+0000 7fe515a00640 1 -- 172.21.15.60:0/2337710542 wait complete. 2024-02-26T10:48:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.716+0000 7fe515a00640 1 Processor -- start 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.716+0000 7fe515a00640 1 -- start start 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.716+0000 7fe515a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101034b0 0x7fe5101996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.716+0000 7fe515a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5101046a0 0x7fe510199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.716+0000 7fe515a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe510105060 0x7fe5101a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.716+0000 7fe515a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51010f190 con 0x7fe5101046a0 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.716+0000 7fe515a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51010f010 con 0x7fe510105060 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.716+0000 7fe515a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51010f310 con 0x7fe5101034b0 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101034b0 0x7fe5101996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe510105060 0x7fe5101a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5101046a0 0x7fe510199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101034b0 0x7fe5101996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:35716/0 (socket says 172.21.15.60:35716) 2024-02-26T10:48:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50ea00640 1 -- 172.21.15.60:0/2093747620 learned_addr learned my addr 172.21.15.60:0/2093747620 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50ea00640 1 -- 172.21.15.60:0/2093747620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe510105060 msgr2=0x7fe5101a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50ea00640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe510105060 0x7fe5101a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50ea00640 1 -- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5101046a0 msgr2=0x7fe510199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50ea00640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5101046a0 0x7fe510199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50ea00640 1 -- 172.21.15.60:0/2093747620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5101a12e0 con 0x7fe5101034b0 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50f400640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe510105060 0x7fe5101a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.717+0000 7fe50e000640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5101046a0 0x7fe510199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.718+0000 7fe50ea00640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101034b0 0x7fe5101996c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe50000e9c0 tx=0x7fe50000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.718+0000 7fe507400640 1 -- 172.21.15.60:0/2093747620 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50000cd80 con 0x7fe5101034b0 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.718+0000 7fe507400640 1 -- 172.21.15.60:0/2093747620 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5000035f0 con 0x7fe5101034b0 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.718+0000 7fe515a00640 1 -- 172.21.15.60:0/2093747620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5101a15d0 con 0x7fe5101034b0 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.718+0000 7fe507400640 1 -- 172.21.15.60:0/2093747620 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500010690 con 0x7fe5101034b0 2024-02-26T10:48:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.718+0000 7fe515a00640 1 -- 172.21.15.60:0/2093747620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5101a1b40 con 0x7fe5101034b0 2024-02-26T10:48:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.721+0000 7fe507400640 1 -- 172.21.15.60:0/2093747620 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe5000037a0 con 0x7fe5101034b0 2024-02-26T10:48:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.721+0000 7fe518877740 1 Processor -- start 2024-02-26T10:48:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.721+0000 7fe518877740 1 -- start start 2024-02-26T10:48:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe518877740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 0x559043825450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe518877740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x55904382a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe518877740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904382ab90 0x55904382d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe518877740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043782340 con 0x559043825a20 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe518877740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904372b520 con 0x55904382ab90 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe518877740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904377e3e0 con 0x559043825030 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe50e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904382ab90 0x55904382d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe50ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x55904382a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe50e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904382ab90 0x55904382d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:54704/0 (socket says 172.21.15.60:54704) 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe50e000640 1 -- 172.21.15.60:0/165562244 learned_addr learned my addr 172.21.15.60:0/165562244 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.722+0000 7fe50f400640 1 --2- 172.21.15.60:0/165562244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 0x559043825450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe50e000640 1 -- 172.21.15.60:0/165562244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 msgr2=0x559043825450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe50e000640 1 --2- 172.21.15.60:0/165562244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 0x559043825450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe50e000640 1 -- 172.21.15.60:0/165562244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 msgr2=0x55904382a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe50e000640 1 --2- 172.21.15.60:0/165562244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x55904382a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe50e000640 1 -- 172.21.15.60:0/165562244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55904373f390 con 0x55904382ab90 2024-02-26T10:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe507400640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4f40a1ea0 0x7fe4f40a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe507400640 1 -- 172.21.15.60:0/2093747620 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe5000ce4c0 con 0x7fe5101034b0 2024-02-26T10:48:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe50f400640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4f40a1ea0 0x7fe4f40a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe50e000640 1 --2- 172.21.15.60:0/165562244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904382ab90 0x55904382d020 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005ab0 tx=0x7fe4fc009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3b85838a8a52da server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.723+0000 7fe4fbe00640 1 -- 172.21.15.60:0/165562244 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc045070 con 0x55904382ab90 2024-02-26T10:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.724+0000 7fe4fbe00640 1 -- 172.21.15.60:0/165562244 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc02cd30 con 0x55904382ab90 2024-02-26T10:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.724+0000 7fe4fbe00640 1 -- 172.21.15.60:0/165562244 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc005130 con 0x55904382ab90 2024-02-26T10:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.724+0000 7fe518877740 1 -- 172.21.15.60:0/165562244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904382ab90 msgr2=0x55904382d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.724+0000 7fe50f400640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4f40a1ea0 0x7fe4f40a4360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x559043826280 tx=0x7fe508009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.724+0000 7fe518877740 1 --2- 172.21.15.60:0/165562244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904382ab90 0x55904382d020 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005ab0 tx=0x7fe4fc009560 comp rx=0 tx=0).stop 2024-02-26T10:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.725+0000 7fe518877740 1 -- 172.21.15.60:0/165562244 shutdown_connections 2024-02-26T10:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.725+0000 7fe518877740 1 --2- 172.21.15.60:0/165562244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904382ab90 0x55904382d020 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.725+0000 7fe518877740 1 --2- 172.21.15.60:0/165562244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x55904382a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.725+0000 7fe518877740 1 --2- 172.21.15.60:0/165562244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 0x559043825450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.725+0000 7fe518877740 1 -- 172.21.15.60:0/165562244 >> 172.21.15.60:0/165562244 conn(0x5590438205a0 msgr2=0x5590438209a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.725+0000 7fe518877740 1 -- 172.21.15.60:0/165562244 shutdown_connections 2024-02-26T10:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.725+0000 7fe518877740 1 -- 172.21.15.60:0/165562244 wait complete. 2024-02-26T10:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.725+0000 7fe518877740 1 -- start start 2024-02-26T10:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe518877740 1 Processor -- start 2024-02-26T10:48:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe518877740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x55904382f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe518877740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x55904382f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe518877740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 0x5590438368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe518877740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904382dae0 con 0x559043825a20 2024-02-26T10:48:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe518877740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904373e980 con 0x559043825030 2024-02-26T10:48:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe518877740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904382dc60 con 0x55904382ab90 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe518877740 10 client.0 _ll_register_callbacks cb 0x55904377a310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe50ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x55904382f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe50ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x55904382f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:54706/0 (socket says 172.21.15.60:54706) 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe50ea00640 1 -- 172.21.15.60:0/2453453249 learned_addr learned my addr 172.21.15.60:0/2453453249 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.726+0000 7fe50e000640 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x55904382f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe50f400640 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 0x5590438368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe50ea00640 1 -- 172.21.15.60:0/2453453249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 msgr2=0x5590438368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe50ea00640 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 0x5590438368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe50ea00640 1 -- 172.21.15.60:0/2453453249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 msgr2=0x55904382f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe50ea00640 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x55904382f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe50ea00640 1 -- 172.21.15.60:0/2453453249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559043497210 con 0x559043825030 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe50ea00640 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x55904382f390 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe5000a3050 tx=0x7fe50009abb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab7cef5480ce44e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe4f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe4f8c00640 1 -- 172.21.15.60:0/2453453249 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50009e070 con 0x559043825030 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe4f8c00640 1 -- 172.21.15.60:0/2453453249 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe5000ce160 con 0x559043825030 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.727+0000 7fe518877740 1 -- 172.21.15.60:0/2453453249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4fc02cd30 con 0x559043825030 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.728+0000 7fe4f8c00640 1 -- 172.21.15.60:0/2453453249 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe50009b4e0 con 0x559043825030 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.728+0000 7fe518877740 10 client.26129 fetch_fsmap learned FSMap version 30 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.728+0000 7fe518877740 1 -- 172.21.15.60:0/2453453249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559043563620 con 0x559043825030 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.728+0000 7fe4f8c00640 1 -- 172.21.15.60:0/2453453249 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc02cd30 con 0x559043825030 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.728+0000 7fe4f8c00640 1 -- 172.21.15.60:0/2453453249 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe50009a230 con 0x559043825030 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.728+0000 7fe518877740 10 client.26129 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.728+0000 7fe518877740 10 client.26129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.728+0000 7fe518877740 20 client.26129 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.728+0000 7fe518877740 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55904381bb40 0x55904381dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.729+0000 7fe518877740 4 client.26129 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.729+0000 7fe518877740 1 -- 172.21.15.60:0/2453453249 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55904378f880 con 0x55904381bb40 2024-02-26T10:48:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.729+0000 7fe50e000640 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55904381bb40 0x55904381dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.729+0000 7fe50e000640 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55904381bb40 0x55904381dfe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3244a1c9a93a07 server_cookie=159cfeca95c39ddb in_seq=0 out_seq=0 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.729+0000 7fe4f8c00640 10 client.26129 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe4f8c00640 1 -- 172.21.15.60:0/2453453249 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55904378f880 con 0x55904381bb40 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe4f8c00640 10 client.26129 handle_command_reply: tid=0 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe518877740 1 client.26129 shutdown 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe518877740 1 -- 172.21.15.60:0/2453453249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 msgr2=0x55904382f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe518877740 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x55904382f390 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe5000a3050 tx=0x7fe50009abb0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe518877740 1 -- 172.21.15.60:0/2453453249 shutdown_connections 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe518877740 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55904381bb40 0x55904381dfe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe518877740 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 0x5590438368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe518877740 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x55904382f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe4f8c00640 0 client.26129 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe518877740 1 --2- 172.21.15.60:0/2453453249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x55904382f390 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.730+0000 7fe518877740 1 -- 172.21.15.60:0/2453453249 >> 172.21.15.60:0/2453453249 conn(0x5590438205a0 msgr2=0x559043816fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.731+0000 7fe518877740 1 -- 172.21.15.60:0/2453453249 shutdown_connections 2024-02-26T10:48:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.731+0000 7fe518877740 1 -- 172.21.15.60:0/2453453249 wait complete. 2024-02-26T10:48:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.731+0000 7fe518877740 20 client.26129 trim_cache size 0 max 16384 2024-02-26T10:48:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe518877740 1 Processor -- start 2024-02-26T10:48:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe518877740 1 -- start start 2024-02-26T10:48:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe518877740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 0x5590438262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe518877740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825a20 0x559043836830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe518877740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55904382ab90 0x559043826de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe518877740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101a9fc0 con 0x55904382ab90 2024-02-26T10:48:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe518877740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101a7aa0 con 0x559043825a20 2024-02-26T10:48:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe518877740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101a5b40 con 0x559043825030 2024-02-26T10:48:22.742 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe50e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825a20 0x559043836830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe50e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825a20 0x559043836830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:54716/0 (socket says 172.21.15.60:54716) 2024-02-26T10:48:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe50e000640 1 -- 172.21.15.60:0/359626529 learned_addr learned my addr 172.21.15.60:0/359626529 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.748+0000 7fe50ea00640 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 0x5590438262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe50f400640 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55904382ab90 0x559043826de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe50e000640 1 -- 172.21.15.60:0/359626529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 msgr2=0x5590438262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe50e000640 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 0x5590438262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe50e000640 1 -- 172.21.15.60:0/359626529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55904382ab90 msgr2=0x559043826de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe50e000640 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55904382ab90 0x559043826de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe50e000640 1 -- 172.21.15.60:0/359626529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55904384c6a0 con 0x559043825a20 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe50f400640 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55904382ab90 0x559043826de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe50ea00640 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 0x5590438262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe50e000640 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825a20 0x559043836830 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc02c7b0 tx=0x7fe4fc02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7987d5b0239edb6d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe4fb400640 1 -- 172.21.15.60:0/359626529 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc055070 con 0x559043825a20 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe4fb400640 1 -- 172.21.15.60:0/359626529 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc02cd30 con 0x559043825a20 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.749+0000 7fe4fb400640 1 -- 172.21.15.60:0/359626529 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc045040 con 0x559043825a20 2024-02-26T10:48:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.750+0000 7fe518877740 1 -- 172.21.15.60:0/359626529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825a20 msgr2=0x559043836830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.750+0000 7fe518877740 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825a20 0x559043836830 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc02c7b0 tx=0x7fe4fc02bc30 comp rx=0 tx=0).stop 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.750+0000 7fe518877740 1 -- 172.21.15.60:0/359626529 shutdown_connections 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.750+0000 7fe518877740 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825a20 0x559043836830 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.750+0000 7fe518877740 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55904382ab90 0x559043826de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.750+0000 7fe518877740 1 --2- 172.21.15.60:0/359626529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559043825030 0x5590438262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.750+0000 7fe518877740 1 -- 172.21.15.60:0/359626529 >> 172.21.15.60:0/359626529 conn(0x5590438205a0 msgr2=0x559043820980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.750+0000 7fe518877740 1 -- 172.21.15.60:0/359626529 shutdown_connections 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.750+0000 7fe518877740 1 -- 172.21.15.60:0/359626529 wait complete. 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe518877740 1 -- start start 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe518877740 1 Processor -- start 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe518877740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x559043833380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe518877740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x5590438338c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe518877740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 0x559043899570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe518877740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101a6140 con 0x559043825a20 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe518877740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101a7440 con 0x559043825030 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe518877740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101a6780 con 0x55904382ab90 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe518877740 10 client.0 _ll_register_callbacks cb 0x7fe5080093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe50ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x559043833380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.751+0000 7fe50ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x559043833380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:54722/0 (socket says 172.21.15.60:54722) 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe50ea00640 1 -- 172.21.15.60:0/3825300879 learned_addr learned my addr 172.21.15.60:0/3825300879 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe50e000640 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x5590438338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe50f400640 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 0x559043899570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe50ea00640 1 -- 172.21.15.60:0/3825300879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 msgr2=0x559043899570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe50ea00640 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 0x559043899570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe50ea00640 1 -- 172.21.15.60:0/3825300879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 msgr2=0x5590438338c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe50ea00640 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x5590438338c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe50ea00640 1 -- 172.21.15.60:0/3825300879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559043836990 con 0x559043825030 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe50ea00640 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x559043833380 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe500012020 tx=0x7fe5000cbc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b075b1d0bc05d9f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe4f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe4f8c00640 1 -- 172.21.15.60:0/3825300879 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50009e070 con 0x559043825030 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe4f8c00640 1 -- 172.21.15.60:0/3825300879 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe5000ce160 con 0x559043825030 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.752+0000 7fe518877740 1 -- 172.21.15.60:0/3825300879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4fc02cd30 con 0x559043825030 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe4f8c00640 1 -- 172.21.15.60:0/3825300879 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe50009a920 con 0x559043825030 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe518877740 10 client.26135 fetch_fsmap learned FSMap version 30 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe518877740 1 -- 172.21.15.60:0/3825300879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55904389a990 con 0x559043825030 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe4f8c00640 1 -- 172.21.15.60:0/3825300879 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc02cd30 con 0x559043825030 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe4f8c00640 1 -- 172.21.15.60:0/3825300879 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe500099ea0 con 0x559043825030 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe518877740 10 client.26135 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe518877740 10 client.26135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe518877740 20 client.26135 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe518877740 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55904389aec0 0x55904389d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe518877740 4 client.26135 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:48:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe518877740 1 -- 172.21.15.60:0/3825300879 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559043789bd0 con 0x55904389aec0 2024-02-26T10:48:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.753+0000 7fe50e000640 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55904389aec0 0x55904389d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.754+0000 7fe50e000640 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55904389aec0 0x55904389d2e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee79aadd31a113d server_cookie=a0b620ca276f8a6e in_seq=0 out_seq=0 2024-02-26T10:48:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.754+0000 7fe4f8c00640 10 client.26135 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe4f8c00640 1 -- 172.21.15.60:0/3825300879 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559043789bd0 con 0x55904389aec0 2024-02-26T10:48:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe4f8c00640 10 client.26135 handle_command_reply: tid=0 2024-02-26T10:48:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 client.26135 shutdown 2024-02-26T10:48:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 -- 172.21.15.60:0/3825300879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 msgr2=0x559043833380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x559043833380 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe500012020 tx=0x7fe5000cbc30 comp rx=0 tx=0).stop 2024-02-26T10:48:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 -- 172.21.15.60:0/3825300879 shutdown_connections 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55904389aec0 0x55904389d2e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55904382ab90 0x559043899570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559043825a20 0x5590438338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 --2- 172.21.15.60:0/3825300879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559043825030 0x559043833380 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe4f8c00640 0 client.26135 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 -- 172.21.15.60:0/3825300879 >> 172.21.15.60:0/3825300879 conn(0x5590438205a0 msgr2=0x559043826de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 -- 172.21.15.60:0/3825300879 shutdown_connections 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.799+0000 7fe518877740 1 -- 172.21.15.60:0/3825300879 wait complete. 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe518877740 20 client.26135 trim_cache size 0 max 16384 2024-02-26T10:48:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:48:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:48:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:48:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:48:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 -- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4f40a1ea0 msgr2=0x7fe4f40a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4f40a1ea0 0x7fe4f40a4360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x559043826280 tx=0x7fe508009500 comp rx=0 tx=0).stop 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 -- 172.21.15.60:0/2093747620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101034b0 msgr2=0x7fe5101996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101034b0 0x7fe5101996c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe50000e9c0 tx=0x7fe50000ee80 comp rx=0 tx=0).stop 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 -- 172.21.15.60:0/2093747620 shutdown_connections 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4f40a1ea0 0x7fe4f40a4360 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe510105060 0x7fe5101a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5101046a0 0x7fe510199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 --2- 172.21.15.60:0/2093747620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5101034b0 0x7fe5101996c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.800+0000 7fe515a00640 1 -- 172.21.15.60:0/2093747620 >> 172.21.15.60:0/2093747620 conn(0x7fe5100fea20 msgr2=0x7fe5101078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.801+0000 7fe515a00640 1 -- 172.21.15.60:0/2093747620 shutdown_connections 2024-02-26T10:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:22.801+0000 7fe515a00640 1 -- 172.21.15.60:0/2093747620 wait complete. 2024-02-26T10:48:24.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:24 smithi113 ceph-mon[36126]: pgmap v498: 129 pgs: 129 active+clean; 1012 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 21 MiB/s wr, 276 op/s 2024-02-26T10:48:24.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:24 smithi060 ceph-mon[31539]: pgmap v498: 129 pgs: 129 active+clean; 1012 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 21 MiB/s wr, 276 op/s 2024-02-26T10:48:24.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:24 smithi100 ceph-mon[36051]: pgmap v498: 129 pgs: 129 active+clean; 1012 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 21 MiB/s wr, 276 op/s 2024-02-26T10:48:24.635 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-26T10:48:24.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-26T10:48:26.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:26 smithi113 ceph-mon[36126]: pgmap v499: 129 pgs: 129 active+clean; 471 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 330 op/s 2024-02-26T10:48:26.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:26 smithi060 ceph-mon[31539]: pgmap v499: 129 pgs: 129 active+clean; 471 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 330 op/s 2024-02-26T10:48:26.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:26 smithi100 ceph-mon[36051]: pgmap v499: 129 pgs: 129 active+clean; 471 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 330 op/s 2024-02-26T10:48:28.426 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:28 smithi100 ceph-mon[36051]: pgmap v500: 129 pgs: 129 active+clean; 337 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 255 op/s 2024-02-26T10:48:28.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:28 smithi113 ceph-mon[36126]: pgmap v500: 129 pgs: 129 active+clean; 337 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 255 op/s 2024-02-26T10:48:28.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:28 smithi060 ceph-mon[31539]: pgmap v500: 129 pgs: 129 active+clean; 337 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 255 op/s 2024-02-26T10:48:30.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:30 smithi113 ceph-mon[36126]: pgmap v501: 129 pgs: 129 active+clean; 146 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 328 op/s 2024-02-26T10:48:30.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:30 smithi060 ceph-mon[31539]: pgmap v501: 129 pgs: 129 active+clean; 146 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 328 op/s 2024-02-26T10:48:30.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:30 smithi100 ceph-mon[36051]: pgmap v501: 129 pgs: 129 active+clean; 146 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 328 op/s 2024-02-26T10:48:32.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:32 smithi113 ceph-mon[36126]: pgmap v502: 129 pgs: 129 active+clean; 106 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 372 op/s 2024-02-26T10:48:32.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:32 smithi060 ceph-mon[31539]: pgmap v502: 129 pgs: 129 active+clean; 106 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 372 op/s 2024-02-26T10:48:32.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:32 smithi100 ceph-mon[36051]: pgmap v502: 129 pgs: 129 active+clean; 106 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 372 op/s 2024-02-26T10:48:34.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:34 smithi113 ceph-mon[36126]: pgmap v503: 129 pgs: 129 active+clean; 109 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 395 op/s 2024-02-26T10:48:34.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:34 smithi060 ceph-mon[31539]: pgmap v503: 129 pgs: 129 active+clean; 109 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 395 op/s 2024-02-26T10:48:34.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:34 smithi100 ceph-mon[36051]: pgmap v503: 129 pgs: 129 active+clean; 109 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 395 op/s 2024-02-26T10:48:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.990+0000 7f2c2ea00640 1 -- 172.21.15.60:0/550376350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c28101290 msgr2=0x7f2c281016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.990+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/550376350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c28101290 0x7f2c281016b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009900 tx=0x7f2c1802bc30 comp rx=0 tx=0).stop 2024-02-26T10:48:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.990+0000 7f2c2ea00640 1 -- 172.21.15.60:0/550376350 shutdown_connections 2024-02-26T10:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.990+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/550376350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c28102480 0x7f2c28102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.990+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/550376350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c28102e40 0x7f2c281096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.990+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/550376350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c28101290 0x7f2c281016b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.990+0000 7f2c2ea00640 1 -- 172.21.15.60:0/550376350 >> 172.21.15.60:0/550376350 conn(0x7f2c280fc820 msgr2=0x7f2c280fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.991+0000 7f2c2ea00640 1 -- 172.21.15.60:0/550376350 shutdown_connections 2024-02-26T10:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.991+0000 7f2c2ea00640 1 -- 172.21.15.60:0/550376350 wait complete. 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.991+0000 7f2c2ea00640 1 Processor -- start 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c2ea00640 1 -- start start 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c2ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c28101290 0x7f2c28199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c2ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c28102480 0x7f2c28199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c2ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c28102e40 0x7f2c281a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c2ea00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2810cf70 con 0x7f2c28102e40 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c2ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2810cdf0 con 0x7f2c28102480 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c2ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2810d0f0 con 0x7f2c28101290 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c28102480 0x7f2c28199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c27e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c28101290 0x7f2c28199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c28102480 0x7f2c28199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42680/0 (socket says 172.21.15.60:42680) 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c27400640 1 -- 172.21.15.60:0/1533782545 learned_addr learned my addr 172.21.15.60:0/1533782545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.992+0000 7f2c2ca00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c28102e40 0x7f2c281a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c27e00640 1 -- 172.21.15.60:0/1533782545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c28102480 msgr2=0x7f2c28199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c27e00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c28102480 0x7f2c28199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c27e00640 1 -- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c28102e40 msgr2=0x7f2c281a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c27e00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c28102e40 0x7f2c281a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c27e00640 1 -- 172.21.15.60:0/1533782545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c281a1320 con 0x7f2c28101290 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c27400640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c28102480 0x7f2c28199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c2ca00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c28102e40 0x7f2c281a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c27e00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c28101290 0x7f2c28199700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009230 tx=0x7f2c1802c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c24c00640 1 -- 172.21.15.60:0/1533782545 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18043020 con 0x7f2c28101290 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c24c00640 1 -- 172.21.15.60:0/1533782545 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c180054e0 con 0x7f2c28101290 2024-02-26T10:48:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c2ea00640 1 -- 172.21.15.60:0/1533782545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c281a15b0 con 0x7f2c28101290 2024-02-26T10:48:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c24c00640 1 -- 172.21.15.60:0/1533782545 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18003b60 con 0x7f2c28101290 2024-02-26T10:48:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.993+0000 7f2c2ea00640 1 -- 172.21.15.60:0/1533782545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c281a1ad0 con 0x7f2c28101290 2024-02-26T10:48:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.997+0000 7f2c24c00640 1 -- 172.21.15.60:0/1533782545 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2c18046020 con 0x7f2c28101290 2024-02-26T10:48:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.997+0000 7f2c31a10740 1 Processor -- start 2024-02-26T10:48:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.997+0000 7f2c31a10740 1 -- start start 2024-02-26T10:48:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.997+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 0x560db5208c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.997+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db520dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.997+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 0x560db5210810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.997+0000 7f2c31a10740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db516e340 con 0x560db5208820 2024-02-26T10:48:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.997+0000 7f2c31a10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db5117520 con 0x560db5209210 2024-02-26T10:48:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.997+0000 7f2c31a10740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db516a3e0 con 0x560db520e380 2024-02-26T10:48:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db520dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db520dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42696/0 (socket says 172.21.15.60:42696) 2024-02-26T10:48:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c27e00640 1 -- 172.21.15.60:0/2417395862 learned_addr learned my addr 172.21.15.60:0/2417395862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c2ca00640 1 --2- 172.21.15.60:0/2417395862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 0x560db5208c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c27e00640 1 -- 172.21.15.60:0/2417395862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 msgr2=0x560db5210810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c27e00640 1 --2- 172.21.15.60:0/2417395862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 0x560db5210810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c27e00640 1 -- 172.21.15.60:0/2417395862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 msgr2=0x560db5208c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c27e00640 1 --2- 172.21.15.60:0/2417395862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 0x560db5208c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c27e00640 1 -- 172.21.15.60:0/2417395862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db512b390 con 0x560db5209210 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c24c00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c0c0a1e20 0x7f2c0c0a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.998+0000 7f2c27e00640 1 --2- 172.21.15.60:0/2417395862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db520dd10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2c180f81b0 tx=0x7f2c180f81e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9ef2bafc29303f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.999+0000 7f2c1d200640 1 -- 172.21.15.60:0/2417395862 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c180b8e90 con 0x560db5209210 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.999+0000 7f2c1d200640 1 -- 172.21.15.60:0/2417395862 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c180bd2f0 con 0x560db5209210 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.999+0000 7f2c1d200640 1 -- 172.21.15.60:0/2417395862 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c180d2200 con 0x560db5209210 2024-02-26T10:48:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.999+0000 7f2c24c00640 1 -- 172.21.15.60:0/1533782545 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2c180f0150 con 0x7f2c28101290 2024-02-26T10:48:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.999+0000 7f2c31a10740 1 -- 172.21.15.60:0/2417395862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 msgr2=0x560db520dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.999+0000 7f2c31a10740 1 --2- 172.21.15.60:0/2417395862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db520dd10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2c180f81b0 tx=0x7f2c180f81e0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:35.999+0000 7f2c2ca00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c0c0a1e20 0x7f2c0c0a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c31a10740 1 -- 172.21.15.60:0/2417395862 shutdown_connections 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c31a10740 1 --2- 172.21.15.60:0/2417395862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db520dd10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c31a10740 1 --2- 172.21.15.60:0/2417395862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 0x560db5210810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c31a10740 1 --2- 172.21.15.60:0/2417395862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 0x560db5208c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c31a10740 1 -- 172.21.15.60:0/2417395862 >> 172.21.15.60:0/2417395862 conn(0x560db5203d90 msgr2=0x560db5204190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c31a10740 1 -- 172.21.15.60:0/2417395862 shutdown_connections 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c31a10740 1 -- 172.21.15.60:0/2417395862 wait complete. 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c2ca00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c0c0a1e20 0x7f2c0c0a42e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x560db5209a70 tx=0x7f2c20009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c31a10740 1 -- start start 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.000+0000 7f2c31a10740 1 Processor -- start 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 0x560db5212730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db5212c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 0x560db5219cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c31a10740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db5117520 con 0x560db5208820 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c31a10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db512a980 con 0x560db5209210 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c31a10740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db52110f0 con 0x560db520e380 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c31a10740 10 client.0 _ll_register_callbacks cb 0x560db5166310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db5212c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db5212c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42710/0 (socket says 172.21.15.60:42710) 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c27e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 0x560db5212730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c27400640 1 -- 172.21.15.60:0/1911717498 learned_addr learned my addr 172.21.15.60:0/1911717498 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.001+0000 7f2c2ca00640 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 0x560db5219cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c27400640 1 -- 172.21.15.60:0/1911717498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 msgr2=0x560db5219cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c27400640 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 0x560db5219cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c27400640 1 -- 172.21.15.60:0/1911717498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 msgr2=0x560db5212730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c27400640 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 0x560db5212730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c27400640 1 -- 172.21.15.60:0/1911717498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560db4e83210 con 0x560db5209210 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c2ca00640 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 0x560db5219cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c27400640 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db5212c70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2c14011d00 tx=0x7f2c1400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c5e2fc4c5acb13b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c11600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c11600640 1 -- 172.21.15.60:0/1911717498 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14009e10 con 0x560db5209210 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c11600640 1 -- 172.21.15.60:0/1911717498 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2c14004050 con 0x560db5209210 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.002+0000 7f2c31a10740 1 -- 172.21.15.60:0/1911717498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c180bd2f0 con 0x560db5209210 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c11600640 1 -- 172.21.15.60:0/1911717498 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c14004250 con 0x560db5209210 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c31a10740 10 client.26147 fetch_fsmap learned FSMap version 30 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c31a10740 1 -- 172.21.15.60:0/1911717498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560db4f4f620 con 0x560db5209210 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c11600640 1 -- 172.21.15.60:0/1911717498 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c180bd2f0 con 0x560db5209210 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c11600640 1 -- 172.21.15.60:0/1911717498 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f2c14018070 con 0x560db5209210 2024-02-26T10:48:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c31a10740 10 client.26147 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c31a10740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c31a10740 20 client.26147 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560db521e6a0 0x560db5220b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c31a10740 4 client.26147 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.003+0000 7f2c31a10740 1 -- 172.21.15.60:0/1911717498 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560db517b880 con 0x560db521e6a0 2024-02-26T10:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.004+0000 7f2c27e00640 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560db521e6a0 0x560db5220b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.004+0000 7f2c27e00640 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560db521e6a0 0x560db5220b00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4177525efbd047e server_cookie=c51d6c9b0918107c in_seq=0 out_seq=0 2024-02-26T10:48:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.004+0000 7f2c11600640 10 client.26147 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c11600640 1 -- 172.21.15.60:0/1911717498 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560db517b880 con 0x560db521e6a0 2024-02-26T10:48:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c11600640 10 client.26147 handle_command_reply: tid=0 2024-02-26T10:48:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 client.26147 shutdown 2024-02-26T10:48:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 -- 172.21.15.60:0/1911717498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 msgr2=0x560db5212c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db5212c70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2c14011d00 tx=0x7f2c1400f790 comp rx=0 tx=0).stop 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 -- 172.21.15.60:0/1911717498 shutdown_connections 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560db521e6a0 0x560db5220b00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5209210 0x560db5212c70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db520e380 0x560db5219cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1911717498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5208820 0x560db5212730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c11600640 0 client.26147 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 -- 172.21.15.60:0/1911717498 >> 172.21.15.60:0/1911717498 conn(0x560db5203d90 msgr2=0x560db5202fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 -- 172.21.15.60:0/1911717498 shutdown_connections 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 1 -- 172.21.15.60:0/1911717498 wait complete. 2024-02-26T10:48:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.005+0000 7f2c31a10740 20 client.26147 trim_cache size 0 max 16384 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c31a10740 1 Processor -- start 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c31a10740 1 -- start start 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5203d90 0x560db5212730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5212c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5209210 0x560db5219c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c31a10740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c281ae4d0 con 0x560db5209210 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c31a10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c281abfb0 con 0x560db5208820 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c31a10740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c281aa050 con 0x560db5203d90 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5212c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c2ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5209210 0x560db5219c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5212c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42716/0 (socket says 172.21.15.60:42716) 2024-02-26T10:48:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.021+0000 7f2c27e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5203d90 0x560db5212730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.022+0000 7f2c27400640 1 -- 172.21.15.60:0/952834804 learned_addr learned my addr 172.21.15.60:0/952834804 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.022+0000 7f2c27400640 1 -- 172.21.15.60:0/952834804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5203d90 msgr2=0x560db5212730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.022+0000 7f2c27400640 1 --2- 172.21.15.60:0/952834804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5203d90 0x560db5212730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.022+0000 7f2c27400640 1 -- 172.21.15.60:0/952834804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5209210 msgr2=0x560db5219c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.022+0000 7f2c27400640 1 --2- 172.21.15.60:0/952834804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5209210 0x560db5219c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.022+0000 7f2c27400640 1 -- 172.21.15.60:0/952834804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db5225d30 con 0x560db5208820 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.022+0000 7f2c27400640 1 --2- 172.21.15.60:0/952834804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5212c70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2c1401a7f0 tx=0x7f2c14005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83a40d7500511565 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.022+0000 7f2c13e00640 1 -- 172.21.15.60:0/952834804 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14009c10 con 0x560db5208820 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c13e00640 1 -- 172.21.15.60:0/952834804 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c14005510 con 0x560db5208820 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c13e00640 1 -- 172.21.15.60:0/952834804 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14028bb0 con 0x560db5208820 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c31a10740 1 -- 172.21.15.60:0/952834804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 msgr2=0x560db5212c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c31a10740 1 --2- 172.21.15.60:0/952834804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5212c70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2c1401a7f0 tx=0x7f2c14005770 comp rx=0 tx=0).stop 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c31a10740 1 -- 172.21.15.60:0/952834804 shutdown_connections 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c31a10740 1 --2- 172.21.15.60:0/952834804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5212c70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c31a10740 1 --2- 172.21.15.60:0/952834804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5209210 0x560db5219c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c31a10740 1 --2- 172.21.15.60:0/952834804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5203d90 0x560db5212730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c31a10740 1 -- 172.21.15.60:0/952834804 >> 172.21.15.60:0/952834804 conn(0x560db520de80 msgr2=0x560db520a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c31a10740 1 -- 172.21.15.60:0/952834804 shutdown_connections 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.023+0000 7f2c31a10740 1 -- 172.21.15.60:0/952834804 wait complete. 2024-02-26T10:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.024+0000 7f2c31a10740 1 -- start start 2024-02-26T10:48:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.024+0000 7f2c31a10740 1 Processor -- start 2024-02-26T10:48:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.024+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5203d90 0x560db52194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5219a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c31a10740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5209210 0x560db5217af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c31a10740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c281aa650 con 0x560db5203d90 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c31a10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c281ab950 con 0x560db5208820 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c31a10740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c281aac90 con 0x560db5209210 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c31a10740 10 client.0 _ll_register_callbacks cb 0x7f2c180f3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5219a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c2ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5209210 0x560db5217af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5219a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42728/0 (socket says 172.21.15.60:42728) 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27400640 1 -- 172.21.15.60:0/1090653406 learned_addr learned my addr 172.21.15.60:0/1090653406 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27e00640 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5203d90 0x560db52194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27400640 1 -- 172.21.15.60:0/1090653406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5209210 msgr2=0x560db5217af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27400640 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5209210 0x560db5217af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27400640 1 -- 172.21.15.60:0/1090653406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5203d90 msgr2=0x560db52194e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27400640 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5203d90 0x560db52194e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27400640 1 -- 172.21.15.60:0/1090653406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560db52181e0 con 0x560db5208820 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c27400640 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5219a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c1401ab40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c266e0dd56d16ec7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.025+0000 7f2c11600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.026+0000 7f2c11600640 1 -- 172.21.15.60:0/1090653406 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14009e20 con 0x560db5208820 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.026+0000 7f2c31a10740 1 -- 172.21.15.60:0/1090653406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c14005510 con 0x560db5208820 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.026+0000 7f2c11600640 1 -- 172.21.15.60:0/1090653406 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2c14005040 con 0x560db5208820 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.026+0000 7f2c11600640 1 -- 172.21.15.60:0/1090653406 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c14040050 con 0x560db5208820 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.026+0000 7f2c31a10740 10 client.26153 fetch_fsmap learned FSMap version 30 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.026+0000 7f2c31a10740 1 -- 172.21.15.60:0/1090653406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560db52755e0 con 0x560db5208820 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.026+0000 7f2c11600640 1 -- 172.21.15.60:0/1090653406 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c14005510 con 0x560db5208820 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.026+0000 7f2c11600640 1 -- 172.21.15.60:0/1090653406 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f2c14018040 con 0x560db5208820 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.027+0000 7f2c31a10740 10 client.26153 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.027+0000 7f2c31a10740 10 client.26153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.027+0000 7f2c31a10740 20 client.26153 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.027+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560db520e380 0x560db5277f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.027+0000 7f2c31a10740 4 client.26153 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:48:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.027+0000 7f2c31a10740 1 -- 172.21.15.60:0/1090653406 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560db5175bd0 con 0x560db520e380 2024-02-26T10:48:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.027+0000 7f2c27e00640 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560db520e380 0x560db5277f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.027+0000 7f2c27e00640 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560db520e380 0x560db5277f10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6087369f1846925 server_cookie=963d7f5365f0ee4a in_seq=0 out_seq=0 2024-02-26T10:48:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.027+0000 7f2c11600640 10 client.26153 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.139+0000 7f2c11600640 1 -- 172.21.15.60:0/1090653406 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560db5175bd0 con 0x560db520e380 2024-02-26T10:48:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.139+0000 7f2c11600640 10 client.26153 handle_command_reply: tid=0 2024-02-26T10:48:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.139+0000 7f2c31a10740 1 client.26153 shutdown 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 -- 172.21.15.60:0/1090653406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 msgr2=0x560db5219a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5219a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c1401ab40 comp rx=0 tx=0).stop 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 -- 172.21.15.60:0/1090653406 shutdown_connections 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x560db520e380 0x560db5277f10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560db5208820 0x560db5219a20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560db5209210 0x560db5217af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 --2- 172.21.15.60:0/1090653406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560db5203d90 0x560db52194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c11600640 0 client.26153 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 -- 172.21.15.60:0/1090653406 >> 172.21.15.60:0/1090653406 conn(0x560db520de80 msgr2=0x560db520a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 -- 172.21.15.60:0/1090653406 shutdown_connections 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 1 -- 172.21.15.60:0/1090653406 wait complete. 2024-02-26T10:48:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.140+0000 7f2c31a10740 20 client.26153 trim_cache size 0 max 16384 2024-02-26T10:48:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:48:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:48:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:48:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 -- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c0c0a1e20 msgr2=0x7f2c0c0a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c0c0a1e20 0x7f2c0c0a42e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x560db5209a70 tx=0x7f2c20009500 comp rx=0 tx=0).stop 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 -- 172.21.15.60:0/1533782545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c28101290 msgr2=0x7f2c28199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c28101290 0x7f2c28199700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009230 tx=0x7f2c1802c5d0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 -- 172.21.15.60:0/1533782545 shutdown_connections 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2c0c0a1e20 0x7f2c0c0a42e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c28102480 0x7f2c28199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c28102e40 0x7f2c281a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 --2- 172.21.15.60:0/1533782545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c28101290 0x7f2c28199700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 -- 172.21.15.60:0/1533782545 >> 172.21.15.60:0/1533782545 conn(0x7f2c280fc820 msgr2=0x7f2c28073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 -- 172.21.15.60:0/1533782545 shutdown_connections 2024-02-26T10:48:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:36.141+0000 7f2c2ea00640 1 -- 172.21.15.60:0/1533782545 wait complete. 2024-02-26T10:48:36.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:36 smithi113 ceph-mon[36126]: pgmap v504: 129 pgs: 129 active+clean; 101 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 542 op/s 2024-02-26T10:48:36.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:36 smithi060 ceph-mon[31539]: pgmap v504: 129 pgs: 129 active+clean; 101 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 542 op/s 2024-02-26T10:48:36.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:36 smithi100 ceph-mon[36051]: pgmap v504: 129 pgs: 129 active+clean; 101 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 542 op/s 2024-02-26T10:48:38.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:38 smithi113 ceph-mon[36126]: pgmap v505: 129 pgs: 129 active+clean; 101 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 481 op/s 2024-02-26T10:48:38.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:38 smithi060 ceph-mon[31539]: pgmap v505: 129 pgs: 129 active+clean; 101 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 481 op/s 2024-02-26T10:48:38.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:38 smithi100 ceph-mon[36051]: pgmap v505: 129 pgs: 129 active+clean; 101 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 481 op/s 2024-02-26T10:48:40.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:40 smithi113 ceph-mon[36126]: pgmap v506: 129 pgs: 129 active+clean; 89 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 597 op/s 2024-02-26T10:48:40.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:40 smithi060 ceph-mon[31539]: pgmap v506: 129 pgs: 129 active+clean; 89 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 597 op/s 2024-02-26T10:48:40.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:40 smithi100 ceph-mon[36051]: pgmap v506: 129 pgs: 129 active+clean; 89 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 597 op/s 2024-02-26T10:48:42.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:42 smithi113 ceph-mon[36126]: pgmap v507: 129 pgs: 129 active+clean; 86 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 631 op/s 2024-02-26T10:48:42.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:42 smithi060 ceph-mon[31539]: pgmap v507: 129 pgs: 129 active+clean; 86 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 631 op/s 2024-02-26T10:48:42.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:42 smithi100 ceph-mon[36051]: pgmap v507: 129 pgs: 129 active+clean; 86 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 631 op/s 2024-02-26T10:48:44.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:44 smithi113 ceph-mon[36126]: pgmap v508: 129 pgs: 129 active+clean; 86 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 589 op/s 2024-02-26T10:48:44.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:44 smithi060 ceph-mon[31539]: pgmap v508: 129 pgs: 129 active+clean; 86 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 589 op/s 2024-02-26T10:48:44.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:44 smithi100 ceph-mon[36051]: pgmap v508: 129 pgs: 129 active+clean; 86 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 589 op/s 2024-02-26T10:48:46.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:46 smithi113 ceph-mon[36126]: pgmap v509: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 712 op/s 2024-02-26T10:48:46.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:46 smithi060 ceph-mon[31539]: pgmap v509: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 712 op/s 2024-02-26T10:48:46.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:46 smithi100 ceph-mon[36051]: pgmap v509: 129 pgs: 129 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 712 op/s 2024-02-26T10:48:47.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:47 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:48:47.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:47 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:48:47.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:47 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:48:48.482 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:48 smithi113 ceph-mon[36126]: pgmap v510: 129 pgs: 129 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 576 op/s 2024-02-26T10:48:48.482 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:48:48.482 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:48:48.482 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:48:48.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:48 smithi060 ceph-mon[31539]: pgmap v510: 129 pgs: 129 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 576 op/s 2024-02-26T10:48:48.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:48:48.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:48:48.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:48:48.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:48 smithi100 ceph-mon[36051]: pgmap v510: 129 pgs: 129 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 576 op/s 2024-02-26T10:48:48.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:48:48.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:48:48.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:48:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.290+0000 7f942be00640 1 -- 172.21.15.60:0/3013804393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101b30 msgr2=0x7f942c103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.290+0000 7f942be00640 1 --2- 172.21.15.60:0/3013804393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101b30 0x7f942c103f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f941c009ca0 tx=0x7f941c02c100 comp rx=0 tx=0).stop 2024-02-26T10:48:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.291+0000 7f942be00640 1 -- 172.21.15.60:0/3013804393 shutdown_connections 2024-02-26T10:48:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.291+0000 7f942be00640 1 --2- 172.21.15.60:0/3013804393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c106d90 0x7f942c1091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.291+0000 7f942be00640 1 --2- 172.21.15.60:0/3013804393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c104460 0x7f942c106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.291+0000 7f942be00640 1 --2- 172.21.15.60:0/3013804393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101b30 0x7f942c103f20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.291+0000 7f942be00640 1 -- 172.21.15.60:0/3013804393 >> 172.21.15.60:0/3013804393 conn(0x7f942c0fb540 msgr2=0x7f942c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.291+0000 7f942be00640 1 -- 172.21.15.60:0/3013804393 shutdown_connections 2024-02-26T10:48:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.291+0000 7f942be00640 1 -- 172.21.15.60:0/3013804393 wait complete. 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942be00640 1 Processor -- start 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942be00640 1 -- start start 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c101b30 0x7f942c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c104460 0x7f942c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c106d90 0x7f942c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942be00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c10cd40 con 0x7f942c106d90 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c10cbc0 con 0x7f942c104460 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c10cec0 con 0x7f942c101b30 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c101b30 0x7f942c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f9422000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c104460 0x7f942c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c101b30 0x7f942c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:55778/0 (socket says 172.21.15.60:55778) 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942aa00640 1 -- 172.21.15.60:0/3648209525 learned_addr learned my addr 172.21.15.60:0/3648209525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f9422000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c104460 0x7f942c199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:50556/0 (socket says 172.21.15.60:50556) 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.292+0000 7f942b400640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c106d90 0x7f942c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.293+0000 7f942aa00640 1 -- 172.21.15.60:0/3648209525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c104460 msgr2=0x7f942c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.293+0000 7f942aa00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c104460 0x7f942c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.293+0000 7f942aa00640 1 -- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c106d90 msgr2=0x7f942c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.293+0000 7f942aa00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c106d90 0x7f942c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.293+0000 7f942aa00640 1 -- 172.21.15.60:0/3648209525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c1a1210 con 0x7f942c101b30 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.293+0000 7f942aa00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c101b30 0x7f942c1995f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f941c0325f0 tx=0x7f941c035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.293+0000 7f9423e00640 1 -- 172.21.15.60:0/3648209525 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c039040 con 0x7f942c101b30 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.293+0000 7f9423e00640 1 -- 172.21.15.60:0/3648209525 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f941c044070 con 0x7f942c101b30 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.293+0000 7f9423e00640 1 -- 172.21.15.60:0/3648209525 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c004610 con 0x7f942c101b30 2024-02-26T10:48:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.294+0000 7f942be00640 1 -- 172.21.15.60:0/3648209525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c1a14a0 con 0x7f942c101b30 2024-02-26T10:48:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.294+0000 7f942be00640 1 -- 172.21.15.60:0/3648209525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c1a19b0 con 0x7f942c101b30 2024-02-26T10:48:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.297+0000 7f9423e00640 1 -- 172.21.15.60:0/3648209525 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f941c003970 con 0x7f942c101b30 2024-02-26T10:48:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.297+0000 7f9434a5b740 1 Processor -- start 2024-02-26T10:48:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.297+0000 7f9434a5b740 1 -- start start 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f585fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5864ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f58679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9434a5b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f57c5340 con 0x5581f585f9e0 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9434a5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f576e520 con 0x5581f5865540 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9434a5b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f57c13e0 con 0x5581f58603d0 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f942aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5864ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9422000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f58679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9422000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f58679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:50572/0 (socket says 172.21.15.60:50572) 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9422000640 1 -- 172.21.15.60:0/3208639110 learned_addr learned my addr 172.21.15.60:0/3208639110 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f942b400640 1 --2- 172.21.15.60:0/3208639110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f585fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9422000640 1 -- 172.21.15.60:0/3208639110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 msgr2=0x5581f5864ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9422000640 1 --2- 172.21.15.60:0/3208639110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5864ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9422000640 1 -- 172.21.15.60:0/3208639110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 msgr2=0x5581f585fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9422000640 1 --2- 172.21.15.60:0/3208639110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f585fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f9422000640 1 -- 172.21.15.60:0/3208639110 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581f5782390 con 0x5581f5865540 2024-02-26T10:48:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.298+0000 7f942aa00640 1 --2- 172.21.15.60:0/3208639110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5864ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.299+0000 7f9422000640 1 --2- 172.21.15.60:0/3208639110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f58679d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f941400bcd0 tx=0x7f94140107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=935b735d647a9439 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.299+0000 7f941be00640 1 -- 172.21.15.60:0/3208639110 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941401c680 con 0x5581f5865540 2024-02-26T10:48:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.299+0000 7f941be00640 1 -- 172.21.15.60:0/3208639110 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f941401cd00 con 0x5581f5865540 2024-02-26T10:48:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.299+0000 7f941be00640 1 -- 172.21.15.60:0/3208639110 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414011d40 con 0x5581f5865540 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.299+0000 7f9423e00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94100a1e70 0x7f94100a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.299+0000 7f9423e00640 1 -- 172.21.15.60:0/3648209525 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f941c0f07d0 con 0x7f942c101b30 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.299+0000 7f942b400640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94100a1e70 0x7f94100a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f9434a5b740 1 -- 172.21.15.60:0/3208639110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 msgr2=0x5581f58679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3208639110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f58679d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f941400bcd0 tx=0x7f94140107a0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f9434a5b740 1 -- 172.21.15.60:0/3208639110 shutdown_connections 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3208639110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f58679d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3208639110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5864ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3208639110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f585fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f9434a5b740 1 -- 172.21.15.60:0/3208639110 >> 172.21.15.60:0/3208639110 conn(0x5581f585af50 msgr2=0x5581f585b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f9434a5b740 1 -- 172.21.15.60:0/3208639110 shutdown_connections 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f9434a5b740 1 -- 172.21.15.60:0/3208639110 wait complete. 2024-02-26T10:48:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.300+0000 7f942b400640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94100a1e70 0x7f94100a4330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5581f5860c30 tx=0x7f942400b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.301+0000 7f9434a5b740 1 -- start start 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.301+0000 7f9434a5b740 1 Processor -- start 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.301+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f5869ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.301+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f58603d0 0x5581f586a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.301+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f5865540 0x5581f5871260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.301+0000 7f9434a5b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f58684f0 con 0x5581f585f9e0 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.301+0000 7f9434a5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f5781980 con 0x5581f58603d0 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.301+0000 7f9434a5b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f5868670 con 0x5581f5865540 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.301+0000 7f9434a5b740 10 client.0 _ll_register_callbacks cb 0x5581f57bd310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f942aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f5869ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9422000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f58603d0 0x5581f586a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9422000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f58603d0 0x5581f586a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:50582/0 (socket says 172.21.15.60:50582) 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9422000640 1 -- 172.21.15.60:0/3212276470 learned_addr learned my addr 172.21.15.60:0/3212276470 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f942b400640 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f5865540 0x5581f5871260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9422000640 1 -- 172.21.15.60:0/3212276470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f5865540 msgr2=0x5581f5871260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9422000640 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f5865540 0x5581f5871260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9422000640 1 -- 172.21.15.60:0/3212276470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 msgr2=0x5581f5869ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9422000640 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f5869ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9422000640 1 -- 172.21.15.60:0/3212276470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581f54da210 con 0x5581f58603d0 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f942aa00640 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f5869ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9422000640 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f58603d0 0x5581f586a220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9414010ca0 tx=0x7f9414011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03ca327790721e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.302+0000 7f9418c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.303+0000 7f9418c00640 1 -- 172.21.15.60:0/3212276470 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941402a060 con 0x5581f58603d0 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.303+0000 7f9418c00640 1 -- 172.21.15.60:0/3212276470 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f9414005440 con 0x5581f58603d0 2024-02-26T10:48:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.303+0000 7f9434a5b740 1 -- 172.21.15.60:0/3212276470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f941401cd00 con 0x5581f58603d0 2024-02-26T10:48:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.303+0000 7f9418c00640 1 -- 172.21.15.60:0/3212276470 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9414031050 con 0x5581f58603d0 2024-02-26T10:48:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.303+0000 7f9434a5b740 10 client.26165 fetch_fsmap learned FSMap version 30 2024-02-26T10:48:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.303+0000 7f9434a5b740 1 -- 172.21.15.60:0/3212276470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5581f55a6620 con 0x5581f58603d0 2024-02-26T10:48:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.303+0000 7f9418c00640 1 -- 172.21.15.60:0/3212276470 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f941401cd00 con 0x5581f58603d0 2024-02-26T10:48:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f9418c00640 1 -- 172.21.15.60:0/3212276470 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f9414024030 con 0x5581f58603d0 2024-02-26T10:48:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f9434a5b740 10 client.26165 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:48:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f9434a5b740 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f9434a5b740 20 client.26165 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5581f58757b0 0x5581f5877ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f9434a5b740 4 client.26165 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f9434a5b740 1 -- 172.21.15.60:0/3212276470 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581f57d2880 con 0x5581f58757b0 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f942aa00640 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5581f58757b0 0x5581f5877ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f942aa00640 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5581f58757b0 0x5581f5877ba0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b159a6ce244864a7 server_cookie=44661b0c055c423a in_seq=0 out_seq=0 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.304+0000 7f9418c00640 10 client.26165 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9418c00640 1 -- 172.21.15.60:0/3212276470 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581f57d2880 con 0x5581f58757b0 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9418c00640 10 client.26165 handle_command_reply: tid=0 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 client.26165 shutdown 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 -- 172.21.15.60:0/3212276470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f58603d0 msgr2=0x5581f586a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f58603d0 0x5581f586a220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9414010ca0 tx=0x7f9414011c70 comp rx=0 tx=0).stop 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 -- 172.21.15.60:0/3212276470 shutdown_connections 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5581f58757b0 0x5581f5877ba0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f58603d0 0x5581f586a220 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f5865540 0x5581f5871260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3212276470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f5869ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9418c00640 0 client.26165 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 -- 172.21.15.60:0/3212276470 >> 172.21.15.60:0/3212276470 conn(0x5581f585af50 msgr2=0x5581f585a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.305+0000 7f9434a5b740 1 -- 172.21.15.60:0/3212276470 shutdown_connections 2024-02-26T10:48:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.306+0000 7f9434a5b740 1 -- 172.21.15.60:0/3212276470 wait complete. 2024-02-26T10:48:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.306+0000 7f9434a5b740 20 client.26165 trim_cache size 0 max 16384 2024-02-26T10:48:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9434a5b740 1 Processor -- start 2024-02-26T10:48:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9434a5b740 1 -- start start 2024-02-26T10:48:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f585f9e0 0x5581f585fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5871220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f5865540 0x5581f5869ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9434a5b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1ae3c0 con 0x5581f5865540 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9434a5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1abea0 con 0x5581f585f9e0 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9434a5b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1a9f40 con 0x5581f58603d0 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f942aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f585f9e0 0x5581f585fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9422000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5871220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f942aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f585f9e0 0x5581f585fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:50588/0 (socket says 172.21.15.60:50588) 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f942aa00640 1 -- 172.21.15.60:0/3779129174 learned_addr learned my addr 172.21.15.60:0/3779129174 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f942b400640 1 --2- 172.21.15.60:0/3779129174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f5865540 0x5581f5869ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9422000640 1 -- 172.21.15.60:0/3779129174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f585f9e0 msgr2=0x5581f585fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9422000640 1 --2- 172.21.15.60:0/3779129174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f585f9e0 0x5581f585fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9422000640 1 -- 172.21.15.60:0/3779129174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f5865540 msgr2=0x5581f5869ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9422000640 1 --2- 172.21.15.60:0/3779129174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f5865540 0x5581f5869ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9422000640 1 -- 172.21.15.60:0/3779129174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581f585bcc0 con 0x5581f58603d0 2024-02-26T10:48:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f942aa00640 1 --2- 172.21.15.60:0/3779129174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f585f9e0 0x5581f585fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.321+0000 7f9422000640 1 --2- 172.21.15.60:0/3779129174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5871220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9414001eb0 tx=0x7f94140107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48740486cfdb734b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f941b400640 1 -- 172.21.15.60:0/3779129174 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941402a030 con 0x5581f58603d0 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f941b400640 1 -- 172.21.15.60:0/3779129174 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f941400b8c0 con 0x5581f58603d0 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f941b400640 1 -- 172.21.15.60:0/3779129174 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414011860 con 0x5581f58603d0 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f9434a5b740 1 -- 172.21.15.60:0/3779129174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 msgr2=0x5581f5871220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3779129174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5871220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9414001eb0 tx=0x7f94140107a0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f9434a5b740 1 -- 172.21.15.60:0/3779129174 shutdown_connections 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3779129174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f5865540 0x5581f5869ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3779129174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f5871220 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f9434a5b740 1 --2- 172.21.15.60:0/3779129174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f585f9e0 0x5581f585fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f9434a5b740 1 -- 172.21.15.60:0/3779129174 >> 172.21.15.60:0/3779129174 conn(0x5581f585af50 msgr2=0x5581f587c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f9434a5b740 1 -- 172.21.15.60:0/3779129174 shutdown_connections 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.322+0000 7f9434a5b740 1 -- 172.21.15.60:0/3779129174 wait complete. 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.323+0000 7f9434a5b740 1 -- start start 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.323+0000 7f9434a5b740 1 Processor -- start 2024-02-26T10:48:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.323+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f586f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.323+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f586f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.323+0000 7f9434a5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f5870600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.323+0000 7f9434a5b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1aa540 con 0x5581f585f9e0 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.323+0000 7f9434a5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1ab840 con 0x5581f5865540 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.323+0000 7f9434a5b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1aab80 con 0x5581f58603d0 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.323+0000 7f9434a5b740 10 client.0 _ll_register_callbacks cb 0x7f941400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f586f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f9422000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f586f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f5870600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f9422000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f586f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:55820/0 (socket says 172.21.15.60:55820) 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f9422000640 1 -- 172.21.15.60:0/1820384385 learned_addr learned my addr 172.21.15.60:0/1820384385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f586f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33444/0 (socket says 172.21.15.60:33444) 2024-02-26T10:48:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942b400640 1 -- 172.21.15.60:0/1820384385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 msgr2=0x5581f586f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942b400640 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f586f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942b400640 1 -- 172.21.15.60:0/1820384385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 msgr2=0x5581f586f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942b400640 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f586f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942b400640 1 -- 172.21.15.60:0/1820384385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581f5870bf0 con 0x5581f5865540 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942aa00640 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f586f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f9422000640 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f586f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f942b400640 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f5870600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9424004520 tx=0x7f94240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d456d4964fe5e14d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.324+0000 7f9418c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.325+0000 7f9418c00640 1 -- 172.21.15.60:0/1820384385 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942401e930 con 0x5581f5865540 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.325+0000 7f9434a5b740 1 -- 172.21.15.60:0/1820384385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f941400b8c0 con 0x5581f5865540 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.325+0000 7f9418c00640 1 -- 172.21.15.60:0/1820384385 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f942401ead0 con 0x5581f5865540 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.325+0000 7f9418c00640 1 -- 172.21.15.60:0/1820384385 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9424034050 con 0x5581f5865540 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.325+0000 7f9434a5b740 10 client.26171 fetch_fsmap learned FSMap version 30 2024-02-26T10:48:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.325+0000 7f9434a5b740 1 -- 172.21.15.60:0/1820384385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5581f58cc2e0 con 0x5581f5865540 2024-02-26T10:48:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.325+0000 7f9418c00640 1 -- 172.21.15.60:0/1820384385 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f941400b8c0 con 0x5581f5865540 2024-02-26T10:48:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.325+0000 7f9418c00640 1 -- 172.21.15.60:0/1820384385 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f9424004b10 con 0x5581f5865540 2024-02-26T10:48:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.326+0000 7f9434a5b740 10 client.26171 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:48:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.326+0000 7f9434a5b740 10 client.26171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:48:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.326+0000 7f9434a5b740 20 client.26171 populate_metadata read hostname 'smithi060' 2024-02-26T10:48:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.326+0000 7f9434a5b740 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5581f58ccce0 0x5581f58cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:48:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.326+0000 7f9434a5b740 4 client.26171 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:48:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.326+0000 7f9434a5b740 1 -- 172.21.15.60:0/1820384385 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5581f57ccbd0 con 0x5581f58ccce0 2024-02-26T10:48:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.326+0000 7f942aa00640 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5581f58ccce0 0x5581f58cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:48:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.326+0000 7f942aa00640 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5581f58ccce0 0x5581f58cf100 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94575d5e160a7ae0 server_cookie=5f6b117ddee7f0e5 in_seq=0 out_seq=0 2024-02-26T10:48:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.326+0000 7f9418c00640 10 client.26171 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9418c00640 1 -- 172.21.15.60:0/1820384385 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5581f57ccbd0 con 0x5581f58ccce0 2024-02-26T10:48:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9418c00640 10 client.26171 handle_command_reply: tid=0 2024-02-26T10:48:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9434a5b740 1 client.26171 shutdown 2024-02-26T10:48:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9434a5b740 1 -- 172.21.15.60:0/1820384385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 msgr2=0x5581f5870600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9434a5b740 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f5870600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9424004520 tx=0x7f94240046c0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9434a5b740 1 -- 172.21.15.60:0/1820384385 shutdown_connections 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9434a5b740 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x5581f58ccce0 0x5581f58cf100 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9434a5b740 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581f5865540 0x5581f5870600 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9434a5b740 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5581f58603d0 0x5581f586f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.467+0000 7f9434a5b740 1 --2- 172.21.15.60:0/1820384385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581f585f9e0 0x5581f586f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.468+0000 7f9434a5b740 1 -- 172.21.15.60:0/1820384385 >> 172.21.15.60:0/1820384385 conn(0x5581f585af50 msgr2=0x5581f587c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.468+0000 7f9418c00640 0 client.26171 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.468+0000 7f9434a5b740 1 -- 172.21.15.60:0/1820384385 shutdown_connections 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.468+0000 7f9434a5b740 1 -- 172.21.15.60:0/1820384385 wait complete. 2024-02-26T10:48:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.468+0000 7f9434a5b740 20 client.26171 trim_cache size 0 max 16384 2024-02-26T10:48:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:48:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:48:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:48:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.469+0000 7f942be00640 1 -- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94100a1e70 msgr2=0x7f94100a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.469+0000 7f942be00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94100a1e70 0x7f94100a4330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5581f5860c30 tx=0x7f942400b2e0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.469+0000 7f942be00640 1 -- 172.21.15.60:0/3648209525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c101b30 msgr2=0x7f942c1995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.469+0000 7f942be00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c101b30 0x7f942c1995f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f941c0325f0 tx=0x7f941c035d10 comp rx=0 tx=0).stop 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.469+0000 7f942be00640 1 -- 172.21.15.60:0/3648209525 shutdown_connections 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.469+0000 7f942be00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f94100a1e70 0x7f94100a4330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.469+0000 7f942be00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c104460 0x7f942c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.469+0000 7f942be00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c106d90 0x7f942c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.469+0000 7f942be00640 1 --2- 172.21.15.60:0/3648209525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f942c101b30 0x7f942c1995f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.470+0000 7f942be00640 1 -- 172.21.15.60:0/3648209525 >> 172.21.15.60:0/3648209525 conn(0x7f942c0fb540 msgr2=0x7f942c101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:48:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.470+0000 7f942be00640 1 -- 172.21.15.60:0/3648209525 shutdown_connections 2024-02-26T10:48:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:48:49.470+0000 7f942be00640 1 -- 172.21.15.60:0/3648209525 wait complete. 2024-02-26T10:48:50.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:50 smithi113 ceph-mon[36126]: pgmap v511: 129 pgs: 129 active+clean; 83 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 642 op/s 2024-02-26T10:48:50.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:50 smithi060 ceph-mon[31539]: pgmap v511: 129 pgs: 129 active+clean; 83 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 642 op/s 2024-02-26T10:48:50.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:50 smithi100 ceph-mon[36051]: pgmap v511: 129 pgs: 129 active+clean; 83 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 642 op/s 2024-02-26T10:48:52.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:52 smithi113 ceph-mon[36126]: pgmap v512: 129 pgs: 129 active+clean; 79 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 628 op/s 2024-02-26T10:48:52.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:52 smithi060 ceph-mon[31539]: pgmap v512: 129 pgs: 129 active+clean; 79 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 628 op/s 2024-02-26T10:48:52.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:52 smithi100 ceph-mon[36051]: pgmap v512: 129 pgs: 129 active+clean; 79 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 628 op/s 2024-02-26T10:48:54.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:54 smithi113 ceph-mon[36126]: pgmap v513: 129 pgs: 129 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 546 op/s 2024-02-26T10:48:54.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:54 smithi060 ceph-mon[31539]: pgmap v513: 129 pgs: 129 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 546 op/s 2024-02-26T10:48:54.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:54 smithi100 ceph-mon[36051]: pgmap v513: 129 pgs: 129 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 546 op/s 2024-02-26T10:48:56.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:56 smithi113 ceph-mon[36126]: pgmap v514: 129 pgs: 129 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 684 op/s 2024-02-26T10:48:56.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:56 smithi060 ceph-mon[31539]: pgmap v514: 129 pgs: 129 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 684 op/s 2024-02-26T10:48:56.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:56 smithi100 ceph-mon[36051]: pgmap v514: 129 pgs: 129 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 684 op/s 2024-02-26T10:48:58.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:48:58 smithi113 ceph-mon[36126]: pgmap v515: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 527 op/s 2024-02-26T10:48:58.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:48:58 smithi060 ceph-mon[31539]: pgmap v515: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 527 op/s 2024-02-26T10:48:58.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:48:58 smithi100 ceph-mon[36051]: pgmap v515: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 527 op/s 2024-02-26T10:49:00.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:00 smithi113 ceph-mon[36126]: pgmap v516: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 601 op/s 2024-02-26T10:49:00.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:00 smithi060 ceph-mon[31539]: pgmap v516: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 601 op/s 2024-02-26T10:49:00.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:00 smithi100 ceph-mon[36051]: pgmap v516: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 601 op/s 2024-02-26T10:49:02.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:02 smithi113 ceph-mon[36126]: pgmap v517: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 617 op/s 2024-02-26T10:49:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:02 smithi060 ceph-mon[31539]: pgmap v517: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 617 op/s 2024-02-26T10:49:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.628+0000 7fac2e600640 1 -- 172.21.15.60:0/2824317363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac28101b60 msgr2=0x7fac281083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.628+0000 7fac2e600640 1 --2- 172.21.15.60:0/2824317363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac28101b60 0x7fac281083f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac20009ca0 tx=0x7fac2002c0f0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.628+0000 7fac2e600640 1 -- 172.21.15.60:0/2824317363 shutdown_connections 2024-02-26T10:49:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.628+0000 7fac2e600640 1 --2- 172.21.15.60:0/2824317363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac28101b60 0x7fac281083f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.628+0000 7fac2e600640 1 --2- 172.21.15.60:0/2824317363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac281011a0 0x7fac28101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.628+0000 7fac2e600640 1 --2- 172.21.15.60:0/2824317363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280fffb0 0x7fac281003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.628+0000 7fac2e600640 1 -- 172.21.15.60:0/2824317363 >> 172.21.15.60:0/2824317363 conn(0x7fac280fb540 msgr2=0x7fac280fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.628+0000 7fac2e600640 1 -- 172.21.15.60:0/2824317363 shutdown_connections 2024-02-26T10:49:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.628+0000 7fac2e600640 1 -- 172.21.15.60:0/2824317363 wait complete. 2024-02-26T10:49:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.629+0000 7fac2e600640 1 Processor -- start 2024-02-26T10:49:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.629+0000 7fac2e600640 1 -- start start 2024-02-26T10:49:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.629+0000 7fac2e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac280fffb0 0x7fac28199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.629+0000 7fac2e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281011a0 0x7fac28199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac2e600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac28101b60 0x7fac281a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac2e600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2810bc40 con 0x7fac28101b60 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac2e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2810bac0 con 0x7fac281011a0 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac2e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2810bdc0 con 0x7fac280fffb0 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281011a0 0x7fac28199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281011a0 0x7fac28199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40754/0 (socket says 172.21.15.60:40754) 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac1fe00640 1 -- 172.21.15.60:0/3161454799 learned_addr learned my addr 172.21.15.60:0/3161454799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac2d200640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac280fffb0 0x7fac28199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac2dc00640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac28101b60 0x7fac281a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac1fe00640 1 -- 172.21.15.60:0/3161454799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac280fffb0 msgr2=0x7fac28199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac1fe00640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac280fffb0 0x7fac28199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac1fe00640 1 -- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac28101b60 msgr2=0x7fac281a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac1fe00640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac28101b60 0x7fac281a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac1fe00640 1 -- 172.21.15.60:0/3161454799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac281a1160 con 0x7fac281011a0 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac2dc00640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac28101b60 0x7fac281a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.630+0000 7fac2d200640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac280fffb0 0x7fac28199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.631+0000 7fac1fe00640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281011a0 0x7fac28199a80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d950 tx=0x7fac1000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.631+0000 7fac26000640 1 -- 172.21.15.60:0/3161454799 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10017030 con 0x7fac281011a0 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.631+0000 7fac2e600640 1 -- 172.21.15.60:0/3161454799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac281a1450 con 0x7fac281011a0 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.631+0000 7fac26000640 1 -- 172.21.15.60:0/3161454799 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac10004020 con 0x7fac281011a0 2024-02-26T10:49:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.631+0000 7fac26000640 1 -- 172.21.15.60:0/3161454799 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100047a0 con 0x7fac281011a0 2024-02-26T10:49:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.631+0000 7fac2e600640 1 -- 172.21.15.60:0/3161454799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac281a19a0 con 0x7fac281011a0 2024-02-26T10:49:02.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:02 smithi100 ceph-mon[36051]: pgmap v517: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 617 op/s 2024-02-26T10:49:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.634+0000 7fac32c37740 1 Processor -- start 2024-02-26T10:49:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.634+0000 7fac26000640 1 -- 172.21.15.60:0/3161454799 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fac100041d0 con 0x7fac281011a0 2024-02-26T10:49:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.634+0000 7fac32c37740 1 -- start start 2024-02-26T10:49:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac32c37740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae5fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac32c37740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae603d0 0x55ddfae64ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac32c37740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae65540 0x55ddfae679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac32c37740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddfadc5340 con 0x55ddfae5f9e0 2024-02-26T10:49:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac32c37740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddfad6e520 con 0x55ddfae603d0 2024-02-26T10:49:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac32c37740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddfadc13e0 con 0x55ddfae65540 2024-02-26T10:49:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac2dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae5fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac2d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae603d0 0x55ddfae64ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac2d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae603d0 0x55ddfae64ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40762/0 (socket says 172.21.15.60:40762) 2024-02-26T10:49:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac2d200640 1 -- 172.21.15.60:0/1790952870 learned_addr learned my addr 172.21.15.60:0/1790952870 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac1fe00640 1 --2- 172.21.15.60:0/1790952870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae65540 0x55ddfae679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac2d200640 1 -- 172.21.15.60:0/1790952870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae65540 msgr2=0x55ddfae679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac2d200640 1 --2- 172.21.15.60:0/1790952870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae65540 0x55ddfae679d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac2d200640 1 -- 172.21.15.60:0/1790952870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 msgr2=0x55ddfae5fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac2d200640 1 --2- 172.21.15.60:0/1790952870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae5fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.635+0000 7fac2d200640 1 -- 172.21.15.60:0/1790952870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddfad82390 con 0x55ddfae603d0 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.636+0000 7fac2d200640 1 --2- 172.21.15.60:0/1790952870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae603d0 0x55ddfae64ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fac180114c0 tx=0x7fac18011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a8dc617599471 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.636+0000 7fac1e000640 1 -- 172.21.15.60:0/1790952870 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18004020 con 0x55ddfae603d0 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.636+0000 7fac1e000640 1 -- 172.21.15.60:0/1790952870 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18005500 con 0x55ddfae603d0 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.636+0000 7fac1e000640 1 -- 172.21.15.60:0/1790952870 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18004e10 con 0x55ddfae603d0 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.636+0000 7fac26000640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac0c0a1d70 0x7fac0c0a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.636+0000 7fac26000640 1 -- 172.21.15.60:0/3161454799 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fac100cdaf0 con 0x7fac281011a0 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.636+0000 7fac2dc00640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac0c0a1d70 0x7fac0c0a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 -- 172.21.15.60:0/1790952870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae603d0 msgr2=0x55ddfae64ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 --2- 172.21.15.60:0/1790952870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae603d0 0x55ddfae64ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fac180114c0 tx=0x7fac18011980 comp rx=0 tx=0).stop 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 -- 172.21.15.60:0/1790952870 shutdown_connections 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 --2- 172.21.15.60:0/1790952870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae603d0 0x55ddfae64ed0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 --2- 172.21.15.60:0/1790952870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae65540 0x55ddfae679d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 --2- 172.21.15.60:0/1790952870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae5fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 -- 172.21.15.60:0/1790952870 >> 172.21.15.60:0/1790952870 conn(0x55ddfae5af50 msgr2=0x55ddfae5b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 -- 172.21.15.60:0/1790952870 shutdown_connections 2024-02-26T10:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 -- 172.21.15.60:0/1790952870 wait complete. 2024-02-26T10:49:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac2dc00640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac0c0a1d70 0x7fac0c0a4230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fac2002c630 tx=0x7fac20032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.637+0000 7fac32c37740 1 -- start start 2024-02-26T10:49:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac32c37740 1 Processor -- start 2024-02-26T10:49:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac32c37740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae5f9e0 0x55ddfae69d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac32c37740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac32c37740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae65540 0x55ddfae71260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac32c37740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddfae684f0 con 0x55ddfae65540 2024-02-26T10:49:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac32c37740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddfad81980 con 0x55ddfae5f9e0 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac32c37740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddfae68670 con 0x55ddfae603d0 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac32c37740 10 client.0 _ll_register_callbacks cb 0x55ddfadbd310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac2d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae5f9e0 0x55ddfae69d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac2d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae5f9e0 0x55ddfae69d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40774/0 (socket says 172.21.15.60:40774) 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac2d200640 1 -- 172.21.15.60:0/3472189306 learned_addr learned my addr 172.21.15.60:0/3472189306 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac1fe00640 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.638+0000 7fac2dc00640 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae65540 0x55ddfae71260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.639+0000 7fac2d200640 1 -- 172.21.15.60:0/3472189306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 msgr2=0x55ddfae6a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.639+0000 7fac2d200640 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.639+0000 7fac2d200640 1 -- 172.21.15.60:0/3472189306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae65540 msgr2=0x55ddfae71260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.639+0000 7fac2d200640 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae65540 0x55ddfae71260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.639+0000 7fac2d200640 1 -- 172.21.15.60:0/3472189306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddfaada210 con 0x55ddfae5f9e0 2024-02-26T10:49:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.639+0000 7fac1fe00640 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.639+0000 7fac2d200640 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae5f9e0 0x55ddfae69d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac18011e80 tx=0x7fac18005290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d06557aa19554e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.639+0000 7fac16a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.639+0000 7fac16a00640 1 -- 172.21.15.60:0/3472189306 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18028030 con 0x55ddfae5f9e0 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.640+0000 7fac32c37740 1 -- 172.21.15.60:0/3472189306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac18005500 con 0x55ddfae5f9e0 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.640+0000 7fac16a00640 1 -- 172.21.15.60:0/3472189306 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fac18016040 con 0x55ddfae5f9e0 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.640+0000 7fac16a00640 1 -- 172.21.15.60:0/3472189306 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac18032050 con 0x55ddfae5f9e0 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.640+0000 7fac32c37740 10 client.26183 fetch_fsmap learned FSMap version 30 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.640+0000 7fac32c37740 1 -- 172.21.15.60:0/3472189306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ddfaba6620 con 0x55ddfae5f9e0 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.640+0000 7fac16a00640 1 -- 172.21.15.60:0/3472189306 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18005500 con 0x55ddfae5f9e0 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.640+0000 7fac16a00640 1 -- 172.21.15.60:0/3472189306 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fac18023470 con 0x55ddfae5f9e0 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.640+0000 7fac32c37740 10 client.26183 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.640+0000 7fac32c37740 10 client.26183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.641+0000 7fac32c37740 20 client.26183 populate_metadata read hostname 'smithi060' 2024-02-26T10:49:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.641+0000 7fac32c37740 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ddfae75900 0x55ddfae77d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.641+0000 7fac32c37740 4 client.26183 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.641+0000 7fac32c37740 1 -- 172.21.15.60:0/3472189306 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddfadd2880 con 0x55ddfae75900 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.641+0000 7fac1fe00640 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ddfae75900 0x55ddfae77d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.641+0000 7fac1fe00640 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ddfae75900 0x55ddfae77d10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e3e95cfc4d5e46 server_cookie=1c1d0e8c7ed8789a in_seq=0 out_seq=0 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.641+0000 7fac16a00640 10 client.26183 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.642+0000 7fac16a00640 1 -- 172.21.15.60:0/3472189306 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ddfadd2880 con 0x55ddfae75900 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.642+0000 7fac16a00640 10 client.26183 handle_command_reply: tid=0 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.642+0000 7fac32c37740 1 client.26183 shutdown 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 -- 172.21.15.60:0/3472189306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae5f9e0 msgr2=0x55ddfae69d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae5f9e0 0x55ddfae69d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac18011e80 tx=0x7fac18005290 comp rx=0 tx=0).stop 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 -- 172.21.15.60:0/3472189306 shutdown_connections 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ddfae75900 0x55ddfae77d10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae65540 0x55ddfae71260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 --2- 172.21.15.60:0/3472189306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae5f9e0 0x55ddfae69d50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac16a00640 0 client.26183 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:49:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 -- 172.21.15.60:0/3472189306 >> 172.21.15.60:0/3472189306 conn(0x55ddfae5af50 msgr2=0x55ddfae5a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 -- 172.21.15.60:0/3472189306 shutdown_connections 2024-02-26T10:49:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 1 -- 172.21.15.60:0/3472189306 wait complete. 2024-02-26T10:49:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.643+0000 7fac32c37740 20 client.26183 trim_cache size 0 max 16384 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.661+0000 7fac32c37740 1 Processor -- start 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.661+0000 7fac32c37740 1 -- start start 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.661+0000 7fac32c37740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.661+0000 7fac32c37740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae61cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.661+0000 7fac32c37740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 0x55ddfae71220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.661+0000 7fac32c37740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281ae400 con 0x55ddfae5f9e0 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.661+0000 7fac32c37740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281abee0 con 0x55ddfae65540 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.661+0000 7fac32c37740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281a9f80 con 0x55ddfae603d0 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae61cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae61cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:58182/0 (socket says 172.21.15.60:58182) 2024-02-26T10:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac1fe00640 1 -- 172.21.15.60:0/2036241816 learned_addr learned my addr 172.21.15.60:0/2036241816 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac2d200640 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac2dc00640 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 0x55ddfae71220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac1fe00640 1 -- 172.21.15.60:0/2036241816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 msgr2=0x55ddfae71220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:49:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac1fe00640 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 0x55ddfae71220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac1fe00640 1 -- 172.21.15.60:0/2036241816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 msgr2=0x55ddfae617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:49:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac1fe00640 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae617b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac1fe00640 1 -- 172.21.15.60:0/2036241816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddfae5bcc0 con 0x55ddfae603d0 2024-02-26T10:49:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac2dc00640 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 0x55ddfae71220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:49:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.662+0000 7fac1fe00640 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae61cf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac100cb0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b31083db364dbf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac1d600640 1 -- 172.21.15.60:0/2036241816 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10099f90 con 0x55ddfae603d0 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac1d600640 1 -- 172.21.15.60:0/2036241816 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac1009ab50 con 0x55ddfae603d0 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac1d600640 1 -- 172.21.15.60:0/2036241816 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10099db0 con 0x55ddfae603d0 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac32c37740 1 -- 172.21.15.60:0/2036241816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 msgr2=0x55ddfae61cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac32c37740 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae61cf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac100cb0f0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac32c37740 1 -- 172.21.15.60:0/2036241816 shutdown_connections 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac32c37740 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 0x55ddfae71220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac32c37740 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae61cf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac32c37740 1 --2- 172.21.15.60:0/2036241816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac32c37740 1 -- 172.21.15.60:0/2036241816 >> 172.21.15.60:0/2036241816 conn(0x55ddfae5af50 msgr2=0x55ddfae5b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.663+0000 7fac32c37740 1 -- 172.21.15.60:0/2036241816 shutdown_connections 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.664+0000 7fac32c37740 1 -- 172.21.15.60:0/2036241816 wait complete. 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.664+0000 7fac32c37740 1 -- start start 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.664+0000 7fac32c37740 1 Processor -- start 2024-02-26T10:49:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac32c37740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae6f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac32c37740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac32c37740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 0x55ddfae705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac32c37740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281aa580 con 0x55ddfae5f9e0 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac32c37740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281ab880 con 0x55ddfae65540 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac32c37740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281aabc0 con 0x55ddfae603d0 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac32c37740 10 client.0 _ll_register_callbacks cb 0x7fac18009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac2d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae6f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac2d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae6f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44432/0 (socket says 172.21.15.60:44432) 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac2dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 0x55ddfae705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac2d200640 1 -- 172.21.15.60:0/4154240379 learned_addr learned my addr 172.21.15.60:0/4154240379 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac2d200640 1 -- 172.21.15.60:0/4154240379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 msgr2=0x55ddfae6f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:49:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac2d200640 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac2d200640 1 -- 172.21.15.60:0/4154240379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 msgr2=0x55ddfae705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:49:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac2d200640 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 0x55ddfae705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac2d200640 1 -- 172.21.15.60:0/4154240379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddfae70bb0 con 0x55ddfae5f9e0 2024-02-26T10:49:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.665+0000 7fac1fe00640 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.666+0000 7fac2d200640 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae6f070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fac18023840 tx=0x7fac18005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe92467470747de server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.666+0000 7fac16a00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.666+0000 7fac16a00640 1 -- 172.21.15.60:0/4154240379 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1803b060 con 0x55ddfae5f9e0 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.666+0000 7fac16a00640 1 -- 172.21.15.60:0/4154240379 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fac18023bb0 con 0x55ddfae5f9e0 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.666+0000 7fac32c37740 1 -- 172.21.15.60:0/4154240379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac1009ab50 con 0x55ddfae5f9e0 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.666+0000 7fac16a00640 1 -- 172.21.15.60:0/4154240379 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac18042050 con 0x55ddfae5f9e0 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.666+0000 7fac32c37740 10 client.16050 fetch_fsmap learned FSMap version 30 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.666+0000 7fac32c37740 1 -- 172.21.15.60:0/4154240379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ddfaecc490 con 0x55ddfae5f9e0 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.666+0000 7fac16a00640 1 -- 172.21.15.60:0/4154240379 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac1009ab50 con 0x55ddfae5f9e0 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac16a00640 1 -- 172.21.15.60:0/4154240379 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fac18004680 con 0x55ddfae5f9e0 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac32c37740 10 client.16050 fetch_fsmap finished waiting for FSMap version 30 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac32c37740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac32c37740 20 client.16050 populate_metadata read hostname 'smithi060' 2024-02-26T10:49:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac32c37740 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ddfaecce10 0x55ddfaecf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac32c37740 4 client.16050 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-26T10:49:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac32c37740 1 -- 172.21.15.60:0/4154240379 --> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ddfadccbd0 con 0x55ddfaecce10 2024-02-26T10:49:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac1fe00640 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ddfaecce10 0x55ddfaecf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac1fe00640 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ddfaecce10 0x55ddfaecf230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c3fc7d0cdb9911 server_cookie=367133eeaa4d0b3b in_seq=0 out_seq=0 2024-02-26T10:49:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.667+0000 7fac16a00640 10 client.16050 ms_handle_connect on v2:172.21.15.100:6838/2377114018 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac16a00640 1 -- 172.21.15.60:0/4154240379 <== mds.0 v2:172.21.15.100:6838/2377114018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ddfadccbd0 con 0x55ddfaecce10 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac16a00640 10 client.16050 handle_command_reply: tid=0 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 client.16050 shutdown 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 -- 172.21.15.60:0/4154240379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 msgr2=0x55ddfae6f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae6f070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fac18023840 tx=0x7fac18005f10 comp rx=0 tx=0).stop 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 -- 172.21.15.60:0/4154240379 shutdown_connections 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.100:6838/2377114018,v1:172.21.15.100:6839/2377114018] conn(0x55ddfaecce10 0x55ddfaecf230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddfae65540 0x55ddfae705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ddfae603d0 0x55ddfae6f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac16a00640 0 client.16050 ms_handle_reset on v2:172.21.15.100:6838/2377114018 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 --2- 172.21.15.60:0/4154240379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ddfae5f9e0 0x55ddfae6f070 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 -- 172.21.15.60:0/4154240379 >> 172.21.15.60:0/4154240379 conn(0x55ddfae5af50 msgr2=0x55ddfae617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 -- 172.21.15.60:0/4154240379 shutdown_connections 2024-02-26T10:49:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.778+0000 7fac32c37740 1 -- 172.21.15.60:0/4154240379 wait complete. 2024-02-26T10:49:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac32c37740 20 client.16050 trim_cache size 0 max 16384 2024-02-26T10:49:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:{ 2024-02-26T10:49:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "message": "", 2024-02-26T10:49:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout: "return_code": 0 2024-02-26T10:49:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stdout:} 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 -- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac0c0a1d70 msgr2=0x7fac0c0a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac0c0a1d70 0x7fac0c0a4230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fac2002c630 tx=0x7fac20032590 comp rx=0 tx=0).stop 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 -- 172.21.15.60:0/3161454799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281011a0 msgr2=0x7fac28199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281011a0 0x7fac28199a80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d950 tx=0x7fac1000de10 comp rx=0 tx=0).stop 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 -- 172.21.15.60:0/3161454799 shutdown_connections 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fac0c0a1d70 0x7fac0c0a4230 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac281011a0 0x7fac28199a80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac28101b60 0x7fac281a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 --2- 172.21.15.60:0/3161454799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac280fffb0 0x7fac28199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 -- 172.21.15.60:0/3161454799 >> 172.21.15.60:0/3161454799 conn(0x7fac280fb540 msgr2=0x7fac280fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.779+0000 7fac2e600640 1 -- 172.21.15.60:0/3161454799 shutdown_connections 2024-02-26T10:49:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi060.front.sepia.ceph.com.smithi060.stderr:2024-02-26T10:49:02.780+0000 7fac2e600640 1 -- 172.21.15.60:0/3161454799 wait complete. 2024-02-26T10:49:04.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:04 smithi113 ceph-mon[36126]: pgmap v518: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.0 MiB/s wr, 540 op/s 2024-02-26T10:49:04.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:04 smithi060 ceph-mon[31539]: pgmap v518: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.0 MiB/s wr, 540 op/s 2024-02-26T10:49:04.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:04 smithi100 ceph-mon[36051]: pgmap v518: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.0 MiB/s wr, 540 op/s 2024-02-26T10:49:06.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:06 smithi113 ceph-mon[36126]: pgmap v519: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 665 op/s 2024-02-26T10:49:06.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:06 smithi060 ceph-mon[31539]: pgmap v519: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 665 op/s 2024-02-26T10:49:06.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:06 smithi100 ceph-mon[36051]: pgmap v519: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 665 op/s 2024-02-26T10:49:08.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:08 smithi113 ceph-mon[36126]: pgmap v520: 129 pgs: 129 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1012 KiB/s wr, 512 op/s 2024-02-26T10:49:08.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:08 smithi060 ceph-mon[31539]: pgmap v520: 129 pgs: 129 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1012 KiB/s wr, 512 op/s 2024-02-26T10:49:08.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:08 smithi100 ceph-mon[36051]: pgmap v520: 129 pgs: 129 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1012 KiB/s wr, 512 op/s 2024-02-26T10:49:09.316 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-26T10:49:09.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-26T10:49:10.509 DEBUG:teuthology.parallel:result is None 2024-02-26T10:49:10.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-26T10:49:10.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:10 smithi113 ceph-mon[36126]: pgmap v521: 129 pgs: 129 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 614 op/s 2024-02-26T10:49:10.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:10 smithi060 ceph-mon[31539]: pgmap v521: 129 pgs: 129 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 614 op/s 2024-02-26T10:49:10.607 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-26T10:49:10.607 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-02-26T10:49:10.620 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T10:49:10.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:10 smithi100 ceph-mon[36051]: pgmap v521: 129 pgs: 129 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 614 op/s 2024-02-26T10:49:10.647 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-02-26T10:49:10.660 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-02-26T10:49:10.660 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-02-26T10:49:10.661 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-26T10:49:10.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:49:10.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.926+0000 7f3ea7200640 1 -- 172.21.15.60:0/1353892844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ea81088b0 msgr2=0x7f3ea8108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.926+0000 7f3ea7200640 1 --2- 172.21.15.60:0/1353892844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ea81088b0 0x7f3ea8108d30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009900 tx=0x7f3e9002bc50 comp rx=0 tx=0).stop 2024-02-26T10:49:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.926+0000 7f3ea7200640 1 -- 172.21.15.60:0/1353892844 shutdown_connections 2024-02-26T10:49:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.926+0000 7f3ea7200640 1 --2- 172.21.15.60:0/1353892844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ea81088b0 0x7f3ea8108d30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.926+0000 7f3ea7200640 1 --2- 172.21.15.60:0/1353892844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8109270 0x7f3ea810fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.926+0000 7f3ea7200640 1 --2- 172.21.15.60:0/1353892844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea81076c0 0x7f3ea8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.926+0000 7f3ea7200640 1 -- 172.21.15.60:0/1353892844 >> 172.21.15.60:0/1353892844 conn(0x7f3ea8076670 msgr2=0x7f3ea8078a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.926+0000 7f3ea7200640 1 -- 172.21.15.60:0/1353892844 shutdown_connections 2024-02-26T10:49:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.926+0000 7f3ea7200640 1 -- 172.21.15.60:0/1353892844 wait complete. 2024-02-26T10:49:10.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.927+0000 7f3ea7200640 1 Processor -- start 2024-02-26T10:49:10.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.927+0000 7f3ea7200640 1 -- start start 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.927+0000 7f3ea7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ea81076c0 0x7f3ea819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.927+0000 7f3ea7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea81088b0 0x7f3ea819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea7200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8109270 0x7f3ea81a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea7200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8113390 con 0x7f3ea8109270 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8113210 con 0x7f3ea81076c0 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8113510 con 0x7f3ea81088b0 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8109270 0x7f3ea81a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ea81076c0 0x7f3ea819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8109270 0x7f3ea81a4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41660/0 (socket says 172.21.15.60:41660) 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea6800640 1 -- 172.21.15.60:0/208218340 learned_addr learned my addr 172.21.15.60:0/208218340 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3e9fe00640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea81088b0 0x7f3ea819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea6800640 1 -- 172.21.15.60:0/208218340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea81088b0 msgr2=0x7f3ea819dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea6800640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea81088b0 0x7f3ea819dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea6800640 1 -- 172.21.15.60:0/208218340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ea81076c0 msgr2=0x7f3ea819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea6800640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ea81076c0 0x7f3ea819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.928+0000 7f3ea6800640 1 -- 172.21.15.60:0/208218340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea81a55c0 con 0x7f3ea8109270 2024-02-26T10:49:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.929+0000 7f3ea6800640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8109270 0x7f3ea81a4f50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000eb10 tx=0x7f3ea000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.929+0000 7f3e9ea00640 1 -- 172.21.15.60:0/208218340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0004b20 con 0x7f3ea8109270 2024-02-26T10:49:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.929+0000 7f3e9ea00640 1 -- 172.21.15.60:0/208218340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ea0004cc0 con 0x7f3ea8109270 2024-02-26T10:49:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.929+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea81a58b0 con 0x7f3ea8109270 2024-02-26T10:49:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.929+0000 7f3e9ea00640 1 -- 172.21.15.60:0/208218340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0010760 con 0x7f3ea8109270 2024-02-26T10:49:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.929+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea81a5d80 con 0x7f3ea8109270 2024-02-26T10:49:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.931+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea810e9e0 con 0x7f3ea8109270 2024-02-26T10:49:10.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.932+0000 7f3e9ea00640 1 -- 172.21.15.60:0/208218340 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3ea0005020 con 0x7f3ea8109270 2024-02-26T10:49:10.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.934+0000 7f3e9ea00640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3e8c0a1d80 0x7f3e8c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:10.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.934+0000 7f3e9ea00640 1 -- 172.21.15.60:0/208218340 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f3ea00ce770 con 0x7f3ea8109270 2024-02-26T10:49:10.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.934+0000 7f3ea5e00640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3e8c0a1d80 0x7f3e8c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:10.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.935+0000 7f3ea5e00640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3e8c0a1d80 0x7f3e8c0a4240 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8108710 tx=0x7f3e98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:10.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:10.938+0000 7f3e9ea00640 1 -- 172.21.15.60:0/208218340 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ea009b790 con 0x7f3ea8109270 2024-02-26T10:49:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.143+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ea8061e60 con 0x7f3ea8109270 2024-02-26T10:49:11.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.147+0000 7f3e9ea00640 1 -- 172.21.15.60:0/208218340 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7f3ea009b090 con 0x7f3ea8109270 2024-02-26T10:49:11.141 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:49:11.141 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:49:11.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3e8c0a1d80 msgr2=0x7f3e8c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3e8c0a1d80 0x7f3e8c0a4240 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8108710 tx=0x7f3e98005f90 comp rx=0 tx=0).stop 2024-02-26T10:49:11.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8109270 msgr2=0x7f3ea81a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8109270 0x7f3ea81a4f50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000eb10 tx=0x7f3ea000c2d0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 shutdown_connections 2024-02-26T10:49:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3e8c0a1d80 0x7f3e8c0a4240 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8109270 0x7f3ea81a4f50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea81088b0 0x7f3ea819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 --2- 172.21.15.60:0/208218340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ea81076c0 0x7f3ea819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 >> 172.21.15.60:0/208218340 conn(0x7f3ea8076670 msgr2=0x7f3ea810bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 shutdown_connections 2024-02-26T10:49:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.150+0000 7f3ea7200640 1 -- 172.21.15.60:0/208218340 wait complete. 2024-02-26T10:49:11.165 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:49:11.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:49:11.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.401+0000 7f3fdc000640 1 -- 172.21.15.60:0/687775267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd4107880 msgr2=0x7f3fd4107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.401+0000 7f3fdc000640 1 --2- 172.21.15.60:0/687775267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd4107880 0x7f3fd4107ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009860 tx=0x7f3fc802bb70 comp rx=0 tx=0).stop 2024-02-26T10:49:11.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.402+0000 7f3fdc000640 1 -- 172.21.15.60:0/687775267 shutdown_connections 2024-02-26T10:49:11.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.402+0000 7f3fdc000640 1 --2- 172.21.15.60:0/687775267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4109430 0x7f3fd410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.402+0000 7f3fdc000640 1 --2- 172.21.15.60:0/687775267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fd4108a70 0x7f3fd4108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.402+0000 7f3fdc000640 1 --2- 172.21.15.60:0/687775267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd4107880 0x7f3fd4107ca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.402+0000 7f3fdc000640 1 -- 172.21.15.60:0/687775267 >> 172.21.15.60:0/687775267 conn(0x7f3fd4076820 msgr2=0x7f3fd4078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:11.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.402+0000 7f3fdc000640 1 -- 172.21.15.60:0/687775267 shutdown_connections 2024-02-26T10:49:11.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.402+0000 7f3fdc000640 1 -- 172.21.15.60:0/687775267 wait complete. 2024-02-26T10:49:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.403+0000 7f3fdc000640 1 Processor -- start 2024-02-26T10:49:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.403+0000 7f3fdc000640 1 -- start start 2024-02-26T10:49:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.403+0000 7f3fdc000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4107880 0x7f3fd419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.403+0000 7f3fdc000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fd4108a70 0x7f3fd419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.403+0000 7f3fdc000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd4109430 0x7f3fd41a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.403+0000 7f3fdc000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41134d0 con 0x7f3fd4107880 2024-02-26T10:49:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.403+0000 7f3fdc000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4113350 con 0x7f3fd4109430 2024-02-26T10:49:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.403+0000 7f3fdc000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4113650 con 0x7f3fd4108a70 2024-02-26T10:49:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fd9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4107880 0x7f3fd419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fd9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4107880 0x7f3fd419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41688/0 (socket says 172.21.15.60:41688) 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fd9600640 1 -- 172.21.15.60:0/213819512 learned_addr learned my addr 172.21.15.60:0/213819512 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fd8c00640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fd4108a70 0x7f3fd419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fda000640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd4109430 0x7f3fd41a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fd9600640 1 -- 172.21.15.60:0/213819512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fd4108a70 msgr2=0x7f3fd419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fd9600640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fd4108a70 0x7f3fd419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fd9600640 1 -- 172.21.15.60:0/213819512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd4109430 msgr2=0x7f3fd41a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fd9600640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd4109430 0x7f3fd41a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fd9600640 1 -- 172.21.15.60:0/213819512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd41a57f0 con 0x7f3fd4107880 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.404+0000 7f3fda000640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd4109430 0x7f3fd41a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.405+0000 7f3fd9600640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4107880 0x7f3fd419dbd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80349e0 tx=0x7f3fc8032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.405+0000 7f3fd2000640 1 -- 172.21.15.60:0/213819512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8041020 con 0x7f3fd4107880 2024-02-26T10:49:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.405+0000 7f3fd2000640 1 -- 172.21.15.60:0/213819512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fc802c9f0 con 0x7f3fd4107880 2024-02-26T10:49:11.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.405+0000 7f3fd2000640 1 -- 172.21.15.60:0/213819512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8039040 con 0x7f3fd4107880 2024-02-26T10:49:11.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.405+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd41a5a80 con 0x7f3fd4107880 2024-02-26T10:49:11.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.405+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd41a5f70 con 0x7f3fd4107880 2024-02-26T10:49:11.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.406+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd410ec70 con 0x7f3fd4107880 2024-02-26T10:49:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.408+0000 7f3fd2000640 1 -- 172.21.15.60:0/213819512 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3fc8044020 con 0x7f3fd4107880 2024-02-26T10:49:11.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.410+0000 7f3fd2000640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fbc0a1d80 0x7f3fbc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:11.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.411+0000 7f3fd2000640 1 -- 172.21.15.60:0/213819512 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f3fc80ef9f0 con 0x7f3fd4107880 2024-02-26T10:49:11.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.411+0000 7f3fd8c00640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fbc0a1d80 0x7f3fbc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:11.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.411+0000 7f3fd8c00640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fbc0a1d80 0x7f3fbc0a4240 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3fd419f0e0 tx=0x7f3fc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:11.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.415+0000 7f3fd2000640 1 -- 172.21.15.60:0/213819512 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3fc804b020 con 0x7f3fd4107880 2024-02-26T10:49:11.581 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:11 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/208218340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:49:11.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:11 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/208218340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:49:11.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.621+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fd4061e60 con 0x7f3fd4107880 2024-02-26T10:49:11.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.621+0000 7f3fd2000640 1 -- 172.21.15.60:0/213819512 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fc80bca10 con 0x7f3fd4107880 2024-02-26T10:49:11.615 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:49:11.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.624+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fbc0a1d80 msgr2=0x7f3fbc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.624+0000 7f3fdc000640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fbc0a1d80 0x7f3fbc0a4240 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3fd419f0e0 tx=0x7f3fc4005f90 comp rx=0 tx=0).stop 2024-02-26T10:49:11.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.624+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4107880 msgr2=0x7f3fd419dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.624+0000 7f3fdc000640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4107880 0x7f3fd419dbd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80349e0 tx=0x7f3fc8032430 comp rx=0 tx=0).stop 2024-02-26T10:49:11.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.625+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 shutdown_connections 2024-02-26T10:49:11.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.625+0000 7f3fdc000640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f3fbc0a1d80 0x7f3fbc0a4240 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.625+0000 7f3fdc000640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd4109430 0x7f3fd41a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.625+0000 7f3fdc000640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fd4108a70 0x7f3fd419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.625+0000 7f3fdc000640 1 --2- 172.21.15.60:0/213819512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4107880 0x7f3fd419dbd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.625+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 >> 172.21.15.60:0/213819512 conn(0x7f3fd4076820 msgr2=0x7f3fd410be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:11.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.625+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 shutdown_connections 2024-02-26T10:49:11.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.625+0000 7f3fdc000640 1 -- 172.21.15.60:0/213819512 wait complete. 2024-02-26T10:49:11.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:11 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/208218340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:49:11.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:49:11.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.874+0000 7fd3a5800640 1 -- 172.21.15.60:0/3709036830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a0075c20 msgr2=0x7fd3a0076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.874+0000 7fd3a5800640 1 --2- 172.21.15.60:0/3709036830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a0075c20 0x7fd3a0076040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd390009ca0 tx=0x7fd39002c100 comp rx=0 tx=0).stop 2024-02-26T10:49:11.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.874+0000 7fd3a5800640 1 -- 172.21.15.60:0/3709036830 shutdown_connections 2024-02-26T10:49:11.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.874+0000 7fd3a5800640 1 --2- 172.21.15.60:0/3709036830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a0077800 0x7fd3a00780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.874+0000 7fd3a5800640 1 --2- 172.21.15.60:0/3709036830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a0076e60 0x7fd3a00772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.874+0000 7fd3a5800640 1 --2- 172.21.15.60:0/3709036830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a0075c20 0x7fd3a0076040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.874+0000 7fd3a5800640 1 -- 172.21.15.60:0/3709036830 >> 172.21.15.60:0/3709036830 conn(0x7fd3a00fea10 msgr2=0x7fd3a0100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:11.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.874+0000 7fd3a5800640 1 -- 172.21.15.60:0/3709036830 shutdown_connections 2024-02-26T10:49:11.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.874+0000 7fd3a5800640 1 -- 172.21.15.60:0/3709036830 wait complete. 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.875+0000 7fd3a5800640 1 Processor -- start 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd3a5800640 1 -- start start 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd3a5800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a0075c20 0x7fd3a019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd3a5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a0076e60 0x7fd3a019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd3a5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a0077800 0x7fd3a01a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd3a5800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0113530 con 0x7fd3a0075c20 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd3a5800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a01133b0 con 0x7fd3a0076e60 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd3a5800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a01136b0 con 0x7fd3a0077800 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a0075c20 0x7fd3a019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a0075c20 0x7fd3a019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41720/0 (socket says 172.21.15.60:41720) 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd39ea00640 1 -- 172.21.15.60:0/1355738329 learned_addr learned my addr 172.21.15.60:0/1355738329 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd39f400640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a0077800 0x7fd3a01a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.876+0000 7fd39e000640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a0076e60 0x7fd3a019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd39ea00640 1 -- 172.21.15.60:0/1355738329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a0077800 msgr2=0x7fd3a01a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd39ea00640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a0077800 0x7fd3a01a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd39ea00640 1 -- 172.21.15.60:0/1355738329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a0076e60 msgr2=0x7fd3a019e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd39ea00640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a0076e60 0x7fd3a019e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd39ea00640 1 -- 172.21.15.60:0/1355738329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a01a5770 con 0x7fd3a0075c20 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd39f400640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a0077800 0x7fd3a01a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd39ea00640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a0075c20 0x7fd3a019db50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd390009dc0 tx=0x7fd390035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd397400640 1 -- 172.21.15.60:0/1355738329 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390039040 con 0x7fd3a0075c20 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd397400640 1 -- 172.21.15.60:0/1355738329 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd390044070 con 0x7fd3a0075c20 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a01a5a00 con 0x7fd3a0075c20 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.877+0000 7fd397400640 1 -- 172.21.15.60:0/1355738329 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390004600 con 0x7fd3a0075c20 2024-02-26T10:49:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.878+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a01a5ef0 con 0x7fd3a0075c20 2024-02-26T10:49:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.879+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a010fe10 con 0x7fd3a0075c20 2024-02-26T10:49:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.881+0000 7fd397400640 1 -- 172.21.15.60:0/1355738329 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd390032890 con 0x7fd3a0075c20 2024-02-26T10:49:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.882+0000 7fd397400640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3840a1d80 0x7fd3840a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.882+0000 7fd397400640 1 -- 172.21.15.60:0/1355738329 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fd3900f0870 con 0x7fd3a0075c20 2024-02-26T10:49:11.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.882+0000 7fd39e000640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3840a1d80 0x7fd3840a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:11.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.883+0000 7fd39e000640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3840a1d80 0x7fd3840a4240 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd3a019f060 tx=0x7fd388005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:11.886+0000 7fd397400640 1 -- 172.21.15.60:0/1355738329 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd3900bd8f0 con 0x7fd3a0075c20 2024-02-26T10:49:12.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.089+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3a0061e60 con 0x7fd3a0075c20 2024-02-26T10:49:12.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.090+0000 7fd397400640 1 -- 172.21.15.60:0/1355738329 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd3900bd1f0 con 0x7fd3a0075c20 2024-02-26T10:49:12.083 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:49:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3840a1d80 msgr2=0x7fd3840a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3840a1d80 0x7fd3840a4240 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd3a019f060 tx=0x7fd388005f90 comp rx=0 tx=0).stop 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a0075c20 msgr2=0x7fd3a019db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a0075c20 0x7fd3a019db50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd390009dc0 tx=0x7fd390035d90 comp rx=0 tx=0).stop 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 shutdown_connections 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd3840a1d80 0x7fd3840a4240 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3a0076e60 0x7fd3a019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a0077800 0x7fd3a01a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 --2- 172.21.15.60:0/1355738329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a0075c20 0x7fd3a019db50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 >> 172.21.15.60:0/1355738329 conn(0x7fd3a00fea10 msgr2=0x7fd3a0100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 shutdown_connections 2024-02-26T10:49:12.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.092+0000 7fd3a5800640 1 -- 172.21.15.60:0/1355738329 wait complete. 2024-02-26T10:49:12.106 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:49:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.343+0000 7faa1b200640 1 -- 172.21.15.60:0/3147852527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c102350 msgr2=0x7faa1c1027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.343+0000 7faa1b200640 1 --2- 172.21.15.60:0/3147852527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c102350 0x7faa1c1027d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faa04009900 tx=0x7faa0402bc30 comp rx=0 tx=0).stop 2024-02-26T10:49:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.343+0000 7faa1b200640 1 -- 172.21.15.60:0/3147852527 shutdown_connections 2024-02-26T10:49:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.343+0000 7faa1b200640 1 --2- 172.21.15.60:0/3147852527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa1c102d10 0x7faa1c1095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.343+0000 7faa1b200640 1 --2- 172.21.15.60:0/3147852527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c102350 0x7faa1c1027d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.343+0000 7faa1b200640 1 --2- 172.21.15.60:0/3147852527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c101160 0x7faa1c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.343+0000 7faa1b200640 1 -- 172.21.15.60:0/3147852527 >> 172.21.15.60:0/3147852527 conn(0x7faa1c0fc6d0 msgr2=0x7faa1c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.343+0000 7faa1b200640 1 -- 172.21.15.60:0/3147852527 shutdown_connections 2024-02-26T10:49:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.344+0000 7faa1b200640 1 -- 172.21.15.60:0/3147852527 wait complete. 2024-02-26T10:49:12.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.345+0000 7faa1b200640 1 Processor -- start 2024-02-26T10:49:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.345+0000 7faa1b200640 1 -- start start 2024-02-26T10:49:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.345+0000 7faa1b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c101160 0x7faa1c075010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.345+0000 7faa1b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa1c102350 0x7faa1c073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.345+0000 7faa1b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c102d10 0x7faa1c073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.345+0000 7faa1b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c10ce40 con 0x7faa1c102350 2024-02-26T10:49:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.345+0000 7faa1b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c10ccc0 con 0x7faa1c102d10 2024-02-26T10:49:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.345+0000 7faa1b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c10cfc0 con 0x7faa1c101160 2024-02-26T10:49:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa19e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c101160 0x7faa1c075010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa1a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c102d10 0x7faa1c073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa19e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c101160 0x7faa1c075010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:40580/0 (socket says 172.21.15.60:40580) 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa19e00640 1 -- 172.21.15.60:0/3477782267 learned_addr learned my addr 172.21.15.60:0/3477782267 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa1a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c102d10 0x7faa1c073b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:44028/0 (socket says 172.21.15.60:44028) 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa11400640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa1c102350 0x7faa1c073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa19e00640 1 -- 172.21.15.60:0/3477782267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c102d10 msgr2=0x7faa1c073b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa19e00640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c102d10 0x7faa1c073b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa19e00640 1 -- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa1c102350 msgr2=0x7faa1c073650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa19e00640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa1c102350 0x7faa1c073650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa19e00640 1 -- 172.21.15.60:0/3477782267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c074420 con 0x7faa1c101160 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.346+0000 7faa1a800640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c102d10 0x7faa1c073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:49:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.347+0000 7faa19e00640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c101160 0x7faa1c075010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9c0 tx=0x7faa0c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:12.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.347+0000 7faa13400640 1 -- 172.21.15.60:0/3477782267 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00cd80 con 0x7faa1c101160 2024-02-26T10:49:12.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.347+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c1a15b0 con 0x7faa1c101160 2024-02-26T10:49:12.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.347+0000 7faa13400640 1 -- 172.21.15.60:0/3477782267 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa0c0035f0 con 0x7faa1c101160 2024-02-26T10:49:12.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.347+0000 7faa13400640 1 -- 172.21.15.60:0/3477782267 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c010690 con 0x7faa1c101160 2024-02-26T10:49:12.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.347+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c1a1ad0 con 0x7faa1c101160 2024-02-26T10:49:12.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.348+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c067f70 con 0x7faa1c101160 2024-02-26T10:49:12.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.350+0000 7faa13400640 1 -- 172.21.15.60:0/3477782267 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7faa0c0037a0 con 0x7faa1c101160 2024-02-26T10:49:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.351+0000 7faa13400640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa000a1d80 0x7faa000a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.351+0000 7faa13400640 1 -- 172.21.15.60:0/3477782267 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7faa0c0ce590 con 0x7faa1c101160 2024-02-26T10:49:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.352+0000 7faa11400640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa000a1d80 0x7faa000a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.352+0000 7faa11400640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa000a1d80 0x7faa000a4240 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faa0402c240 tx=0x7faa0402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:12.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.356+0000 7faa13400640 1 -- 172.21.15.60:0/3477782267 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa0c09b5b0 con 0x7faa1c101160 2024-02-26T10:49:12.362 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:12 smithi060 ceph-mon[31539]: pgmap v522: 129 pgs: 129 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 588 op/s 2024-02-26T10:49:12.362 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:12 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/213819512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:49:12.362 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:12 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1355738329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:49:12.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.561+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa1c02d120 con 0x7faa1c101160 2024-02-26T10:49:12.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.562+0000 7faa13400640 1 -- 172.21.15.60:0/3477782267 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa0c09aeb0 con 0x7faa1c101160 2024-02-26T10:49:12.555 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa000a1d80 msgr2=0x7faa000a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa000a1d80 0x7faa000a4240 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faa0402c240 tx=0x7faa0402df90 comp rx=0 tx=0).stop 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c101160 msgr2=0x7faa1c075010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c101160 0x7faa1c075010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9c0 tx=0x7faa0c00ee80 comp rx=0 tx=0).stop 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 shutdown_connections 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7faa000a1d80 0x7faa000a4240 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c102d10 0x7faa1c073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa1c102350 0x7faa1c073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 --2- 172.21.15.60:0/3477782267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa1c101160 0x7faa1c075010 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 >> 172.21.15.60:0/3477782267 conn(0x7faa1c0fc6d0 msgr2=0x7faa1c105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.565+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 shutdown_connections 2024-02-26T10:49:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.566+0000 7faa1b200640 1 -- 172.21.15.60:0/3477782267 wait complete. 2024-02-26T10:49:12.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:49:12.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:12 smithi113 ceph-mon[36126]: pgmap v522: 129 pgs: 129 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 588 op/s 2024-02-26T10:49:12.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:12 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/213819512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:49:12.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:12 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1355738329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:49:12.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:12 smithi100 ceph-mon[36051]: pgmap v522: 129 pgs: 129 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 588 op/s 2024-02-26T10:49:12.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:12 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/213819512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:49:12.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:12 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1355738329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:49:12.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.801+0000 7fde80e00640 1 -- 172.21.15.60:0/3493796196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde7c102330 msgr2=0x7fde7c1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.801+0000 7fde80e00640 1 --2- 172.21.15.60:0/3493796196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde7c102330 0x7fde7c1027b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fde64009860 tx=0x7fde6402bb70 comp rx=0 tx=0).stop 2024-02-26T10:49:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.801+0000 7fde80e00640 1 -- 172.21.15.60:0/3493796196 shutdown_connections 2024-02-26T10:49:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.801+0000 7fde80e00640 1 --2- 172.21.15.60:0/3493796196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde7c102330 0x7fde7c1027b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.801+0000 7fde80e00640 1 --2- 172.21.15.60:0/3493796196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde7c102cf0 0x7fde7c109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.801+0000 7fde80e00640 1 --2- 172.21.15.60:0/3493796196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde7c101140 0x7fde7c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.801+0000 7fde80e00640 1 -- 172.21.15.60:0/3493796196 >> 172.21.15.60:0/3493796196 conn(0x7fde7c0fc6d0 msgr2=0x7fde7c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.801+0000 7fde80e00640 1 -- 172.21.15.60:0/3493796196 shutdown_connections 2024-02-26T10:49:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.801+0000 7fde80e00640 1 -- 172.21.15.60:0/3493796196 wait complete. 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.802+0000 7fde80e00640 1 Processor -- start 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde80e00640 1 -- start start 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde80e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde7c101140 0x7fde7c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde80e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde7c102330 0x7fde7c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde80e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde7c102cf0 0x7fde7c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde80e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c10cdd0 con 0x7fde7c102330 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde80e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c10cc50 con 0x7fde7c102cf0 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde80e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c10cf50 con 0x7fde7c101140 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde7c102330 0x7fde7c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde7c102330 0x7fde7c199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41756/0 (socket says 172.21.15.60:41756) 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde7aa00640 1 -- 172.21.15.60:0/3920798329 learned_addr learned my addr 172.21.15.60:0/3920798329 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde7b400640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde7c101140 0x7fde7c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.803+0000 7fde7be00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde7c102cf0 0x7fde7c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde7aa00640 1 -- 172.21.15.60:0/3920798329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde7c101140 msgr2=0x7fde7c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde7aa00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde7c101140 0x7fde7c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde7aa00640 1 -- 172.21.15.60:0/3920798329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde7c102cf0 msgr2=0x7fde7c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde7aa00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde7c102cf0 0x7fde7c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde7aa00640 1 -- 172.21.15.60:0/3920798329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c1a1230 con 0x7fde7c102330 2024-02-26T10:49:12.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde7aa00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde7c102330 0x7fde7c199b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde640349e0 tx=0x7fde64032450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:12.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde73e00640 1 -- 172.21.15.60:0/3920798329 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64041030 con 0x7fde7c102330 2024-02-26T10:49:12.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde73e00640 1 -- 172.21.15.60:0/3920798329 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde6402c510 con 0x7fde7c102330 2024-02-26T10:49:12.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c1a14c0 con 0x7fde7c102330 2024-02-26T10:49:12.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.804+0000 7fde73e00640 1 -- 172.21.15.60:0/3920798329 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64039040 con 0x7fde7c102330 2024-02-26T10:49:12.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.805+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c1a19b0 con 0x7fde7c102330 2024-02-26T10:49:12.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.806+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7c1a1c10 con 0x7fde7c102330 2024-02-26T10:49:12.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.808+0000 7fde73e00640 1 -- 172.21.15.60:0/3920798329 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fde64044070 con 0x7fde7c102330 2024-02-26T10:49:12.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.809+0000 7fde73e00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fde600a1d80 0x7fde600a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:12.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.809+0000 7fde73e00640 1 -- 172.21.15.60:0/3920798329 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fde640ef6c0 con 0x7fde7c102330 2024-02-26T10:49:12.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.809+0000 7fde7b400640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fde600a1d80 0x7fde600a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:12.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.810+0000 7fde7b400640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fde600a1d80 0x7fde600a4240 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fde7c102190 tx=0x7fde6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:12.813+0000 7fde73e00640 1 -- 172.21.15.60:0/3920798329 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde640bc6e0 con 0x7fde7c102330 2024-02-26T10:49:13.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.014+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fde7c1a1de0 con 0x7fde7c102330 2024-02-26T10:49:13.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.015+0000 7fde73e00640 1 -- 172.21.15.60:0/3920798329 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fde640bbfe0 con 0x7fde7c102330 2024-02-26T10:49:13.008 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:49:13.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fde600a1d80 msgr2=0x7fde600a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:13.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fde600a1d80 0x7fde600a4240 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fde7c102190 tx=0x7fde6c009260 comp rx=0 tx=0).stop 2024-02-26T10:49:13.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde7c102330 msgr2=0x7fde7c199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:13.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde7c102330 0x7fde7c199b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde640349e0 tx=0x7fde64032450 comp rx=0 tx=0).stop 2024-02-26T10:49:13.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 shutdown_connections 2024-02-26T10:49:13.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fde600a1d80 0x7fde600a4240 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:13.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde7c102cf0 0x7fde7c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:13.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde7c102330 0x7fde7c199b50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:13.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 --2- 172.21.15.60:0/3920798329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde7c101140 0x7fde7c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:13.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 >> 172.21.15.60:0/3920798329 conn(0x7fde7c0fc6d0 msgr2=0x7fde7c073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:13.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 shutdown_connections 2024-02-26T10:49:13.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.018+0000 7fde80e00640 1 -- 172.21.15.60:0/3920798329 wait complete. 2024-02-26T10:49:13.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-26T10:49:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.263+0000 7fd67fe00640 1 -- 172.21.15.60:0/2119757226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd680108f50 msgr2=0x7fd68010b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.263+0000 7fd67fe00640 1 --2- 172.21.15.60:0/2119757226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd680108f50 0x7fd68010b3e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd678009ca0 tx=0x7fd67802c180 comp rx=0 tx=0).stop 2024-02-26T10:49:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.263+0000 7fd67fe00640 1 -- 172.21.15.60:0/2119757226 shutdown_connections 2024-02-26T10:49:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.263+0000 7fd67fe00640 1 --2- 172.21.15.60:0/2119757226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd680073c40 0x7fd6800740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.263+0000 7fd67fe00640 1 --2- 172.21.15.60:0/2119757226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd680108f50 0x7fd68010b3e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.263+0000 7fd67fe00640 1 --2- 172.21.15.60:0/2119757226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd680075220 0x7fd680073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.263+0000 7fd67fe00640 1 -- 172.21.15.60:0/2119757226 >> 172.21.15.60:0/2119757226 conn(0x7fd6800fc5f0 msgr2=0x7fd6800fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.263+0000 7fd67fe00640 1 -- 172.21.15.60:0/2119757226 shutdown_connections 2024-02-26T10:49:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.264+0000 7fd67fe00640 1 -- 172.21.15.60:0/2119757226 wait complete. 2024-02-26T10:49:13.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.264+0000 7fd67fe00640 1 Processor -- start 2024-02-26T10:49:13.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67fe00640 1 -- start start 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd680073c40 0x7fd680199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd680075220 0x7fd680199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd680108f50 0x7fd6801a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68010ef40 con 0x7fd680075220 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68010edc0 con 0x7fd680108f50 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68010f0c0 con 0x7fd680073c40 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd680073c40 0x7fd680199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd680075220 0x7fd680199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd680073c40 0x7fd680199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:40606/0 (socket says 172.21.15.60:40606) 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67ea00640 1 -- 172.21.15.60:0/891895749 learned_addr learned my addr 172.21.15.60:0/891895749 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.265+0000 7fd67f400640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd680108f50 0x7fd6801a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd67ea00640 1 -- 172.21.15.60:0/891895749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd680108f50 msgr2=0x7fd6801a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd67ea00640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd680108f50 0x7fd6801a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd67ea00640 1 -- 172.21.15.60:0/891895749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd680075220 msgr2=0x7fd680199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd67ea00640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd680075220 0x7fd680199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd67ea00640 1 -- 172.21.15.60:0/891895749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6801a1170 con 0x7fd680073c40 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd67ea00640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd680073c40 0x7fd680199550 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd67000e9f0 tx=0x7fd67000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd677400640 1 -- 172.21.15.60:0/891895749 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67000cd80 con 0x7fd680073c40 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6801a1460 con 0x7fd680073c40 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd677400640 1 -- 172.21.15.60:0/891895749 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd670010910 con 0x7fd680073c40 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd677400640 1 -- 172.21.15.60:0/891895749 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6700188f0 con 0x7fd680073c40 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.266+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6801a1a00 con 0x7fd680073c40 2024-02-26T10:49:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.267+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd680107d10 con 0x7fd680073c40 2024-02-26T10:49:13.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.270+0000 7fd677400640 1 -- 172.21.15.60:0/891895749 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd670010ac0 con 0x7fd680073c40 2024-02-26T10:49:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.271+0000 7fd677400640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6640a1d80 0x7fd6640a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.271+0000 7fd677400640 1 -- 172.21.15.60:0/891895749 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fd6700ce980 con 0x7fd680073c40 2024-02-26T10:49:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.271+0000 7fd67e000640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6640a1d80 0x7fd6640a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.272+0000 7fd67e000640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6640a1d80 0x7fd6640a4240 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd68019aa60 tx=0x7fd66c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:13.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.273+0000 7fd677400640 1 -- 172.21.15.60:0/891895749 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd67009b350 con 0x7fd680073c40 2024-02-26T10:49:13.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.525+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fd680061e60 con 0x7fd680073c40 2024-02-26T10:49:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.536+0000 7fd677400640 1 -- 172.21.15.60:0/891895749 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+736396 (secure 0 0 0) 0x7fd67009f3f0 con 0x7fd680073c40 2024-02-26T10:49:13.533 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:49:13.533 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:49:13.533 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 20, 2024-02-26T10:49:13.533 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.533 INFO:teuthology.orchestra.run.smithi060.stdout: "active_gid": 14152, 2024-02-26T10:49:13.533 INFO:teuthology.orchestra.run.smithi060.stdout: "active_name": "x", 2024-02-26T10:49:13.533 INFO:teuthology.orchestra.run.smithi060.stdout: "active_addrs": { 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6800", 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3493132094 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6801", 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3493132094 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.534 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: "active_addr": "172.21.15.60:6801/3493132094", 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: "active_change": "2024-02-26T10:34:36.734063+0000", 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: "active_mgr_features": 4540701547738038271, 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: "available": true, 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: "standbys": [ 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 24112, 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "z", 2024-02-26T10:49:13.535 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_features": 4540701547738038271, 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.536 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.537 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.538 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.539 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.541 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.542 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to", 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.544 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.545 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.546 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.547 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-02-26T10:49:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.550 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.551 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.552 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-02-26T10:49:13.553 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.554 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.555 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.556 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-02-26T10:49:13.557 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.558 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.559 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-02-26T10:49:13.560 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.561 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T10:49:13.562 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.563 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.564 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.565 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.566 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.567 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.568 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.569 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-02-26T10:49:13.570 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T10:49:13.572 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:13.573 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.574 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-02-26T10:49:13.575 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.576 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:13.577 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-02-26T10:49:13.578 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.579 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.580 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.581 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.582 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T10:49:13.583 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-02-26T10:49:13.585 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.586 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.587 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.588 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-02-26T10:49:13.589 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-02-26T10:49:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.591 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.592 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T10:49:13.593 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.594 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.595 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-02-26T10:49:13.596 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-02-26T10:49:13.596 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.596 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.596 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.596 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T10:49:13.596 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.596 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.597 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T10:49:13.598 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.598 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.598 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.598 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-02-26T10:49:13.598 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.598 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.598 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.599 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-02-26T10:49:13.600 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.600 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.600 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.600 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-02-26T10:49:13.600 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-02-26T10:49:13.600 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.600 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.600 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.601 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T10:49:13.601 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.601 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.601 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.601 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-02-26T10:49:13.601 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.601 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.601 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.602 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-02-26T10:49:13.602 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-02-26T10:49:13.602 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.602 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.602 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.602 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T10:49:13.602 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.602 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.602 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.603 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-02-26T10:49:13.603 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.603 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.603 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.603 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-02-26T10:49:13.603 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-02-26T10:49:13.603 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T10:49:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.604 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-02-26T10:49:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-02-26T10:49:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T10:49:13.605 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.606 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.606 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.606 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-02-26T10:49:13.606 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.606 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.606 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.606 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-02-26T10:49:13.606 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.607 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.608 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.609 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.610 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.611 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-02-26T10:49:13.612 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.613 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-02-26T10:49:13.614 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-02-26T10:49:13.615 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.617 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.618 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T10:49:13.619 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.620 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.621 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.623 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T10:49:13.624 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.625 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T10:49:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T10:49:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T10:49:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-02-26T10:49:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-02-26T10:49:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T10:49:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-02-26T10:49:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T10:49:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T10:49:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-02-26T10:49:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.686 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:13 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3477782267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:49:13.686 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:13 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3920798329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:49:13.687 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:13 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3477782267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:49:13.687 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:13 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3920798329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:49:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T10:49:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T10:49:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-02-26T10:49:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T10:49:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-02-26T10:49:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-02-26T10:49:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T10:49:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T10:49:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T10:49:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T10:49:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T10:49:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-02-26T10:49:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-02-26T10:49:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-02-26T10:49:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-02-26T10:49:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-02-26T10:49:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-02-26T10:49:13.787 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.788 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.789 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.790 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.791 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.792 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.793 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.794 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.796 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.797 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.798 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.799 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.800 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.801 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "k8sevents", 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_event_retention_days": { 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ceph_event_retention_days", 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.802 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "config_check_secs": { 2024-02-26T10:49:13.803 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_check_secs", 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "10", 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.804 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.805 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.806 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.807 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.808 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.810 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.811 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.812 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.813 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.814 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.815 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.816 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.818 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.820 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.821 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-02-26T10:49:13.822 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.823 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-02-26T10:49:13.842 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.843 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.844 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.845 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.846 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.847 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.848 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.849 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.851 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.853 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.854 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.855 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.856 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.857 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.858 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.859 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.860 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.861 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.862 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.864 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.866 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.867 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.869 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.870 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-02-26T10:49:13.871 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-02-26T10:49:13.872 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.873 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.874 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.875 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.876 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.877 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.878 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.878 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.878 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.878 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.878 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.878 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.879 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.880 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.880 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.880 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.880 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.880 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.880 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.880 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.880 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.881 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.882 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.882 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-02-26T10:49:13.882 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-02-26T10:49:13.882 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.882 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.882 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.882 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-02-26T10:49:13.882 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.883 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.884 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:13.884 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.884 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.884 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.884 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-02-26T10:49:13.884 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.884 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.884 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.885 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.885 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.885 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-02-26T10:49:13.885 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.885 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.885 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.885 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.886 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.887 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.888 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.889 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.889 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.889 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.889 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.889 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.889 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.889 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.889 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.890 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.890 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.890 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.890 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.890 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.890 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.890 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.891 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.891 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.891 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.891 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.891 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.891 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.891 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.891 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.891 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.892 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.893 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.894 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.895 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.896 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.897 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T10:49:13.898 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-02-26T10:49:13.899 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-02-26T10:49:13.900 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T10:49:13.901 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.902 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.903 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.904 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.905 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.906 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.907 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.908 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.909 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.911 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.912 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.913 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.913 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.913 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.913 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.913 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.913 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.913 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.914 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.915 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.915 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.915 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.915 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.915 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.915 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.916 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.917 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.917 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.917 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.917 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.917 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.917 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.917 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.917 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.918 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.919 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.920 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.920 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.920 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.920 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.920 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.920 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.920 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.921 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.921 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.921 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.921 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.921 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.921 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.921 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.921 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.922 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.923 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.924 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.925 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.927 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.928 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.930 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.931 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.932 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.933 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.934 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.935 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.936 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.937 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.938 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.939 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-02-26T10:49:13.940 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.941 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.942 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.943 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.944 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.945 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-02-26T10:49:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.947 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.948 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.949 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.950 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.951 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.952 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.953 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-02-26T10:49:13.954 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.955 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.956 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.957 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.958 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.959 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-02-26T10:49:13.960 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.961 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.962 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.964 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.966 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.967 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.968 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.969 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.970 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.971 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.972 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.973 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.974 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.975 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.976 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.977 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.979 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.981 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T10:49:13.982 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.983 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.984 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.985 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.986 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:13.988 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.989 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:13.990 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.991 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.992 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.993 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.994 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-02-26T10:49:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-02-26T10:49:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-02-26T10:49:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-02-26T10:49:14.004 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-02-26T10:49:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_no_wait": { 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_no_wait", 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-02-26T10:49:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 24116, 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "y", 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_features": 4540701547738038271, 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-02-26T10:49:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to", 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-02-26T10:49:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-02-26T10:49:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-02-26T10:49:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-02-26T10:49:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T10:49:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-02-26T10:49:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-02-26T10:49:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-02-26T10:49:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T10:49:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-02-26T10:49:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-02-26T10:49:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T10:49:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-02-26T10:49:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T10:49:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-02-26T10:49:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T10:49:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.099 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.100 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-02-26T10:49:14.101 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.102 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.103 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.104 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.105 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T10:49:14.106 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.107 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-02-26T10:49:14.108 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.109 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.111 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.112 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T10:49:14.113 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.114 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.115 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-02-26T10:49:14.116 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.117 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.119 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.120 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.121 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.122 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.123 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-02-26T10:49:14.124 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-02-26T10:49:14.125 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.127 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-02-26T10:49:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.132 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.133 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T10:49:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.137 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T10:49:14.138 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.139 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.140 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-02-26T10:49:14.141 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-02-26T10:49:14.142 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T10:49:14.144 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.145 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.146 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.147 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.148 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-02-26T10:49:14.149 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.151 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T10:49:14.152 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.153 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.154 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.155 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-02-26T10:49:14.156 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.158 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.160 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.161 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.163 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.164 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.165 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T10:49:14.166 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.167 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.168 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.169 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.170 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.171 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.172 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T10:49:14.174 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T10:49:14.175 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.176 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.177 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.178 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.180 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T10:49:14.181 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.182 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.183 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.185 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T10:49:14.186 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T10:49:14.187 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.188 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.189 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.190 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.191 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.192 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.193 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.193 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.193 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.193 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.193 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.193 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.193 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.193 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.194 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.195 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T10:49:14.196 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.197 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.198 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.198 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T10:49:14.198 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T10:49:14.198 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.198 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.198 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.198 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.198 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.199 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T10:49:14.200 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.201 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.202 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.203 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.204 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.205 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.206 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.207 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T10:49:14.208 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.209 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.210 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.211 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.212 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.213 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T10:49:14.214 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.215 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.216 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.218 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-02-26T10:49:14.219 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.220 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-02-26T10:49:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T10:49:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.236 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-02-26T10:49:14.236 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-02-26T10:49:14.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.236 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.236 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.236 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-02-26T10:49:14.236 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.236 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.236 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.239 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.241 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.242 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-02-26T10:49:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.244 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.247 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.248 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.249 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.250 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-02-26T10:49:14.251 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.252 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.253 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.254 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.255 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.256 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.257 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.258 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.259 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.260 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.263 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-02-26T10:49:14.264 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.266 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.267 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-02-26T10:49:14.268 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.269 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.270 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-02-26T10:49:14.271 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-02-26T10:49:14.272 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.274 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.275 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.276 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.277 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.278 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.279 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.280 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T10:49:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.282 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T10:49:14.283 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.284 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.285 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.286 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.287 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.288 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.289 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.290 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.291 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.292 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.293 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.294 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.295 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.297 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "k8sevents", 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.298 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_event_retention_days": { 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ceph_event_retention_days", 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.299 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "config_check_secs": { 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_check_secs", 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "10", 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.300 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.301 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.302 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.303 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.304 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.305 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.306 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.307 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T10:49:14.308 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.309 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.311 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.312 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.313 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-02-26T10:49:14.314 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.316 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T10:49:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.318 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.319 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.320 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.321 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.322 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.322 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.322 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.322 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.322 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.322 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.322 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.322 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.323 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.324 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.325 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.326 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.327 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.328 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.329 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.330 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.331 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.333 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.334 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.335 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.336 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.337 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.338 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.339 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.340 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.341 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.342 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.343 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.344 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.345 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.346 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.349 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.350 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.351 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.352 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.354 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.355 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.356 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.357 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.358 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.360 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-02-26T10:49:14.361 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.364 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.365 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.367 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.368 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.370 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.371 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.372 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.373 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.375 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.376 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.377 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.378 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.380 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.381 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.382 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T10:49:14.383 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.384 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.386 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:13 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3477782267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:49:14.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:13 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3920798329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.388 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.390 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.391 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T10:49:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.393 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.394 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T10:49:14.395 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.396 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.397 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.398 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.399 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T10:49:14.400 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.402 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.403 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.405 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.406 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-02-26T10:49:14.407 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.409 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.410 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-02-26T10:49:14.411 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.412 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.415 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.416 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.416 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.416 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.416 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.416 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.416 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.416 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.416 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.417 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.417 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.417 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.417 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.417 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.417 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.417 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.418 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.419 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T10:49:14.421 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.422 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.423 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.424 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.425 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.426 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.427 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.428 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.429 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.430 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.431 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.432 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.434 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.435 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-02-26T10:49:14.436 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.437 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.438 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.440 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.441 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.442 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.443 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.444 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.445 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.446 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.447 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-02-26T10:49:14.448 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-02-26T10:49:14.449 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.451 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.452 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-02-26T10:49:14.453 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.454 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.454 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.454 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.454 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6640a1d80 msgr2=0x7fd6640a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6640a1d80 0x7fd6640a4240 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd68019aa60 tx=0x7fd66c005f90 comp rx=0 tx=0).stop 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd680073c40 msgr2=0x7fd680199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd680073c40 0x7fd680199550 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd67000e9f0 tx=0x7fd67000eeb0 comp rx=0 tx=0).stop 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 shutdown_connections 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fd6640a1d80 0x7fd6640a4240 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd680108f50 0x7fd6801a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd680075220 0x7fd680199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 --2- 172.21.15.60:0/891895749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd680073c40 0x7fd680199550 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.545+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 >> 172.21.15.60:0/891895749 conn(0x7fd6800fc5f0 msgr2=0x7fd680104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.546+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 shutdown_connections 2024-02-26T10:49:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:13.546+0000 7fd67fe00640 1 -- 172.21.15.60:0/891895749 wait complete. 2024-02-26T10:49:14.458 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-02-26T10:49:14.458 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-02-26T10:49:14.458 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.459 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.460 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.461 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.462 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.463 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.464 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.466 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.467 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.468 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.469 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.470 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.472 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.473 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.474 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.477 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.478 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.479 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.480 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.481 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.482 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.483 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-02-26T10:49:14.484 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.485 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.486 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.487 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.488 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.489 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.490 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.491 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.492 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.493 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T10:49:14.495 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.496 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-02-26T10:49:14.498 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.499 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.500 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.501 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.502 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.503 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.505 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.507 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_no_wait": { 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_no_wait", 2024-02-26T10:49:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-02-26T10:49:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "modules": [ 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-02-26T10:49:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "dashboard", 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "iostat", 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "nfs", 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "restful" 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-02-26T10:49:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to", 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-02-26T10:49:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-02-26T10:49:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T10:49:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T10:49:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-02-26T10:49:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T10:49:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T10:49:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-02-26T10:49:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-02-26T10:49:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-02-26T10:49:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-02-26T10:49:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-02-26T10:49:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T10:49:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-02-26T10:49:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.604 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.605 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.606 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T10:49:14.607 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.608 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.609 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.611 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.612 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.613 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T10:49:14.614 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.615 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-02-26T10:49:14.616 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-02-26T10:49:14.617 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.619 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.620 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.621 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.622 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.623 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-02-26T10:49:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.626 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-02-26T10:49:14.626 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-02-26T10:49:14.626 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.626 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.626 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.627 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-02-26T10:49:14.627 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.627 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.627 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.627 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-02-26T10:49:14.627 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.628 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.628 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.628 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-02-26T10:49:14.628 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-02-26T10:49:14.629 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.629 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.629 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.629 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.629 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.629 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.629 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.630 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T10:49:14.630 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.630 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.630 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.630 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-02-26T10:49:14.631 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-02-26T10:49:14.631 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.631 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.631 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.631 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.631 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.632 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.632 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.632 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-02-26T10:49:14.632 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.632 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.632 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.633 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-02-26T10:49:14.633 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-02-26T10:49:14.633 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.633 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.633 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.634 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.634 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.634 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.634 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.634 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-02-26T10:49:14.634 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.634 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.635 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.635 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-02-26T10:49:14.635 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-02-26T10:49:14.635 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.637 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.638 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-02-26T10:49:14.639 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.640 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.641 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.642 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T10:49:14.645 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.646 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.647 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.649 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T10:49:14.650 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T10:49:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.652 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T10:49:14.653 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.654 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-02-26T10:49:14.655 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.657 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T10:49:14.658 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-02-26T10:49:14.660 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.661 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.662 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-02-26T10:49:14.663 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.665 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T10:49:14.666 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.667 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.668 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.669 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.670 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.671 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.674 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.675 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-02-26T10:49:14.676 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.678 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.679 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T10:49:14.680 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.683 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.684 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.685 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.686 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.687 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T10:49:14.688 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.689 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.690 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.691 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.693 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T10:49:14.695 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.697 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.698 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.699 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.700 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.701 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T10:49:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.703 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.704 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.705 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.706 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.707 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.708 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T10:49:14.709 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.712 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.713 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.714 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.715 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.716 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T10:49:14.717 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.718 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.719 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.720 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.721 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.722 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.723 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T10:49:14.724 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T10:49:14.725 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.727 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.728 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.729 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.730 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-02-26T10:49:14.731 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.732 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.733 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.734 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.735 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.736 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.737 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.738 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.739 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.740 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.741 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.742 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.742 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T10:49:14.742 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T10:49:14.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.742 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.742 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.743 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T10:49:14.743 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.743 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.743 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.743 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.743 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.743 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.743 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.744 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.745 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.746 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.747 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.749 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.750 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.751 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.752 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.753 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.754 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.757 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.760 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.761 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.762 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.763 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.764 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-02-26T10:49:14.765 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.766 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.767 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.768 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.769 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.770 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.772 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.773 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.774 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T10:49:14.775 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.776 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.777 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-02-26T10:49:14.778 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.779 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.780 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.781 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.782 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.783 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.784 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.785 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.786 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.787 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-02-26T10:49:14.788 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.790 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-02-26T10:49:14.793 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-02-26T10:49:14.794 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.795 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.796 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.797 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.798 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.801 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.802 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.803 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.804 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.805 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.806 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "k8sevents", 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.807 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_event_retention_days": { 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ceph_event_retention_days", 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "config_check_secs": { 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_check_secs", 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "10", 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.810 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.811 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.812 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.813 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.814 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.815 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.816 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.817 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.818 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.819 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.820 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.821 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.822 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.823 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.824 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-02-26T10:49:14.825 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.826 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.827 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T10:49:14.828 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.829 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.830 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.831 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.832 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.833 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.834 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.835 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.836 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.837 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.838 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.839 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.842 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.843 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.844 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.845 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.846 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.848 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.849 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.850 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.852 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.853 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.854 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.855 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.856 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.857 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.858 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.859 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.860 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.862 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.863 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.864 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.865 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.866 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.867 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.868 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.869 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.871 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.872 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.873 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.874 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.875 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-02-26T10:49:14.876 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.878 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.879 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.880 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.881 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.882 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.883 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.884 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.885 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.886 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T10:49:14.887 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.888 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.889 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T10:49:14.890 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.891 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.892 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.893 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.894 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.895 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.896 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.897 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.898 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.899 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.900 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.901 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.903 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.904 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.905 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.906 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.907 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.908 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.909 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.910 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.911 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.912 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-02-26T10:49:14.913 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T10:49:14.914 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.915 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.916 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.917 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.918 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.919 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.920 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T10:49:14.921 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-02-26T10:49:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.923 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.924 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.925 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.926 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.928 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.930 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.931 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.932 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.933 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.934 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.935 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.936 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.937 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.938 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.939 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.940 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.941 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.961 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.962 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.963 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.964 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.965 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.966 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.967 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.968 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.969 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.970 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.971 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.972 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.973 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.974 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.975 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.977 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.978 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.979 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.980 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.981 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.982 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.983 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.984 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.985 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-02-26T10:49:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_no_wait": { 2024-02-26T10:49:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_no_wait", 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T10:49:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T10:49:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T10:49:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "services": { 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "dashboard": "https://172.21.15.60:8443/" 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "always_on_modules": { 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "octopus": [ 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T10:49:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "pacific": [ 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T10:49:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "quincy": [ 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T10:49:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "reef": [ 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "squid": [ 2024-02-26T10:49:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "last_failure_osd_epoch": 3, 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "active_clients": [ 2024-02-26T10:49:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4152931325 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "libcephsqlite", 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3500886609 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 988911145 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 806578503 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T10:49:15.031 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:49:15.037 DEBUG:tasks.check_counter:Getting stats from a 2024-02-26T10:49:15.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-26T10:49:15.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:14 smithi060 ceph-mon[31539]: pgmap v523: 129 pgs: 129 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1003 KiB/s wr, 483 op/s 2024-02-26T10:49:15.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:14 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/891895749' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T10:49:15.039 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:14 smithi100 ceph-mon[36051]: pgmap v523: 129 pgs: 129 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1003 KiB/s wr, 483 op/s 2024-02-26T10:49:15.039 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:14 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/891895749' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T10:49:15.039 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:14 smithi113 ceph-mon[36126]: pgmap v523: 129 pgs: 129 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1003 KiB/s wr, 483 op/s 2024-02-26T10:49:15.039 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:14 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/891895749' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T10:49:15.510 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:49:16.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:16 smithi113 ceph-mon[36126]: pgmap v524: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 477 op/s 2024-02-26T10:49:16.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:16 smithi060 ceph-mon[31539]: pgmap v524: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 477 op/s 2024-02-26T10:49:16.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:16 smithi100 ceph-mon[36051]: pgmap v524: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 477 op/s 2024-02-26T10:49:18.458 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:49:18.458 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:18.458 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-02-26T10:49:18.458 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-02-26T10:49:18.458 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.004700613, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001021101, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:18.459 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 116, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 111, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 718752, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 66080, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.048174658, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.015615800, 2024-02-26T10:49:18.460 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.037600900, 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.005005054, 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 110, 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.013547265, 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000123156 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 718752, 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 66080 2024-02-26T10:49:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 5, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 86, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 3184, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 92640, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.016373970, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.011906681, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.002783114, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:18.462 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 85, 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.010834702, 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000127467 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 92640 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.463 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:18.464 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:18.465 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 149854, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 28, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:18.466 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-02-26T10:49:18.467 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 117, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 702394, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 117, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 117, 2024-02-26T10:49:18.468 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 702394, 2024-02-26T10:49:18.469 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-02-26T10:49:18.469 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:18.469 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:18.469 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:18.469 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:18.469 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:18.469 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:49:18.515 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:18 smithi060 ceph-mon[31539]: pgmap v525: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 673 KiB/s wr, 296 op/s 2024-02-26T10:49:18.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:18 smithi113 ceph-mon[36126]: pgmap v525: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 673 KiB/s wr, 296 op/s 2024-02-26T10:49:18.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:18 smithi100 ceph-mon[36051]: pgmap v525: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 673 KiB/s wr, 296 op/s 2024-02-26T10:49:19.426 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-02-26T10:49:19.426 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-02-26T10:49:19.426 DEBUG:tasks.check_counter:Getting stats from d 2024-02-26T10:49:19.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-26T10:49:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:49:20.580 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:20 smithi113 ceph-mon[36126]: pgmap v526: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 672 KiB/s wr, 260 op/s 2024-02-26T10:49:20.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:20 smithi060 ceph-mon[31539]: pgmap v526: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 672 KiB/s wr, 260 op/s 2024-02-26T10:49:20.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:20 smithi100 ceph-mon[36051]: pgmap v526: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 672 KiB/s wr, 260 op/s 2024-02-26T10:49:22.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:22 smithi060 ceph-mon[31539]: pgmap v527: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 218 KiB/s wr, 133 op/s 2024-02-26T10:49:22.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:22 smithi100 ceph-mon[36051]: pgmap v527: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 218 KiB/s wr, 133 op/s 2024-02-26T10:49:22.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:22 smithi113 ceph-mon[36126]: pgmap v527: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 218 KiB/s wr, 133 op/s 2024-02-26T10:49:23.044 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:49:23.044 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:23.044 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-02-26T10:49:23.044 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.003805809, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000865941, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.045 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 127, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 120, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 723680, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 70896, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.043626489, 2024-02-26T10:49:23.046 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.012256901, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.035535088, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.005099268, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 120, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.014932751, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000124439 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 723680, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 70896 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.047 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 90, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 160, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 93248, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.017192633, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.012620031, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.002586914, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 88, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.007990109, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000090796 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.048 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 93248 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:23.049 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:23.050 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:23.051 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 149854, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 28, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-02-26T10:49:23.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 124, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 703304, 2024-02-26T10:49:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 124, 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 124, 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 703304, 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:23.054 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:49:23.966 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-02-26T10:49:23.966 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-02-26T10:49:23.966 DEBUG:tasks.check_counter:Getting stats from g 2024-02-26T10:49:23.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-26T10:49:24.454 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:49:24.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:24 smithi060 ceph-mon[31539]: pgmap v528: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 135 KiB/s wr, 57 op/s 2024-02-26T10:49:24.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:24 smithi100 ceph-mon[36051]: pgmap v528: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 135 KiB/s wr, 57 op/s 2024-02-26T10:49:24.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:24 smithi113 ceph-mon[36126]: pgmap v528: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 135 KiB/s wr, 57 op/s 2024-02-26T10:49:26.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:26 smithi060 ceph-mon[31539]: pgmap v529: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 116 KiB/s wr, 45 op/s 2024-02-26T10:49:26.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:26 smithi100 ceph-mon[36051]: pgmap v529: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 116 KiB/s wr, 45 op/s 2024-02-26T10:49:26.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:26 smithi113 ceph-mon[36126]: pgmap v529: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 116 KiB/s wr, 45 op/s 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 4, 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 4, 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 3024, 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 448, 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.004784242, 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000051343, 2024-02-26T10:49:26.863 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001635241, 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3, 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.001577043, 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000525681 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.864 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 119, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 114, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 719568, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 67952, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.046956686, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.016686118, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.033760757, 2024-02-26T10:49:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.004920985, 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 113, 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.022176250, 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000196250 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 719568, 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 67952 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.866 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 87, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 160, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 92720, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.017688870, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.012808021, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.002829337, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 85, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.007509441, 2024-02-26T10:49:26.867 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000088346 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 92720 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-02-26T10:49:26.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:26.869 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:26.870 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 149871, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 29, 2024-02-26T10:49:26.871 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:26.872 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 120, 2024-02-26T10:49:26.873 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 702784, 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 120, 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 120, 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 702784, 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:26.874 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:49:27.816 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-02-26T10:49:27.816 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-02-26T10:49:27.816 DEBUG:tasks.check_counter:Getting stats from j 2024-02-26T10:49:27.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-26T10:49:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T10:49:28.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:28 smithi060 ceph-mon[31539]: pgmap v530: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-02-26T10:49:28.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:28 smithi100 ceph-mon[36051]: pgmap v530: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-02-26T10:49:28.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:28 smithi113 ceph-mon[36126]: pgmap v530: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-02-26T10:49:30.593 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T10:49:30.593 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:30.593 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-02-26T10:49:30.593 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-02-26T10:49:30.593 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-02-26T10:49:30.593 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.004611478, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001174094, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:30.594 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 5, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 94, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 3184, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 94048, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.019510143, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.013370560, 2024-02-26T10:49:30.595 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.004066362, 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 93, 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.010590333, 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000113874 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 94048 2024-02-26T10:49:30.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 126, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 121, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 721472, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 72320, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.043923420, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.013783649, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.031356967, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.005471957, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 120, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.013001907, 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000108349 2024-02-26T10:49:30.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 721472, 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 72320 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:30.598 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:30.599 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:30.600 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 149854, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 28, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:30.601 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:30.602 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 127, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 703694, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 127, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 127, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 703694, 2024-02-26T10:49:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-02-26T10:49:30.604 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T10:49:30.604 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T10:49:30.604 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T10:49:30.604 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:30.604 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T10:49:30.604 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T10:49:30.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:30 smithi060 ceph-mon[31539]: pgmap v531: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-02-26T10:49:30.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:30 smithi100 ceph-mon[36051]: pgmap v531: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-02-26T10:49:30.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:30 smithi113 ceph-mon[36126]: pgmap v531: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-02-26T10:49:31.559 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-26T10:49:31.560 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-26T10:49:31.560 DEBUG:tasks.check_counter:Getting stats from b 2024-02-26T10:49:31.560 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-26T10:49:31.808 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:49:32.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:32 smithi060 ceph-mon[31539]: pgmap v532: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-02-26T10:49:32.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:32 smithi100 ceph-mon[36051]: pgmap v532: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-02-26T10:49:32.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:32 smithi113 ceph-mon[36126]: pgmap v532: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-02-26T10:49:33.568 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-02-26T10:49:33.568 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 57896, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 104635, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 21712973, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 118316276, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 47, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 7, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 4.470740146, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 2.063697913, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 3.120901004, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.492716598, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 104632, 2024-02-26T10:49:33.569 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 11.674863429, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000111580 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 92185, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.015837503, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000171 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 16229776, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 45895776 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 28611, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 29326, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 9663994, 2024-02-26T10:49:33.570 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 51777740, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 38, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 1.829467253, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.693166413, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.685351348, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.425621595, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 29326, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 6.468761877, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000220581 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 23254, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.004375209, 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000188 2024-02-26T10:49:33.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 234672 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 22883, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 24819, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 7773316, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 25308293, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 10, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 6, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 1.539885888, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.605978396, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.486131518, 2024-02-26T10:49:33.572 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.410623453, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 24818, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 10.209659530, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000411381 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 13378, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.002259255, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000168 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 788592, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 80928 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-02-26T10:49:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 10750, 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 25.078684067, 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002332900 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 17629, 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 2.341819316, 2024-02-26T10:49:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000132839 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 25767, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 25636, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 25636, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 52.865531108, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002062159 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 131, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 6, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 2, 2024-02-26T10:49:33.575 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 2545, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 8, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 2, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 308, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 18, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 280, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 10, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 281, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 2852, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 2, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 2, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 4, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 29839, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 28923, 2024-02-26T10:49:33.576 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 51, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 186, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 6, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 10, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 150, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 2348, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 99, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 155, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 5528, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 16, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-02-26T10:49:33.577 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 5528, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 571, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 4, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 567, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 4957, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 194, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 3023, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 23184, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 43, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T10:49:33.578 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 13354, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 10087, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 129, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 13449 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 210, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 300, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 3, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-02-26T10:49:33.579 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 9789, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 9773, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 20, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 4, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 151, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 10, 2024-02-26T10:49:33.580 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 50962, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 47584, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 47584, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 3378, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 7515, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 70, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 65, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 65, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 5, 2024-02-26T10:49:33.581 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 1, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 9, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 100032094, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 105834208, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4115, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 154.911009452, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.037645445 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-02-26T10:49:33.582 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 35, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 12323, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 12288, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 34, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 1615, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 1581, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 308, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 22608, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 22300, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 2, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 12229, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 12227, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 134980, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 215420 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.583 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 33450, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 1261, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 25767, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 276, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 2120, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 10460, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 13.906918000, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001329533 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 180, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.286545462, 2024-02-26T10:49:33.584 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001591919 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 48, 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.436476960, 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.009093270 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2086, 2024-02-26T10:49:33.585 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.255765935, 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000601997 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1, 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000201567, 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000201567 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.586 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.588 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-02-26T10:49:33.588 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.588 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.588 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.588 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-02-26T10:49:33.588 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1370, 2024-02-26T10:49:33.588 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 5.472587653, 2024-02-26T10:49:33.588 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.003994589 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 67, 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.068941343, 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001028975 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 18, 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.005317815, 2024-02-26T10:49:33.589 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000295434 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1281, 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.212257185, 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000946336 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 128, 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.823433025, 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.014245570 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.590 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1312, 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 16.061920107, 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.012242317 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.591 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 46, 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.188961550, 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004107859 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-02-26T10:49:33.592 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-02-26T10:49:33.593 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 9, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.023483333, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002609259 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 95, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.108949656, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001146838 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 8535, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 12.013771517, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001407588 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-02-26T10:49:33.594 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 4, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 3, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 1, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 13566, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 13566, 2024-02-26T10:49:33.595 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 232, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 360, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 360, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:33.596 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:33.597 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 307206, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 132, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 968, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 11, 2024-02-26T10:49:33.598 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 8984, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 172, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 353285, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 3484, 2024-02-26T10:49:33.599 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 51775, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 128133335, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 37, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 51775, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 51775, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 13008.613437118, 2024-02-26T10:49:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.251252794 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 51775, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 86674 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 51775, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 5799, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 45976, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 1203, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 19763, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 258, 2024-02-26T10:49:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 4369, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 186, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 8, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 18904, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 5540, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 30927, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-02-26T10:49:33.602 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 50, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-02-26T10:49:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 95, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 10, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 12, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 12, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-02-26T10:49:33.604 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 5245, 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 17, 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 204 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-02-26T10:49:33.605 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 24, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 16, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 18338, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 9773, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 109386, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 27336021, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 109386, 2024-02-26T10:49:33.606 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 109386, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 27336021, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-02-26T10:49:33.607 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 51775, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 129186775, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 51775, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 129186775, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-02-26T10:49:33.608 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 51775, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 51775, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 51775, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 51775, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 9773, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1053569, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 9773, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 254, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1053569, 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:33.610 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x5645d05cc6e0": { 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 50962, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 101639904, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 50962, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:33.611 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:33.612 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 4115, 2024-02-26T10:49:33.612 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 101639904, 2024-02-26T10:49:33.612 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:33.612 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:33.612 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:33.612 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:33.612 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:33.612 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:33.612 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-02-26T10:49:34.396 INFO:tasks.check_counter:Daemon mds.b mds.exported=150 2024-02-26T10:49:34.396 INFO:tasks.check_counter:Daemon mds.b mds.imported=99 2024-02-26T10:49:34.396 DEBUG:tasks.check_counter:Getting stats from e 2024-02-26T10:49:34.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-26T10:49:34.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:34 smithi060 ceph-mon[31539]: pgmap v533: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:34.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:34 smithi100 ceph-mon[36051]: pgmap v533: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:49:34.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:34 smithi113 ceph-mon[36126]: pgmap v533: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:36.374 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-02-26T10:49:36.374 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:36.374 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 430, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 430, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 12697115, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 135980, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 7, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.043163790, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.013653028, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.015803595, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.009487953, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 429, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.017393550, 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000040544 2024-02-26T10:49:36.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 426, 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 414, 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 13419700, 2024-02-26T10:49:36.376 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 163867, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.054212116, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.013866365, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.030454104, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.011139118, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 413, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.020916783, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000050645 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.377 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 765712, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 73232 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 458, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 550, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 4788525, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 447248, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-02-26T10:49:36.378 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.057411522, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.018567743, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.016701576, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.016649726, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 547, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.027867534, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000050946 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.379 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 302240 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2006, 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.051901591, 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000025873 2024-02-26T10:49:36.380 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 0, 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 0, 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-02-26T10:49:36.381 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 0, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 3620, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-02-26T10:49:36.382 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 3620, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 3620, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 1, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-02-26T10:49:36.383 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-02-26T10:49:36.384 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-02-26T10:49:36.385 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-02-26T10:49:36.386 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-02-26T10:49:36.387 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 0, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 13599, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 0, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 19, 2024-02-26T10:49:36.388 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 3, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 34001491, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 34000353, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 13599 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-02-26T10:49:36.389 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 101, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 2985, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 2884, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 99, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 598, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 499, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 3620, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 6090, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 2470, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 0, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 0, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 63716, 2024-02-26T10:49:36.390 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 215420 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 0, 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 0, 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 0, 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.391 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.393 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.394 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.395 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.396 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.396 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-02-26T10:49:36.396 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.396 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.396 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.398 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.399 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.401 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-02-26T10:49:36.402 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 2, 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 1, 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 1, 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 0, 2024-02-26T10:49:36.403 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 0, 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 2149, 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:36.404 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:36.405 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:36.406 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 217497, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 150, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:36.407 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 8984, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 172, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 2920587, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 26029, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-02-26T10:49:36.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 1167, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 21, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 3, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 1167, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1167, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.876950349, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001608355 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-02-26T10:49:36.409 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1167, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1168 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 1167, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 1167, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 0, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 683, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 0, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 482, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 0, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 0, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-02-26T10:49:36.410 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 1, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-02-26T10:49:36.411 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-02-26T10:49:36.412 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 95, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 4, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 8, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 8, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-02-26T10:49:36.413 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 0, 2024-02-26T10:49:36.415 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 2, 2024-02-26T10:49:36.415 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 0 2024-02-26T10:49:36.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 0, 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-02-26T10:49:36.416 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 1310, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 30777176, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 1310, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:36.417 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1310, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 30777176, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:36.418 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 1167, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 29807192, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1167, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 29807192, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-02-26T10:49:36.419 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 1167, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 1167, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1167, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1167, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-02-26T10:49:36.420 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:36.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x5632341ca960": { 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:36.422 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:36.423 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:36.423 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:36.423 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:36.423 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:36.423 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-02-26T10:49:36.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:36 smithi100 ceph-mon[36051]: pgmap v534: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:36.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:36 smithi113 ceph-mon[36126]: pgmap v534: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:36.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:36 smithi060 ceph-mon[31539]: pgmap v534: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:37.090 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-02-26T10:49:37.091 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-02-26T10:49:37.091 DEBUG:tasks.check_counter:Getting stats from h 2024-02-26T10:49:37.091 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-26T10:49:37.330 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:49:38.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:38 smithi100 ceph-mon[36051]: pgmap v535: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:38.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:38 smithi113 ceph-mon[36126]: pgmap v535: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:38.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:38 smithi060 ceph-mon[31539]: pgmap v535: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:39.067 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-02-26T10:49:39.067 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:39.067 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 7906, 2024-02-26T10:49:39.067 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 8961, 2024-02-26T10:49:39.067 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 2100540, 2024-02-26T10:49:39.067 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 16093024, 2024-02-26T10:49:39.067 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 21, 2024-02-26T10:49:39.067 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-02-26T10:49:39.068 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.496938954, 2024-02-26T10:49:39.068 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.204456455, 2024-02-26T10:49:39.068 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.229966931, 2024-02-26T10:49:39.068 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.077541819, 2024-02-26T10:49:39.068 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:39.068 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 8959, 2024-02-26T10:49:39.068 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.312437579, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000034874 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 11176, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001653953, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000147 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 928, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 736 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 15473, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 26597, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 6353738, 2024-02-26T10:49:39.069 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 24693129, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 9, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 6, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 1.151348425, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.532722405, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.662340336, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.080776787, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 26595, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.801530484, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000030138 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 27014, 2024-02-26T10:49:39.070 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.004645216, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000171 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 5654240, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 15183728 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 15521, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 14881, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 6025081, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 13197717, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 9, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 6, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.985290090, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.438412134, 2024-02-26T10:49:39.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.544928726, 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.100188408, 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 14880, 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.455411270, 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000030605 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 23259, 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.003925423, 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000168 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 773328, 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 107776 2024-02-26T10:49:39.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5714, 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 6.621044004, 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001158740 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-02-26T10:49:39.073 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5353, 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.527545114, 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000098551 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 7739, 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 7620, 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 7620, 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 21.582209549, 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002832310 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.074 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 119, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 1, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 11, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 302, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 114, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 83, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 105, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 139, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 850, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 0, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 0, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 9, 2024-02-26T10:49:39.075 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 11764, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 10660, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 93, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 232, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 4, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 10, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 75, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 287, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 101, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 698, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-02-26T10:49:39.076 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 239, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 8, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 2862, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 12460942, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 239, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 18, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 18, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 221, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 175, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 1381, 2024-02-26T10:49:39.077 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 5444, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 170, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 4582, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 2528, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 144, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 4777 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 255, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 271, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 88, 2024-02-26T10:49:39.078 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 2606, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 2450, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 65, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 8, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 76, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-02-26T10:49:39.079 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 13599, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 13599, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 19, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-02-26T10:49:39.080 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 19, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 3, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 34001491, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2971, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 27.391847078, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.009219739 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 121, 2024-02-26T10:49:39.081 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 3001, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 2880, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 120, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 613, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 493, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 304, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 6543, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 6239, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 0, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 3297, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 3297, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 104416, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 207228 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 11699, 2024-02-26T10:49:39.082 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 1477, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 7739, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 280, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 2353, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1082, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.837018288, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001697798 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 137, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.567554825, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004142735 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.083 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 184, 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.996208783, 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.005414178 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1247, 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.923937400, 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000740928 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.084 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.086 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-02-26T10:49:39.086 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.086 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.086 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.086 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-02-26T10:49:39.086 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.086 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 358, 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.575899854, 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001608658 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 347, 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.550216507, 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001585638 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-02-26T10:49:39.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 33, 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.019853386, 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000601617 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 454, 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.178217209, 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002595192 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.088 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 412, 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 4.364258398, 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.010592860 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-02-26T10:49:39.089 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 534, 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 5.226557646, 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.009787561 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.091 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 174, 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.705659470, 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004055514 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-02-26T10:49:39.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-02-26T10:49:39.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 320, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.570903591, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001784073 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2338, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 4.065924192, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001739060 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-02-26T10:49:39.094 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 2, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 1, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 1, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 4225, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 4225, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 2153, 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-02-26T10:49:39.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:39.096 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:39.097 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 314644, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 221, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 1232, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 14, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 8984, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 172, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 679451, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 15548, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.099 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 8661, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 30338995, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 33, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 8661, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 8661, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 61.896353640, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.007146559 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-02-26T10:49:39.100 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 8661, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 8834 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 8661, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 441, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 8220, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 0, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 10, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 201, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 3172, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 118, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-02-26T10:49:39.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 4614, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 240, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 20, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T10:49:39.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 203, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-02-26T10:49:39.103 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 95, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 4, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 12, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 12, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 177, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 2, 2024-02-26T10:49:39.104 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 77 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 23, 2024-02-26T10:49:39.105 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 13, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 4595, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 2450, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 38896, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 10373070, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 38896, 2024-02-26T10:49:39.106 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 38896, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 10373070, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-02-26T10:49:39.107 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 8661, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 30598789, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 8661, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 30598789, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:39.108 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 8661, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 8661, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 8661, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 8661, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.109 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 2450, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 259810, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 2450, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 201, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 259810, 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:39.110 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x559fcae4e6e0": { 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 13599, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 29807187, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 13599, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:39.111 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2971, 2024-02-26T10:49:39.112 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 29807187, 2024-02-26T10:49:39.112 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:39.112 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:39.112 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:39.112 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:39.112 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:39.112 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:39.112 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-02-26T10:49:39.779 INFO:tasks.check_counter:Daemon mds.h mds.exported=75 2024-02-26T10:49:39.779 INFO:tasks.check_counter:Daemon mds.h mds.imported=101 2024-02-26T10:49:39.779 DEBUG:tasks.check_counter:Getting stats from k 2024-02-26T10:49:39.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-26T10:49:40.019 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T10:49:40.594 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:40 smithi100 ceph-mon[36051]: pgmap v536: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:40.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:40 smithi113 ceph-mon[36126]: pgmap v536: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:40.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:40 smithi060 ceph-mon[31539]: pgmap v536: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:41.884 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-02-26T10:49:41.884 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:41.884 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 129, 2024-02-26T10:49:41.884 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 124, 2024-02-26T10:49:41.884 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 722288, 2024-02-26T10:49:41.884 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 74192, 2024-02-26T10:49:41.884 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 0, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.023650762, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.007643583, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.015603164, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.002934692, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 123, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.006700905, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000054478 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:41.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 722288, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 74192 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 96, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 160, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 172064, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.006123265, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.004157310, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000895722, 2024-02-26T10:49:41.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 94, 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.008402048, 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000089383 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 172064 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:41.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 4, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 4, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 3024, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 448, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 2, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.000686243, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000049661, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.000641070, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000784307, 2024-02-26T10:49:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000261435 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 0, 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-02-26T10:49:41.889 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:41.890 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:41.891 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 148818, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 28, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 928, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 2, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-02-26T10:49:41.893 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 130, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 704084, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 130, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 130, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 704084, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-02-26T10:49:41.894 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-02-26T10:49:41.895 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-02-26T10:49:41.895 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:41.895 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-02-26T10:49:41.895 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-02-26T10:49:42.594 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-02-26T10:49:42.594 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-02-26T10:49:42.594 DEBUG:tasks.check_counter:Getting stats from c 2024-02-26T10:49:42.594 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-26T10:49:42.674 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:42 smithi100 ceph-mon[36051]: pgmap v537: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:42.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:42 smithi113 ceph-mon[36126]: pgmap v537: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:42.841 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:49:42.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:42 smithi060 ceph-mon[31539]: pgmap v537: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:44.626 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:44 smithi113 ceph-mon[36126]: pgmap v538: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 950, 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 937, 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 30367909, 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 330328, 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 2, 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.118912107, 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.030834041, 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.057504267, 2024-02-26T10:49:44.749 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.027386883, 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 936, 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.039488951, 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000042189 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 766144, 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 74016 2024-02-26T10:49:44.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 171, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 261, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 34578076, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 280982, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 5, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.031408755, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.007192087, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.018576542, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.003853566, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 260, 2024-02-26T10:49:44.751 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.017210498, 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000066194 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 228128 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 456, 2024-02-26T10:49:44.752 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 458, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 37864818, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 145015, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 5, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.057085280, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.013828960, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.026652715, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.012770657, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 456, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.022442785, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000049216 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:44.753 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-02-26T10:49:44.754 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2281, 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.209310633, 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000091762 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 0, 2024-02-26T10:49:44.755 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 0, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 0, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 0, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 9827, 2024-02-26T10:49:44.756 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 5924, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 3903, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 1365, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 2, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 0, 2024-02-26T10:49:44.757 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 4, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 234, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-02-26T10:49:44.758 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 0, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 0 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-02-26T10:49:44.759 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-02-26T10:49:44.760 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 0, 2024-02-26T10:49:44.761 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 50962, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 0, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 5, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 16, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 100032094, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 105834208, 2024-02-26T10:49:44.762 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 105830450, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 50962 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 26, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 12266, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 12240, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 21, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 1584, 2024-02-26T10:49:44.763 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 1563, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 9827, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 22215, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 12388, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 0, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 0, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 0, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 71576, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 215420 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 0, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 0, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 0, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 0, 2024-02-26T10:49:44.764 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 0, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.765 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-02-26T10:49:44.766 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.767 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.768 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.768 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.768 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-02-26T10:49:44.768 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.768 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.768 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.768 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.768 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-02-26T10:49:44.768 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.769 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.770 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.771 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.772 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.773 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.774 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-02-26T10:49:44.775 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 4, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 3, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 2, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 0, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 0, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 4075, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:44.776 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:44.777 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:44.778 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 222472, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 70, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 8984, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 172, 2024-02-26T10:49:44.779 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 7002568, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 40029, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 1428, 2024-02-26T10:49:44.780 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 26, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 4, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 1428, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1428, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 3.155927864, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002210033 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1428, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1429 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 1428, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 1428, 2024-02-26T10:49:44.781 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 886, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 539, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-02-26T10:49:44.782 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 2, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-02-26T10:49:44.783 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-02-26T10:49:44.784 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 95, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 4, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 9, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 9, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 0, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 2, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 0 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 0, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-02-26T10:49:44.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 0, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 0, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 0, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:44.786 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 1573, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 102658914, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 1573, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1573, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 102658914, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:44.787 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 1428, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 101644349, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1428, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 101644349, 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 1428, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 1428, 2024-02-26T10:49:44.789 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1428, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 1428, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:44.790 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 0, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 0, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x557db19e2960": { 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:44.791 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 0, 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 0, 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:44.792 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-02-26T10:49:44.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:44 smithi060 ceph-mon[31539]: pgmap v538: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:44.873 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:44 smithi100 ceph-mon[36051]: pgmap v538: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:45.552 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-02-26T10:49:45.552 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-02-26T10:49:45.553 DEBUG:tasks.check_counter:Getting stats from f 2024-02-26T10:49:45.553 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-26T10:49:45.802 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:49:46.683 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:46 smithi113 ceph-mon[36126]: pgmap v539: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:46.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:46 smithi060 ceph-mon[31539]: pgmap v539: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:46.873 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:46 smithi100 ceph-mon[36051]: pgmap v539: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:47.563 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-02-26T10:49:47.563 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:47.563 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 1043, 2024-02-26T10:49:47.563 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 1045, 2024-02-26T10:49:47.563 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 14794157, 2024-02-26T10:49:47.563 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 331681, 2024-02-26T10:49:47.563 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-02-26T10:49:47.563 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 2, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.098217938, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.030794220, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.029593068, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.029356215, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1043, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.041279413, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000039577 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.564 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 179, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 167, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 4969640, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 85990, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 4, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 3, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.029705319, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.008466114, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.020158712, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.004040736, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 166, 2024-02-26T10:49:47.565 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.010985951, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000066180 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 766256, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 74480 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 60, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 152, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 16650, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 325188, 2024-02-26T10:49:47.566 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 6, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 4, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.011238993, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.005312146, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.002788156, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.001692820, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 151, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.011600467, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000076824 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.567 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 308016 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1999, 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.051475284, 2024-02-26T10:49:47.568 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000025750 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 0, 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 0, 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-02-26T10:49:47.569 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 0, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 0, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 2475, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 0, 2024-02-26T10:49:47.570 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 2475, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 2475, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 1, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-02-26T10:49:47.571 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T10:49:47.572 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 0, 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 0 2024-02-26T10:49:47.573 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 0, 2024-02-26T10:49:47.574 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-02-26T10:49:47.575 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 8889, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 12, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 2, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-02-26T10:49:47.576 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 4194304, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 22882940, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 22881802, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 8889 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 47, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 2506, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 2459, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 45, 2024-02-26T10:49:47.577 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 450, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 405, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 2475, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 4865, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 2390, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 0, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 0, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 0, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 56136, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 215420 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 0, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 0, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 0, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 0, 2024-02-26T10:49:47.578 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 0, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.579 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.580 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.581 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.582 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.583 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.584 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-02-26T10:49:47.586 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-02-26T10:49:47.587 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-02-26T10:49:47.588 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 2, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 1, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 2, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 0, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 0, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 4078, 2024-02-26T10:49:47.589 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:47.590 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:47.591 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 198592, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 87, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:47.592 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 8984, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 172, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 1913128, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 14935, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-02-26T10:49:47.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 1133, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 21, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 3, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 1133, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1133, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1.222972014, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001079410 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1133, 2024-02-26T10:49:47.594 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 1134 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 1133, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 1133, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 0, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 642, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 0, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 489, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 0, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 0, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-02-26T10:49:47.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 1, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T10:49:47.596 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-02-26T10:49:47.597 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 95, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 4, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 6, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 6, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 0, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 2, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 0 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 0, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-02-26T10:49:47.598 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 0, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 0, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 0, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.599 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 1278, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 19655059, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 1278, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1278, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 19655059, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.600 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 1133, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 18689589, 2024-02-26T10:49:47.601 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1133, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 18689589, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:47.602 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 1133, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 1133, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1133, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 1133, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:47.603 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 0, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 0, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x55dda9810960": { 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-02-26T10:49:47.604 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:47.605 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-02-26T10:49:48.482 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-02-26T10:49:48.482 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-02-26T10:49:48.482 DEBUG:tasks.check_counter:Getting stats from i 2024-02-26T10:49:48.482 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-26T10:49:48.729 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:49:48.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:48 smithi113 ceph-mon[36126]: pgmap v540: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:48.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:48 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:49:48.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:48 smithi060 ceph-mon[31539]: pgmap v540: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:48.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:48 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:49:48.873 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:48 smithi100 ceph-mon[36051]: pgmap v540: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:48.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:48 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T10:49:50.495 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 9288, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 8239, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 2845979, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 17040038, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 23, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 6, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.530940226, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.208995204, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.272091111, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.083396667, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 8235, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.237634542, 2024-02-26T10:49:50.496 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000028856 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 11192, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.001523560, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000136 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 3328 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 9363, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 7494, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 4503500, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 2328273, 2024-02-26T10:49:50.497 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 9, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 5, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.501610985, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.188806492, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.349862694, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.042745742, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 7494, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.240750628, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000032125 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 13404, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.001863476, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000139 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.498 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 307552 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 9781, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 17185, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 4605172, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 12701310, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 7, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 6, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.757767528, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.334046538, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.409411023, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.083970755, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 17184, 2024-02-26T10:49:50.499 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.432680645, 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000025179 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 15313, 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.002694720, 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000175 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 3648208, 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 8762912 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 1, 2024-02-26T10:49:50.500 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-MDSRank": { 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 3457, 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 3.217669855, 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000930769 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "finisher-PurgeQueue": { 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "queue_len": 0, 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "complete_latency": { 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4477, 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.428815734, 2024-02-26T10:49:50.501 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000095781 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "mds": { 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "request": 4266, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "reply": 4228, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "reply_latency": { 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 4228, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 8.359934538, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001977278 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_reply": 0, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "forward": 38, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_complete": 0, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_fetch_keys": 0, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_commit": 11, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_split": 0, 2024-02-26T10:49:50.502 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_merge": 0, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes": 127, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_top": 47, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_bottom": 34, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pin_tail": 46, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_pinned": 57, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_expired": 593, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "inodes_with_caps": 1, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "caps": 1, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "subtrees": 5, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse": 6698, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_hit": 5909, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_forward": 33, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_discover": 206, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_dir_fetch": 0, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_remote_ino": 1, 2024-02-26T10:49:50.503 INFO:teuthology.orchestra.run.smithi113.stdout: "traverse_lock": 5, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "load_cent": 0, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "q": 0, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "exported": 72, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "exported_inodes": 41, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "imported": 97, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "imported_inodes": 1823, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_dir_fetch": 0, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_backtrace_fetch": 1397, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "openino_peer_discover": 10, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rfiles": 2862, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rbytes": 12460942, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "root_rsnaps": 0, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_fetch": 1397, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_set_tag": 0, 2024-02-26T10:49:50.504 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_inotable_repaired": 0, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_inodes": 43, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_dirfrag_rstats": 43, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_file_inodes": 1354, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_inode_file_caps": 39, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_revoke": 630, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_grant": 3051, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_trunc": 31, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps": 2352, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_caps_dirty": 1613, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_cap_release": 91, 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: "process_request_cap_release": 3387 2024-02-26T10:49:50.505 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_cache": { 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update": 0, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_update_receipt": 0, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_try_discover": 0, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_send_discover": 217, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "dir_handle_discover": 111, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays": 35, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_delayed": 0, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "num_strays_enqueuing": 0, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_created": 2224, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_enqueued": 2164, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_reintegrated": 20, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "strays_migrated": 8, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_processing": 0, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_enqueued": 0, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "num_recovering_prioritized": 0, 2024-02-26T10:49:50.506 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_started": 0, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "recovery_completed": 0, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_exportdir": 72, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_flush": 0, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragmentdir": 0, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_fragstats": 0, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "ireq_inodestats": 0 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_log": { 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "evlrg": 0, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "evadd": 8889, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "evex": 0, 2024-02-26T10:49:50.507 INFO:teuthology.orchestra.run.smithi113.stdout: "evtrm": 0, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "ev": 8889, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "evexg": 0, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "evexd": 0, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "segadd": 12, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "segex": 0, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "segtrm": 0, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "seg": 12, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "segmjr": 2, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "segexg": 0, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "segexd": 0, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "expos": 4194304, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "wrpos": 22882940, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "rdpos": 0, 2024-02-26T10:49:50.508 INFO:teuthology.orchestra.run.smithi113.stdout: "jlat": { 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1202, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 4.813098812, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.004004241 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "replayed": 0 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_mem": { 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "ino": 72, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "ino+": 2490, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "ino-": 2418, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "dir": 71, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "dir+": 427, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "dir-": 356, 2024-02-26T10:49:50.509 INFO:teuthology.orchestra.run.smithi113.stdout: "dn": 127, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "dn+": 5142, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "dn-": 5015, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "cap": 1, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "cap+": 3157, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "cap-": 3156, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "rss": 95324, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "heap": 207228 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_server": { 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_client_request": 6002, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "dispatch_server_request": 721, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_request": 4266, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_client_session": 139, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "handle_peer_request": 1237, 2024-02-26T10:49:50.510 INFO:teuthology.orchestra.run.smithi113.stdout: "req_create_latency": { 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 167, 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.230335383, 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001379253 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getattr_latency": { 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 33, 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.086109375, 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.002609375 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getfilelock_latency": { 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "req_link_latency": { 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 43, 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.279438574, 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.006498571 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookup_latency": { 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1075, 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.780955422, 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000726470 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookuphash_latency": { 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.512 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupino_latency": { 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 1, 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000129167, 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000129167 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupname_latency": { 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupparent_latency": { 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.513 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.514 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lookupsnap_latency": { 2024-02-26T10:49:50.514 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.514 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.514 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.514 INFO:teuthology.orchestra.run.smithi113.stdout: "req_lssnap_latency": { 2024-02-26T10:49:50.514 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.514 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.514 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mkdir_latency": { 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 90, 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.094252677, 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001047251 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mknod_latency": { 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 77, 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.088921800, 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001154828 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.515 INFO:teuthology.orchestra.run.smithi113.stdout: "req_mksnap_latency": { 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: "req_open_latency": { 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 3, 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.002507040, 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000835680 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: "req_readdir_latency": { 2024-02-26T10:49:50.516 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 238, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.200813365, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000843753 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rename_latency": { 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 95, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.991934948, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.010441420 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "req_renamesnap_latency": { 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "req_snapdiff_latency": { 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.517 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmdir_latency": { 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 182, 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 2.217723698, 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.012185295 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmsnap_latency": { 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.518 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "req_rmxattr_latency": { 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setattr_latency": { 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 31, 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.044095754, 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001422443 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setdirlayout_latency": { 2024-02-26T10:49:50.519 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setfilelock_latency": { 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setlayout_latency": { 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.520 INFO:teuthology.orchestra.run.smithi113.stdout: "req_setxattr_latency": { 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "req_symlink_latency": { 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 90, 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.095086919, 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001056521 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "req_unlink_latency": { 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 2103, 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 3.247630416, 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.001544284 2024-02-26T10:49:50.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_revoke_eviction": 0, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "cap_acquisition_throttle": 0, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "req_getvxattr_latency": { 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_sessions": { 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "session_count": 1, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "session_add": 2, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "session_remove": 1, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_open": 1, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "sessions_stale": 0, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "total_load": 2826, 2024-02-26T10:49:50.522 INFO:teuthology.orchestra.run.smithi113.stdout: "average_load": 2826, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "avg_session_uptime": 239, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "mdthresh_evicted": 0 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:50.523 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:50.524 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:50.525 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:50.525 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:50.525 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:50.525 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:50.525 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:50.525 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:50.525 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:50.525 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-02-26T10:49:50.525 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 293254, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 150, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:50.527 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 8984, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 172, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 356971, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 6859, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-02-26T10:49:50.528 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "objecter": { 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "op_active": 0, 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "op_laggy": 0, 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send": 7325, 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "op_send_bytes": 19157618, 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "op_resend": 27, 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "op_reply": 7325, 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "op_latency": { 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 7325, 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 43.018008987, 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.005872765 2024-02-26T10:49:50.529 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "op_inflight": 0, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "oplen_avg": { 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 7325, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 7486 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "op": 7325, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "op_r": 1531, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "op_w": 5794, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "op_rmw": 0, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "op_pg": 0, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_stat": 0, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_create": 10, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_read": 134, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_write": 1336, 2024-02-26T10:49:50.530 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writefull": 83, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_writesame": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_append": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_zero": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_truncate": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_delete": 4227, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_mapext": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_sparse_read": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_clonerange": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_getxattr": 1397, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_setxattr": 20, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_cmpxattr": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_rmxattr": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_resetxattrs": 0, 2024-02-26T10:49:50.531 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_call": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_watch": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_notify": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_pgls_filter": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "osdop_other": 34, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_active": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_send": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_resend": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "linger_ping": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_active": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_send": 0, 2024-02-26T10:49:50.532 INFO:teuthology.orchestra.run.smithi113.stdout: "poolop_resend": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_active": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_send": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "poolstat_resend": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_active": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_send": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "statfs_resend": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "command_active": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "command_send": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "command_resend": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "map_epoch": 95, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "map_full": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "map_inc": 4, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_sessions": 12, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_open": 12, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_session_close": 0, 2024-02-26T10:49:50.533 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_laggy": 0, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_wr": 175, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_rd": 0, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_del": 70 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "oft": { 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_objs": 1, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_kv_pairs": 0, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_updates": 0, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "omap_total_removes": 0 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "osd-slow-ops": { 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "slow_ops_count": 0 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "purge_queue": { 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops": 0, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_ops_high_water": 23, 2024-02-26T10:49:50.534 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing": 0, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executing_high_water": 13, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed_ops": 4211, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_executed": 2164, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "pq_item_in_journal": 0 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 28428, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 9077117, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 28428, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:50.535 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 28428, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 9077117, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_bytes": { 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:50.536 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:50.537 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:50 smithi113 ceph-mon[36126]: pgmap v541: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:50.537 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:49:50.537 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:49:50.537 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:50 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:49:50.537 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:50.537 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 7325, 2024-02-26T10:49:50.537 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 19391046, 2024-02-26T10:49:50.537 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 7325, 2024-02-26T10:49:50.537 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 19391046, 2024-02-26T10:49:50.537 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-objecter_ops": { 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 1024, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 0, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 0, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 0, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 7325, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 7325, 2024-02-26T10:49:50.538 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 7325, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 7325, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle": { 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 2164, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 233428, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 2164, 2024-02-26T10:49:50.539 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 134, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 233428, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-write_buf_throttle-0x558449e806e0": { 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 3758096384, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:50.540 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 8889, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 18688636, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 8889, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 1202, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 18688636, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:50.541 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-02-26T10:49:50.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:50 smithi060 ceph-mon[31539]: pgmap v541: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:50.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:49:50.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:49:50.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:50 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:49:50.873 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:50 smithi100 ceph-mon[36051]: pgmap v541: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:50.873 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T10:49:50.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T10:49:50.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:50 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' 2024-02-26T10:49:51.301 INFO:tasks.check_counter:Daemon mds.i mds.exported=72 2024-02-26T10:49:51.302 INFO:tasks.check_counter:Daemon mds.i mds.imported=97 2024-02-26T10:49:51.302 DEBUG:tasks.check_counter:Getting stats from l 2024-02-26T10:49:51.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893 shell --fsid 51464d50-d492-11ee-95c3-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-26T10:49:51.541 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T10:49:52.660 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:52 smithi113 ceph-mon[36126]: pgmap v542: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:52.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:52 smithi060 ceph-mon[31539]: pgmap v542: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:52.873 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:52 smithi100 ceph-mon[36051]: pgmap v542: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:53.226 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-02-26T10:49:53.226 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T10:49:53.226 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 4, 2024-02-26T10:49:53.226 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 4, 2024-02-26T10:49:53.226 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 3024, 2024-02-26T10:49:53.226 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 448, 2024-02-26T10:49:53.226 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 4, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 0, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.001354347, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.000041079, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.000537535, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 3, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000646107, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000215369 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:53.227 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 2, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 100, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 320, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 172592, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 4, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 2, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.005996194, 2024-02-26T10:49:53.228 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.004057753, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.000758768, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 97, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.008141176, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000083929 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-26T10:49:53.229 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 172592 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 133, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 128, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 723376, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 76688, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 2, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 1, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.024659201, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.007956602, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.015996720, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.003474278, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 127, 2024-02-26T10:49:53.230 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.006974806, 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000054919 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 723376, 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 76688 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 0, 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.231 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T10:49:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-02-26T10:49:53.233 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 148841, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 28, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-02-26T10:49:53.234 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 928, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 2, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-02-26T10:49:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 135, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 704631, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 135, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 135, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 704631, 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-02-26T10:49:53.236 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-02-26T10:49:53.237 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-02-26T10:49:53.237 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-02-26T10:49:53.237 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:53.237 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-02-26T10:49:53.237 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-02-26T10:49:53.948 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-02-26T10:49:53.948 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-02-26T10:49:53.949 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-26T10:49:53.965 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-26T10:49:53.965 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:49:53.965 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:49:54.028 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:49:54.029 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:49:54.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:49:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.330+0000 7f2cfb200640 1 -- 172.21.15.60:0/860293647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf4073cc0 msgr2=0x7f2cf4074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.330+0000 7f2cfb200640 1 --2- 172.21.15.60:0/860293647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf4073cc0 0x7f2cf4074140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009860 tx=0x7f2ce402bb70 comp rx=0 tx=0).stop 2024-02-26T10:49:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.330+0000 7f2cfb200640 1 -- 172.21.15.60:0/860293647 shutdown_connections 2024-02-26T10:49:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.330+0000 7f2cfb200640 1 --2- 172.21.15.60:0/860293647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf4108f60 0x7f2cf410b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.330+0000 7f2cfb200640 1 --2- 172.21.15.60:0/860293647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf4073cc0 0x7f2cf4074140 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.330+0000 7f2cfb200640 1 --2- 172.21.15.60:0/860293647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf40752a0 0x7f2cf40736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.330+0000 7f2cfb200640 1 -- 172.21.15.60:0/860293647 >> 172.21.15.60:0/860293647 conn(0x7f2cf40fc5f0 msgr2=0x7f2cf40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.331+0000 7f2cfb200640 1 -- 172.21.15.60:0/860293647 shutdown_connections 2024-02-26T10:49:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.331+0000 7f2cfb200640 1 -- 172.21.15.60:0/860293647 wait complete. 2024-02-26T10:49:54.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.332+0000 7f2cfb200640 1 Processor -- start 2024-02-26T10:49:54.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.332+0000 7f2cfb200640 1 -- start start 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.332+0000 7f2cfb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf4073cc0 0x7f2cf41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.332+0000 7f2cfb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf40752a0 0x7f2cf4199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.332+0000 7f2cfb200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf4108f60 0x7f2cf41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.332+0000 7f2cfb200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf410ef20 con 0x7f2cf4108f60 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.332+0000 7f2cfb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf410eda0 con 0x7f2cf40752a0 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cfb200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf410f0a0 con 0x7f2cf4073cc0 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf40752a0 0x7f2cf4199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf4073cc0 0x7f2cf41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf40752a0 0x7f2cf4199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:46920/0 (socket says 172.21.15.60:46920) 2024-02-26T10:49:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9400640 1 -- 172.21.15.60:0/1075687429 learned_addr learned my addr 172.21.15.60:0/1075687429 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cfa800640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf4108f60 0x7f2cf41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9e00640 1 -- 172.21.15.60:0/1075687429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf40752a0 msgr2=0x7f2cf4199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9e00640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf40752a0 0x7f2cf4199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9e00640 1 -- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf4108f60 msgr2=0x7f2cf41a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9e00640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf4108f60 0x7f2cf41a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9e00640 1 -- 172.21.15.60:0/1075687429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf41a11c0 con 0x7f2cf4073cc0 2024-02-26T10:49:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9400640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf40752a0 0x7f2cf4199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:49:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.333+0000 7f2cf9e00640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf4073cc0 0x7f2cf41995a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800e790 tx=0x7f2ce800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:54.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.334+0000 7f2ceea00640 1 -- 172.21.15.60:0/1075687429 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800eea0 con 0x7f2cf4073cc0 2024-02-26T10:49:54.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.334+0000 7f2ceea00640 1 -- 172.21.15.60:0/1075687429 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ce800ccc0 con 0x7f2cf4073cc0 2024-02-26T10:49:54.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.334+0000 7f2ceea00640 1 -- 172.21.15.60:0/1075687429 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8018780 con 0x7f2cf4073cc0 2024-02-26T10:49:54.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.334+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf41a14b0 con 0x7f2cf4073cc0 2024-02-26T10:49:54.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.334+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf41a1a00 con 0x7f2cf4073cc0 2024-02-26T10:49:54.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.335+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf4107c90 con 0x7f2cf4073cc0 2024-02-26T10:49:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.337+0000 7f2ceea00640 1 -- 172.21.15.60:0/1075687429 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2ce8010430 con 0x7f2cf4073cc0 2024-02-26T10:49:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.338+0000 7f2ceea00640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2cdc0a1d80 0x7f2cdc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.338+0000 7f2ceea00640 1 -- 172.21.15.60:0/1075687429 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f2ce80ce360 con 0x7f2cf4073cc0 2024-02-26T10:49:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.338+0000 7f2cf9400640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2cdc0a1d80 0x7f2cdc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.339+0000 7f2cf9400640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2cdc0a1d80 0x7f2cdc0a4240 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2cf419aab0 tx=0x7f2ce402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.342+0000 7f2ceea00640 1 -- 172.21.15.60:0/1075687429 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2ce809ad30 con 0x7f2cf4073cc0 2024-02-26T10:49:54.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.547+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cf4061e60 con 0x7f2cf4073cc0 2024-02-26T10:49:54.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.550+0000 7f2ceea00640 1 -- 172.21.15.60:0/1075687429 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7f2ce809edd0 con 0x7f2cf4073cc0 2024-02-26T10:49:54.543 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:49:54.543 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:49:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2cdc0a1d80 msgr2=0x7f2cdc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2cdc0a1d80 0x7f2cdc0a4240 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2cf419aab0 tx=0x7f2ce402e040 comp rx=0 tx=0).stop 2024-02-26T10:49:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf4073cc0 msgr2=0x7f2cf41995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf4073cc0 0x7f2cf41995a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800e790 tx=0x7f2ce800ec50 comp rx=0 tx=0).stop 2024-02-26T10:49:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 shutdown_connections 2024-02-26T10:49:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f2cdc0a1d80 0x7f2cdc0a4240 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf40752a0 0x7f2cf4199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf4108f60 0x7f2cf41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 --2- 172.21.15.60:0/1075687429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cf4073cc0 0x7f2cf41995a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 >> 172.21.15.60:0/1075687429 conn(0x7f2cf40fc5f0 msgr2=0x7f2cf4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.552+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 shutdown_connections 2024-02-26T10:49:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.553+0000 7f2cfb200640 1 -- 172.21.15.60:0/1075687429 wait complete. 2024-02-26T10:49:54.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:54 smithi060 ceph-mon[31539]: pgmap v543: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:54.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:37:25.424456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:42:40.368004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:42:41.581951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T10:42:45.736477+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T10:49:54.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:49:54.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.800+0000 7fbcfd600640 1 -- 172.21.15.60:0/1687702116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf8107880 msgr2=0x7fbcf8107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.800+0000 7fbcfd600640 1 --2- 172.21.15.60:0/1687702116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf8107880 0x7fbcf8107ca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009900 tx=0x7fbce802bc30 comp rx=0 tx=0).stop 2024-02-26T10:49:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.800+0000 7fbcfd600640 1 -- 172.21.15.60:0/1687702116 shutdown_connections 2024-02-26T10:49:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.800+0000 7fbcfd600640 1 --2- 172.21.15.60:0/1687702116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8109430 0x7fbcf810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.800+0000 7fbcfd600640 1 --2- 172.21.15.60:0/1687702116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcf8108a70 0x7fbcf8108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.800+0000 7fbcfd600640 1 --2- 172.21.15.60:0/1687702116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf8107880 0x7fbcf8107ca0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.800+0000 7fbcfd600640 1 -- 172.21.15.60:0/1687702116 >> 172.21.15.60:0/1687702116 conn(0x7fbcf8076820 msgr2=0x7fbcf8078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.800+0000 7fbcfd600640 1 -- 172.21.15.60:0/1687702116 shutdown_connections 2024-02-26T10:49:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.801+0000 7fbcfd600640 1 -- 172.21.15.60:0/1687702116 wait complete. 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.802+0000 7fbcfd600640 1 Processor -- start 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.802+0000 7fbcfd600640 1 -- start start 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.802+0000 7fbcfd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8107880 0x7fbcf819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.802+0000 7fbcfd600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcf8108a70 0x7fbcf819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.802+0000 7fbcfd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf8109430 0x7fbcf81a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.802+0000 7fbcfd600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8113550 con 0x7fbcf8108a70 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.802+0000 7fbcfd600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf81133d0 con 0x7fbcf8107880 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.802+0000 7fbcfd600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf81136d0 con 0x7fbcf8109430 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcf8108a70 0x7fbcf819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcf8108a70 0x7fbcf819e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48602/0 (socket says 172.21.15.60:48602) 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf6000640 1 -- 172.21.15.60:0/3410891197 learned_addr learned my addr 172.21.15.60:0/3410891197 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf6a00640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8107880 0x7fbcf819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf7400640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf8109430 0x7fbcf81a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf6000640 1 -- 172.21.15.60:0/3410891197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf8109430 msgr2=0x7fbcf81a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf6000640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf8109430 0x7fbcf81a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf6000640 1 -- 172.21.15.60:0/3410891197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8107880 msgr2=0x7fbcf819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf6000640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8107880 0x7fbcf819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf6000640 1 -- 172.21.15.60:0/3410891197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf81a5780 con 0x7fbcf8108a70 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.803+0000 7fbcf7400640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf8109430 0x7fbcf81a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.804+0000 7fbcf6000640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcf8108a70 0x7fbcf819e0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbce400e9f0 tx=0x7fbce400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.804+0000 7fbcef400640 1 -- 172.21.15.60:0/3410891197 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce400cd80 con 0x7fbcf8108a70 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.804+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf81a5a70 con 0x7fbcf8108a70 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.804+0000 7fbcef400640 1 -- 172.21.15.60:0/3410891197 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbce4010910 con 0x7fbcf8108a70 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.804+0000 7fbcef400640 1 -- 172.21.15.60:0/3410891197 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce40188f0 con 0x7fbcf8108a70 2024-02-26T10:49:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.804+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf81a5fc0 con 0x7fbcf8108a70 2024-02-26T10:49:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.805+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf810eba0 con 0x7fbcf8108a70 2024-02-26T10:49:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.808+0000 7fbcef400640 1 -- 172.21.15.60:0/3410891197 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbce4010ac0 con 0x7fbcf8108a70 2024-02-26T10:49:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.809+0000 7fbcef400640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcdc0a1d80 0x7fbcdc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.809+0000 7fbcef400640 1 -- 172.21.15.60:0/3410891197 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fbce40ce5d0 con 0x7fbcf8108a70 2024-02-26T10:49:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.809+0000 7fbcf6a00640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcdc0a1d80 0x7fbcdc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.810+0000 7fbcf6a00640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcdc0a1d80 0x7fbcdc0a4240 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbce802c170 tx=0x7fbce802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:54.814+0000 7fbcef400640 1 -- 172.21.15.60:0/3410891197 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbce409b5f0 con 0x7fbcf8108a70 2024-02-26T10:49:54.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:54 smithi113 ceph-mon[36126]: pgmap v543: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:54.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:54 smithi100 ceph-mon[36051]: pgmap v543: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:55.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.014+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbcf8061e60 con 0x7fbcf8108a70 2024-02-26T10:49:55.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.015+0000 7fbcef400640 1 -- 172.21.15.60:0/3410891197 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbce409aef0 con 0x7fbcf8108a70 2024-02-26T10:49:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcdc0a1d80 msgr2=0x7fbcdc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcdc0a1d80 0x7fbcdc0a4240 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbce802c170 tx=0x7fbce802df90 comp rx=0 tx=0).stop 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcf8108a70 msgr2=0x7fbcf819e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcf8108a70 0x7fbcf819e0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbce400e9f0 tx=0x7fbce400eeb0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 shutdown_connections 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbcdc0a1d80 0x7fbcdc0a4240 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcf8109430 0x7fbcf81a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcf8108a70 0x7fbcf819e0a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 --2- 172.21.15.60:0/3410891197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcf8107880 0x7fbcf819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 >> 172.21.15.60:0/3410891197 conn(0x7fbcf8076820 msgr2=0x7fbcf810bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 shutdown_connections 2024-02-26T10:49:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.018+0000 7fbcfd600640 1 -- 172.21.15.60:0/3410891197 wait complete. 2024-02-26T10:49:55.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:49:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.271+0000 7fdca6a00640 1 -- 172.21.15.60:0/3996280646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0107880 msgr2=0x7fdca0107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.271+0000 7fdca6a00640 1 --2- 172.21.15.60:0/3996280646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0107880 0x7fdca0107ca0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009be0 tx=0x7fdc9002bef0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.271+0000 7fdca6a00640 1 -- 172.21.15.60:0/3996280646 shutdown_connections 2024-02-26T10:49:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.271+0000 7fdca6a00640 1 --2- 172.21.15.60:0/3996280646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdca0109430 0x7fdca010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.271+0000 7fdca6a00640 1 --2- 172.21.15.60:0/3996280646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdca0108a70 0x7fdca0108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.271+0000 7fdca6a00640 1 --2- 172.21.15.60:0/3996280646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0107880 0x7fdca0107ca0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.271+0000 7fdca6a00640 1 -- 172.21.15.60:0/3996280646 >> 172.21.15.60:0/3996280646 conn(0x7fdca0076820 msgr2=0x7fdca0078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.271+0000 7fdca6a00640 1 -- 172.21.15.60:0/3996280646 shutdown_connections 2024-02-26T10:49:55.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.271+0000 7fdca6a00640 1 -- 172.21.15.60:0/3996280646 wait complete. 2024-02-26T10:49:55.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.272+0000 7fdca6a00640 1 Processor -- start 2024-02-26T10:49:55.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.272+0000 7fdca6a00640 1 -- start start 2024-02-26T10:49:55.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdca6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdca0107880 0x7fdca019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:55.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdca6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0108a70 0x7fdca019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdca6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdca0109430 0x7fdca01a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdca6a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca0113550 con 0x7fdca0109430 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdca6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca01133d0 con 0x7fdca0107880 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdca6a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca01136d0 con 0x7fdca0108a70 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdc9f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0108a70 0x7fdca019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdc9fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdca0107880 0x7fdca019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdc9f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0108a70 0x7fdca019e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:52132/0 (socket says 172.21.15.60:52132) 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdc9f400640 1 -- 172.21.15.60:0/2223379283 learned_addr learned my addr 172.21.15.60:0/2223379283 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.273+0000 7fdca4a00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdca0109430 0x7fdca01a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdc9f400640 1 -- 172.21.15.60:0/2223379283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdca0107880 msgr2=0x7fdca019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdc9f400640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdca0107880 0x7fdca019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdc9f400640 1 -- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdca0109430 msgr2=0x7fdca01a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdc9f400640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdca0109430 0x7fdca01a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdc9f400640 1 -- 172.21.15.60:0/2223379283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca01a5780 con 0x7fdca0108a70 2024-02-26T10:49:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdc9f400640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0108a70 0x7fdca019e0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800e9c0 tx=0x7fdc8800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdc9cc00640 1 -- 172.21.15.60:0/2223379283 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8800ccc0 con 0x7fdca0108a70 2024-02-26T10:49:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca01a5a70 con 0x7fdca0108a70 2024-02-26T10:49:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdc9cc00640 1 -- 172.21.15.60:0/2223379283 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc88005500 con 0x7fdca0108a70 2024-02-26T10:49:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.274+0000 7fdc9cc00640 1 -- 172.21.15.60:0/2223379283 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88010430 con 0x7fdca0108a70 2024-02-26T10:49:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.275+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca01a5fc0 con 0x7fdca0108a70 2024-02-26T10:49:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.276+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca010eba0 con 0x7fdca0108a70 2024-02-26T10:49:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.279+0000 7fdc9cc00640 1 -- 172.21.15.60:0/2223379283 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fdc880046c0 con 0x7fdca0108a70 2024-02-26T10:49:55.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.280+0000 7fdc9cc00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdc840a1d80 0x7fdc840a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:55.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.281+0000 7fdc9cc00640 1 -- 172.21.15.60:0/2223379283 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fdc880ce450 con 0x7fdca0108a70 2024-02-26T10:49:55.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.281+0000 7fdc9fe00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdc840a1d80 0x7fdc840a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:55.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.281+0000 7fdc9fe00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdc840a1d80 0x7fdc840a4240 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdc90034fd0 tx=0x7fdc90000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:55.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.284+0000 7fdc9cc00640 1 -- 172.21.15.60:0/2223379283 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc8809b4d0 con 0x7fdca0108a70 2024-02-26T10:49:55.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.489+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdca0061e60 con 0x7fdca0108a70 2024-02-26T10:49:55.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.490+0000 7fdc9cc00640 1 -- 172.21.15.60:0/2223379283 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdc8809add0 con 0x7fdca0108a70 2024-02-26T10:49:55.483 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:49:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.492+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdc840a1d80 msgr2=0x7fdc840a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.492+0000 7fdca6a00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdc840a1d80 0x7fdc840a4240 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdc90034fd0 tx=0x7fdc90000f40 comp rx=0 tx=0).stop 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.492+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0108a70 msgr2=0x7fdca019e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.492+0000 7fdca6a00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0108a70 0x7fdca019e0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800e9c0 tx=0x7fdc8800ee80 comp rx=0 tx=0).stop 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.493+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 shutdown_connections 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.493+0000 7fdca6a00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fdc840a1d80 0x7fdc840a4240 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.493+0000 7fdca6a00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdca0109430 0x7fdca01a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.493+0000 7fdca6a00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdca0108a70 0x7fdca019e0a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.493+0000 7fdca6a00640 1 --2- 172.21.15.60:0/2223379283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdca0107880 0x7fdca019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.493+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 >> 172.21.15.60:0/2223379283 conn(0x7fdca0076820 msgr2=0x7fdca010bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.493+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 shutdown_connections 2024-02-26T10:49:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.493+0000 7fdca6a00640 1 -- 172.21.15.60:0/2223379283 wait complete. 2024-02-26T10:49:55.504 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:49:55.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.741+0000 7f495d200640 1 -- 172.21.15.60:0/121240954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958104570 msgr2=0x7f49581049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.741+0000 7f495d200640 1 --2- 172.21.15.60:0/121240954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958104570 0x7f49581049f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4944009860 tx=0x7f494402bb70 comp rx=0 tx=0).stop 2024-02-26T10:49:55.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.741+0000 7f495d200640 1 -- 172.21.15.60:0/121240954 shutdown_connections 2024-02-26T10:49:55.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.741+0000 7f495d200640 1 --2- 172.21.15.60:0/121240954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958104570 0x7f49581049f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.741+0000 7f495d200640 1 --2- 172.21.15.60:0/121240954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4958104f30 0x7f495810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.741+0000 7f495d200640 1 --2- 172.21.15.60:0/121240954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4958103380 0x7f49581037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.741+0000 7f495d200640 1 -- 172.21.15.60:0/121240954 >> 172.21.15.60:0/121240954 conn(0x7f49580fe910 msgr2=0x7f4958100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:55.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.741+0000 7f495d200640 1 -- 172.21.15.60:0/121240954 shutdown_connections 2024-02-26T10:49:55.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.741+0000 7f495d200640 1 -- 172.21.15.60:0/121240954 wait complete. 2024-02-26T10:49:55.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.742+0000 7f495d200640 1 Processor -- start 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f495d200640 1 -- start start 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f495d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958103380 0x7f4958199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f495d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4958104570 0x7f4958199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f495d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4958104f30 0x7f49581a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f495d200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495810f010 con 0x7f4958104f30 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f495d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495810ee90 con 0x7f4958103380 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f495d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495810f190 con 0x7f4958104570 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f4957400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958103380 0x7f4958199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f4957400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958103380 0x7f4958199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:46970/0 (socket says 172.21.15.60:46970) 2024-02-26T10:49:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.743+0000 7f4957400640 1 -- 172.21.15.60:0/1142308168 learned_addr learned my addr 172.21.15.60:0/1142308168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f4956a00640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4958104570 0x7f4958199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f4957e00640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4958104f30 0x7f49581a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f4957400640 1 -- 172.21.15.60:0/1142308168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4958104570 msgr2=0x7f4958199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f4957400640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4958104570 0x7f4958199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f4957400640 1 -- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4958104f30 msgr2=0x7f49581a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f4957400640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4958104f30 0x7f49581a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f4957400640 1 -- 172.21.15.60:0/1142308168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49581a1260 con 0x7f4958103380 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f4956a00640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4958104570 0x7f4958199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f4957400640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958103380 0x7f4958199640 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f494800c3a0 tx=0x7f494800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f494fe00640 1 -- 172.21.15.60:0/1142308168 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948003970 con 0x7f4958103380 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f494fe00640 1 -- 172.21.15.60:0/1142308168 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4948004020 con 0x7f4958103380 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49581a1550 con 0x7f4958103380 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.744+0000 7f494fe00640 1 -- 172.21.15.60:0/1142308168 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800b9f0 con 0x7f4958103380 2024-02-26T10:49:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.745+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49581a1a70 con 0x7f4958103380 2024-02-26T10:49:55.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.746+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495810a720 con 0x7f4958103380 2024-02-26T10:49:55.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.748+0000 7f494fe00640 1 -- 172.21.15.60:0/1142308168 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f494801d020 con 0x7f4958103380 2024-02-26T10:49:55.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.749+0000 7f494fe00640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f493c0a1d80 0x7f493c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:55.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.749+0000 7f494fe00640 1 -- 172.21.15.60:0/1142308168 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f49480ce3d0 con 0x7f4958103380 2024-02-26T10:49:55.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.749+0000 7f4956a00640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f493c0a1d80 0x7f493c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.750+0000 7f4956a00640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f493c0a1d80 0x7f493c0a4240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4944009830 tx=0x7f4944005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.754+0000 7f494fe00640 1 -- 172.21.15.60:0/1142308168 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f494809b3f0 con 0x7f4958103380 2024-02-26T10:49:55.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:55 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1075687429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:49:55.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:55 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/3410891197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:49:55.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:55 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1075687429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:49:55.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:55 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/3410891197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:49:55.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:55 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1075687429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:49:55.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:55 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/3410891197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T10:49:55.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.959+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4958061e60 con 0x7f4958103380 2024-02-26T10:49:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.960+0000 7f494fe00640 1 -- 172.21.15.60:0/1142308168 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f494809acf0 con 0x7f4958103380 2024-02-26T10:49:55.953 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T10:49:55.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f493c0a1d80 msgr2=0x7f493c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f493c0a1d80 0x7f493c0a4240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4944009830 tx=0x7f4944005d20 comp rx=0 tx=0).stop 2024-02-26T10:49:55.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958103380 msgr2=0x7f4958199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:55.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958103380 0x7f4958199640 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f494800c3a0 tx=0x7f494800c860 comp rx=0 tx=0).stop 2024-02-26T10:49:55.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 shutdown_connections 2024-02-26T10:49:55.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f493c0a1d80 0x7f493c0a4240 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4958104f30 0x7f49581a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4958104570 0x7f4958199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 --2- 172.21.15.60:0/1142308168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4958103380 0x7f4958199640 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:55.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 >> 172.21.15.60:0/1142308168 conn(0x7f49580fe910 msgr2=0x7f49581078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:55.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 shutdown_connections 2024-02-26T10:49:55.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:55.963+0000 7f495d200640 1 -- 172.21.15.60:0/1142308168 wait complete. 2024-02-26T10:49:55.977 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T10:49:56.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.213+0000 7fa798c00640 1 -- 172.21.15.60:0/3656670724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794102460 msgr2=0x7fa7941028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.213+0000 7fa798c00640 1 --2- 172.21.15.60:0/3656670724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794102460 0x7fa7941028e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009900 tx=0x7fa77c02bc50 comp rx=0 tx=0).stop 2024-02-26T10:49:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.213+0000 7fa798c00640 1 -- 172.21.15.60:0/3656670724 shutdown_connections 2024-02-26T10:49:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.213+0000 7fa798c00640 1 --2- 172.21.15.60:0/3656670724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa794102e20 0x7fa7941096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.213+0000 7fa798c00640 1 --2- 172.21.15.60:0/3656670724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794102460 0x7fa7941028e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.213+0000 7fa798c00640 1 --2- 172.21.15.60:0/3656670724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa794101270 0x7fa794101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.213+0000 7fa798c00640 1 -- 172.21.15.60:0/3656670724 >> 172.21.15.60:0/3656670724 conn(0x7fa7940fc800 msgr2=0x7fa7940fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.214+0000 7fa798c00640 1 -- 172.21.15.60:0/3656670724 shutdown_connections 2024-02-26T10:49:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.214+0000 7fa798c00640 1 -- 172.21.15.60:0/3656670724 wait complete. 2024-02-26T10:49:56.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa798c00640 1 Processor -- start 2024-02-26T10:49:56.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa798c00640 1 -- start start 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa798c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794101270 0x7fa794199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa798c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa794102460 0x7fa794199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa798c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa794102e20 0x7fa7941a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa798c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79410cf50 con 0x7fa794101270 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa798c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79410cdd0 con 0x7fa794102460 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa798c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79410d0d0 con 0x7fa794102e20 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa791e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794101270 0x7fa794199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa791e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794101270 0x7fa794199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48672/0 (socket says 172.21.15.60:48672) 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa791e00640 1 -- 172.21.15.60:0/1631933287 learned_addr learned my addr 172.21.15.60:0/1631933287 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.215+0000 7fa78be00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa794102460 0x7fa794199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.216+0000 7fa792800640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa794102e20 0x7fa7941a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.216+0000 7fa791e00640 1 -- 172.21.15.60:0/1631933287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa794102e20 msgr2=0x7fa7941a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.216+0000 7fa791e00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa794102e20 0x7fa7941a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.216+0000 7fa791e00640 1 -- 172.21.15.60:0/1631933287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa794102460 msgr2=0x7fa794199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.216+0000 7fa791e00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa794102460 0x7fa794199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.216+0000 7fa791e00640 1 -- 172.21.15.60:0/1631933287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7941a1380 con 0x7fa794101270 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.216+0000 7fa791e00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794101270 0x7fa794199760 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa78400e9c0 tx=0x7fa78400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.216+0000 7fa78aa00640 1 -- 172.21.15.60:0/1631933287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78400cd80 con 0x7fa794101270 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.217+0000 7fa78aa00640 1 -- 172.21.15.60:0/1631933287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7840035f0 con 0x7fa794101270 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.217+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7941a1670 con 0x7fa794101270 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.217+0000 7fa78aa00640 1 -- 172.21.15.60:0/1631933287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784010690 con 0x7fa794101270 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.217+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7941a1b90 con 0x7fa794101270 2024-02-26T10:49:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.218+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7941a1e50 con 0x7fa794101270 2024-02-26T10:49:56.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.220+0000 7fa78aa00640 1 -- 172.21.15.60:0/1631933287 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa7840037a0 con 0x7fa794101270 2024-02-26T10:49:56.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.221+0000 7fa78aa00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa7780a1d80 0x7fa7780a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:56.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.222+0000 7fa78aa00640 1 -- 172.21.15.60:0/1631933287 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fa7840ce540 con 0x7fa794101270 2024-02-26T10:49:56.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.222+0000 7fa78be00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa7780a1d80 0x7fa7780a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:56.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.222+0000 7fa78be00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa7780a1d80 0x7fa7780a4240 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa77c02c260 tx=0x7fa77c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:56.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.226+0000 7fa78aa00640 1 -- 172.21.15.60:0/1631933287 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa78409b560 con 0x7fa794101270 2024-02-26T10:49:56.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.434+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa7941a2680 con 0x7fa794101270 2024-02-26T10:49:56.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.435+0000 7fa78aa00640 1 -- 172.21.15.60:0/1631933287 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa78409ae60 con 0x7fa794101270 2024-02-26T10:49:56.428 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.437+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa7780a1d80 msgr2=0x7fa7780a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.437+0000 7fa798c00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa7780a1d80 0x7fa7780a4240 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa77c02c260 tx=0x7fa77c000bc0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.437+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794101270 msgr2=0x7fa794199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.437+0000 7fa798c00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794101270 0x7fa794199760 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa78400e9c0 tx=0x7fa78400ee80 comp rx=0 tx=0).stop 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.438+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 shutdown_connections 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.438+0000 7fa798c00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa7780a1d80 0x7fa7780a4240 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.438+0000 7fa798c00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa794102460 0x7fa794199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.438+0000 7fa798c00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa794102e20 0x7fa7941a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.438+0000 7fa798c00640 1 --2- 172.21.15.60:0/1631933287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa794101270 0x7fa794199760 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.438+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 >> 172.21.15.60:0/1631933287 conn(0x7fa7940fc800 msgr2=0x7fa794073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.438+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 shutdown_connections 2024-02-26T10:49:56.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.438+0000 7fa798c00640 1 -- 172.21.15.60:0/1631933287 wait complete. 2024-02-26T10:49:56.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:49:56.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.682+0000 7ff12ca00640 1 -- 172.21.15.60:0/1230493004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128101c20 msgr2=0x7ff128104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.682+0000 7ff12ca00640 1 --2- 172.21.15.60:0/1230493004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128101c20 0x7ff128104010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff118009900 tx=0x7ff11802bc30 comp rx=0 tx=0).stop 2024-02-26T10:49:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.682+0000 7ff12ca00640 1 -- 172.21.15.60:0/1230493004 shutdown_connections 2024-02-26T10:49:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.682+0000 7ff12ca00640 1 --2- 172.21.15.60:0/1230493004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff128106e80 0x7ff128109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.682+0000 7ff12ca00640 1 --2- 172.21.15.60:0/1230493004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff128104550 0x7ff128106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.682+0000 7ff12ca00640 1 --2- 172.21.15.60:0/1230493004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128101c20 0x7ff128104010 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.682+0000 7ff12ca00640 1 -- 172.21.15.60:0/1230493004 >> 172.21.15.60:0/1230493004 conn(0x7ff1280fb5d0 msgr2=0x7ff1280fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.682+0000 7ff12ca00640 1 -- 172.21.15.60:0/1230493004 shutdown_connections 2024-02-26T10:49:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.682+0000 7ff12ca00640 1 -- 172.21.15.60:0/1230493004 wait complete. 2024-02-26T10:49:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.683+0000 7ff12ca00640 1 Processor -- start 2024-02-26T10:49:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff12ca00640 1 -- start start 2024-02-26T10:49:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff12ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff128101c20 0x7ff128101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff12ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128104550 0x7ff1280ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff12ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff128106e80 0x7ff128100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff12ca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12810ce20 con 0x7ff128101c20 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff12ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12810cca0 con 0x7ff128104550 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff12ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12810cfa0 con 0x7ff128106e80 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff126800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff128101c20 0x7ff128101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff126800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff128101c20 0x7ff128101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48692/0 (socket says 172.21.15.60:48692) 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff126800640 1 -- 172.21.15.60:0/408238978 learned_addr learned my addr 172.21.15.60:0/408238978 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff11fe00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128104550 0x7ff1280ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.684+0000 7ff127200640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff128106e80 0x7ff128100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff11fe00640 1 -- 172.21.15.60:0/408238978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff128106e80 msgr2=0x7ff128100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff11fe00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff128106e80 0x7ff128100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff11fe00640 1 -- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff128101c20 msgr2=0x7ff128101590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff11fe00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff128101c20 0x7ff128101590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff11fe00640 1 -- 172.21.15.60:0/408238978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1281009a0 con 0x7ff128104550 2024-02-26T10:49:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff126800640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff128101c20 0x7ff128101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:49:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff11fe00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128104550 0x7ff1280ffbd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff11000e9c0 tx=0x7ff11000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff11f400640 1 -- 172.21.15.60:0/408238978 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11000ccc0 con 0x7ff128104550 2024-02-26T10:49:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff11f400640 1 -- 172.21.15.60:0/408238978 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff110005020 con 0x7ff128104550 2024-02-26T10:49:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1281a15b0 con 0x7ff128104550 2024-02-26T10:49:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.685+0000 7ff11f400640 1 -- 172.21.15.60:0/408238978 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110004a00 con 0x7ff128104550 2024-02-26T10:49:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.686+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1281a1b00 con 0x7ff128104550 2024-02-26T10:49:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.687+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1280fd0c0 con 0x7ff128104550 2024-02-26T10:49:56.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.688+0000 7ff11f400640 1 -- 172.21.15.60:0/408238978 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff1100051d0 con 0x7ff128104550 2024-02-26T10:49:56.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.690+0000 7ff11f400640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff10c0a1e00 0x7ff10c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:56.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.691+0000 7ff11f400640 1 -- 172.21.15.60:0/408238978 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7ff1100ce110 con 0x7ff128104550 2024-02-26T10:49:56.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.691+0000 7ff126800640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff10c0a1e00 0x7ff10c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:56.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.692+0000 7ff126800640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff10c0a1e00 0x7ff10c0a42c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff11802c8e0 tx=0x7ff11802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:56.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.694+0000 7ff11f400640 1 -- 172.21.15.60:0/408238978 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff11009b130 con 0x7ff128104550 2024-02-26T10:49:56.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:56 smithi113 ceph-mon[36126]: pgmap v544: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:56.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:56 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2223379283' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:49:56.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:56 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1142308168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:49:56.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:56 smithi060 ceph-mon[31539]: pgmap v544: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:56.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:56 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2223379283' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:49:56.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:56 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1142308168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:49:56.873 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:56 smithi100 ceph-mon[36051]: pgmap v544: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-02-26T10:49:56.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:56 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2223379283' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T10:49:56.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:56 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1142308168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T10:49:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.914+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff12802ce30 con 0x7ff128104550 2024-02-26T10:49:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.915+0000 7ff11f400640 1 -- 172.21.15.60:0/408238978 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v95) v1 ==== 81+0+587 (secure 0 0 0) 0x7ff11009aa30 con 0x7ff128104550 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/603652410 2024-02-27T10:34:36.733932+0000 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3463459063 2024-02-27T10:34:36.733932+0000 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2587364905 2024-02-27T10:34:36.733932+0000 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/603652410 2024-02-27T10:34:36.733932+0000 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/116674454 2024-02-27T10:34:01.770755+0000 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/61419351 2024-02-27T10:34:01.770755+0000 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3598708487 2024-02-27T10:34:36.733932+0000 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1853267621 2024-02-27T10:34:01.770755+0000 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/2948072642 2024-02-27T10:34:01.770755+0000 2024-02-26T10:49:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/2948072642 2024-02-27T10:34:01.770755+0000 2024-02-26T10:49:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:listed 10 entries 2024-02-26T10:49:56.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.917+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff10c0a1e00 msgr2=0x7ff10c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.917+0000 7ff12ca00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff10c0a1e00 0x7ff10c0a42c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff11802c8e0 tx=0x7ff11802df90 comp rx=0 tx=0).stop 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.917+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128104550 msgr2=0x7ff1280ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.917+0000 7ff12ca00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128104550 0x7ff1280ffbd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff11000e9c0 tx=0x7ff11000ee80 comp rx=0 tx=0).stop 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.918+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 shutdown_connections 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.918+0000 7ff12ca00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7ff10c0a1e00 0x7ff10c0a42c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.918+0000 7ff12ca00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff128104550 0x7ff1280ffbd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.918+0000 7ff12ca00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff128106e80 0x7ff128100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.918+0000 7ff12ca00640 1 --2- 172.21.15.60:0/408238978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff128101c20 0x7ff128101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.918+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 >> 172.21.15.60:0/408238978 conn(0x7ff1280fb5d0 msgr2=0x7ff128102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.918+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 shutdown_connections 2024-02-26T10:49:56.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:56.918+0000 7ff12ca00640 1 -- 172.21.15.60:0/408238978 wait complete. 2024-02-26T10:49:56.928 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-26T10:49:56.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:49:56.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-26T10:49:57.079 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T10:49:57.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:49:57.079 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-26T10:49:57.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:49:57.162 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-02-26T10:49:57.239 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T10:49:57.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:49:57.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:49:57.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:49:57.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-02-26T10:49:57.441 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-26T10:49:57.441 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T10:49:57.441 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:49:57.482 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499600k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/e5a6a3e60f08ffabc67bde51f5ab3e6cb951febb92d0339a5aaed24f9bd43ce4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/e5a6a3e60f08ffabc67bde51f5ab3e6cb951febb92d0339a5aaed24f9bd43ce4/diff,workdir=/var/lib/containers/storage/overlay/e5a6a3e60f08ffabc67bde51f5ab3e6cb951febb92d0339a5aaed24f9bd43ce4/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/253346cbc385c073ac6a4e758d751db061614c97e9468042cadd4e85c852efcc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/253346cbc385c073ac6a4e758d751db061614c97e9468042cadd4e85c852efcc/diff,workdir=/var/lib/containers/storage/overlay/253346cbc385c073ac6a4e758d751db061614c97e9468042cadd4e85c852efcc/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/5968bd15e2e17a7da34c339a27aec6445d79e494fa14e24057a93d18c55db7fa/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/5968bd15e2e17a7da34c339a27aec6445d79e494fa14e24057a93d18c55db7fa/diff,workdir=/var/lib/containers/storage/overlay/5968bd15e2e17a7da34c339a27aec6445d79e494fa14e24057a93d18c55db7fa/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/44542eaa013e36ad5fe2a9e94d1644c5153bc2b7503f2c40139155a5b4b93e9f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/44542eaa013e36ad5fe2a9e94d1644c5153bc2b7503f2c40139155a5b4b93e9f/diff,workdir=/var/lib/containers/storage/overlay/44542eaa013e36ad5fe2a9e94d1644c5153bc2b7503f2c40139155a5b4b93e9f/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6f03bbcd9f96db8db8eba6b64f88f78d3b2a7eb1900cde3bb579a40c2295a226/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/6f03bbcd9f96db8db8eba6b64f88f78d3b2a7eb1900cde3bb579a40c2295a226/diff,workdir=/var/lib/containers/storage/overlay/6f03bbcd9f96db8db8eba6b64f88f78d3b2a7eb1900cde3bb579a40c2295a226/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6fdc2eac3ca88b63d2eaa8bdb04ce624c7f78459321234b3323415ae258cc352/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/6fdc2eac3ca88b63d2eaa8bdb04ce624c7f78459321234b3323415ae258cc352/diff,workdir=/var/lib/containers/storage/overlay/6fdc2eac3ca88b63d2eaa8bdb04ce624c7f78459321234b3323415ae258cc352/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d308c945b169c9948d1a8d4250b3ce26905169c88f61290ddaad1fbea09232c3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/d308c945b169c9948d1a8d4250b3ce26905169c88f61290ddaad1fbea09232c3/diff,workdir=/var/lib/containers/storage/overlay/d308c945b169c9948d1a8d4250b3ce26905169c88f61290ddaad1fbea09232c3/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/17694ec7e0747de64d1a13343c8351b46ee55b1e23bdc7a7f8325ba8fcb56a5d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/17694ec7e0747de64d1a13343c8351b46ee55b1e23bdc7a7f8325ba8fcb56a5d/diff,workdir=/var/lib/containers/storage/overlay/17694ec7e0747de64d1a13343c8351b46ee55b1e23bdc7a7f8325ba8fcb56a5d/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ce76e8d301ac2be53c72fb3c83f140e61071f479e87a17faa5e459f8fed3bf58/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/ce76e8d301ac2be53c72fb3c83f140e61071f479e87a17faa5e459f8fed3bf58/diff,workdir=/var/lib/containers/storage/overlay/ce76e8d301ac2be53c72fb3c83f140e61071f479e87a17faa5e459f8fed3bf58/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/da6919f976f6b017679e7e0151a951ca362a3f0c383d3d66b90ad2aff796cbeb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/645MDNCVQCKXUEWSKGE6R5D5YE:/var/lib/containers/storage/overlay/l/KJ4DNLHE4IBAQKTC476FPY32DW,upperdir=/var/lib/containers/storage/overlay/da6919f976f6b017679e7e0151a951ca362a3f0c383d3d66b90ad2aff796cbeb/diff,workdir=/var/lib/containers/storage/overlay/da6919f976f6b017679e7e0151a951ca362a3f0c383d3d66b90ad2aff796cbeb/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499600k,nr_inodes=819200,mode=755 0 0 2024-02-26T10:49:57.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:49:57.482 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-02-26T10:49:57.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:49:57.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-02-26T10:49:57.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:57 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1631933287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:49:57.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:57 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/408238978' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T10:49:57.699 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2024-02-26T10:49:57.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:49:57.704 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-26T10:49:57.722 INFO:tasks.cephadm:Teardown begin 2024-02-26T10:49:57.722 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-26T10:49:57.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:49:57.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:57 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1631933287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:49:57.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:57 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/408238978' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T10:49:57.873 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:57 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1631933287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T10:49:57.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:57 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/408238978' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T10:49:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.937+0000 7fe9d6a00640 1 -- 172.21.15.60:0/607860299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 msgr2=0x7fe9d01014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.937+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/607860299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 0x7fe9d01014f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009900 tx=0x7fe9c402bc40 comp rx=0 tx=0).stop 2024-02-26T10:49:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.937+0000 7fe9d6a00640 1 -- 172.21.15.60:0/607860299 shutdown_connections 2024-02-26T10:49:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.937+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/607860299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d0102c80 0x7fe9d0109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:57.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.937+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/607860299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d01022c0 0x7fe9d0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:57.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.937+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/607860299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 0x7fe9d01014f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:57.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.937+0000 7fe9d6a00640 1 -- 172.21.15.60:0/607860299 >> 172.21.15.60:0/607860299 conn(0x7fe9d00fc640 msgr2=0x7fe9d00fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:57.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.937+0000 7fe9d6a00640 1 -- 172.21.15.60:0/607860299 shutdown_connections 2024-02-26T10:49:57.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.937+0000 7fe9d6a00640 1 -- 172.21.15.60:0/607860299 wait complete. 2024-02-26T10:49:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.938+0000 7fe9d6a00640 1 Processor -- start 2024-02-26T10:49:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.938+0000 7fe9d6a00640 1 -- start start 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 0x7fe9d01973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d01022c0 0x7fe9d0197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d0102c80 0x7fe9d019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d6a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d010cd60 con 0x7fe9d01010d0 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d010cbe0 con 0x7fe9d0102c80 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d6a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d010cee0 con 0x7fe9d01022c0 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 0x7fe9d01973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 0x7fe9d01973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48722/0 (socket says 172.21.15.60:48722) 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d5600640 1 -- 172.21.15.60:0/1163576019 learned_addr learned my addr 172.21.15.60:0/1163576019 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d6000640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d0102c80 0x7fe9d019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d4c00640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d01022c0 0x7fe9d0197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d5600640 1 -- 172.21.15.60:0/1163576019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d01022c0 msgr2=0x7fe9d0197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d5600640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d01022c0 0x7fe9d0197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d5600640 1 -- 172.21.15.60:0/1163576019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d0102c80 msgr2=0x7fe9d019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d5600640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d0102c80 0x7fe9d019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d5600640 1 -- 172.21.15.60:0/1163576019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d019f000 con 0x7fe9d01010d0 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.939+0000 7fe9d6000640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d0102c80 0x7fe9d019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.940+0000 7fe9d5600640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 0x7fe9d01973e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4000c00 tx=0x7fe9c402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.940+0000 7fe9ce000640 1 -- 172.21.15.60:0/1163576019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4041070 con 0x7fe9d01010d0 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.940+0000 7fe9ce000640 1 -- 172.21.15.60:0/1163576019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9c4004450 con 0x7fe9d01010d0 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.940+0000 7fe9ce000640 1 -- 172.21.15.60:0/1163576019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4034440 con 0x7fe9d01010d0 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.940+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d019f290 con 0x7fe9d01010d0 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.940+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d019f780 con 0x7fe9d01010d0 2024-02-26T10:49:57.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.941+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d0108470 con 0x7fe9d01010d0 2024-02-26T10:49:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.945+0000 7fe9ce000640 1 -- 172.21.15.60:0/1163576019 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe9c4044070 con 0x7fe9d01010d0 2024-02-26T10:49:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.946+0000 7fe9ce000640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe9b80a1d80 0x7fe9b80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.946+0000 7fe9ce000640 1 -- 172.21.15.60:0/1163576019 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe9c40eff30 con 0x7fe9d01010d0 2024-02-26T10:49:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.946+0000 7fe9d4c00640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe9b80a1d80 0x7fe9b80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.947+0000 7fe9d4c00640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe9b80a1d80 0x7fe9b80a4240 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe9d01988f0 tx=0x7fe9bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:57.949+0000 7fe9ce000640 1 -- 172.21.15.60:0/1163576019 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9c40bcfe0 con 0x7fe9d01010d0 2024-02-26T10:49:58.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.181+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9d002ce30 con 0x7fe9d01010d0 2024-02-26T10:49:58.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.183+0000 7fe9ce000640 1 -- 172.21.15.60:0/1163576019 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11368 (secure 0 0 0) 0x7fe9c40bc8e0 con 0x7fe9d01010d0 2024-02-26T10:49:58.176 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:49:58.176 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:48:05.053038+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":80,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":80,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:49:58.176 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-02-26T10:49:58.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe9b80a1d80 msgr2=0x7fe9b80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe9b80a1d80 0x7fe9b80a4240 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe9d01988f0 tx=0x7fe9bc005f90 comp rx=0 tx=0).stop 2024-02-26T10:49:58.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 msgr2=0x7fe9d01973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 0x7fe9d01973e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4000c00 tx=0x7fe9c402c650 comp rx=0 tx=0).stop 2024-02-26T10:49:58.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 shutdown_connections 2024-02-26T10:49:58.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe9b80a1d80 0x7fe9b80a4240 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d0102c80 0x7fe9d019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d01022c0 0x7fe9d0197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 --2- 172.21.15.60:0/1163576019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d01010d0 0x7fe9d01973e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.185+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 >> 172.21.15.60:0/1163576019 conn(0x7fe9d00fc640 msgr2=0x7fe9d0105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:58.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.186+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 shutdown_connections 2024-02-26T10:49:58.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.186+0000 7fe9d6a00640 1 -- 172.21.15.60:0/1163576019 wait complete. 2024-02-26T10:49:58.196 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:49:58.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.427+0000 7f9058e00640 1 -- 172.21.15.60:0/2359592164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9054100280 msgr2=0x7f90541006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.427+0000 7f9058e00640 1 --2- 172.21.15.60:0/2359592164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9054100280 0x7f90541006e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f903c009860 tx=0x7f903c02bb70 comp rx=0 tx=0).stop 2024-02-26T10:49:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.427+0000 7f9058e00640 1 -- 172.21.15.60:0/2359592164 shutdown_connections 2024-02-26T10:49:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.427+0000 7f9058e00640 1 --2- 172.21.15.60:0/2359592164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9054100c20 0x7f90541010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.427+0000 7f9058e00640 1 --2- 172.21.15.60:0/2359592164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9054100280 0x7f90541006e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.427+0000 7f9058e00640 1 --2- 172.21.15.60:0/2359592164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90540ff920 0x7f90540ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.427+0000 7f9058e00640 1 -- 172.21.15.60:0/2359592164 >> 172.21.15.60:0/2359592164 conn(0x7f90540fb540 msgr2=0x7f90540fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:58.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.427+0000 7f9058e00640 1 -- 172.21.15.60:0/2359592164 shutdown_connections 2024-02-26T10:49:58.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.427+0000 7f9058e00640 1 -- 172.21.15.60:0/2359592164 wait complete. 2024-02-26T10:49:58.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.428+0000 7f9058e00640 1 Processor -- start 2024-02-26T10:49:58.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.428+0000 7f9058e00640 1 -- start start 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9058e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90540ff920 0x7f9054199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9058e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9054100280 0x7f9054199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9058e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9054100c20 0x7f90541a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9058e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905410ef50 con 0x7f9054100c20 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9058e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905410edd0 con 0x7f90540ff920 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9058e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905410f0d0 con 0x7f9054100280 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9053e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9054100c20 0x7f90541a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9052a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9054100280 0x7f9054199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9053e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9054100c20 0x7f90541a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48744/0 (socket says 172.21.15.60:48744) 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9053e00640 1 -- 172.21.15.60:0/2182782136 learned_addr learned my addr 172.21.15.60:0/2182782136 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9053400640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90540ff920 0x7f9054199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:58.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9052a00640 1 -- 172.21.15.60:0/2182782136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90540ff920 msgr2=0x7f9054199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9052a00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90540ff920 0x7f9054199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9052a00640 1 -- 172.21.15.60:0/2182782136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9054100c20 msgr2=0x7f90541a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.429+0000 7f9052a00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9054100c20 0x7f90541a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.430+0000 7f9052a00640 1 -- 172.21.15.60:0/2182782136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90541a11b0 con 0x7f9054100280 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.430+0000 7f9052a00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9054100280 0x7f9054199ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f903c02c0b0 tx=0x7f903c032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.430+0000 7f904be00640 1 -- 172.21.15.60:0/2182782136 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c041020 con 0x7f9054100280 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.430+0000 7f904be00640 1 -- 172.21.15.60:0/2182782136 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f903c02c9f0 con 0x7f9054100280 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.430+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90541a1440 con 0x7f9054100280 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.430+0000 7f904be00640 1 -- 172.21.15.60:0/2182782136 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c039040 con 0x7f9054100280 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.430+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90541a1930 con 0x7f9054100280 2024-02-26T10:49:58.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.431+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905410a670 con 0x7f9054100280 2024-02-26T10:49:58.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.433+0000 7f904be00640 1 -- 172.21.15.60:0/2182782136 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f903c044020 con 0x7f9054100280 2024-02-26T10:49:58.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.435+0000 7f904be00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90380a1d80 0x7f90380a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:58.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.435+0000 7f904be00640 1 -- 172.21.15.60:0/2182782136 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f903c0efdd0 con 0x7f9054100280 2024-02-26T10:49:58.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.435+0000 7f9053400640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90380a1d80 0x7f90380a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:58.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.436+0000 7f9053400640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90380a1d80 0x7f90380a4240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9044009530 tx=0x7f9044009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:58.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.437+0000 7f904be00640 1 -- 172.21.15.60:0/2182782136 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f903c04b020 con 0x7f9054100280 2024-02-26T10:49:58.431 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:58 smithi060 ceph-mon[31539]: pgmap v545: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:58.431 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:58 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1163576019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:49:58.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.677+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f905402cff0 con 0x7f9054100280 2024-02-26T10:49:58.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.678+0000 7f904be00640 1 -- 172.21.15.60:0/2182782136 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11368 (secure 0 0 0) 0x7f903c0bcdf0 con 0x7f9054100280 2024-02-26T10:49:58.671 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:49:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2551740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2551740513},{"type":"v1","addr":"172.21.15.60:6839","nonce":2551740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1960047882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1960047882},{"type":"v1","addr":"172.21.15.60:6841","nonce":1960047882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1265971356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1265971356},{"type":"v1","addr":"172.21.15.113:6833","nonce":1265971356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1828064682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1828064682},{"type":"v1","addr":"172.21.15.100:6833","nonce":1828064682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2576791717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2576791717},{"type":"v1","addr":"172.21.15.60:6835","nonce":2576791717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2267710624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2267710624},{"type":"v1","addr":"172.21.15.60:6837","nonce":2267710624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T10:42:42.076607+0000","modified":"2024-02-26T10:48:05.053038+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24553,"mds_2":24530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/1201544557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1201544557},{"type":"v1","addr":"172.21.15.100:6835","nonce":1201544557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6835/2109318506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2109318506},{"type":"v1","addr":"172.21.15.113:6835","nonce":2109318506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6837/3968092815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3968092815},{"type":"v1","addr":"172.21.15.113:6837","nonce":3968092815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":80,"addr":"172.21.15.100:6837/1347855255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1347855255},{"type":"v1","addr":"172.21.15.100:6837","nonce":1347855255}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":80,"addr":"172.21.15.113:6839/2057195735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":2057195735},{"type":"v1","addr":"172.21.15.113:6839","nonce":2057195735}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2377114018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2377114018},{"type":"v1","addr":"172.21.15.100:6839","nonce":2377114018}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:49:58.671 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-02-26T10:49:58.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90380a1d80 msgr2=0x7f90380a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90380a1d80 0x7f90380a4240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9044009530 tx=0x7f9044009260 comp rx=0 tx=0).stop 2024-02-26T10:49:58.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9054100280 msgr2=0x7f9054199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9054100280 0x7f9054199ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f903c02c0b0 tx=0x7f903c032430 comp rx=0 tx=0).stop 2024-02-26T10:49:58.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 shutdown_connections 2024-02-26T10:49:58.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f90380a1d80 0x7f90380a4240 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9054100c20 0x7f90541a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9054100280 0x7f9054199ad0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 --2- 172.21.15.60:0/2182782136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90540ff920 0x7f9054199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 >> 172.21.15.60:0/2182782136 conn(0x7f90540fb540 msgr2=0x7f90540fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:58.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 shutdown_connections 2024-02-26T10:49:58.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.681+0000 7f9058e00640 1 -- 172.21.15.60:0/2182782136 wait complete. 2024-02-26T10:49:58.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:49:58.775 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:58 smithi100 ceph-mon[36051]: pgmap v545: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:58.775 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:58 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1163576019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:49:58.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:58 smithi113 ceph-mon[36126]: pgmap v545: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-02-26T10:49:58.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:58 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1163576019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:49:58.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.923+0000 7fe420200640 1 -- 172.21.15.60:0/1593086422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe418108a90 msgr2=0x7fe418108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.923+0000 7fe420200640 1 --2- 172.21.15.60:0/1593086422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe418108a90 0x7fe418108f10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe404009860 tx=0x7fe40402bb70 comp rx=0 tx=0).stop 2024-02-26T10:49:58.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.923+0000 7fe420200640 1 -- 172.21.15.60:0/1593086422 shutdown_connections 2024-02-26T10:49:58.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.923+0000 7fe420200640 1 --2- 172.21.15.60:0/1593086422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe418108a90 0x7fe418108f10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.923+0000 7fe420200640 1 --2- 172.21.15.60:0/1593086422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe418109450 0x7fe41810fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.923+0000 7fe420200640 1 --2- 172.21.15.60:0/1593086422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4181078a0 0x7fe418107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.923+0000 7fe420200640 1 -- 172.21.15.60:0/1593086422 >> 172.21.15.60:0/1593086422 conn(0x7fe418076870 msgr2=0x7fe418078c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:58.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.923+0000 7fe420200640 1 -- 172.21.15.60:0/1593086422 shutdown_connections 2024-02-26T10:49:58.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.923+0000 7fe420200640 1 -- 172.21.15.60:0/1593086422 wait complete. 2024-02-26T10:49:58.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.924+0000 7fe420200640 1 Processor -- start 2024-02-26T10:49:58.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.924+0000 7fe420200640 1 -- start start 2024-02-26T10:49:58.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe420200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4181078a0 0x7fe41819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:58.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe420200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe418108a90 0x7fe41819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:58.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe420200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe418109450 0x7fe4181a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:58.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe420200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe418113520 con 0x7fe4181078a0 2024-02-26T10:49:58.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe420200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4181133a0 con 0x7fe418109450 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe420200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4181136a0 con 0x7fe418108a90 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4181078a0 0x7fe41819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4181078a0 0x7fe41819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48776/0 (socket says 172.21.15.60:48776) 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe418109450 0x7fe4181a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41d800640 1 -- 172.21.15.60:0/1912560579 learned_addr learned my addr 172.21.15.60:0/1912560579 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41ce00640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe418108a90 0x7fe41819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41d800640 1 -- 172.21.15.60:0/1912560579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe418108a90 msgr2=0x7fe41819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41d800640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe418108a90 0x7fe41819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41d800640 1 -- 172.21.15.60:0/1912560579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe418109450 msgr2=0x7fe4181a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41d800640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe418109450 0x7fe4181a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.925+0000 7fe41d800640 1 -- 172.21.15.60:0/1912560579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4181a5760 con 0x7fe4181078a0 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.926+0000 7fe41d800640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4181078a0 0x7fe41819db40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00e9f0 tx=0x7fe40c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.926+0000 7fe416000640 1 -- 172.21.15.60:0/1912560579 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00ccc0 con 0x7fe4181078a0 2024-02-26T10:49:58.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.926+0000 7fe416000640 1 -- 172.21.15.60:0/1912560579 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe40c0035f0 con 0x7fe4181078a0 2024-02-26T10:49:58.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.926+0000 7fe416000640 1 -- 172.21.15.60:0/1912560579 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00ce60 con 0x7fe4181078a0 2024-02-26T10:49:58.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.926+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4181a5a50 con 0x7fe4181078a0 2024-02-26T10:49:58.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.926+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4181a5f20 con 0x7fe4181078a0 2024-02-26T10:49:58.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.927+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4180684a0 con 0x7fe4181078a0 2024-02-26T10:49:58.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.929+0000 7fe416000640 1 -- 172.21.15.60:0/1912560579 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe40c0037a0 con 0x7fe4181078a0 2024-02-26T10:49:58.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.931+0000 7fe416000640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4000a1e00 0x7fe4000a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:58.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.931+0000 7fe416000640 1 -- 172.21.15.60:0/1912560579 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7fe40c0ce430 con 0x7fe4181078a0 2024-02-26T10:49:58.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.931+0000 7fe41ce00640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4000a1e00 0x7fe4000a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:58.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.932+0000 7fe41ce00640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4000a1e00 0x7fe4000a42c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe404009830 tx=0x7fe404034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:58.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:58.935+0000 7fe416000640 1 -- 172.21.15.60:0/1912560579 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe40c09b450 con 0x7fe4181078a0 2024-02-26T10:49:59.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.140+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe41810ebb0 con 0x7fe4181078a0 2024-02-26T10:49:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.143+0000 7fe416000640 1 -- 172.21.15.60:0/1912560579 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24805 (secure 0 0 0) 0x7fe41810ebb0 con 0x7fe4181078a0 2024-02-26T10:49:59.137 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T10:49:59.137 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"51464d50-d492-11ee-95c3-87774f69a715","created":"2024-02-26T10:33:28.696158+0000","modified":"2024-02-26T10:44:39.229830+0000","last_up_change":"2024-02-26T10:41:43.344878+0000","last_in_change":"2024-02-26T10:41:23.045415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:37:25.424456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:42:40.368004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:42:41.581951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T10:42:45.736477+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d95e5-b12d-4215-b22a-879bb7807b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6803","nonce":810878839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6805","nonce":810878839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6809","nonce":810878839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":810878839},{"type":"v1","addr":"172.21.15.60:6807","nonce":810878839}]},"public_addr":"172.21.15.60:6803/810878839","cluster_addr":"172.21.15.60:6805/810878839","heartbeat_back_addr":"172.21.15.60:6809/810878839","heartbeat_front_addr":"172.21.15.60:6807/810878839","state":["exists","up"]},{"osd":1,"uuid":"63269103-ac90-4d99-af6e-e73cd5b8846d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6801","nonce":3893830913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6803","nonce":3893830913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6807","nonce":3893830913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3893830913},{"type":"v1","addr":"172.21.15.100:6805","nonce":3893830913}]},"public_addr":"172.21.15.100:6801/3893830913","cluster_addr":"172.21.15.100:6803/3893830913","heartbeat_back_addr":"172.21.15.100:6807/3893830913","heartbeat_front_addr":"172.21.15.100:6805/3893830913","state":["exists","up"]},{"osd":2,"uuid":"857e383d-a190-46b0-a523-79126cf32a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6801","nonce":1221563222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6803","nonce":1221563222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6807","nonce":1221563222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1221563222},{"type":"v1","addr":"172.21.15.113:6805","nonce":1221563222}]},"public_addr":"172.21.15.113:6801/1221563222","cluster_addr":"172.21.15.113:6803/1221563222","heartbeat_back_addr":"172.21.15.113:6807/1221563222","heartbeat_front_addr":"172.21.15.113:6805/1221563222","state":["exists","up"]},{"osd":3,"uuid":"ff9ff9aa-4973-4dbb-8e57-3d07c7ef624e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6811","nonce":3095490526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6813","nonce":3095490526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6817","nonce":3095490526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3095490526},{"type":"v1","addr":"172.21.15.60:6815","nonce":3095490526}]},"public_addr":"172.21.15.60:6811/3095490526","cluster_addr":"172.21.15.60:6813/3095490526","heartbeat_back_addr":"172.21.15.60:6817/3095490526","heartbeat_front_addr":"172.21.15.60:6815/3095490526","state":["exists","up"]},{"osd":4,"uuid":"c3719710-9869-4caf-bfcf-cbe453306583","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6809","nonce":2467117088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6811","nonce":2467117088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6815","nonce":2467117088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2467117088},{"type":"v1","addr":"172.21.15.100:6813","nonce":2467117088}]},"public_addr":"172.21.15.100:6809/2467117088","cluster_addr":"172.21.15.100:6811/2467117088","heartbeat_back_addr":"172.21.15.100:6815/2467117088","heartbeat_front_addr":"172.21.15.100:6813/2467117088","state":["exists","up"]},{"osd":5,"uuid":"ecd0dba3-6cb1-4e97-9b63-c83d0485c759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6809","nonce":3892917193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6811","nonce":3892917193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6815","nonce":3892917193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3892917193},{"type":"v1","addr":"172.21.15.113:6813","nonce":3892917193}]},"public_addr":"172.21.15.113:6809/3892917193","cluster_addr":"172.21.15.113:6811/3892917193","heartbeat_back_addr":"172.21.15.113:6815/3892917193","heartbeat_front_addr":"172.21.15.113:6813/3892917193","state":["exists","up"]},{"osd":6,"uuid":"cdacf8ee-41a6-4c16-937f-ff45cfb8aeb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6819","nonce":1002313688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6821","nonce":1002313688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6825","nonce":1002313688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1002313688},{"type":"v1","addr":"172.21.15.60:6823","nonce":1002313688}]},"public_addr":"172.21.15.60:6819/1002313688","cluster_addr":"172.21.15.60:6821/1002313688","heartbeat_back_addr":"172.21.15.60:6825/1002313688","heartbeat_front_addr":"172.21.15.60:6823/1002313688","state":["exists","up"]},{"osd":7,"uuid":"c07a3c8c-b460-46fd-8b14-7e042a98495d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6817","nonce":1677982377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6819","nonce":1677982377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6823","nonce":1677982377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1677982377},{"type":"v1","addr":"172.21.15.100:6821","nonce":1677982377}]},"public_addr":"172.21.15.100:6817/1677982377","cluster_addr":"172.21.15.100:6819/1677982377","heartbeat_back_addr":"172.21.15.100:6823/1677982377","heartbeat_front_addr":"172.21.15.100:6821/1677982377","state":["exists","up"]},{"osd":8,"uuid":"a26d9b49-c7af-40f2-96b1-25018e1e9ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6817","nonce":3414484490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6819","nonce":3414484490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6823","nonce":3414484490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3414484490},{"type":"v1","addr":"172.21.15.113:6821","nonce":3414484490}]},"public_addr":"172.21.15.113:6817/3414484490","cluster_addr":"172.21.15.113:6819/3414484490","heartbeat_back_addr":"172.21.15.113:6823/3414484490","heartbeat_front_addr":"172.21.15.113:6821/3414484490","state":["exists","up"]},{"osd":9,"uuid":"2beba404-c823-40fc-9102-990ad3810ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6827","nonce":3674402617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6829","nonce":3674402617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6833","nonce":3674402617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3674402617},{"type":"v1","addr":"172.21.15.60:6831","nonce":3674402617}]},"public_addr":"172.21.15.60:6827/3674402617","cluster_addr":"172.21.15.60:6829/3674402617","heartbeat_back_addr":"172.21.15.60:6833/3674402617","heartbeat_front_addr":"172.21.15.60:6831/3674402617","state":["exists","up"]},{"osd":10,"uuid":"1d9cc6be-e52a-40fb-b7ac-c91d37b9aca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6825","nonce":304375829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6827","nonce":304375829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6831","nonce":304375829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":304375829},{"type":"v1","addr":"172.21.15.100:6829","nonce":304375829}]},"public_addr":"172.21.15.100:6825/304375829","cluster_addr":"172.21.15.100:6827/304375829","heartbeat_back_addr":"172.21.15.100:6831/304375829","heartbeat_front_addr":"172.21.15.100:6829/304375829","state":["exists","up"]},{"osd":11,"uuid":"7ecbb968-c217-40ea-b64a-91e5de0c7327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6825","nonce":2587204649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6827","nonce":2587204649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6831","nonce":2587204649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2587204649},{"type":"v1","addr":"172.21.15.113:6829","nonce":2587204649}]},"public_addr":"172.21.15.113:6825/2587204649","cluster_addr":"172.21.15.113:6827/2587204649","heartbeat_back_addr":"172.21.15.113:6831/2587204649","heartbeat_front_addr":"172.21.15.113:6829/2587204649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:32.389481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:36:57.099242+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:21.081005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:37:54.089780+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:20.122381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:45.535097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:20.725191+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:39:47.684602+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:13.273110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:40:48.042969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:15.438490+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:41:41.335630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6800/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/3463459063":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/2587364905":"2024-02-27T10:34:36.733932+0000","172.21.15.60:6801/603652410":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/116674454":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/61419351":"2024-02-27T10:34:01.770755+0000","172.21.15.60:0/3598708487":"2024-02-27T10:34:36.733932+0000","172.21.15.60:0/1853267621":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6800/2948072642":"2024-02-27T10:34:01.770755+0000","172.21.15.60:6801/2948072642":"2024-02-27T10:34:01.770755+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:49:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4000a1e00 msgr2=0x7fe4000a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4000a1e00 0x7fe4000a42c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe404009830 tx=0x7fe404034ad0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4181078a0 msgr2=0x7fe41819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4181078a0 0x7fe41819db40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00e9f0 tx=0x7fe40c00eeb0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 shutdown_connections 2024-02-26T10:49:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fe4000a1e00 0x7fe4000a42c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe418109450 0x7fe4181a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe418108a90 0x7fe41819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 --2- 172.21.15.60:0/1912560579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4181078a0 0x7fe41819db40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 >> 172.21.15.60:0/1912560579 conn(0x7fe418076870 msgr2=0x7fe41810bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.146+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 shutdown_connections 2024-02-26T10:49:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.147+0000 7fe420200640 1 -- 172.21.15.60:0/1912560579 wait complete. 2024-02-26T10:49:59.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-26T10:49:59.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.388+0000 7f841e000640 1 -- 172.21.15.60:0/2163847898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418109130 msgr2=0x7f841810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.388+0000 7f841e000640 1 --2- 172.21.15.60:0/2163847898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418109130 0x7f841810b5c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8410009900 tx=0x7f841002bc30 comp rx=0 tx=0).stop 2024-02-26T10:49:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.388+0000 7f841e000640 1 -- 172.21.15.60:0/2163847898 shutdown_connections 2024-02-26T10:49:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.388+0000 7f841e000640 1 --2- 172.21.15.60:0/2163847898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418109130 0x7f841810b5c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.388+0000 7f841e000640 1 --2- 172.21.15.60:0/2163847898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8418073cc0 0x7f8418074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.388+0000 7f841e000640 1 --2- 172.21.15.60:0/2163847898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84180752a0 0x7f84180736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.388+0000 7f841e000640 1 -- 172.21.15.60:0/2163847898 >> 172.21.15.60:0/2163847898 conn(0x7f84180fc7e0 msgr2=0x7f84180fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.389+0000 7f841e000640 1 -- 172.21.15.60:0/2163847898 shutdown_connections 2024-02-26T10:49:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.389+0000 7f841e000640 1 -- 172.21.15.60:0/2163847898 wait complete. 2024-02-26T10:49:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.389+0000 7f841e000640 1 Processor -- start 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f841e000640 1 -- start start 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f841e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418073cc0 0x7f8418199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f841e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84180752a0 0x7f8418199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f841e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418109130 0x7f84181a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f841e000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841810f0f0 con 0x7f8418073cc0 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f841e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841810ef70 con 0x7f8418109130 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f841e000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841810f270 con 0x7f84180752a0 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f8417400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418073cc0 0x7f8418199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f8417e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418109130 0x7f84181a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f8417400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418073cc0 0x7f8418199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48790/0 (socket says 172.21.15.60:48790) 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f8417400640 1 -- 172.21.15.60:0/1322213121 learned_addr learned my addr 172.21.15.60:0/1322213121 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:59.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.390+0000 7f8416a00640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84180752a0 0x7f8418199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f8417400640 1 -- 172.21.15.60:0/1322213121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84180752a0 msgr2=0x7f8418199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f8417400640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84180752a0 0x7f8418199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f8417400640 1 -- 172.21.15.60:0/1322213121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418109130 msgr2=0x7f84181a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f8417400640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418109130 0x7f84181a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f8417400640 1 -- 172.21.15.60:0/1322213121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84181a13b0 con 0x7f8418073cc0 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f8417400640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418073cc0 0x7f8418199790 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f840800ebc0 tx=0x7f840800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f840fe00640 1 -- 172.21.15.60:0/1322213121 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408010630 con 0x7f8418073cc0 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f840fe00640 1 -- 172.21.15.60:0/1322213121 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8408010cb0 con 0x7f8418073cc0 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84181a16a0 con 0x7f8418073cc0 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f840fe00640 1 -- 172.21.15.60:0/1322213121 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408018bd0 con 0x7f8418073cc0 2024-02-26T10:49:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.391+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84181a1bf0 con 0x7f8418073cc0 2024-02-26T10:49:59.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.393+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8418107e60 con 0x7f8418073cc0 2024-02-26T10:49:59.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.395+0000 7f840fe00640 1 -- 172.21.15.60:0/1322213121 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f84080107d0 con 0x7f8418073cc0 2024-02-26T10:49:59.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.396+0000 7f840fe00640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83fc0a1d80 0x7f83fc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:59.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.396+0000 7f840fe00640 1 -- 172.21.15.60:0/1322213121 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9179+0+0 (secure 0 0 0) 0x7f84080ce790 con 0x7f8418073cc0 2024-02-26T10:49:59.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.396+0000 7f8416a00640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83fc0a1d80 0x7f83fc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:59.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.397+0000 7f8416a00640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83fc0a1d80 0x7f83fc0a4240 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f841819aca0 tx=0x7f8404005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:59.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.401+0000 7f840fe00640 1 -- 172.21.15.60:0/1322213121 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f840809b160 con 0x7f8418073cc0 2024-02-26T10:49:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.641+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8418061e60 con 0x7f8418073cc0 2024-02-26T10:49:59.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.645+0000 7f840fe00640 1 -- 172.21.15.60:0/1322213121 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f840809f200 con 0x7f8418073cc0 2024-02-26T10:49:59.638 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T10:49:59.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83fc0a1d80 msgr2=0x7f83fc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83fc0a1d80 0x7f83fc0a4240 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f841819aca0 tx=0x7f8404005f90 comp rx=0 tx=0).stop 2024-02-26T10:49:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418073cc0 msgr2=0x7f8418199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418073cc0 0x7f8418199790 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f840800ebc0 tx=0x7f840800c6a0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 shutdown_connections 2024-02-26T10:49:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f83fc0a1d80 0x7f83fc0a4240 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418109130 0x7f84181a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84180752a0 0x7f8418199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 --2- 172.21.15.60:0/1322213121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418073cc0 0x7f8418199790 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 >> 172.21.15.60:0/1322213121 conn(0x7f84180fc7e0 msgr2=0x7f8418105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:59.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.649+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 shutdown_connections 2024-02-26T10:49:59.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.650+0000 7f841e000640 1 -- 172.21.15.60:0/1322213121 wait complete. 2024-02-26T10:49:59.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-26T10:49:59.692 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2182782136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:49:59.692 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1912560579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:49:59.692 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b[66645]: ignoring --setuser ceph since I am not root 2024-02-26T10:49:59.692 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b[66645]: ignoring --setgroup ceph since I am not root 2024-02-26T10:49:59.692 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-mds[66649]: -- 172.21.15.100:0/2605665078 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cfc8deda40 con 0x55cfc8dc0800 2024-02-26T10:49:59.692 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b[66645]: starting mds.b at 2024-02-26T10:49:59.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:59 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2182782136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:49:59.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:59 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1912560579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:49:59.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:49:59 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:49:59.642+0000 7fe596600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:49:59.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2182782136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T10:49:59.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1912560579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T10:49:59.831 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f[66346]: ignoring --setuser ceph since I am not root 2024-02-26T10:49:59.831 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f[66346]: ignoring --setgroup ceph since I am not root 2024-02-26T10:49:59.831 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-mds[66350]: -- 172.21.15.113:0/3963345599 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55835a365a40 con 0x55835a338c00 2024-02-26T10:49:59.831 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f[66346]: starting mds.f at 2024-02-26T10:49:59.831 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: ignoring --setuser ceph since I am not root 2024-02-26T10:49:59.831 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: ignoring --setgroup ceph since I am not root 2024-02-26T10:49:59.831 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-mds[66984]: -- 172.21.15.113:0/4096681053 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55983155fa40 con 0x559831532c00 2024-02-26T10:49:59.831 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: starting mds.i at 2024-02-26T10:49:59.832 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-c[66659]: ignoring --setuser ceph since I am not root 2024-02-26T10:49:59.832 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-c[66659]: ignoring --setgroup ceph since I am not root 2024-02-26T10:49:59.832 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-mds[66664]: -- 172.21.15.113:0/3568943765 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a95a0d3a40 con 0x55a95a0a6800 2024-02-26T10:49:59.832 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:49:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-c[66659]: starting mds.c at 2024-02-26T10:49:59.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.883+0000 7f09e1a00640 1 -- 172.21.15.60:0/1609934084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09dc1057a0 msgr2=0x7f09dc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.883+0000 7f09e1a00640 1 --2- 172.21.15.60:0/1609934084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09dc1057a0 0x7f09dc107b90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09d400acf0 tx=0x7f09d402d220 comp rx=0 tx=0).stop 2024-02-26T10:49:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.883+0000 7f09e1a00640 1 -- 172.21.15.60:0/1609934084 shutdown_connections 2024-02-26T10:49:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.883+0000 7f09e1a00640 1 --2- 172.21.15.60:0/1609934084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09dc1057a0 0x7f09dc107b90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.883+0000 7f09e1a00640 1 --2- 172.21.15.60:0/1609934084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09dc100aa0 0x7f09dc100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.883+0000 7f09e1a00640 1 --2- 172.21.15.60:0/1609934084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09dc0676e0 0x7f09dc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.883+0000 7f09e1a00640 1 -- 172.21.15.60:0/1609934084 >> 172.21.15.60:0/1609934084 conn(0x7f09dc0fa8e0 msgr2=0x7f09dc0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:49:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.883+0000 7f09e1a00640 1 -- 172.21.15.60:0/1609934084 shutdown_connections 2024-02-26T10:49:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.883+0000 7f09e1a00640 1 -- 172.21.15.60:0/1609934084 wait complete. 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.884+0000 7f09e1a00640 1 Processor -- start 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.884+0000 7f09e1a00640 1 -- start start 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.884+0000 7f09e1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09dc0676e0 0x7f09dc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.884+0000 7f09e1a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09dc100aa0 0x7f09dc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09e1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09dc1057a0 0x7f09dc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09e1a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc10abe0 con 0x7f09dc100aa0 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09e1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc10aa60 con 0x7f09dc0676e0 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09e1a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc10ad60 con 0x7f09dc1057a0 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09dbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09dc0676e0 0x7f09dc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09dbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09dc0676e0 0x7f09dc199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:47116/0 (socket says 172.21.15.60:47116) 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09dbe00640 1 -- 172.21.15.60:0/4266555490 learned_addr learned my addr 172.21.15.60:0/4266555490 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09db400640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09dc100aa0 0x7f09dc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09e1000640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09dc1057a0 0x7f09dc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09dbe00640 1 -- 172.21.15.60:0/4266555490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09dc1057a0 msgr2=0x7f09dc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09dbe00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09dc1057a0 0x7f09dc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09dbe00640 1 -- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09dc100aa0 msgr2=0x7f09dc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09dbe00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09dc100aa0 0x7f09dc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.885+0000 7f09dbe00640 1 -- 172.21.15.60:0/4266555490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc1a1220 con 0x7f09dc0676e0 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.886+0000 7f09dbe00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09dc0676e0 0x7f09dc199600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00e9f0 tx=0x7f09cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.886+0000 7f09d8c00640 1 -- 172.21.15.60:0/4266555490 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00ccc0 con 0x7f09dc0676e0 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.886+0000 7f09d8c00640 1 -- 172.21.15.60:0/4266555490 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09cc0035f0 con 0x7f09dc0676e0 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.886+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc1a1510 con 0x7f09dc0676e0 2024-02-26T10:49:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.886+0000 7f09d8c00640 1 -- 172.21.15.60:0/4266555490 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00ce60 con 0x7f09dc0676e0 2024-02-26T10:49:59.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.886+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc1a1ab0 con 0x7f09dc0676e0 2024-02-26T10:49:59.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.887+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc1045b0 con 0x7f09dc0676e0 2024-02-26T10:49:59.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.889+0000 7f09d8c00640 1 -- 172.21.15.60:0/4266555490 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f09cc0037a0 con 0x7f09dc0676e0 2024-02-26T10:49:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.891+0000 7f09d8c00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09c00a1e00 0x7f09c00a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:49:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.891+0000 7f09d8c00640 1 -- 172.21.15.60:0/4266555490 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f09cc0ce480 con 0x7f09dc0676e0 2024-02-26T10:49:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.891+0000 7f09db400640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09c00a1e00 0x7f09c00a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:49:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.892+0000 7f09db400640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09c00a1e00 0x7f09c00a42c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f09dc19ab10 tx=0x7f09c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:49:59.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:49:59.894+0000 7f09d8c00640 1 -- 172.21.15.60:0/4266555490 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f09cc09b3a0 con 0x7f09dc0676e0 2024-02-26T10:49:59.981 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: ignoring --setuser ceph since I am not root 2024-02-26T10:49:59.981 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: ignoring --setgroup ceph since I am not root 2024-02-26T10:49:59.981 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-mds[66328]: -- 172.21.15.100:0/2788595697 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5619eac95a40 con 0x5619eac68800 2024-02-26T10:49:59.981 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: starting mds.h at 2024-02-26T10:49:59.982 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e[66010]: ignoring --setuser ceph since I am not root 2024-02-26T10:49:59.982 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e[66010]: ignoring --setgroup ceph since I am not root 2024-02-26T10:49:59.982 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-mds[66014]: -- 172.21.15.100:0/2691797739 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555df55b3a40 con 0x555df5586800 2024-02-26T10:49:59.982 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:49:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e[66010]: starting mds.e at 2024-02-26T10:50:00.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.146+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f09dc061e60 con 0x7f09dc0676e0 2024-02-26T10:50:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:49:59.999+0000 7fe599800700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-26T10:50:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:49:59.999+0000 7fe599800700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-26T10:50:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:49:59.999+0000 7fe599800700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-02-26T10:50:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:49:59.999+0000 7fe599800700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-26T10:50:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:49:59.999+0000 7fe599800700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-02-26T10:50:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.652+0000 7f09d8c00640 1 -- 172.21.15.60:0/4266555490 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f09cc09aca0 con 0x7f09dc0676e0 2024-02-26T10:50:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09c00a1e00 msgr2=0x7f09c00a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09c00a1e00 0x7f09c00a42c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f09dc19ab10 tx=0x7f09c8005f90 comp rx=0 tx=0).stop 2024-02-26T10:50:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09dc0676e0 msgr2=0x7f09dc199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09dc0676e0 0x7f09dc199600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00e9f0 tx=0x7f09cc00eeb0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 shutdown_connections 2024-02-26T10:50:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f09c00a1e00 0x7f09c00a42c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09dc1057a0 0x7f09dc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09dc100aa0 0x7f09dc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 --2- 172.21.15.60:0/4266555490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09dc0676e0 0x7f09dc199600 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.657+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 >> 172.21.15.60:0/4266555490 conn(0x7f09dc0fa8e0 msgr2=0x7f09dc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:50:00.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.658+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 shutdown_connections 2024-02-26T10:50:00.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.658+0000 7f09e1a00640 1 -- 172.21.15.60:0/4266555490 wait complete. 2024-02-26T10:50:00.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: pgmap v546: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1322213121' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1322213121' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-26T10:50:00.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: fs cephfs is degraded 2024-02-26T10:50:00.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-26T10:50:00.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: fs cephfs is offline because no MDS is active for it. 2024-02-26T10:50:00.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/4266555490' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:50:00.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:00 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:50:00.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: pgmap v546: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-26T10:50:00.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1322213121' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T10:50:00.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1322213121' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: fs cephfs is degraded 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: fs cephfs is offline because no MDS is active for it. 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/4266555490' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:50:00.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:00 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: pgmap v546: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1322213121' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1322213121' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-26T10:50:00.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: fs cephfs is degraded 2024-02-26T10:50:00.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-26T10:50:00.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: fs cephfs is offline because no MDS is active for it. 2024-02-26T10:50:00.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/4266555490' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:50:00.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:00 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T10:50:00.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.910+0000 7fa45b800640 1 -- 172.21.15.60:0/4210346601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa454103490 msgr2=0x7fa4541038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:00.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.910+0000 7fa45b800640 1 --2- 172.21.15.60:0/4210346601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa454103490 0x7fa4541038b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa448009900 tx=0x7fa44802bc30 comp rx=0 tx=0).stop 2024-02-26T10:50:00.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.910+0000 7fa45b800640 1 -- 172.21.15.60:0/4210346601 shutdown_connections 2024-02-26T10:50:00.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.910+0000 7fa45b800640 1 --2- 172.21.15.60:0/4210346601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa454105040 0x7fa45410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.910+0000 7fa45b800640 1 --2- 172.21.15.60:0/4210346601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa454104680 0x7fa454104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.910+0000 7fa45b800640 1 --2- 172.21.15.60:0/4210346601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa454103490 0x7fa4541038b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.910+0000 7fa45b800640 1 -- 172.21.15.60:0/4210346601 >> 172.21.15.60:0/4210346601 conn(0x7fa4540fea20 msgr2=0x7fa454100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:50:00.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.910+0000 7fa45b800640 1 -- 172.21.15.60:0/4210346601 shutdown_connections 2024-02-26T10:50:00.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.910+0000 7fa45b800640 1 -- 172.21.15.60:0/4210346601 wait complete. 2024-02-26T10:50:00.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.911+0000 7fa45b800640 1 Processor -- start 2024-02-26T10:50:00.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.911+0000 7fa45b800640 1 -- start start 2024-02-26T10:50:00.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa45b800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa454103490 0x7fa454199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:00.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa45b800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa454104680 0x7fa454199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:00.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa45b800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa454105040 0x7fa4541a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:00.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa45b800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45410f120 con 0x7fa454105040 2024-02-26T10:50:00.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa45b800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45410efa0 con 0x7fa454103490 2024-02-26T10:50:00.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa45b800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45410f2a0 con 0x7fa454104680 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa459800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa454105040 0x7fa4541a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa453e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa454104680 0x7fa454199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa458e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa454103490 0x7fa454199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa453e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa454104680 0x7fa454199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:52322/0 (socket says 172.21.15.60:52322) 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa453e00640 1 -- 172.21.15.60:0/1432990456 learned_addr learned my addr 172.21.15.60:0/1432990456 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.912+0000 7fa453e00640 1 -- 172.21.15.60:0/1432990456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa454103490 msgr2=0x7fa454199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa453e00640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa454103490 0x7fa454199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa453e00640 1 -- 172.21.15.60:0/1432990456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa454105040 msgr2=0x7fa4541a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa453e00640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa454105040 0x7fa4541a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa453e00640 1 -- 172.21.15.60:0/1432990456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4541a13b0 con 0x7fa454104680 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa453e00640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa454104680 0x7fa454199cd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa44000d950 tx=0x7fa44000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa451600640 1 -- 172.21.15.60:0/1432990456 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440017070 con 0x7fa454104680 2024-02-26T10:50:00.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa451600640 1 -- 172.21.15.60:0/1432990456 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa440009c30 con 0x7fa454104680 2024-02-26T10:50:00.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4541a16a0 con 0x7fa454104680 2024-02-26T10:50:00.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa451600640 1 -- 172.21.15.60:0/1432990456 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44000f6b0 con 0x7fa454104680 2024-02-26T10:50:00.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.913+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4541a1c40 con 0x7fa454104680 2024-02-26T10:50:00.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.914+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45410a8d0 con 0x7fa454104680 2024-02-26T10:50:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.917+0000 7fa451600640 1 -- 172.21.15.60:0/1432990456 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa4400109d0 con 0x7fa454104680 2024-02-26T10:50:00.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.919+0000 7fa451600640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa43c0a1e00 0x7fa43c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:00.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.919+0000 7fa451600640 1 -- 172.21.15.60:0/1432990456 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa4400cdd90 con 0x7fa454104680 2024-02-26T10:50:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.919+0000 7fa458e00640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa43c0a1e00 0x7fa43c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.920+0000 7fa458e00640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa43c0a1e00 0x7fa43c0a42c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa448000c00 tx=0x7fa448005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:50:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:00.920+0000 7fa451600640 1 -- 172.21.15.60:0/1432990456 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa440009de0 con 0x7fa454104680 2024-02-26T10:50:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.126+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa45402ce30 con 0x7fa454104680 2024-02-26T10:50:01.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.665+0000 7fa451600640 1 -- 172.21.15.60:0/1432990456 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa44009a6e0 con 0x7fa454104680 2024-02-26T10:50:01.658 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-02-26T10:50:01.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa43c0a1e00 msgr2=0x7fa43c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa43c0a1e00 0x7fa43c0a42c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa448000c00 tx=0x7fa448005fd0 comp rx=0 tx=0).stop 2024-02-26T10:50:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa454104680 msgr2=0x7fa454199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa454104680 0x7fa454199cd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa44000d950 tx=0x7fa44000de10 comp rx=0 tx=0).stop 2024-02-26T10:50:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 shutdown_connections 2024-02-26T10:50:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fa43c0a1e00 0x7fa43c0a42c0 secure :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa448000c00 tx=0x7fa448005fd0 comp rx=0 tx=0).stop 2024-02-26T10:50:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa454105040 0x7fa4541a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa454104680 0x7fa454199cd0 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa44000d950 tx=0x7fa44000de10 comp rx=0 tx=0).stop 2024-02-26T10:50:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 --2- 172.21.15.60:0/1432990456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa454103490 0x7fa454199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 >> 172.21.15.60:0/1432990456 conn(0x7fa4540fea20 msgr2=0x7fa454107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:50:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.670+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 shutdown_connections 2024-02-26T10:50:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.671+0000 7fa45b800640 1 -- 172.21.15.60:0/1432990456 wait complete. 2024-02-26T10:50:01.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-26T10:50:01.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: Cluster is now healthy 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.113:6835/353547173,v1:172.21.15.113:6838/353547173] up:boot 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6838/1796482363,v1:172.21.15.100:6839/1796482363] up:boot 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.113:6834/4016450357,v1:172.21.15.113:6837/4016450357] up:boot 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.113:6836/4112622589,v1:172.21.15.113:6839/4112622589] up:boot 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6834/2707771706,v1:172.21.15.100:6835/2707771706] up:boot 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: mds.? [v2:172.21.15.100:6836/2850408799,v1:172.21.15.100:6837/2850408799] up:boot 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: fsmap 12 up:standby 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T10:50:01.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T10:50:01.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T10:50:01.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T10:50:01.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T10:50:01.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T10:50:01.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/1432990456' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:01.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:01 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:01.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T10:50:01.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: Cluster is now healthy 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.113:6835/353547173,v1:172.21.15.113:6838/353547173] up:boot 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6838/1796482363,v1:172.21.15.100:6839/1796482363] up:boot 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.113:6834/4016450357,v1:172.21.15.113:6837/4016450357] up:boot 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.113:6836/4112622589,v1:172.21.15.113:6839/4112622589] up:boot 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6834/2707771706,v1:172.21.15.100:6835/2707771706] up:boot 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: mds.? [v2:172.21.15.100:6836/2850408799,v1:172.21.15.100:6837/2850408799] up:boot 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: fsmap 12 up:standby 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T10:50:01.831 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T10:50:01.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T10:50:01.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/1432990456' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:01.832 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:01 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:01.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T10:50:01.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T10:50:01.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: Cluster is now healthy 2024-02-26T10:50:01.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T10:50:01.874 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.113:6835/353547173,v1:172.21.15.113:6838/353547173] up:boot 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6838/1796482363,v1:172.21.15.100:6839/1796482363] up:boot 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.113:6834/4016450357,v1:172.21.15.113:6837/4016450357] up:boot 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.113:6836/4112622589,v1:172.21.15.113:6839/4112622589] up:boot 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6834/2707771706,v1:172.21.15.100:6835/2707771706] up:boot 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: mds.? [v2:172.21.15.100:6836/2850408799,v1:172.21.15.100:6837/2850408799] up:boot 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: fsmap 12 up:standby 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: from='mgr.14152 172.21.15.60:0/1870801131' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/1432990456' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:01.875 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:01 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.908+0000 7f367b200640 1 -- 172.21.15.60:0/4172828849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f367c0ffa10 msgr2=0x7f367c0ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.908+0000 7f367b200640 1 --2- 172.21.15.60:0/4172828849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f367c0ffa10 0x7f367c0ffe30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f366c009900 tx=0x7f366c02bc30 comp rx=0 tx=0).stop 2024-02-26T10:50:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.909+0000 7f367b200640 1 -- 172.21.15.60:0/4172828849 shutdown_connections 2024-02-26T10:50:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.909+0000 7f367b200640 1 --2- 172.21.15.60:0/4172828849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f367c100d10 0x7f367c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.909+0000 7f367b200640 1 --2- 172.21.15.60:0/4172828849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f367c100370 0x7f367c1007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.909+0000 7f367b200640 1 --2- 172.21.15.60:0/4172828849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f367c0ffa10 0x7f367c0ffe30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.909+0000 7f367b200640 1 -- 172.21.15.60:0/4172828849 >> 172.21.15.60:0/4172828849 conn(0x7f367c0fb5d0 msgr2=0x7f367c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:50:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.909+0000 7f367b200640 1 -- 172.21.15.60:0/4172828849 shutdown_connections 2024-02-26T10:50:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.909+0000 7f367b200640 1 -- 172.21.15.60:0/4172828849 wait complete. 2024-02-26T10:50:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.910+0000 7f367b200640 1 Processor -- start 2024-02-26T10:50:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.910+0000 7f367b200640 1 -- start start 2024-02-26T10:50:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.910+0000 7f367b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f367c0ffa10 0x7f367c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.910+0000 7f367b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f367c100370 0x7f367c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.910+0000 7f367b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f367c100d10 0x7f367c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f367b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c10f030 con 0x7f367c100370 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f367b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c10eeb0 con 0x7f367c0ffa10 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f367b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c10f1b0 con 0x7f367c100d10 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3673e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f367c100370 0x7f367c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3673e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f367c100370 0x7f367c199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48860/0 (socket says 172.21.15.60:48860) 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3673e00640 1 -- 172.21.15.60:0/2502689225 learned_addr learned my addr 172.21.15.60:0/2502689225 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3679e00640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f367c0ffa10 0x7f367c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f367a800640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f367c100d10 0x7f367c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3673e00640 1 -- 172.21.15.60:0/2502689225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f367c100d10 msgr2=0x7f367c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3673e00640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f367c100d10 0x7f367c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3673e00640 1 -- 172.21.15.60:0/2502689225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f367c0ffa10 msgr2=0x7f367c199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3673e00640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f367c0ffa10 0x7f367c199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3673e00640 1 -- 172.21.15.60:0/2502689225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c1a1250 con 0x7f367c100370 2024-02-26T10:50:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.911+0000 7f3679e00640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f367c0ffa10 0x7f367c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:50:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.912+0000 7f3673e00640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f367c100370 0x7f367c199b70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f366400e9f0 tx=0x7f366400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:50:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.912+0000 7f3672a00640 1 -- 172.21.15.60:0/2502689225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400cd80 con 0x7f367c100370 2024-02-26T10:50:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.912+0000 7f3672a00640 1 -- 172.21.15.60:0/2502689225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3664005500 con 0x7f367c100370 2024-02-26T10:50:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.912+0000 7f3672a00640 1 -- 172.21.15.60:0/2502689225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664010690 con 0x7f367c100370 2024-02-26T10:50:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.912+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367c1a1540 con 0x7f367c100370 2024-02-26T10:50:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.912+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c1a1a40 con 0x7f367c100370 2024-02-26T10:50:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.914+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367c10a8f0 con 0x7f367c100370 2024-02-26T10:50:01.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.917+0000 7f3672a00640 1 -- 172.21.15.60:0/2502689225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f36640046c0 con 0x7f367c100370 2024-02-26T10:50:01.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.918+0000 7f3672a00640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36600a1e00 0x7f36600a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:01.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.918+0000 7f3672a00640 1 -- 172.21.15.60:0/2502689225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8883+0+0 (secure 0 0 0) 0x7f36640ce5a0 con 0x7f367c100370 2024-02-26T10:50:01.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.919+0000 7f3679e00640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36600a1e00 0x7f36600a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:01.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.919+0000 7f3679e00640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36600a1e00 0x7f36600a42c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f366c02c240 tx=0x7f366c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:50:01.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:01.922+0000 7f3672a00640 1 -- 172.21.15.60:0/2502689225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3664052d20 con 0x7f367c100370 2024-02-26T10:50:02.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.151+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f367c02ce30 con 0x7f367c100370 2024-02-26T10:50:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.674+0000 7f3672a00640 1 -- 172.21.15.60:0/2502689225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7f366409b740 con 0x7f367c100370 2024-02-26T10:50:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-02-26T10:50:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.678+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36600a1e00 msgr2=0x7f36600a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.678+0000 7f367b200640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36600a1e00 0x7f36600a42c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f366c02c240 tx=0x7f366c02df90 comp rx=0 tx=0).stop 2024-02-26T10:50:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.678+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f367c100370 msgr2=0x7f367c199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.678+0000 7f367b200640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f367c100370 0x7f367c199b70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f366400e9f0 tx=0x7f366400edf0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.681+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 shutdown_connections 2024-02-26T10:50:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.681+0000 7f367b200640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7f36600a1e00 0x7f36600a42c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.682+0000 7f367b200640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f367c100d10 0x7f367c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.682+0000 7f367b200640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f367c100370 0x7f367c199b70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.682+0000 7f367b200640 1 --2- 172.21.15.60:0/2502689225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f367c0ffa10 0x7f367c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.682+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 >> 172.21.15.60:0/2502689225 conn(0x7f367c0fb5d0 msgr2=0x7f367c0fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:50:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.682+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 shutdown_connections 2024-02-26T10:50:02.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.682+0000 7f367b200640 1 -- 172.21.15.60:0/2502689225 wait complete. 2024-02-26T10:50:02.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-02-26T10:50:02.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.913+0000 7fbb27e00640 1 -- 172.21.15.60:0/1981082527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28102c10 msgr2=0x7fbb281094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:02.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.913+0000 7fbb27e00640 1 --2- 172.21.15.60:0/1981082527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28102c10 0x7fbb281094a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009ca0 tx=0x7fbb2002c0f0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.913+0000 7fbb27e00640 1 -- 172.21.15.60:0/1981082527 shutdown_connections 2024-02-26T10:50:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.913+0000 7fbb27e00640 1 --2- 172.21.15.60:0/1981082527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28102c10 0x7fbb281094a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.913+0000 7fbb27e00640 1 --2- 172.21.15.60:0/1981082527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb28102250 0x7fbb281026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.913+0000 7fbb27e00640 1 --2- 172.21.15.60:0/1981082527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28101060 0x7fbb28101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.913+0000 7fbb27e00640 1 -- 172.21.15.60:0/1981082527 >> 172.21.15.60:0/1981082527 conn(0x7fbb280fc610 msgr2=0x7fbb280fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:50:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.913+0000 7fbb27e00640 1 -- 172.21.15.60:0/1981082527 shutdown_connections 2024-02-26T10:50:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.913+0000 7fbb27e00640 1 -- 172.21.15.60:0/1981082527 wait complete. 2024-02-26T10:50:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.914+0000 7fbb27e00640 1 Processor -- start 2024-02-26T10:50:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.914+0000 7fbb27e00640 1 -- start start 2024-02-26T10:50:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb28101060 0x7fbb281994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28102250 0x7fbb281999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28102c10 0x7fbb281a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb27e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2810ccf0 con 0x7fbb28102c10 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb27e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2810cb70 con 0x7fbb28102250 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb27e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2810ce70 con 0x7fbb28101060 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb26a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb28101060 0x7fbb281994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb26a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb28101060 0x7fbb281994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.60:52354/0 (socket says 172.21.15.60:52354) 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb26000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28102250 0x7fbb281999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb26a00640 1 -- 172.21.15.60:0/614000858 learned_addr learned my addr 172.21.15.60:0/614000858 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.915+0000 7fbb27400640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28102c10 0x7fbb281a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb26a00640 1 -- 172.21.15.60:0/614000858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28102250 msgr2=0x7fbb281999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb26a00640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28102250 0x7fbb281999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb26a00640 1 -- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28102c10 msgr2=0x7fbb281a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb26a00640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28102c10 0x7fbb281a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb26a00640 1 -- 172.21.15.60:0/614000858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb281a10d0 con 0x7fbb28101060 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb26000640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28102250 0x7fbb281999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb27400640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28102c10 0x7fbb281a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb26a00640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb28101060 0x7fbb281994b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e9c0 tx=0x7fbb1800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb1f400640 1 -- 172.21.15.60:0/614000858 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800cd80 con 0x7fbb28101060 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb1f400640 1 -- 172.21.15.60:0/614000858 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb180035f0 con 0x7fbb28101060 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb281a13c0 con 0x7fbb28101060 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb1f400640 1 -- 172.21.15.60:0/614000858 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18010690 con 0x7fbb28101060 2024-02-26T10:50:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.916+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb281a1960 con 0x7fbb28101060 2024-02-26T10:50:02.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.918+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb281a1c20 con 0x7fbb28101060 2024-02-26T10:50:02.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.920+0000 7fbb1f400640 1 -- 172.21.15.60:0/614000858 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 20) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbb180037a0 con 0x7fbb28101060 2024-02-26T10:50:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.921+0000 7fbb1f400640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbb0c0a1e00 0x7fbb0c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:50:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.921+0000 7fbb1f400640 1 -- 172.21.15.60:0/614000858 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8324+0+0 (secure 0 0 0) 0x7fbb180ce220 con 0x7fbb28101060 2024-02-26T10:50:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.921+0000 7fbb26000640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbb0c0a1e00 0x7fbb0c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:50:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.922+0000 7fbb26000640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbb0c0a1e00 0x7fbb0c0a42c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbb2819a9c0 tx=0x7fbb14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:50:02.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:02.925+0000 7fbb1f400640 1 -- 172.21.15.60:0/614000858 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb1809b590 con 0x7fbb28101060 2024-02-26T10:50:03.080 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:02 smithi113 ceph-mon[36126]: pgmap v548: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 9 op/s 2024-02-26T10:50:03.080 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:02 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T10:50:03.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:02 smithi113 ceph-mon[36126]: osdmap e97: 12 total, 12 up, 12 in 2024-02-26T10:50:03.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:02 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2502689225' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:02 smithi060 ceph-mon[31539]: pgmap v548: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 9 op/s 2024-02-26T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:02 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T10:50:03.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:02 smithi060 ceph-mon[31539]: osdmap e97: 12 total, 12 up, 12 in 2024-02-26T10:50:03.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:02 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2502689225' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:03.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.127+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbb281a2310 con 0x7fbb28101060 2024-02-26T10:50:03.123 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:02 smithi100 ceph-mon[36051]: pgmap v548: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 9 op/s 2024-02-26T10:50:03.123 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:02 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T10:50:03.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:02 smithi100 ceph-mon[36051]: osdmap e97: 12 total, 12 up, 12 in 2024-02-26T10:50:03.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:02 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2502689225' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:03.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.699+0000 7fbb1f400640 1 -- 172.21.15.60:0/614000858 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v99) v1 ==== 178+0+0 (secure 0 0 0) 0x7fbb1809ae90 con 0x7fbb28101060 2024-02-26T10:50:03.696 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data_ec' removed 2024-02-26T10:50:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbb0c0a1e00 msgr2=0x7fbb0c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbb0c0a1e00 0x7fbb0c0a42c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbb2819a9c0 tx=0x7fbb14005f90 comp rx=0 tx=0).stop 2024-02-26T10:50:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb28101060 msgr2=0x7fbb281994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:50:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb28101060 0x7fbb281994b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e9c0 tx=0x7fbb1800ee80 comp rx=0 tx=0).stop 2024-02-26T10:50:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 shutdown_connections 2024-02-26T10:50:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:6800/3493132094,v1:172.21.15.60:6801/3493132094] conn(0x7fbb0c0a1e00 0x7fbb0c0a42c0 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbb2819a9c0 tx=0x7fbb14005f90 comp rx=0 tx=0).stop 2024-02-26T10:50:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28102250 0x7fbb281999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28102c10 0x7fbb281a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 --2- 172.21.15.60:0/614000858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb28101060 0x7fbb281994b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:50:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.709+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 >> 172.21.15.60:0/614000858 conn(0x7fbb280fc610 msgr2=0x7fbb28073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:50:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.711+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 shutdown_connections 2024-02-26T10:50:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T10:50:03.711+0000 7fbb27e00640 1 -- 172.21.15.60:0/614000858 wait complete. 2024-02-26T10:50:03.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T10:50:03.821 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T10:50:03.867 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T10:50:03.919 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-26T10:50:03.920 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T10:50:03.977 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T10:50:03.993 DEBUG:teuthology.orchestra.run.smithi113:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T10:50:04.010 INFO:tasks.cephadm:Stopping all daemons... 2024-02-26T10:50:04.010 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-26T10:50:04.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.a 2024-02-26T10:50:04.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:03 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/2502689225' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T10:50:04.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:03 smithi113 ceph-mon[36126]: osdmap e98: 12 total, 12 up, 12 in 2024-02-26T10:50:04.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:03 smithi113 ceph-mon[36126]: from='client.? 172.21.15.60:0/614000858' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:04.081 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:03 smithi113 ceph-mon[36126]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:03 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/2502689225' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T10:50:04.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:03 smithi060 ceph-mon[31539]: osdmap e98: 12 total, 12 up, 12 in 2024-02-26T10:50:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:03 smithi060 ceph-mon[31539]: from='client.? 172.21.15.60:0/614000858' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:04.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:03 smithi060 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:04.123 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:03 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/2502689225' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T10:50:04.123 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:03 smithi100 ceph-mon[36051]: osdmap e98: 12 total, 12 up, 12 in 2024-02-26T10:50:04.123 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:03 smithi100 ceph-mon[36051]: from='client.? 172.21.15.60:0/614000858' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:04.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:03 smithi100 ceph-mon[36051]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T10:50:04.362 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:04 smithi060 systemd[1]: Stopping Ceph mon.a for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:04.960 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:04 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:50:04.679+0000 7fe59d400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:04.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:04 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a[31535]: 2024-02-26T10:50:04.679+0000 7fe59d400700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-26T10:50:05.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:04 smithi060 podman[91624]: 2024-02-26 10:50:04.967657273 +0000 UTC m=+0.487937375 container died 0eaaccfac6f94a25e2b61ff8a6c4cf8e1e69ecd095d2caf69584842c4b8ea18c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-26T10:50:05.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:05 smithi060 podman[91624]: 2024-02-26 10:50:05.539632215 +0000 UTC m=+1.059912310 container cleanup 0eaaccfac6f94a25e2b61ff8a6c4cf8e1e69ecd095d2caf69584842c4b8ea18c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-26T10:50:05.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:05 smithi060 bash[91624]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a 2024-02-26T10:50:05.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 10:50:05 smithi060 podman[91638]: 2024-02-26 10:50:05.766503542 +0000 UTC m=+0.780231370 container remove 0eaaccfac6f94a25e2b61ff8a6c4cf8e1e69ecd095d2caf69584842c4b8ea18c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222) 2024-02-26T10:50:06.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.a.service' 2024-02-26T10:50:06.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:06.541 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-26T10:50:06.541 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-26T10:50:06.541 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.b 2024-02-26T10:50:06.845 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:06 smithi100 systemd[1]: Stopping Ceph mon.b for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:07.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-b[36047]: 2024-02-26T10:50:06.852+0000 7fa31e369700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:07.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-b[36047]: 2024-02-26T10:50:06.852+0000 7fa31e369700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-26T10:50:07.124 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:06 smithi100 podman[69847]: 2024-02-26 10:50:06.996406763 +0000 UTC m=+0.255475317 container died 2aaee5209894be7a125093914b4ad9ebc3bd1fe85273ad8a3c63455225a4bf20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:50:07.566 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:07 smithi100 podman[69847]: 2024-02-26 10:50:07.345299993 +0000 UTC m=+0.604368547 container cleanup 2aaee5209894be7a125093914b4ad9ebc3bd1fe85273ad8a3c63455225a4bf20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-b, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:50:07.566 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:07 smithi100 bash[69847]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-b 2024-02-26T10:50:07.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.b.service' 2024-02-26T10:50:07.816 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:07 smithi100 podman[69861]: 2024-02-26 10:50:07.574015211 +0000 UTC m=+0.569861717 container remove 2aaee5209894be7a125093914b4ad9ebc3bd1fe85273ad8a3c63455225a4bf20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:50:07.816 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:07 smithi100 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.b.service: Deactivated successfully. 2024-02-26T10:50:07.816 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:07 smithi100 systemd[1]: Stopped Ceph mon.b for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:50:07.816 INFO:journalctl@ceph.mon.b.smithi100.stdout:Feb 26 10:50:07 smithi100 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.b.service: Consumed 10.892s CPU time. 2024-02-26T10:50:07.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:07.841 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-26T10:50:07.841 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-26T10:50:07.841 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.c 2024-02-26T10:50:08.172 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:07 smithi113 systemd[1]: Stopping Ceph mon.c for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:08.565 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:08 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-c[36100]: 2024-02-26T10:50:08.180+0000 7fdf02920700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:08.566 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:08 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-c[36100]: 2024-02-26T10:50:08.180+0000 7fdf02920700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-26T10:50:08.566 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:08 smithi113 podman[70323]: 2024-02-26 10:50:08.282558923 +0000 UTC m=+0.232606164 container died 800ead2bfef6bd25db134bbab8cad5d48ea37cdeaace6c3f9ec5184f9d7a9af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, ceph=True) 2024-02-26T10:50:08.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:08 smithi113 podman[70323]: 2024-02-26 10:50:08.672640151 +0000 UTC m=+0.622687368 container cleanup 800ead2bfef6bd25db134bbab8cad5d48ea37cdeaace6c3f9ec5184f9d7a9af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:50:08.830 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:08 smithi113 bash[70323]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-c 2024-02-26T10:50:09.135 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.c.service' 2024-02-26T10:50:09.158 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:08 smithi113 podman[70337]: 2024-02-26 10:50:08.916389473 +0000 UTC m=+0.634926526 container remove 800ead2bfef6bd25db134bbab8cad5d48ea37cdeaace6c3f9ec5184f9d7a9af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mon-c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-02-26T10:50:09.159 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:09 smithi113 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.c.service: Deactivated successfully. 2024-02-26T10:50:09.159 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:09 smithi113 systemd[1]: Stopped Ceph mon.c for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:50:09.159 INFO:journalctl@ceph.mon.c.smithi113.stdout:Feb 26 10:50:09 smithi113 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mon.c.service: Consumed 11.001s CPU time. 2024-02-26T10:50:09.206 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:09.206 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-26T10:50:09.206 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-26T10:50:09.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.x 2024-02-26T10:50:09.612 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:50:09 smithi060 systemd[1]: Stopping Ceph mgr.x for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:10.113 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:50:09 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:50:09.817+0000 7f65bb200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:10.113 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:50:09 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x[31771]: 2024-02-26T10:50:09.817+0000 7f65bb200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-26T10:50:10.113 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:50:09 smithi060 podman[91743]: 2024-02-26 10:50:09.903128963 +0000 UTC m=+0.238146262 container died 397be5014c6afccc5174500766bf811ab97d364fb9c40f0badea427ceb944035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-26T10:50:10.551 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:50:10 smithi060 podman[91743]: 2024-02-26 10:50:10.529584853 +0000 UTC m=+0.864602150 container cleanup 397be5014c6afccc5174500766bf811ab97d364fb9c40f0badea427ceb944035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-26T10:50:10.551 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:50:10 smithi060 bash[91743]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x 2024-02-26T10:50:11.112 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 10:50:10 smithi060 podman[91756]: 2024-02-26 10:50:10.781753279 +0000 UTC m=+0.861477734 container remove 397be5014c6afccc5174500766bf811ab97d364fb9c40f0badea427ceb944035 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6085b91fefeb74bf4310344a87d81d3217ef7893, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-x, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-26T10:50:11.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.x.service' 2024-02-26T10:50:11.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:11.466 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-26T10:50:11.466 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-26T10:50:11.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.y 2024-02-26T10:50:11.790 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:50:11 smithi100 systemd[1]: Stopping Ceph mgr.y for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:12.062 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:50:11 smithi100 podman[69970]: 2024-02-26 10:50:11.798310713 +0000 UTC m=+0.136774276 container died 1cf2df4ef95ea0c9a740693db0195dcebb8e59719a576ed625a121fd8296a308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:50:12.373 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:50:12 smithi100 podman[69970]: 2024-02-26 10:50:12.169358554 +0000 UTC m=+0.507822119 container cleanup 1cf2df4ef95ea0c9a740693db0195dcebb8e59719a576ed625a121fd8296a308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-02-26T10:50:12.374 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 26 10:50:12 smithi100 bash[69970]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-y 2024-02-26T10:50:12.574 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.y.service' 2024-02-26T10:50:12.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:12.611 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-26T10:50:12.611 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-26T10:50:12.611 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.z 2024-02-26T10:50:12.982 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:50:12 smithi113 systemd[1]: Stopping Ceph mgr.z for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:13.280 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:50:12 smithi113 podman[70446]: 2024-02-26 10:50:12.990404068 +0000 UTC m=+0.178583526 container died 9242054c1bbf1531bd27a90e2b14ee73551c86d451115a0e0523b424e8fd005d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240222) 2024-02-26T10:50:13.580 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:50:13 smithi113 podman[70446]: 2024-02-26 10:50:13.395550218 +0000 UTC m=+0.583729668 container cleanup 9242054c1bbf1531bd27a90e2b14ee73551c86d451115a0e0523b424e8fd005d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux ) 2024-02-26T10:50:13.580 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:50:13 smithi113 bash[70446]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z 2024-02-26T10:50:14.049 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.z.service' 2024-02-26T10:50:14.073 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:50:13 smithi113 podman[70460]: 2024-02-26 10:50:13.830889604 +0000 UTC m=+0.837642629 container remove 9242054c1bbf1531bd27a90e2b14ee73551c86d451115a0e0523b424e8fd005d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mgr-z, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-26T10:50:14.073 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:50:13 smithi113 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-02-26T10:50:14.074 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:50:14 smithi113 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-02-26T10:50:14.074 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:50:14 smithi113 systemd[1]: Stopped Ceph mgr.z for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:50:14.074 INFO:journalctl@ceph.mgr.z.smithi113.stdout:Feb 26 10:50:14 smithi113 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mgr.z.service: Consumed 11.658s CPU time. 2024-02-26T10:50:14.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:14.085 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-26T10:50:14.085 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-26T10:50:14.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.0 2024-02-26T10:50:14.612 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:14 smithi060 systemd[1]: Stopping Ceph osd.0 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:15.113 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:14 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0[46547]: 2024-02-26T10:50:14.691+0000 7fa2d2c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:15.113 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:14 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0[46547]: 2024-02-26T10:50:14.691+0000 7fa2d2c00700 -1 osd.0 99 *** Got signal Terminated *** 2024-02-26T10:50:15.113 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:14 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0[46547]: 2024-02-26T10:50:14.691+0000 7fa2d2c00700 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:50:20.362 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:20 smithi060 podman[91861]: 2024-02-26 10:50:20.14007724 +0000 UTC m=+5.612261377 container died c74db9e103bea643c2e665483e21abba30ed82ad86f748531426e5f5ee32a836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:50:20.862 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:20 smithi060 podman[91861]: 2024-02-26 10:50:20.686105972 +0000 UTC m=+6.158290146 container cleanup c74db9e103bea643c2e665483e21abba30ed82ad86f748531426e5f5ee32a836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-26T10:50:20.862 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:20 smithi060 bash[91861]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0 2024-02-26T10:50:21.362 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:20 smithi060 podman[91881]: 2024-02-26 10:50:20.938050156 +0000 UTC m=+0.782686915 container remove c74db9e103bea643c2e665483e21abba30ed82ad86f748531426e5f5ee32a836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-26T10:50:22.094 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:21 smithi060 podman[91966]: 2024-02-26 10:50:21.653219293 +0000 UTC m=+0.038298439 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:50:22.094 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:21 smithi060 podman[91966]: 2024-02-26 10:50:21.789155225 +0000 UTC m=+0.174234356 container create 5aa795625974996e197fb2aac1a8bffde0a3d4450fea6c4a63528e761b4beaca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-26T10:50:22.362 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:22 smithi060 podman[91966]: 2024-02-26 10:50:22.101747939 +0000 UTC m=+0.486827110 container init 5aa795625974996e197fb2aac1a8bffde0a3d4450fea6c4a63528e761b4beaca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-26T10:50:22.362 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:22 smithi060 podman[91966]: 2024-02-26 10:50:22.114208232 +0000 UTC m=+0.499287369 container start 5aa795625974996e197fb2aac1a8bffde0a3d4450fea6c4a63528e761b4beaca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:50:22.363 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:22 smithi060 podman[91966]: 2024-02-26 10:50:22.200755734 +0000 UTC m=+0.585834925 container attach 5aa795625974996e197fb2aac1a8bffde0a3d4450fea6c4a63528e761b4beaca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-26T10:50:22.862 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 10:50:22 smithi060 podman[91966]: 2024-02-26 10:50:22.574770651 +0000 UTC m=+0.959849815 container died 5aa795625974996e197fb2aac1a8bffde0a3d4450fea6c4a63528e761b4beaca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-0-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, ceph=True) 2024-02-26T10:50:23.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.0.service' 2024-02-26T10:50:23.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:23.523 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-26T10:50:23.523 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-26T10:50:23.523 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.1 2024-02-26T10:50:23.815 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:23 smithi100 systemd[1]: Stopping Ceph osd.1 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:24.123 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:23 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1[40652]: 2024-02-26T10:50:23.822+0000 7f825f915700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:24.123 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:23 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1[40652]: 2024-02-26T10:50:23.822+0000 7f825f915700 -1 osd.1 99 *** Got signal Terminated *** 2024-02-26T10:50:24.124 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:23 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1[40652]: 2024-02-26T10:50:23.822+0000 7f825f915700 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:50:29.451 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:29 smithi100 podman[70094]: 2024-02-26 10:50:29.164593469 +0000 UTC m=+5.449398208 container died f1b23795e7390a53f331b96d06094a4d87859b240b495acfc5d84513e867317e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-26T10:50:29.817 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:29 smithi100 podman[70094]: 2024-02-26 10:50:29.548745747 +0000 UTC m=+5.833550483 container cleanup f1b23795e7390a53f331b96d06094a4d87859b240b495acfc5d84513e867317e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-26T10:50:29.817 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:29 smithi100 bash[70094]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1 2024-02-26T10:50:30.123 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:29 smithi100 podman[70111]: 2024-02-26 10:50:29.82530503 +0000 UTC m=+0.653159014 container remove f1b23795e7390a53f331b96d06094a4d87859b240b495acfc5d84513e867317e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:50:30.429 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:30 smithi100 podman[70207]: 2024-02-26 10:50:30.059466709 +0000 UTC m=+0.017503487 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:50:30.429 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:30 smithi100 podman[70207]: 2024-02-26 10:50:30.176287554 +0000 UTC m=+0.134324338 container create 4ce805e7559e8e2da893e44764d6e1e9d9a784269308858dde18cb524d758554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-26T10:50:30.429 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:30 smithi100 podman[70207]: 2024-02-26 10:50:30.437308777 +0000 UTC m=+0.395345541 container init 4ce805e7559e8e2da893e44764d6e1e9d9a784269308858dde18cb524d758554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222) 2024-02-26T10:50:30.724 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:30 smithi100 podman[70207]: 2024-02-26 10:50:30.440805734 +0000 UTC m=+0.398842496 container start 4ce805e7559e8e2da893e44764d6e1e9d9a784269308858dde18cb524d758554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-02-26T10:50:30.724 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:30 smithi100 podman[70207]: 2024-02-26 10:50:30.512391564 +0000 UTC m=+0.470428339 container attach 4ce805e7559e8e2da893e44764d6e1e9d9a784269308858dde18cb524d758554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-26T10:50:31.074 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 26 10:50:30 smithi100 podman[70207]: 2024-02-26 10:50:30.732521894 +0000 UTC m=+0.690558668 container died 4ce805e7559e8e2da893e44764d6e1e9d9a784269308858dde18cb524d758554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:50:31.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.1.service' 2024-02-26T10:50:31.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:31.442 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-26T10:50:31.442 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-26T10:50:31.443 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.2 2024-02-26T10:50:31.580 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:31 smithi113 systemd[1]: Stopping Ceph osd.2 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:32.080 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2[41272]: 2024-02-26T10:50:31.773+0000 7f4c6d7c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:32.080 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2[41272]: 2024-02-26T10:50:31.773+0000 7f4c6d7c4700 -1 osd.2 99 *** Got signal Terminated *** 2024-02-26T10:50:32.080 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2[41272]: 2024-02-26T10:50:31.773+0000 7f4c6d7c4700 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:50:37.392 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:37 smithi113 podman[70582]: 2024-02-26 10:50:37.10960166 +0000 UTC m=+5.466662664 container died fd97e01dd6659700d744eec83728829c3dc0136e05ddc9ad99f7872e49f9045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-26T10:50:37.760 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:37 smithi113 podman[70582]: 2024-02-26 10:50:37.49971254 +0000 UTC m=+5.856773508 container cleanup fd97e01dd6659700d744eec83728829c3dc0136e05ddc9ad99f7872e49f9045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:50:37.760 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:37 smithi113 bash[70582]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2 2024-02-26T10:50:38.080 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:37 smithi113 podman[70598]: 2024-02-26 10:50:37.76811165 +0000 UTC m=+0.661109811 container remove fd97e01dd6659700d744eec83728829c3dc0136e05ddc9ad99f7872e49f9045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-26T10:50:38.426 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:38 smithi113 podman[70691]: 2024-02-26 10:50:38.014064764 +0000 UTC m=+0.017831311 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:50:38.426 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:38 smithi113 podman[70691]: 2024-02-26 10:50:38.127565283 +0000 UTC m=+0.131331805 container create 8e3a00ecec7293a1b3fb1936f9a9f6a2a64cea670df7c1b7e1c9c759b2e61ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-26T10:50:38.721 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:38 smithi113 podman[70691]: 2024-02-26 10:50:38.434185707 +0000 UTC m=+0.437952239 container init 8e3a00ecec7293a1b3fb1936f9a9f6a2a64cea670df7c1b7e1c9c759b2e61ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-26T10:50:38.721 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:38 smithi113 podman[70691]: 2024-02-26 10:50:38.437337533 +0000 UTC m=+0.441104064 container start 8e3a00ecec7293a1b3fb1936f9a9f6a2a64cea670df7c1b7e1c9c759b2e61ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-26T10:50:38.721 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:38 smithi113 podman[70691]: 2024-02-26 10:50:38.539057624 +0000 UTC m=+0.542824173 container attach 8e3a00ecec7293a1b3fb1936f9a9f6a2a64cea670df7c1b7e1c9c759b2e61ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-26T10:50:39.076 INFO:journalctl@ceph.osd.2.smithi113.stdout:Feb 26 10:50:38 smithi113 podman[70691]: 2024-02-26 10:50:38.72932418 +0000 UTC m=+0.733090719 container died 8e3a00ecec7293a1b3fb1936f9a9f6a2a64cea670df7c1b7e1c9c759b2e61ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True) 2024-02-26T10:50:39.427 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.2.service' 2024-02-26T10:50:39.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:39.472 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-26T10:50:39.473 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-26T10:50:39.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.3 2024-02-26T10:50:39.833 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:39 smithi060 systemd[1]: Stopping Ceph osd.3 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:40.362 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3[53325]: 2024-02-26T10:50:40.097+0000 7f902b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:40.362 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3[53325]: 2024-02-26T10:50:40.097+0000 7f902b800700 -1 osd.3 99 *** Got signal Terminated *** 2024-02-26T10:50:40.363 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:40 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3[53325]: 2024-02-26T10:50:40.097+0000 7f902b800700 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:50:41.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:41 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:41.365+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:41.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:41 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:41.520+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:42.373 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:42 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:42.283+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:42.612 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:42 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3[53325]: 2024-02-26T10:50:42.361+0000 7f9022000700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:18.281583+0000 front 2024-02-26T10:50:18.281356+0000 (oldest deadline 2024-02-26T10:50:41.781071+0000) 2024-02-26T10:50:42.613 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:42 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:42.295+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:42.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:42.545+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:42.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:42.390+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:43.612 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:43 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3[53325]: 2024-02-26T10:50:43.378+0000 7f9022000700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:18.281583+0000 front 2024-02-26T10:50:18.281356+0000 (oldest deadline 2024-02-26T10:50:41.781071+0000) 2024-02-26T10:50:43.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:43 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:43.330+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:43.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:43 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:43.307+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:43.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:43 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:43.506+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:43.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:43 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:43.351+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:44.612 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:44 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3[53325]: 2024-02-26T10:50:44.421+0000 7f9022000700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:18.281583+0000 front 2024-02-26T10:50:18.281356+0000 (oldest deadline 2024-02-26T10:50:41.781071+0000) 2024-02-26T10:50:44.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:44 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:44.317+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:44.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:44 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:44.324+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:44.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:44.402+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:44.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:44.479+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:45.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:45 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:45.295+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:45.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:45 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:45.313+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:45.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:45 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:45.387+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:45.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:45 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:45.474+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:46.020 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:45 smithi060 podman[92070]: 2024-02-26 10:50:45.713136927 +0000 UTC m=+5.761141333 container died 5647faff3888a7c4d3e182c958f23dbe939198282571bc811dddf0b2390314fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-26T10:50:46.281 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:46 smithi060 podman[92070]: 2024-02-26 10:50:46.264636607 +0000 UTC m=+6.312641053 container cleanup 5647faff3888a7c4d3e182c958f23dbe939198282571bc811dddf0b2390314fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-26T10:50:46.281 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:46 smithi060 bash[92070]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3 2024-02-26T10:50:46.282 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:46 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:46.255+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:46.612 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:46 smithi060 podman[92101]: 2024-02-26 10:50:46.491513957 +0000 UTC m=+0.770165253 container remove 5647faff3888a7c4d3e182c958f23dbe939198282571bc811dddf0b2390314fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-02-26T10:50:46.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:46 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:46.352+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:46.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:46.348+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:46.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:46.505+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:47.493 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:47 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:47.218+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:47.494 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:47 smithi060 podman[92185]: 2024-02-26 10:50:47.211200084 +0000 UTC m=+0.041510097 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:50:47.494 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:47 smithi060 podman[92185]: 2024-02-26 10:50:47.334253186 +0000 UTC m=+0.164563228 container create b1727f8a62b317e4571e2728bc335937585dbdc0c3fab235294c895ca52760f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-26T10:50:47.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:47 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:47.312+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:47.763 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:47 smithi060 podman[92185]: 2024-02-26 10:50:47.688283401 +0000 UTC m=+0.518593450 container init b1727f8a62b317e4571e2728bc335937585dbdc0c3fab235294c895ca52760f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-26T10:50:47.763 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:47 smithi060 podman[92185]: 2024-02-26 10:50:47.699710917 +0000 UTC m=+0.530020992 container start b1727f8a62b317e4571e2728bc335937585dbdc0c3fab235294c895ca52760f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-26T10:50:47.763 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:47 smithi060 podman[92185]: 2024-02-26 10:50:47.770643054 +0000 UTC m=+0.600953138 container attach b1727f8a62b317e4571e2728bc335937585dbdc0c3fab235294c895ca52760f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS) 2024-02-26T10:50:47.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:47.362+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:47.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:47.518+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:48.498 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:48 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:48.259+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:48.499 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 10:50:48 smithi060 podman[92185]: 2024-02-26 10:50:48.143807557 +0000 UTC m=+0.974117596 container died b1727f8a62b317e4571e2728bc335937585dbdc0c3fab235294c895ca52760f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:50:48.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:48 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:48.336+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:48.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:48 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:48.408+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:48.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:48 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:48.565+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:48.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.3.service' 2024-02-26T10:50:49.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:49.007 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-26T10:50:49.007 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-26T10:50:49.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.4 2024-02-26T10:50:49.327 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:49 smithi100 systemd[1]: Stopping Ceph osd.4 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:49.611 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:49 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:49.247+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:49.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:49 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:49.360+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:49.623 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:49 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4[47215]: 2024-02-26T10:50:49.333+0000 7f58e3633700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:49.623 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:49 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4[47215]: 2024-02-26T10:50:49.333+0000 7f58e3633700 -1 osd.4 99 *** Got signal Terminated *** 2024-02-26T10:50:49.623 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:49 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4[47215]: 2024-02-26T10:50:49.333+0000 7f58e3633700 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:50:49.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:49 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:49.448+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:49.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:49 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:49.578+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:50.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:50 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:50.281+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:50.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:50 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:50.377+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:50.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:50 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:50.278+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:50.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:50.484+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:50.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:50.484+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:50.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:50.552+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:50.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:50.552+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:51.273 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:51 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4[47215]: 2024-02-26T10:50:51.016+0000 7f58dbc1f700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:24.727286+0000 front 2024-02-26T10:50:24.727304+0000 (oldest deadline 2024-02-26T10:50:50.627434+0000) 2024-02-26T10:50:51.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:51 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:51.294+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:51.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:51 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:51.280+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:51.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:51 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:51.389+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:51.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:51 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:51.389+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:50:51.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:51.501+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:51.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:51.501+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:51.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:51.545+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:51.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:51.545+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:52.373 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:52 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4[47215]: 2024-02-26T10:50:52.031+0000 7f58dbc1f700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:24.727286+0000 front 2024-02-26T10:50:24.727304+0000 (oldest deadline 2024-02-26T10:50:50.627434+0000) 2024-02-26T10:50:52.374 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:52 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:52.273+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:52.611 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:52 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:52.248+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:52.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:52.504+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:52.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:52.504+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:52.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:52.496+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:52.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:52.496+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:52.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:52 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:52.406+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:52.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:52 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:52.406+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:50:53.373 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:53 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4[47215]: 2024-02-26T10:50:53.026+0000 7f58dbc1f700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:24.727286+0000 front 2024-02-26T10:50:24.727304+0000 (oldest deadline 2024-02-26T10:50:50.627434+0000) 2024-02-26T10:50:53.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:53 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:53.232+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:53.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:53 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:53.219+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:53.612 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:50:53 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:50:53.145+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:50:53.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:53 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:53.455+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:53.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:53 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:53.455+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:53.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:53 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:53.473+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:53.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:53 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:53.473+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:53.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:50:53 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:50:53.468+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:50:53.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:53 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:53.423+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:53.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:53 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:53.423+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:50:54.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:50:54 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:50:54.103+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:50:54.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:54 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:54.222+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:54.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:54 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:54.222+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:50:54.373 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:54 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4[47215]: 2024-02-26T10:50:54.001+0000 7f58dbc1f700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:24.727286+0000 front 2024-02-26T10:50:24.727304+0000 (oldest deadline 2024-02-26T10:50:50.627434+0000) 2024-02-26T10:50:54.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:54 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:54.204+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:54.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:50:54 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:50:54.499+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:50:54.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:54 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:54.478+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:54.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:54 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:54.478+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:54.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:54 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:54.425+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:54.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:54 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:54.425+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:54.832 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:54 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:54.398+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:54.832 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:54 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:54.398+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:50:55.110 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:54 smithi100 podman[70324]: 2024-02-26 10:50:54.840962819 +0000 UTC m=+5.628820138 container died a7db84214aacf93165c726942acc7373a88f0ad1e74a8a1daf61f222d07e97c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:50:55.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:50:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:50:55.059+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:50:55.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:55.220+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:55.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:55 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:55.220+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:50:55.373 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:55 smithi100 podman[70324]: 2024-02-26 10:50:55.209948829 +0000 UTC m=+5.997806140 container cleanup a7db84214aacf93165c726942acc7373a88f0ad1e74a8a1daf61f222d07e97c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-26T10:50:55.373 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:55 smithi100 bash[70324]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4 2024-02-26T10:50:55.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:55 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:55.168+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:55.374 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:55 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:55.358+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:55.374 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:55 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:55.358+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:50:55.812 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:55 smithi100 podman[70341]: 2024-02-26 10:50:55.511479171 +0000 UTC m=+0.673773725 container remove a7db84214aacf93165c726942acc7373a88f0ad1e74a8a1daf61f222d07e97c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, io.buildah.version=1.33.5) 2024-02-26T10:50:55.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:55 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:55.526+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:55.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:55 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:55.526+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:55.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:50:55 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:50:55.521+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:50:55.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:55 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:55.434+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:55.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:55 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:55.434+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:56.123 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:55 smithi100 podman[70433]: 2024-02-26 10:50:55.820632768 +0000 UTC m=+0.112934191 container create 774057d730fc494f0d0c3a6f944c6b611df511ce26633817a3b5d456eb5a6046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-26T10:50:56.123 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:55 smithi100 podman[70433]: 2024-02-26 10:50:55.725129502 +0000 UTC m=+0.017430940 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:50:56.123 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:56 smithi100 podman[70433]: 2024-02-26 10:50:56.056455412 +0000 UTC m=+0.348756852 container init 774057d730fc494f0d0c3a6f944c6b611df511ce26633817a3b5d456eb5a6046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-26T10:50:56.124 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:56 smithi100 podman[70433]: 2024-02-26 10:50:56.059772024 +0000 UTC m=+0.352073453 container start 774057d730fc494f0d0c3a6f944c6b611df511ce26633817a3b5d456eb5a6046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:50:56.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:50:56 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:50:56.080+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:50:56.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:56 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:56.224+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:56.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:56 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:56.224+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:50:56.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:56 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:56.173+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:56.624 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:56 smithi100 podman[70433]: 2024-02-26 10:50:56.166164349 +0000 UTC m=+0.458465792 container attach 774057d730fc494f0d0c3a6f944c6b611df511ce26633817a3b5d456eb5a6046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T10:50:56.624 INFO:journalctl@ceph.osd.4.smithi100.stdout:Feb 26 10:50:56 smithi100 podman[70433]: 2024-02-26 10:50:56.353878448 +0000 UTC m=+0.646179874 container died 774057d730fc494f0d0c3a6f944c6b611df511ce26633817a3b5d456eb5a6046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True) 2024-02-26T10:50:56.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:56 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:56.407+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:56.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:56 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:56.407+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:50:56.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:50:56 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:50:56.495+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:50:56.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:56 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:56.484+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:56.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:56 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:56.484+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:56.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:56 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:56.452+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:56.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:56 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:56.452+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:56.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.4.service' 2024-02-26T10:50:56.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:50:56.973 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-26T10:50:56.973 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-26T10:50:56.973 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.5 2024-02-26T10:50:57.296 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:57 smithi113 systemd[1]: Stopping Ceph osd.5 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:50:57.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:50:57 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:50:57.101+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:50:57.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:57 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:57.218+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:57.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:57 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:57.218+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:50:57.580 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:50:57 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:50:57.513+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:50:57.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:57 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:57.470+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:57.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:57 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:57.470+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:57.581 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:57 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:57.303+0000 7f5318fba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:50:57.581 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:57 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:57.303+0000 7f5318fba700 -1 osd.5 99 *** Got signal Terminated *** 2024-02-26T10:50:57.581 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:57 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:57.303+0000 7f5318fba700 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:50:57.581 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:57 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:57.479+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:57.581 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:57 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:57.479+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:57.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:57 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:57.171+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:57.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:57 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:57.399+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:57.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:57 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:57.399+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:50:58.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:50:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:50:58.067+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:50:58.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:50:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:50:58.067+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:50:58.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:58.226+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:58.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:58.226+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:50:58.624 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:58 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:58.171+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:58.624 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:58 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:58.171+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:50:58.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:58 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:58.399+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:58.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:58 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:58.399+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:50:58.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:58 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:58.399+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:50:58.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:58 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:58.449+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:58.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:58 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:58.449+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:58.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:58 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:58.449+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.065049+0000 front 2024-02-26T10:50:36.065074+0000 (oldest deadline 2024-02-26T10:50:58.365291+0000) 2024-02-26T10:50:58.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:50:58 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:50:58.562+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:50:58.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:58 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:58.449+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:58.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:58 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:58.449+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:59.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:50:59 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:50:59.099+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:50:59.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:50:59 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:50:59.099+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:50:59.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:59 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:59.214+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:50:59.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:59 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:59.214+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:50:59.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:50:59 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:50:59.214+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:50:59.409 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:59.132+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:50:59.409 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:50:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:50:59.132+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:50:59.677 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:59.417+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:50:59.678 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:59.417+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:50:59.678 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:50:59 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:50:59.417+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:50:59.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:50:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:50:59.525+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:50:59.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:59.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:50:59.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:50:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:50:59.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:50:59.831 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:59.461+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:50:59.831 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:59.461+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:50:59.831 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:50:59 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:50:59.461+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.065049+0000 front 2024-02-26T10:50:36.065074+0000 (oldest deadline 2024-02-26T10:50:58.365291+0000) 2024-02-26T10:51:00.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:00.104+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:00.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:00.104+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:00.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:00.230+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:00.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:00.230+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:00.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:00 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:00.230+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:00.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:00 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:00.118+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:00.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:00 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:00.118+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:00.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:00 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:00.490+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:00.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:00 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:51:00.494+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:51:00.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:00 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:51:00.494+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:51:00.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:00 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:51:00.494+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.065049+0000 front 2024-02-26T10:50:36.065074+0000 (oldest deadline 2024-02-26T10:50:58.365291+0000) 2024-02-26T10:51:00.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:00 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:00.439+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:00.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:00 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:00.439+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:00.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:00 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:00.457+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:00.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:00 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:00.457+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:00.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:00 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:00.457+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:01.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:01 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:01.063+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:01.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:01 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:01.063+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:01.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:01 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:01.257+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:01.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:01 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:01.257+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:01.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:01 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:01.257+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:01.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:01 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:01.089+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:01.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:01 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:01.089+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:01.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:51:01.479+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:13.961208+0000 front 2024-02-26T10:50:13.961459+0000 (oldest deadline 2024-02-26T10:50:40.361597+0000) 2024-02-26T10:51:01.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:51:01.479+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:25.563512+0000 front 2024-02-26T10:50:25.563385+0000 (oldest deadline 2024-02-26T10:50:49.663445+0000) 2024-02-26T10:51:01.830 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5[47808]: 2024-02-26T10:51:01.479+0000 7f53115a6700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.065049+0000 front 2024-02-26T10:50:36.065074+0000 (oldest deadline 2024-02-26T10:50:58.365291+0000) 2024-02-26T10:51:01.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:01.528+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:01.831 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:01.528+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:01.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:01.435+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:01.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:01 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:01.435+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:01.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:01 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:01.467+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:01.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:01 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:01.467+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:01.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:01 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:01.467+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:02.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:02 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:02.064+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:02.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:02 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:02.064+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:02.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:02 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:02.214+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:02.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:02 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:02.214+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:02.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:02 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:02.214+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:02.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:02 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:02.116+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:02.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:02 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:02.116+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:02.673 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:02.556+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:02.674 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:02.556+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:02.674 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:02.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:02.674 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:02.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:02.674 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:02 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:02.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:02.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:02 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:02.459+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:02.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:02 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:02.459+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:02.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:02 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:02.459+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:02.989 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:02 smithi113 podman[70809]: 2024-02-26 10:51:02.682005609 +0000 UTC m=+5.503592330 container died d55eb5fd563e5cabd4fac5bb76ca160e9e8ffc81c3d47d8321d6251683d880e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222) 2024-02-26T10:51:03.323 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:03 smithi113 podman[70809]: 2024-02-26 10:51:03.095597366 +0000 UTC m=+5.917184068 container cleanup d55eb5fd563e5cabd4fac5bb76ca160e9e8ffc81c3d47d8321d6251683d880e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-26T10:51:03.323 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:03 smithi113 bash[70809]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5 2024-02-26T10:51:03.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:03 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:03.069+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:03.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:03 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:03.069+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:03.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:03 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:03.192+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:03.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:03 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:03.192+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:03.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:03 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:03.192+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:03.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:03 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:03.129+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:03.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:03 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:03.129+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:03.580 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:03 smithi113 podman[70829]: 2024-02-26 10:51:03.331384704 +0000 UTC m=+0.651466942 container remove d55eb5fd563e5cabd4fac5bb76ca160e9e8ffc81c3d47d8321d6251683d880e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-26T10:51:03.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:03 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:03.420+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:03.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:03 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:03.420+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:03.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:03 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:03.420+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:03.861 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:03 smithi113 podman[70919]: 2024-02-26 10:51:03.655740437 +0000 UTC m=+0.017917671 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:51:03.861 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:03 smithi113 podman[70919]: 2024-02-26 10:51:03.765597359 +0000 UTC m=+0.127774596 container create 9fc70182b03dc1a21238c1ac489221f293e639db7e46256114d17e1658c9d644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-02-26T10:51:03.861 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:03 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:03.596+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:03.862 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:03 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:03.596+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:03.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:03 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:03.437+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:03.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:03 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:03.437+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:03.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:03 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:03.437+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:04.285 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:04 smithi113 podman[70919]: 2024-02-26 10:51:04.001812744 +0000 UTC m=+0.363989987 container init 9fc70182b03dc1a21238c1ac489221f293e639db7e46256114d17e1658c9d644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-26T10:51:04.286 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:04 smithi113 podman[70919]: 2024-02-26 10:51:04.005028885 +0000 UTC m=+0.367206115 container start 9fc70182b03dc1a21238c1ac489221f293e639db7e46256114d17e1658c9d644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:51:04.286 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:04 smithi113 podman[70919]: 2024-02-26 10:51:04.101882023 +0000 UTC m=+0.464059277 container attach 9fc70182b03dc1a21238c1ac489221f293e639db7e46256114d17e1658c9d644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-26T10:51:04.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:04 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:04.063+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:04.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:04 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:04.063+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:04.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:04 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:04.178+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:04.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:04 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:04.178+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:04.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:04 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:04.178+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:04.580 INFO:journalctl@ceph.osd.5.smithi113.stdout:Feb 26 10:51:04 smithi113 podman[70919]: 2024-02-26 10:51:04.294341258 +0000 UTC m=+0.656518499 container died 9fc70182b03dc1a21238c1ac489221f293e639db7e46256114d17e1658c9d644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:51:04.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:04.445+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:04.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:04.445+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:04.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:04.445+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:04.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:04.174+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:04.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:04.174+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:04.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:04.419+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:04.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:04.419+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:04.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:04 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:04.419+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:04.975 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:04.603+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:04.976 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:04 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:04.603+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:05.003 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.5.service' 2024-02-26T10:51:05.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:51:05.050 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-26T10:51:05.050 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-26T10:51:05.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.6 2024-02-26T10:51:05.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:05 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:05.062+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:05.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:05 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:05.062+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:05.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:05 smithi060 systemd[1]: Stopping Ceph osd.6 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:51:05.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:05 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:05.166+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:05.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:05 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:05.166+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:05.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:05 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:05.166+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:05.429 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:05.145+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:05.429 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:05.145+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:05.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:05 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:05.566+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:05.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:05 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:05.566+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:05.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:05 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:05.414+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:05.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:05 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:05.414+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:05.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:05 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:05.414+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:05.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:05.437+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:05.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:05.437+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:05.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:05 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:05.437+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:06.039 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:05 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:05.675+0000 7f1a8c800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:51:06.039 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:05 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:05.675+0000 7f1a8c800700 -1 osd.6 99 *** Got signal Terminated *** 2024-02-26T10:51:06.039 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:05 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:05.675+0000 7f1a8c800700 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:51:06.039 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:06 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:06.045+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:06.039 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:06 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:06.045+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:06.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:06 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:06.195+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:06.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:06 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:06.195+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:06.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:06 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:06.195+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:06.414 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:06.155+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:06.414 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:06.155+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:06.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:06 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:06.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:06.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:06 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:06.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:06.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:06 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:06.441+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:06.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:06 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:06.441+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:06.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:06 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:06.441+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:06.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:06.422+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:06.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:06.422+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:06.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:06 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:06.422+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:07.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:07.002+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:07.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:07.002+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:07.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:07.002+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.634711+0000 front 2024-02-26T10:50:41.635184+0000 (oldest deadline 2024-02-26T10:51:06.934607+0000) 2024-02-26T10:51:07.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:07.231+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:07.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:07.231+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:07.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:07.231+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:07.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:07.193+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:07.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:07.193+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:07.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:07.401+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:07.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:07.401+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:07.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:07 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:07.401+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:07.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:07 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:07.550+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:07.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:07 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:07.550+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:07.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:07 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:07.441+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:07.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:07 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:07.441+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:07.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:07 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:07.441+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:08.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:07.957+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:08.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:07.957+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:08.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:07 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:07.957+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.634711+0000 front 2024-02-26T10:50:41.635184+0000 (oldest deadline 2024-02-26T10:51:06.934607+0000) 2024-02-26T10:51:08.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:08 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:08.201+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:08.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:08 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:08.201+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:08.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:08 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:08.201+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:08.363 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:08 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:08.201+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:08.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:08 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:08.204+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:08.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:08 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:08.204+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:08.623 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:08 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:08.204+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:08.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:08 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:08.447+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:08.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:08 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:08.447+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:08.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:08 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:08.447+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:08.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:08 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:08.552+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:08.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:08 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:08.552+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:08.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:08 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:08.421+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:08.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:08 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:08.421+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:08.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:08 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:08.421+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:09.264 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:08 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:08.984+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:09.264 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:08 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:08.984+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:09.264 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:08 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:08.984+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.634711+0000 front 2024-02-26T10:50:41.635184+0000 (oldest deadline 2024-02-26T10:51:06.934607+0000) 2024-02-26T10:51:09.264 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:09 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:09.219+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:09.264 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:09 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:09.219+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:09.264 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:09 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:09.219+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:09.264 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:09 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:09.219+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:09.456 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:09.180+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:09.456 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:09.180+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:09.456 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:09.180+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:09.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:09 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:09.523+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:09.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:09 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:09.523+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:09.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:09 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:09.405+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:09.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:09 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:09.405+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:09.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:09 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:09.405+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:09.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:09.464+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:09.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:09.464+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:09.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:09.464+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:09.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:09 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:09.464+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:10.290 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:10.007+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.129345+0000 front 2024-02-26T10:50:27.128967+0000 (oldest deadline 2024-02-26T10:50:53.028959+0000) 2024-02-26T10:51:10.290 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:10.007+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:33.029689+0000 front 2024-02-26T10:50:33.030123+0000 (oldest deadline 2024-02-26T10:50:57.129972+0000) 2024-02-26T10:51:10.290 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6[60146]: 2024-02-26T10:51:10.007+0000 7f1a83000700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.634711+0000 front 2024-02-26T10:50:41.635184+0000 (oldest deadline 2024-02-26T10:51:06.934607+0000) 2024-02-26T10:51:10.291 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:10.206+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:10.291 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:10.206+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:10.291 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:10.206+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:10.291 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:10.206+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:10.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:10 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:10.184+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:10.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:10 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:10.184+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:10.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:10 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:10.184+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:10.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:10 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:10.541+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:10.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:10 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:10.541+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:10.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:10 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:10.452+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:10.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:10 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:10.452+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:10.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:10 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:10.452+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:10.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:10 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:10.512+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:10.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:10 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:10.512+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:10.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:10 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:10.512+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:10.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:10 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:10.512+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:11.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:11 smithi060 podman[92289]: 2024-02-26 10:51:11.14821127 +0000 UTC m=+5.604433130 container died 3c4eead1d8969f9e7ce46f7d0943d6f6cff16982b0c262a5995bff04070b4322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:51:11.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:11 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:11.202+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:11.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:11 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:11.202+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:11.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:11 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:11.202+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:11.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:11 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:11.202+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:11.481 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:11 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:11.224+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:11.481 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:11 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:11.224+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:11.482 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:11 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:11.224+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:11.829 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:11 smithi060 podman[92289]: 2024-02-26 10:51:11.619509582 +0000 UTC m=+6.075731465 container cleanup 3c4eead1d8969f9e7ce46f7d0943d6f6cff16982b0c262a5995bff04070b4322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-02-26T10:51:11.829 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:11 smithi060 bash[92289]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6 2024-02-26T10:51:11.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:11 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:11.568+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:11.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:11 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:11.568+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:11.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:11 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:11.568+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:11.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:11 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:11.490+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:11.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:11 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:11.490+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:11.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:11 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:11.490+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:11.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:11 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:11.489+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:11.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:11 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:11.489+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:11.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:11 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:11.489+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:11.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:11 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:11.489+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:12.112 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:11 smithi060 podman[92317]: 2024-02-26 10:51:11.837173063 +0000 UTC m=+0.676815919 container remove 3c4eead1d8969f9e7ce46f7d0943d6f6cff16982b0c262a5995bff04070b4322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-26T10:51:12.513 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:12 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:12.218+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:12.513 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:12 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:12.218+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:12.513 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:12 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:12.218+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:12.602 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:12 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:12.172+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:12.602 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:12 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:12.172+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:12.602 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:12 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:12.172+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:12.602 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:12 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:12.172+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:12.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:12 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:12.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:12.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:12 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:12.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:12.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:12 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:12.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:12.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:12 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:12.510+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:12.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:12 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:12.510+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:12.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:12 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:12.510+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:12.861 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:12 smithi060 podman[92408]: 2024-02-26 10:51:12.520696175 +0000 UTC m=+0.042383042 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:51:12.862 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:12 smithi060 podman[92408]: 2024-02-26 10:51:12.663420046 +0000 UTC m=+0.185106935 container create b2f6acb377b8d29f272a4a1f8e38d706e88d746db205d5873799d9ea0b37e592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-26T10:51:12.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:12 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:12.521+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:12.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:12 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:12.521+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:12.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:12 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:12.521+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:12.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:12 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:12.521+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:13.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:13 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:13.145+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:13.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:13 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:13.145+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:13.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:13 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:13.145+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:13.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:13 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:13.145+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:13.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:12 smithi060 podman[92408]: 2024-02-26 10:51:12.963228709 +0000 UTC m=+0.484915637 container init b2f6acb377b8d29f272a4a1f8e38d706e88d746db205d5873799d9ea0b37e592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-02-26T10:51:13.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:12 smithi060 podman[92408]: 2024-02-26 10:51:12.975043362 +0000 UTC m=+0.496730264 container start b2f6acb377b8d29f272a4a1f8e38d706e88d746db205d5873799d9ea0b37e592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:51:13.362 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:13 smithi060 podman[92408]: 2024-02-26 10:51:13.082858793 +0000 UTC m=+0.604545663 container attach b2f6acb377b8d29f272a4a1f8e38d706e88d746db205d5873799d9ea0b37e592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T10:51:13.478 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:13 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:13.175+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:13.478 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:13 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:13.175+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:13.478 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:13 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:13.175+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:13.815 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 10:51:13 smithi060 podman[92408]: 2024-02-26 10:51:13.42899798 +0000 UTC m=+0.950684898 container died b2f6acb377b8d29f272a4a1f8e38d706e88d746db205d5873799d9ea0b37e592 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-26T10:51:13.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:13 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:13.552+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:13.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:13 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:13.552+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:13.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:13 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:13.552+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:13.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:13 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:13.548+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:13.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:13 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:13.548+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:13.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:13 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:13.548+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:13.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:13 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:13.486+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:13.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:13 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:13.486+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:13.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:13 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:13.486+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:13.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:13 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:13.486+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:14.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.6.service' 2024-02-26T10:51:14.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:51:14.361 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-26T10:51:14.361 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-26T10:51:14.362 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.7 2024-02-26T10:51:14.411 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:14.160+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:14.411 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:14.160+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:14.412 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:14.160+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:14.412 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:14 smithi100 systemd[1]: Stopping Ceph osd.7 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:51:14.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:14 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:14.176+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:14.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:14 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:14.176+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:14.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:14 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:14.176+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:14.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:14 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:14.176+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:14.695 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:14.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:14.695 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:14.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:14.696 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:14.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:14.696 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:14.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:14.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:14 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:14.541+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:14.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:14 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:14.541+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:14.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:14 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:14.541+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:14.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:14 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:14.551+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:14.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:14 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:14.551+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:14.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:14 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:14.551+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:15.123 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:14.703+0000 7faaa43b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:51:15.123 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:14.703+0000 7faaa43b9700 -1 osd.7 99 *** Got signal Terminated *** 2024-02-26T10:51:15.123 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:14 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:14.703+0000 7faaa43b9700 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:51:15.444 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:15.157+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:15.444 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:15.157+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:15.444 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:15.157+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:15.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:15 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:15.225+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:15.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:15 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:15.225+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:15.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:15 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:15.225+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:15.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:15 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:15.225+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:15.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:15 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:15.521+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:15.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:15 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:15.521+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:15.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:15 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:15.521+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:15.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:15 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:15.567+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:15.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:15 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:15.567+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:15.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:15 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:15.567+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:15.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:15 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:15.567+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:15.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:15.452+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:15.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:15.452+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:15.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:15.452+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:15.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:15.452+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:15.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:15.452+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:16.429 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:16.182+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:16.429 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:16.182+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:16.429 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:16.182+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:16.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:16.201+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:16.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:16.201+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:16.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:16.201+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:16.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:16.201+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:16.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:16 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:16.201+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:16.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:16 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:16.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:16.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:16 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:16.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:16.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:16 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:16.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:16.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:16 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:16.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:16.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:16 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:16.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:16.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:16 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:16.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:16.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:16 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:16.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:16.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:16 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:16.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:16.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:16.436+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:16.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:16.436+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:16.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:16.436+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:16.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:16.436+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:16.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:16 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:16.436+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:17.453 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:17 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:17.171+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:17.453 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:17 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:17.171+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:17.453 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:17 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:17.171+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:17.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:17 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:17.242+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:17.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:17 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:17.242+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:17.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:17 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:17.242+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:17.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:17 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:17.242+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:17.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:17 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:17.242+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:17.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:17 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:17.534+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:17.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:17 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:17.534+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:17.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:17 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:17.534+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:17.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:17 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:17.534+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:17.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:17 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:17.512+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:17.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:17 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:17.512+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:17.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:17 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:17.512+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:17.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:17 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:17.512+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:17.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:17 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:17.460+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:17.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:17 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:17.460+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:17.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:17 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:17.460+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:17.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:17 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:17.460+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:17.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:17 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:17.460+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:18.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:18 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:18.121+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:18.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:18 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:18.121+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:18.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:18 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:18.121+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:18.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:18 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:18.264+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:18.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:18 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:18.264+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:18.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:18 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:18.264+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:18.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:18 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:18.264+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:18.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:18 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:18.264+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:18.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:18 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:18.569+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:18.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:18 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:18.569+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:18.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:18 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:18.569+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:18.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:18 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:18.569+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:18.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:18 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:18.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:18.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:18 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:18.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:18.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:18 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:18.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:18.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:18 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:18.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:18.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:18 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:18.419+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:18.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:18 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:18.419+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:18.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:18 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:18.419+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:18.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:18 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:18.419+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:18.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:18 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:18.419+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:19.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:19 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:19.091+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.175728+0000 front 2024-02-26T10:50:28.175633+0000 (oldest deadline 2024-02-26T10:50:49.875698+0000) 2024-02-26T10:51:19.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:19 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:19.091+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.176858+0000 front 2024-02-26T10:50:35.176801+0000 (oldest deadline 2024-02-26T10:50:57.476881+0000) 2024-02-26T10:51:19.373 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:19 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7[54348]: 2024-02-26T10:51:19.091+0000 7faaa09ad700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.279054+0000 front 2024-02-26T10:50:43.278951+0000 (oldest deadline 2024-02-26T10:51:07.978742+0000) 2024-02-26T10:51:19.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:19 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:19.277+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:19.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:19 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:19.277+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:19.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:19 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:19.277+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:19.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:19 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:19.277+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:19.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:19 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:19.277+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:19.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:19 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:19.536+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:19.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:19 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:19.536+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:19.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:19 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:19.536+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:19.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:19 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:19.536+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:19.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:19 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:19.450+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:19.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:19 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:19.450+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:19.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:19 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:19.450+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:19.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:19 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:19.450+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:19.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:19 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:19.424+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:19.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:19 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:19.424+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:19.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:19 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:19.424+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:19.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:19 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:19.424+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:19.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:19 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:19.424+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:20.529 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:20 smithi100 podman[70550]: 2024-02-26 10:51:20.25282805 +0000 UTC m=+5.690343853 container died a741ec77fcabe9b9cb62de267c151131e4b6751ebddcf64306fdd9ac90d7ad0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:51:20.529 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:20 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:20.463+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:20.529 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:20 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:20.463+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:20.529 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:20 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:20.463+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:20.530 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:20 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:20.463+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:20.530 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:20 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:20.463+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:20.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:20 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:20.310+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:20.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:20 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:20.310+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:20.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:20 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:20.310+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:20.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:20 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:20.310+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:20.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:20 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:20.310+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:20.731 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:20 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:20.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:20.731 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:20 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:20.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:20.731 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:20 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:20.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:20.731 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:20 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:20.545+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:20.731 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:20 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:20.480+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:20.732 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:20 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:20.480+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:20.732 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:20 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:20.480+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:20.732 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:20 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:20.480+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:20.855 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:20 smithi100 podman[70550]: 2024-02-26 10:51:20.637215751 +0000 UTC m=+6.074731540 container cleanup a741ec77fcabe9b9cb62de267c151131e4b6751ebddcf64306fdd9ac90d7ad0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:51:20.856 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:20 smithi100 bash[70550]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7 2024-02-26T10:51:21.123 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:20 smithi100 podman[70567]: 2024-02-26 10:51:20.863445017 +0000 UTC m=+0.613002193 container remove a741ec77fcabe9b9cb62de267c151131e4b6751ebddcf64306fdd9ac90d7ad0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:51:21.473 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:21 smithi100 podman[70658]: 2024-02-26 10:51:21.075190389 +0000 UTC m=+0.017459264 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:51:21.474 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:21 smithi100 podman[70658]: 2024-02-26 10:51:21.181014862 +0000 UTC m=+0.123283716 container create 712603053ad9bfa956c36bb131e17091d6f99f08030216995e1d13b15b63f48f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:51:21.474 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:21 smithi100 podman[70658]: 2024-02-26 10:51:21.383120414 +0000 UTC m=+0.325389286 container init 712603053ad9bfa956c36bb131e17091d6f99f08030216995e1d13b15b63f48f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-26T10:51:21.474 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:21 smithi100 podman[70658]: 2024-02-26 10:51:21.386401582 +0000 UTC m=+0.328670438 container start 712603053ad9bfa956c36bb131e17091d6f99f08030216995e1d13b15b63f48f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:51:21.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:21 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:21.348+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:21.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:21 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:21.348+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:21.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:21 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:21.348+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:21.613 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:21 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:21.348+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:21.613 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:21 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:21.348+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:21.745 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:21 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:21.513+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:21.746 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:21 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:21.513+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:21.746 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:21 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:21.513+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:21.746 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:21 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:21.513+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:21.746 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:21 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:21.462+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:21.746 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:21 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:21.462+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:21.746 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:21 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:21.462+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:21.746 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:21 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:21.462+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:21.873 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:21 smithi100 podman[70658]: 2024-02-26 10:51:21.491442799 +0000 UTC m=+0.433711667 container attach 712603053ad9bfa956c36bb131e17091d6f99f08030216995e1d13b15b63f48f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:51:21.873 INFO:journalctl@ceph.osd.7.smithi100.stdout:Feb 26 10:51:21 smithi100 podman[70658]: 2024-02-26 10:51:21.676323901 +0000 UTC m=+0.618592757 container died 712603053ad9bfa956c36bb131e17091d6f99f08030216995e1d13b15b63f48f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222) 2024-02-26T10:51:21.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:21 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:21.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:21.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:21 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:21.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:21.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:21 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:21.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:21.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:21 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:21.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:21.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:21 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:21.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:22.308 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.7.service' 2024-02-26T10:51:22.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:51:22.349 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-26T10:51:22.350 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-26T10:51:22.350 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.8 2024-02-26T10:51:22.558 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:22 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:22.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:22.558 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:22 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:22.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:22.558 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:22 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:22.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:22.558 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:22 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:22.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:22.558 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:22 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:22.480+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:22.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:22 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:22.356+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:22.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:22 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:22.356+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:22.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:22 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:22.356+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:22.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:22 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:22.356+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:22.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:22 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:22.356+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:22.695 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:22 smithi113 systemd[1]: Stopping Ceph osd.8 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:51:22.695 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:22.547+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:22.695 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:22.547+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:22.695 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:22.547+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:22.695 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:22.547+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:22.695 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:22.701+0000 7fd9cf94f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:51:22.695 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:22.701+0000 7fd9cf94f700 -1 osd.8 99 *** Got signal Terminated *** 2024-02-26T10:51:22.695 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:22.701+0000 7fd9cf94f700 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:51:22.695 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:22.447+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:22.696 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:22.447+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:22.696 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:22.447+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:22.696 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:22 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:22.447+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:23.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:23 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:23.323+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:23.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:23 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:23.323+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:23.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:23 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:23.323+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:23.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:23 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:23.323+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:23.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:23 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:23.323+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:23.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:23 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:23.323+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:23.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:23 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:23.597+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:23.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:23 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:23.597+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:23.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:23 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:23.597+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:23.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:23 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:23.597+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:23.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:23 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:23.466+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:23.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:23 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:23.466+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:23.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:23 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:23.466+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:23.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:23 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:23.466+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:23.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:23 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:23.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:23.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:23 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:23.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:23.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:23 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:23.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:23.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:23 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:23.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:23.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:23 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:23.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:24.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:24 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:24.568+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:24.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:24 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:24.568+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:24.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:24 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:24.568+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:24.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:24 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:24.568+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:24.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:24 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:24.490+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:24.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:24 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:24.490+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:24.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:24 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:24.490+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:24.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:24 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:24.490+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:24.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:24 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:24.490+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:24.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:24 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:24.372+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:24.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:24 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:24.372+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:24.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:24 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:24.372+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:24.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:24 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:24.372+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:24.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:24 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:24.372+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:24.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:24 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:24.372+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:24.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:24 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:24.471+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:24.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:24 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:24.471+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:24.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:24 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:24.471+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:24.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:24 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:24.471+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:24.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:24 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:24.471+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:25.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:25.587+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:25.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:25.587+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:25.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:25.587+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:25.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:25.588+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:25.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:25.588+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:59.379001+0000 front 2024-02-26T10:50:59.378977+0000 (oldest deadline 2024-02-26T10:51:25.279210+0000) 2024-02-26T10:51:25.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:25.476+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:25.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:25.476+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:25.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:25.476+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:25.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:25.476+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:25.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:25 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:25.476+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:25.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:25 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:25.422+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:25.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:25 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:25.422+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:25.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:25 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:25.422+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:25.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:25 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:25.422+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:25.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:25 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:25.422+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:25.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:25 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:25.422+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:25.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:25 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:25.509+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:25.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:25 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:25.509+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:25.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:25 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:25.509+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:25.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:25 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:25.509+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:25.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:25 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:25.509+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:25.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:25 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:25.509+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:26.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:26.609+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:26.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:26.609+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:26.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:26.609+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:26.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:26.609+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:26.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:26.609+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:59.379001+0000 front 2024-02-26T10:50:59.378977+0000 (oldest deadline 2024-02-26T10:51:25.279210+0000) 2024-02-26T10:51:26.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:26.432+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:26.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:26.432+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:26.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:26.432+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:26.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:26.432+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:26.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:26 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:26.432+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:26.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:26 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:26.438+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:26.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:26 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:26.438+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:26.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:26 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:26.438+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:26.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:26 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:26.438+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:26.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:26 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:26.438+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:26.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:26 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:26.438+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:26.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:26 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:26.500+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:26.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:26 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:26.500+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:26.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:26 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:26.500+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:26.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:26 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:26.500+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:26.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:26 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:26.500+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:26.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:26 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:26.500+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:27.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:27.563+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.873889+0000 front 2024-02-26T10:50:27.874013+0000 (oldest deadline 2024-02-26T10:50:52.573966+0000) 2024-02-26T10:51:27.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:27.563+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.674906+0000 front 2024-02-26T10:50:36.675007+0000 (oldest deadline 2024-02-26T10:51:01.375070+0000) 2024-02-26T10:51:27.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:27.563+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:41.375741+0000 front 2024-02-26T10:50:41.375814+0000 (oldest deadline 2024-02-26T10:51:11.376202+0000) 2024-02-26T10:51:27.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:27.563+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:51.877428+0000 front 2024-02-26T10:50:51.877538+0000 (oldest deadline 2024-02-26T10:51:15.377429+0000) 2024-02-26T10:51:27.830 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8[54367]: 2024-02-26T10:51:27.563+0000 7fd9c7f3b700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:59.379001+0000 front 2024-02-26T10:50:59.378977+0000 (oldest deadline 2024-02-26T10:51:25.279210+0000) 2024-02-26T10:51:27.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:27.479+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:27.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:27.479+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:27.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:27.479+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:27.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:27.479+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:27.831 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:27 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:27.479+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:27.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:27 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:27.475+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:27.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:27 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:27.475+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:27.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:27 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:27.475+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:27.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:27 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:27.475+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:27.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:27 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:27.475+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:27.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:27 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:27.475+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:27.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:27 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:27.541+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:27.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:27 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:27.541+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:27.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:27 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:27.541+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:27.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:27 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:27.541+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:27.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:27 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:27.541+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:27.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:27 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:27.541+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:28.330 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:28 smithi113 podman[71035]: 2024-02-26 10:51:28.063877444 +0000 UTC m=+5.503655929 container died 1043ad3a44029b40e2dd1a930b628690cfc2840dc9ea427d197adbf410f8a7e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:51:28.729 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:28 smithi113 podman[71035]: 2024-02-26 10:51:28.469447284 +0000 UTC m=+5.909225794 container cleanup 1043ad3a44029b40e2dd1a930b628690cfc2840dc9ea427d197adbf410f8a7e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-02-26T10:51:28.729 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:28 smithi113 bash[71035]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8 2024-02-26T10:51:28.729 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:28 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:28.500+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:28.729 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:28 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:28.500+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:28.729 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:28 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:28.500+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:28.729 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:28 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:28.500+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:28.729 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:28 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:28.500+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:28.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:28 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:28.443+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:28.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:28 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:28.443+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:28.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:28 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:28.443+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:28.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:28 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:28.443+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:28.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:28 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:28.443+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:28.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:28 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:28.443+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:28.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:28 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:28.537+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:28.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:28 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:28.537+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:28.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:28 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:28.537+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:28.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:28 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:28.537+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:28.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:28 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:28.537+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:28.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:28 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:28.537+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:29.077 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:28 smithi113 podman[71053]: 2024-02-26 10:51:28.737386177 +0000 UTC m=+0.668120872 container remove 1043ad3a44029b40e2dd1a930b628690cfc2840dc9ea427d197adbf410f8a7e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-26T10:51:29.329 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:29 smithi113 podman[71143]: 2024-02-26 10:51:28.988868091 +0000 UTC m=+0.017354924 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:51:29.330 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:29 smithi113 podman[71143]: 2024-02-26 10:51:29.104921189 +0000 UTC m=+0.133408005 container create bcea4227eb9b6011a6d0ec2309d1870d936b9ca9cc2fb3580dadcfaaf0d15760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8-deactivate, ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T10:51:29.674 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:29 smithi113 podman[71143]: 2024-02-26 10:51:29.383036019 +0000 UTC m=+0.411522856 container init bcea4227eb9b6011a6d0ec2309d1870d936b9ca9cc2fb3580dadcfaaf0d15760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:51:29.675 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:29 smithi113 podman[71143]: 2024-02-26 10:51:29.386283236 +0000 UTC m=+0.414770058 container start bcea4227eb9b6011a6d0ec2309d1870d936b9ca9cc2fb3580dadcfaaf0d15760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:51:29.675 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:29 smithi113 podman[71143]: 2024-02-26 10:51:29.490915317 +0000 UTC m=+0.519402154 container attach bcea4227eb9b6011a6d0ec2309d1870d936b9ca9cc2fb3580dadcfaaf0d15760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:51:29.675 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:29 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:29.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:29.675 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:29 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:29.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:29.675 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:29 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:29.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:29.675 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:29 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:29.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:29.675 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:29 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:29.487+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:29.774 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:29 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:29.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:29.774 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:29 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:29.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:29.775 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:29 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:29.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:29.775 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:29 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:29.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:29.775 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:29 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:29.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:29.775 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:29 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:29.496+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:29.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:29.459+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:29.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:29.459+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:29.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:29.459+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:29.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:29.459+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:29.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:29.459+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:29.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:29 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:29.459+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:29.961 INFO:journalctl@ceph.osd.8.smithi113.stdout:Feb 26 10:51:29 smithi113 podman[71143]: 2024-02-26 10:51:29.683354366 +0000 UTC m=+0.711841197 container died bcea4227eb9b6011a6d0ec2309d1870d936b9ca9cc2fb3580dadcfaaf0d15760 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-26T10:51:30.321 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.8.service' 2024-02-26T10:51:30.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:51:30.364 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-26T10:51:30.364 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-26T10:51:30.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.9 2024-02-26T10:51:30.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:30.499+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:30.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:30.499+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:30.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:30.499+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:30.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:30.499+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:30.581 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:30 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:30.499+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:30.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 systemd[1]: Stopping Ceph osd.9 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:51:30.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:30.508+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:30.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:30.508+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:30.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:30.508+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:30.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:30.508+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:30.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:30.508+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:30.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:30.508+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:30.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:30.497+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:30.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:30.497+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:30.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:30.497+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:30.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:30.497+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:30.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:30.497+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:30.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:30 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:30.497+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:31.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:30.988+0000 7effb9400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:51:31.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:30.988+0000 7effb9400700 -1 osd.9 99 *** Got signal Terminated *** 2024-02-26T10:51:31.362 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:30 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:30.988+0000 7effb9400700 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:51:31.829 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:31.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:31.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:31.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:31.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:31.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:31.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:31.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:31.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:31.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:31.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:31 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:31.525+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:31.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:31 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:31.525+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:31.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:31 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:31.525+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:31.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:31 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:31.525+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:31.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:31 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:31.525+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:31.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:31 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:31.525+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:31.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:31 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:31.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:31.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:31 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:31.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:31.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:31 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:31.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:31.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:31 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:31.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:31.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:31 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:31.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:31.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:31 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:31.476+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:32.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:32 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:32.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:32.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:32 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:32.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:32.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:32 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:32.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:32.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:32 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:32.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:32.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:32 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:32.509+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:32.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:32.488+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:32.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:32.488+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:32.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:32.488+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:32.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:32.488+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:32.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:32.488+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:32.863 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:32 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:32.489+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:32.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:32 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:32.430+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:32.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:32 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:32.430+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:32.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:32 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:32.430+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:32.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:32 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:32.430+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:32.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:32 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:32.430+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:32.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:32 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:32.430+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:32.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:32 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:32.430+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:33.829 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:33 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:33.542+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:33.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:33 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:33.542+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:33.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:33 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:33.542+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:33.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:33 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:33.542+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:33.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:33 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:33.542+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:33.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:33.513+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:33.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:33.513+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:33.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:33.513+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:33.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:33.513+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:33.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:33.513+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:33.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:33 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:33.513+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:33.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:33 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:33.428+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:33.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:33 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:33.428+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:33.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:33 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:33.428+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:33.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:33 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:33.428+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:33.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:33 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:33.428+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:33.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:33 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:33.428+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:33.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:33 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:33.428+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:34.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:34 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:34.528+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:34.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:34 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:34.528+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:34.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:34 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:34.528+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:34.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:34 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:34.528+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:34.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:34 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:34.528+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:34.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:34 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:34.528+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:34.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:34 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:34.541+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:34.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:34 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:34.541+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:34.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:34 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:34.541+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:34.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:34 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:34.541+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:34.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:34 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:34.541+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:34.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:34 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:34.541+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:34.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:34 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:34.393+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:34.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:34 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:34.393+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:34.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:34 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:34.393+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:34.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:34 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:34.393+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:34.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:34 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:34.393+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:34.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:34 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:34.393+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:34.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:34 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:34.393+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:35.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:35 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:35.377+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:35.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:35 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:35.377+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:35.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:35 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:35.377+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:35.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:35 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:35.377+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:35.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:35 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:35.377+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:35.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:35 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:35.377+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:35.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:35 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:35.377+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:35.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:35 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:35.522+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:35.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:35 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:35.522+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:35.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:35 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:35.522+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:35.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:35 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:35.522+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:35.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:35 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:35.522+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:35.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:35 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:35.522+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:35.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:35 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:35.574+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.574212+0000 front 2024-02-26T10:50:17.574272+0000 (oldest deadline 2024-02-26T10:50:41.674404+0000) 2024-02-26T10:51:35.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:35 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:35.574+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.078042+0000 front 2024-02-26T10:50:28.078111+0000 (oldest deadline 2024-02-26T10:50:53.378263+0000) 2024-02-26T10:51:35.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:35 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:35.574+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.280845+0000 front 2024-02-26T10:50:36.280965+0000 (oldest deadline 2024-02-26T10:50:58.581106+0000) 2024-02-26T10:51:35.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:35 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:35.574+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:43.185152+0000 front 2024-02-26T10:50:43.185451+0000 (oldest deadline 2024-02-26T10:51:07.285395+0000) 2024-02-26T10:51:35.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:35 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:35.574+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:53.188285+0000 front 2024-02-26T10:50:53.188567+0000 (oldest deadline 2024-02-26T10:51:15.488449+0000) 2024-02-26T10:51:35.862 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:35 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9[66746]: 2024-02-26T10:51:35.574+0000 7effb4c00700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.793647+0000 front 2024-02-26T10:51:01.793726+0000 (oldest deadline 2024-02-26T10:51:22.893938+0000) 2024-02-26T10:51:36.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:36 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:36.518+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:36.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:36 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:36.518+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:36.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:36 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:36.518+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:36.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:36 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:36.518+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:36.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:36 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:36.518+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:36.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:36 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:36.518+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:36.861 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:36 smithi060 podman[92512]: 2024-02-26 10:51:36.613641391 +0000 UTC m=+5.765312667 container died f8d98b53910a5757b23a576e818b49b5a5f662cdf13f4660dfc299cdd2869495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-26T10:51:36.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:36 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:36.400+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:36.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:36 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:36.400+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:36.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:36 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:36.400+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:36.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:36 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:36.400+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:36.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:36 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:36.400+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:36.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:36 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:36.400+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:36.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:36 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:36.400+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:37.234 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:37 smithi060 podman[92512]: 2024-02-26 10:51:37.064259909 +0000 UTC m=+6.215931178 container cleanup f8d98b53910a5757b23a576e818b49b5a5f662cdf13f4660dfc299cdd2869495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:51:37.234 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:37 smithi060 bash[92512]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9 2024-02-26T10:51:37.611 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:37 smithi060 podman[92530]: 2024-02-26 10:51:37.240206107 +0000 UTC m=+0.613861732 container remove f8d98b53910a5757b23a576e818b49b5a5f662cdf13f4660dfc299cdd2869495 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-26T10:51:37.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:37 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:37.557+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:37.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:37 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:37.557+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:37.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:37 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:37.557+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:37.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:37 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:37.557+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:37.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:37 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:37.557+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:37.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:37 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:37.557+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:37.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:37 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:37.398+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:37.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:37 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:37.398+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:37.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:37 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:37.398+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:37.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:37 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:37.398+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:37.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:37 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:37.398+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:37.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:37 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:37.398+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:37.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:37 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:37.398+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:38.272 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:38 smithi060 podman[92620]: 2024-02-26 10:51:37.93274523 +0000 UTC m=+0.034554586 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:51:38.272 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:38 smithi060 podman[92620]: 2024-02-26 10:51:38.04134783 +0000 UTC m=+0.143157227 container create c34c59a0b5180f75909b260fafa492b2f936e8ba479819a9ccf2e2cd55a8c71c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2) 2024-02-26T10:51:38.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:38 smithi060 podman[92620]: 2024-02-26 10:51:38.411243695 +0000 UTC m=+0.513053066 container init c34c59a0b5180f75909b260fafa492b2f936e8ba479819a9ccf2e2cd55a8c71c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-02-26T10:51:38.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:38 smithi060 podman[92620]: 2024-02-26 10:51:38.422180723 +0000 UTC m=+0.523990107 container start c34c59a0b5180f75909b260fafa492b2f936e8ba479819a9ccf2e2cd55a8c71c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-26T10:51:38.612 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:38 smithi060 podman[92620]: 2024-02-26 10:51:38.493611368 +0000 UTC m=+0.595420740 container attach c34c59a0b5180f75909b260fafa492b2f936e8ba479819a9ccf2e2cd55a8c71c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-26T10:51:38.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:38 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:38.510+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:38.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:38 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:38.510+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:38.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:38 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:38.510+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:38.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:38 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:38.510+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:38.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:38 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:38.510+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:38.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:38 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:38.510+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:38.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:38 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:38.423+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:38.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:38 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:38.423+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:38.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:38 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:38.423+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:38.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:38 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:38.423+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:38.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:38 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:38.423+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:38.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:38 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:38.423+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:38.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:38 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:38.423+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:39.266 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 10:51:38 smithi060 podman[92620]: 2024-02-26 10:51:38.888347313 +0000 UTC m=+0.990156694 container died c34c59a0b5180f75909b260fafa492b2f936e8ba479819a9ccf2e2cd55a8c71c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-9-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-02-26T10:51:39.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.9.service' 2024-02-26T10:51:39.748 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:51:39.748 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-26T10:51:39.748 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-26T10:51:39.748 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.10 2024-02-26T10:51:39.779 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:39 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:39.445+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:39.779 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:39 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:39.445+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:39.779 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:39 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:39.445+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:39.779 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:39 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:39.445+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:39.779 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:39 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:39.445+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:39.779 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:39 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:39.445+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:39.780 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:39 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:39.445+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:39.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:39 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:39.470+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:39.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:39 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:39.470+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:39.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:39 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:39.470+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:39.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:39 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:39.470+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:39.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:39 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:39.470+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:39.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:39 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:39.470+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:40.055 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:39 smithi100 systemd[1]: Stopping Ceph osd.10 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:51:40.055 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.062+0000 7fb7232da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:51:40.056 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.062+0000 7fb7232da700 -1 osd.10 99 *** Got signal Terminated *** 2024-02-26T10:51:40.373 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.062+0000 7fb7232da700 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:51:40.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:40 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:40.424+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:40.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:40 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:40.424+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:40.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:40 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:40.424+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:40.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:40 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:40.424+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:40.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:40 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:40.424+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:40.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:40 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:40.424+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:40.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.409+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:40.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.409+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:40.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.409+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:40.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.409+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:40.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.409+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:40.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.409+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:40.874 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:40 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:40.409+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:41.829 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:41 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:41.388+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:41.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:41 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:41.388+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:41.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:41 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:41.388+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:41.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:41 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:41.388+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:41.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:41 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:41.388+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:41.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:41 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:41.388+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:41.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:41 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:41.412+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:41.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:41 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:41.412+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:41.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:41 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:41.412+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:41.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:41 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:41.412+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:41.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:41 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:41.412+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:41.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:41 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:41.412+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:41.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:41 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:41.412+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:42.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:42 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:42.375+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:42.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:42 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:42.375+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:42.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:42 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:42.375+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:42.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:42 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:42.375+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:42.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:42 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:42.375+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:42.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:42 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:42.375+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:42.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:42 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:42.375+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:42.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:42.361+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:42.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:42.361+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:42.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:42.361+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:42.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:42.361+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:42.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:42.361+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:42.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:42.361+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:42.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:42 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:42.361+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:43.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:43 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:43.356+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:43.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:43 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:43.356+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:43.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:43 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:43.356+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:43.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:43 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:43.356+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:43.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:43 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:43.356+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:43.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:43 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:43.356+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:43.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:43 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:43.357+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:43.829 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:43 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:43.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:43.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:43 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:43.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:43.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:43 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:43.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:43.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:43 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:43.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:43.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:43 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:43.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:43.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:43 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:43.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:43.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:43 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:43.404+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:44.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:44 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:44.331+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:17.879028+0000 front 2024-02-26T10:50:17.878956+0000 (oldest deadline 2024-02-26T10:50:41.378926+0000) 2024-02-26T10:51:44.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:44 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:44.331+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:27.180436+0000 front 2024-02-26T10:50:27.180465+0000 (oldest deadline 2024-02-26T10:50:51.280598+0000) 2024-02-26T10:51:44.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:44 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:44.331+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:35.381718+0000 front 2024-02-26T10:50:35.381656+0000 (oldest deadline 2024-02-26T10:50:57.681788+0000) 2024-02-26T10:51:44.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:44 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:44.331+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T10:50:44.683627+0000 front 2024-02-26T10:50:44.683550+0000 (oldest deadline 2024-02-26T10:51:09.383439+0000) 2024-02-26T10:51:44.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:44 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:44.331+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:52.884192+0000 front 2024-02-26T10:50:52.884236+0000 (oldest deadline 2024-02-26T10:51:15.184362+0000) 2024-02-26T10:51:44.623 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:44 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:44.331+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:51:01.586026+0000 front 2024-02-26T10:51:01.586081+0000 (oldest deadline 2024-02-26T10:51:24.485996+0000) 2024-02-26T10:51:44.624 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:44 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10[61395]: 2024-02-26T10:51:44.331+0000 7fb71f8ce700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:07.987347+0000 front 2024-02-26T10:51:07.987290+0000 (oldest deadline 2024-02-26T10:51:31.987439+0000) 2024-02-26T10:51:44.829 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:44.454+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:44.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:44.454+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:44.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:44.454+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:44.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:44.454+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:44.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:44.454+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:44.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:44.454+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:44.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:44.454+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:45.797 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:45 smithi100 podman[70775]: 2024-02-26 10:51:45.551242273 +0000 UTC m=+5.600609569 container died dd719bd8767290519147c1859344011ab38d239e49464c9380f26c6010448e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:51:45.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:45 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:45.451+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:45.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:45 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:45.451+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:45.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:45 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:45.451+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:45.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:45 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:45.451+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:45.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:45 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:45.451+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:45.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:45 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:45.451+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:45.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:45 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:45.451+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:46.072 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:45 smithi100 podman[70775]: 2024-02-26 10:51:45.887901085 +0000 UTC m=+5.937268383 container cleanup dd719bd8767290519147c1859344011ab38d239e49464c9380f26c6010448e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:51:46.072 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:45 smithi100 bash[70775]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10 2024-02-26T10:51:46.373 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:46 smithi100 podman[70792]: 2024-02-26 10:51:46.080734474 +0000 UTC m=+0.528259371 container remove dd719bd8767290519147c1859344011ab38d239e49464c9380f26c6010448e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-26T10:51:46.829 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:46.401+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:46.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:46.401+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:46.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:46.401+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:46.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:46.401+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:46.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:46.401+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:46.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:46.401+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:46.830 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:46 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:46.401+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:46.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:46 smithi100 podman[70879]: 2024-02-26 10:51:46.489728273 +0000 UTC m=+0.105387023 container create f0b42fd04470ea8fe6a972f673bc0e73bd7e8718703e443d775c979283841616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-26T10:51:46.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:46 smithi100 podman[70879]: 2024-02-26 10:51:46.402185741 +0000 UTC m=+0.017844516 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:51:46.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:46 smithi100 podman[70879]: 2024-02-26 10:51:46.676005929 +0000 UTC m=+0.291664701 container init f0b42fd04470ea8fe6a972f673bc0e73bd7e8718703e443d775c979283841616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-26T10:51:46.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:46 smithi100 podman[70879]: 2024-02-26 10:51:46.679206671 +0000 UTC m=+0.294865441 container start f0b42fd04470ea8fe6a972f673bc0e73bd7e8718703e443d775c979283841616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10-deactivate, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:51:46.873 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:46 smithi100 podman[70879]: 2024-02-26 10:51:46.73373383 +0000 UTC m=+0.349392611 container attach f0b42fd04470ea8fe6a972f673bc0e73bd7e8718703e443d775c979283841616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10-deactivate, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-26T10:51:47.244 INFO:journalctl@ceph.osd.10.smithi100.stdout:Feb 26 10:51:46 smithi100 podman[70879]: 2024-02-26 10:51:46.962419971 +0000 UTC m=+0.578078733 container died f0b42fd04470ea8fe6a972f673bc0e73bd7e8718703e443d775c979283841616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-26T10:51:47.522 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.10.service' 2024-02-26T10:51:47.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:51:47.603 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-26T10:51:47.603 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-26T10:51:47.603 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.11 2024-02-26T10:51:47.632 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.352+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:47.632 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.352+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:47.632 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.352+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:47.632 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.352+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:47.632 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.352+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:47.632 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.352+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:47.632 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.352+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:47.923 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 systemd[1]: Stopping Ceph osd.11 for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:51:48.327 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.930+0000 7fcda63f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:51:48.327 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.930+0000 7fcda63f2700 -1 osd.11 99 *** Got signal Terminated *** 2024-02-26T10:51:48.327 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:47 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:47.930+0000 7fcda63f2700 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T10:51:48.327 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:48 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:48.334+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:48.327 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:48 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:48.334+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:48.327 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:48 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:48.334+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:48.327 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:48 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:48.334+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:48.327 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:48 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:48.334+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:48.327 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:48 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:48.334+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:48.328 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:48 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:48.334+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:49.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:49 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:49.312+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:49.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:49 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:49.312+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:49.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:49 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:49.312+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:49.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:49 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:49.312+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:49.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:49 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:49.312+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:49.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:49 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:49.312+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:49.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:49 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:49.312+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:50.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:50.297+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:50.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:50.297+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:50.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:50.297+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:50.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:50.297+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:50.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:50.297+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:50.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:50.297+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:50.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:50 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:50.297+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:51.685 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:51.345+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:51.685 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:51.345+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:51.686 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:51.345+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:51.686 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:51.345+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:51.686 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:51.345+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:51.686 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:51.345+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:51.686 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:51.345+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:51.686 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:51 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:51.345+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6820 osd.8 since back 2024-02-26T10:51:24.728537+0000 front 2024-02-26T10:51:24.728502+0000 (oldest deadline 2024-02-26T10:51:50.628560+0000) 2024-02-26T10:51:52.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:52.296+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T10:50:15.215308+0000 front 2024-02-26T10:50:15.215381+0000 (oldest deadline 2024-02-26T10:50:40.515278+0000) 2024-02-26T10:51:52.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:52.296+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-02-26T10:50:28.116908+0000 front 2024-02-26T10:50:28.116966+0000 (oldest deadline 2024-02-26T10:50:50.417052+0000) 2024-02-26T10:51:52.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:52.296+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-02-26T10:50:36.317990+0000 front 2024-02-26T10:50:36.317814+0000 (oldest deadline 2024-02-26T10:51:01.617956+0000) 2024-02-26T10:51:52.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:52.296+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-02-26T10:50:54.321272+0000 front 2024-02-26T10:50:54.321361+0000 (oldest deadline 2024-02-26T10:51:15.921981+0000) 2024-02-26T10:51:52.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:52.296+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-02-26T10:50:58.123493+0000 front 2024-02-26T10:50:58.123462+0000 (oldest deadline 2024-02-26T10:51:24.023586+0000) 2024-02-26T10:51:52.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:52.296+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T10:51:10.425577+0000 front 2024-02-26T10:51:10.425363+0000 (oldest deadline 2024-02-26T10:51:33.925190+0000) 2024-02-26T10:51:52.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:52.296+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-02-26T10:51:19.027219+0000 front 2024-02-26T10:51:19.027286+0000 (oldest deadline 2024-02-26T10:51:41.927169+0000) 2024-02-26T10:51:52.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:52 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11[61478]: 2024-02-26T10:51:52.296+0000 7fcda29e6700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.113:6820 osd.8 since back 2024-02-26T10:51:24.728537+0000 front 2024-02-26T10:51:24.728502+0000 (oldest deadline 2024-02-26T10:51:50.628560+0000) 2024-02-26T10:51:53.580 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:53 smithi113 podman[71248]: 2024-02-26 10:51:53.28496433 +0000 UTC m=+5.474543905 container died f7aed9b784c2d04df00cb78300f91c1715e767b5743dff5a0e1276fd36301977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-26T10:51:54.079 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:53 smithi113 podman[71248]: 2024-02-26 10:51:53.597736222 +0000 UTC m=+5.787315796 container cleanup f7aed9b784c2d04df00cb78300f91c1715e767b5743dff5a0e1276fd36301977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-26T10:51:54.080 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:53 smithi113 bash[71248]: ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11 2024-02-26T10:51:54.080 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:53 smithi113 podman[71277]: 2024-02-26 10:51:53.835203321 +0000 UTC m=+0.553029411 container remove f7aed9b784c2d04df00cb78300f91c1715e767b5743dff5a0e1276fd36301977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:51:54.561 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:54 smithi113 podman[71369]: 2024-02-26 10:51:54.158839424 +0000 UTC m=+0.017734535 image pull 07b056410b3767055232636912482890c135a7573ee79d27ba8d5f4a2a801763 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a 2024-02-26T10:51:54.561 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:54 smithi113 podman[71369]: 2024-02-26 10:51:54.266378253 +0000 UTC m=+0.125273358 container create d33b266870078bb406210d947b888cf881daf69268309a458cd4fdad2c55cf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:51:54.561 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:54 smithi113 podman[71369]: 2024-02-26 10:51:54.486490884 +0000 UTC m=+0.345386000 container init d33b266870078bb406210d947b888cf881daf69268309a458cd4fdad2c55cf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11-deactivate, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222) 2024-02-26T10:51:54.561 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:54 smithi113 podman[71369]: 2024-02-26 10:51:54.489704795 +0000 UTC m=+0.348599906 container start d33b266870078bb406210d947b888cf881daf69268309a458cd4fdad2c55cf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:51:54.561 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:54 smithi113 podman[71369]: 2024-02-26 10:51:54.569529062 +0000 UTC m=+0.428424193 container attach d33b266870078bb406210d947b888cf881daf69268309a458cd4fdad2c55cf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-26T10:51:54.829 INFO:journalctl@ceph.osd.11.smithi113.stdout:Feb 26 10:51:54 smithi113 podman[71369]: 2024-02-26 10:51:54.772630764 +0000 UTC m=+0.631525876 container died d33b266870078bb406210d947b888cf881daf69268309a458cd4fdad2c55cf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-26T10:51:55.329 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@osd.11.service' 2024-02-26T10:51:55.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:51:55.411 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-26T10:51:55.411 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-26T10:51:55.411 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.a 2024-02-26T10:51:55.861 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 10:51:55 smithi060 systemd[1]: Stopping Ceph mds.a for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:51:56.362 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 10:51:56 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-a[73281]: 2024-02-26T10:51:56.046+0000 7f7bc9200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:51:56.362 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 10:51:56 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-a[73281]: 2024-02-26T10:51:56.046+0000 7f7bc9200700 -1 mds.a *** got signal Terminated *** 2024-02-26T10:51:57.588 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 10:51:57 smithi060 podman[92736]: 2024-02-26 10:51:57.313471467 +0000 UTC m=+1.424152874 container died 5647321050e4e803871688f16e4aab95f6883ce625f2052eee64f3908cfd0170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0) 2024-02-26T10:51:57.861 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 10:51:57 smithi060 podman[92736]: 2024-02-26 10:51:57.707661619 +0000 UTC m=+1.818343006 container cleanup 5647321050e4e803871688f16e4aab95f6883ce625f2052eee64f3908cfd0170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-a, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-26T10:51:57.862 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 10:51:57 smithi060 bash[92736]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-a 2024-02-26T10:51:58.231 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 10:51:57 smithi060 podman[92750]: 2024-02-26 10:51:57.983713746 +0000 UTC m=+0.653794036 container remove 5647321050e4e803871688f16e4aab95f6883ce625f2052eee64f3908cfd0170 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-a, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:51:58.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.a.service' 2024-02-26T10:51:58.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:51:58.388 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-26T10:51:58.388 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-26T10:51:58.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.d 2024-02-26T10:51:58.861 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:51:58 smithi060 systemd[1]: Stopping Ceph mds.d for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:51:59.361 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:51:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d[72277]: 2024-02-26T10:51:58.917+0000 7fb683c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:51:59.362 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:51:58 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d[72277]: 2024-02-26T10:51:58.917+0000 7fb683c00700 -1 mds.d *** got signal Terminated *** 2024-02-26T10:52:02.534 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:52:02 smithi060 podman[92856]: 2024-02-26 10:52:02.262951369 +0000 UTC m=+3.457397449 container died 15451d5bb6800e0bb5ab2b18e9584c8c9090b63b5678e202fab0fda18c62e576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:52:02.861 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:52:02 smithi060 podman[92856]: 2024-02-26 10:52:02.661053201 +0000 UTC m=+3.855499275 container cleanup 15451d5bb6800e0bb5ab2b18e9584c8c9090b63b5678e202fab0fda18c62e576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS) 2024-02-26T10:52:02.862 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:52:02 smithi060 bash[92856]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d 2024-02-26T10:52:03.361 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 10:52:02 smithi060 podman[92870]: 2024-02-26 10:52:02.937225379 +0000 UTC m=+0.657880683 container remove 15451d5bb6800e0bb5ab2b18e9584c8c9090b63b5678e202fab0fda18c62e576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222) 2024-02-26T10:52:03.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.d.service' 2024-02-26T10:52:03.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:03.619 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-26T10:52:03.619 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-26T10:52:03.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.g 2024-02-26T10:52:04.111 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:52:03 smithi060 systemd[1]: Stopping Ceph mds.g for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:04.611 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:52:04 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g[72899]: 2024-02-26T10:52:04.362+0000 7fd713000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:04.611 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:52:04 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g[72899]: 2024-02-26T10:52:04.362+0000 7fd713000700 -1 mds.g *** got signal Terminated *** 2024-02-26T10:52:08.647 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:52:08 smithi060 podman[92975]: 2024-02-26 10:52:08.379877151 +0000 UTC m=+4.282208815 container died 07cd66bddb7afb2319197e8782400f69303204b3ae9a974329e9111ea347e5ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-26T10:52:08.908 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:52:08 smithi060 podman[92975]: 2024-02-26 10:52:08.756473665 +0000 UTC m=+4.658805325 container cleanup 07cd66bddb7afb2319197e8782400f69303204b3ae9a974329e9111ea347e5ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T10:52:08.908 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:52:08 smithi060 bash[92975]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g 2024-02-26T10:52:09.361 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 10:52:08 smithi060 podman[92989]: 2024-02-26 10:52:08.915747231 +0000 UTC m=+0.513697659 container remove 07cd66bddb7afb2319197e8782400f69303204b3ae9a974329e9111ea347e5ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-g, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, CEPH_POINT_RELEASE=) 2024-02-26T10:52:09.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.g.service' 2024-02-26T10:52:09.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:09.538 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-26T10:52:09.538 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-26T10:52:09.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.j 2024-02-26T10:52:10.111 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:52:09 smithi060 systemd[1]: Stopping Ceph mds.j for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:10.611 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:52:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j[71912]: 2024-02-26T10:52:10.191+0000 7f1a28000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:10.611 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:52:10 smithi060 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j[71912]: 2024-02-26T10:52:10.191+0000 7f1a28000700 -1 mds.j *** got signal Terminated *** 2024-02-26T10:52:14.299 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:52:14 smithi060 podman[93096]: 2024-02-26 10:52:14.017816572 +0000 UTC m=+3.961579021 container died d938fc1db71a026c61c412bea42b03867debf04cd9f7639a393364e9c98898f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:52:14.611 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:52:14 smithi060 podman[93096]: 2024-02-26 10:52:14.476773718 +0000 UTC m=+4.420536160 container cleanup d938fc1db71a026c61c412bea42b03867debf04cd9f7639a393364e9c98898f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-26T10:52:14.611 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:52:14 smithi060 bash[93096]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j 2024-02-26T10:52:15.111 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 10:52:14 smithi060 podman[93109]: 2024-02-26 10:52:14.711154971 +0000 UTC m=+0.674903640 container remove d938fc1db71a026c61c412bea42b03867debf04cd9f7639a393364e9c98898f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:52:15.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.j.service' 2024-02-26T10:52:15.339 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:15.340 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-26T10:52:15.340 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-26T10:52:15.340 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.b 2024-02-26T10:52:15.623 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:52:15 smithi100 systemd[1]: Stopping Ceph mds.b for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:15.888 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:52:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b[66645]: 2024-02-26T10:52:15.651+0000 7f9049207700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:15.888 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:52:15 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b[66645]: 2024-02-26T10:52:15.651+0000 7f9049207700 -1 mds.b *** got signal Terminated *** 2024-02-26T10:52:19.994 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:52:19 smithi100 podman[70987]: 2024-02-26 10:52:19.661494021 +0000 UTC m=+4.120830927 container died ccbf93bac5b91c86b99630adf11d1bfac608cbfdc1592cc16244131625ce58b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-26T10:52:20.319 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:52:20 smithi100 podman[70987]: 2024-02-26 10:52:20.068832012 +0000 UTC m=+4.528168945 container cleanup ccbf93bac5b91c86b99630adf11d1bfac608cbfdc1592cc16244131625ce58b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-26T10:52:20.320 INFO:journalctl@ceph.mds.b.smithi100.stdout:Feb 26 10:52:20 smithi100 bash[70987]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-b 2024-02-26T10:52:20.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.b.service' 2024-02-26T10:52:20.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:20.468 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-26T10:52:20.468 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-26T10:52:20.468 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.e 2024-02-26T10:52:20.623 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:52:20 smithi100 systemd[1]: Stopping Ceph mds.e for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:21.123 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:52:20 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e[66010]: 2024-02-26T10:52:20.807+0000 7fa083245700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:21.123 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:52:20 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e[66010]: 2024-02-26T10:52:20.807+0000 7fa083245700 -1 mds.e *** got signal Terminated *** 2024-02-26T10:52:24.030 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:52:23 smithi100 podman[71121]: 2024-02-26 10:52:23.718681756 +0000 UTC m=+3.024702777 container died e36bf7a5fde2888107c2c3daee0ea14c045769d794bd715db3731aab02576fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-26T10:52:24.030 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:52:24 smithi100 podman[71121]: 2024-02-26 10:52:24.038905132 +0000 UTC m=+3.344926147 container cleanup e36bf7a5fde2888107c2c3daee0ea14c045769d794bd715db3731aab02576fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, ceph=True, io.buildah.version=1.33.5) 2024-02-26T10:52:24.373 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:52:24 smithi100 bash[71121]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e 2024-02-26T10:52:24.373 INFO:journalctl@ceph.mds.e.smithi100.stdout:Feb 26 10:52:24 smithi100 podman[71136]: 2024-02-26 10:52:24.248484334 +0000 UTC m=+0.525929353 container remove e36bf7a5fde2888107c2c3daee0ea14c045769d794bd715db3731aab02576fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-02-26T10:52:24.460 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.e.service' 2024-02-26T10:52:24.494 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:24.494 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-26T10:52:24.494 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-26T10:52:24.494 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.h 2024-02-26T10:52:24.834 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:52:24 smithi100 systemd[1]: Stopping Ceph mds.h for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:25.123 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:52:24 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:52:24.842+0000 7f5a6bfb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:25.123 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:52:24 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h[66324]: 2024-02-26T10:52:24.842+0000 7f5a6bfb5700 -1 mds.h *** got signal Terminated *** 2024-02-26T10:52:27.976 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:52:27 smithi100 podman[71242]: 2024-02-26 10:52:27.720382524 +0000 UTC m=+3.009816711 container died bc785a3a01e38a0b314ef42f0ae1e99a121559c900a32a28e6de52940f5982fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:52:28.318 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:52:28 smithi100 podman[71242]: 2024-02-26 10:52:28.08410851 +0000 UTC m=+3.373542697 container cleanup bc785a3a01e38a0b314ef42f0ae1e99a121559c900a32a28e6de52940f5982fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91) 2024-02-26T10:52:28.318 INFO:journalctl@ceph.mds.h.smithi100.stdout:Feb 26 10:52:28 smithi100 bash[71242]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-h 2024-02-26T10:52:28.456 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.h.service' 2024-02-26T10:52:28.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:28.491 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-26T10:52:28.491 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-26T10:52:28.492 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.k 2024-02-26T10:52:28.873 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:52:28 smithi100 systemd[1]: Stopping Ceph mds.k for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:28.873 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:52:28 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-k[65694]: 2024-02-26T10:52:28.812+0000 7f6783b6e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:28.873 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:52:28 smithi100 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-k[65694]: 2024-02-26T10:52:28.812+0000 7f6783b6e700 -1 mds.k *** got signal Terminated *** 2024-02-26T10:52:30.313 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:52:30 smithi100 podman[71362]: 2024-02-26 10:52:30.042992748 +0000 UTC m=+1.332406534 container died 25c2b7bad73b39e9d72ca0a28682ed8f53a68124acdaedbdc719ccbffdc3aa81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-k, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:52:30.623 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:52:30 smithi100 podman[71362]: 2024-02-26 10:52:30.412757596 +0000 UTC m=+1.702171382 container cleanup 25c2b7bad73b39e9d72ca0a28682ed8f53a68124acdaedbdc719ccbffdc3aa81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-k, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T10:52:30.623 INFO:journalctl@ceph.mds.k.smithi100.stdout:Feb 26 10:52:30 smithi100 bash[71362]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-k 2024-02-26T10:52:30.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.k.service' 2024-02-26T10:52:30.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:30.853 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-26T10:52:30.853 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-26T10:52:30.853 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.c 2024-02-26T10:52:31.145 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:52:30 smithi113 systemd[1]: Stopping Ceph mds.c for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:31.579 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:52:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-c[66659]: 2024-02-26T10:52:31.153+0000 7ffb18f27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:31.579 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:52:31 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-c[66659]: 2024-02-26T10:52:31.153+0000 7ffb18f27700 -1 mds.c *** got signal Terminated *** 2024-02-26T10:52:35.531 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:52:35 smithi113 podman[71474]: 2024-02-26 10:52:35.167665412 +0000 UTC m=+4.119749184 container died 6f16811c682444bab7796bafdd63192fd2db53584f5a8ac18a43a6afa218af07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:52:35.829 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:52:35 smithi113 podman[71474]: 2024-02-26 10:52:35.622048379 +0000 UTC m=+4.574132131 container cleanup 6f16811c682444bab7796bafdd63192fd2db53584f5a8ac18a43a6afa218af07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_CLEAN=True) 2024-02-26T10:52:35.829 INFO:journalctl@ceph.mds.c.smithi113.stdout:Feb 26 10:52:35 smithi113 bash[71474]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-c 2024-02-26T10:52:36.087 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.c.service' 2024-02-26T10:52:36.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:36.126 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-26T10:52:36.126 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-26T10:52:36.126 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.f 2024-02-26T10:52:36.451 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:36 smithi113 systemd[1]: Stopping Ceph mds.f for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:36.829 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:36 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f[66346]: 2024-02-26T10:52:36.458+0000 7f8195019700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:36.829 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:36 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f[66346]: 2024-02-26T10:52:36.458+0000 7f8195019700 -1 mds.f *** got signal Terminated *** 2024-02-26T10:52:40.034 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:39 smithi113 podman[71606]: 2024-02-26 10:52:39.716412931 +0000 UTC m=+3.368513008 container died 8d5f20c4882e2349bb607baf781b6714aaccf5ec2748ee15800480745c79d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222) 2024-02-26T10:52:40.326 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:40 smithi113 podman[71606]: 2024-02-26 10:52:40.150491511 +0000 UTC m=+3.802591594 container cleanup 8d5f20c4882e2349bb607baf781b6714aaccf5ec2748ee15800480745c79d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222) 2024-02-26T10:52:40.326 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:40 smithi113 bash[71606]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f 2024-02-26T10:52:40.559 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.f.service' 2024-02-26T10:52:40.579 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:40 smithi113 podman[71620]: 2024-02-26 10:52:40.334933875 +0000 UTC m=+0.620878748 container remove 8d5f20c4882e2349bb607baf781b6714aaccf5ec2748ee15800480745c79d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-f, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-26T10:52:40.579 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:40 smithi113 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.f.service: Deactivated successfully. 2024-02-26T10:52:40.579 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:40 smithi113 systemd[1]: Stopped Ceph mds.f for 51464d50-d492-11ee-95c3-87774f69a715. 2024-02-26T10:52:40.579 INFO:journalctl@ceph.mds.f.smithi113.stdout:Feb 26 10:52:40 smithi113 systemd[1]: ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.f.service: Consumed 3.274s CPU time. 2024-02-26T10:52:40.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:40.595 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-26T10:52:40.595 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-26T10:52:40.595 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.i 2024-02-26T10:52:40.920 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:52:40 smithi113 systemd[1]: Stopping Ceph mds.i for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:41.329 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:52:40 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:52:40.928+0000 7f8849c44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:41.329 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:52:40 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i[66980]: 2024-02-26T10:52:40.928+0000 7f8849c44700 -1 mds.i *** got signal Terminated *** 2024-02-26T10:52:43.971 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:52:43 smithi113 podman[71727]: 2024-02-26 10:52:43.715387743 +0000 UTC m=+2.890768524 container died a587f82898b496712c4e35bf940a91ba4764121bf2de33daa90a66419fc68cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, CEPH_POINT_RELEASE=) 2024-02-26T10:52:44.329 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:52:44 smithi113 podman[71727]: 2024-02-26 10:52:44.087555678 +0000 UTC m=+3.262936452 container cleanup a587f82898b496712c4e35bf940a91ba4764121bf2de33daa90a66419fc68cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-26T10:52:44.329 INFO:journalctl@ceph.mds.i.smithi113.stdout:Feb 26 10:52:44 smithi113 bash[71727]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-i 2024-02-26T10:52:44.485 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.i.service' 2024-02-26T10:52:44.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:44.519 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-26T10:52:44.519 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-26T10:52:44.519 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.l 2024-02-26T10:52:45.079 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:52:44 smithi113 systemd[1]: Stopping Ceph mds.l for 51464d50-d492-11ee-95c3-87774f69a715... 2024-02-26T10:52:45.079 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:52:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-l[66040]: 2024-02-26T10:52:44.848+0000 7f5770919700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T10:52:45.079 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:52:44 smithi113 ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-l[66040]: 2024-02-26T10:52:44.848+0000 7f5770919700 -1 mds.l *** got signal Terminated *** 2024-02-26T10:52:47.366 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:52:47 smithi113 podman[71847]: 2024-02-26 10:52:47.120673164 +0000 UTC m=+2.373442329 container died 2110a133da3d24a4f6c7563bd9dceccff00d82eee098862e3bbbf70a1923b055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-l, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T10:52:47.741 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:52:47 smithi113 podman[71847]: 2024-02-26 10:52:47.474221935 +0000 UTC m=+2.726991087 container cleanup 2110a133da3d24a4f6c7563bd9dceccff00d82eee098862e3bbbf70a1923b055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:52cbbe1863dbabdde64eecf19ae24a4297ed3602011ebc98f420004c7a80336a, name=ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-l, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240226.064629-6085b91, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-26T10:52:47.741 INFO:journalctl@ceph.mds.l.smithi113.stdout:Feb 26 10:52:47 smithi113 bash[71847]: ceph-51464d50-d492-11ee-95c3-87774f69a715-mds-l 2024-02-26T10:52:47.880 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51464d50-d492-11ee-95c3-87774f69a715@mds.l.service' 2024-02-26T10:52:47.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T10:52:47.914 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-26T10:52:47.914 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51464d50-d492-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T10:52:48.256 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:52:53.500 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51464d50-d492-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T10:52:53.692 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:52:56.146 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51464d50-d492-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T10:52:56.335 INFO:teuthology.orchestra.run.smithi113.stdout:Deleting cluster with fsid: 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:52:58.804 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T10:52:58.916 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T10:52:58.952 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T10:52:58.981 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-26T10:52:58.983 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/remote/smithi060/crash 2024-02-26T10:52:58.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/crash -- . 2024-02-26T10:52:59.085 INFO:teuthology.orchestra.run.smithi060.stderr:tar: /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T10:52:59.086 INFO:teuthology.orchestra.run.smithi060.stderr:tar: Error is not recoverable: exiting now 2024-02-26T10:52:59.091 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/remote/smithi100/crash 2024-02-26T10:52:59.092 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/crash -- . 2024-02-26T10:52:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T10:52:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-02-26T10:52:59.139 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/remote/smithi113/crash 2024-02-26T10:52:59.140 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/crash -- . 2024-02-26T10:52:59.177 INFO:teuthology.orchestra.run.smithi113.stderr:tar: /var/lib/ceph/51464d50-d492-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T10:52:59.177 INFO:teuthology.orchestra.run.smithi113.stderr:tar: Error is not recoverable: exiting now 2024-02-26T10:52:59.178 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-26T10:52:59.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-26T10:52:59.291 INFO:tasks.cephadm:Compressing logs... 2024-02-26T10:52:59.292 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T10:52:59.296 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T10:52:59.299 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T10:52:59.332 INFO:teuthology.orchestra.run.smithi113.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-02-26T10:52:59.332 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-02-26T10:52:59.332 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.l.log 2024-02-26T10:52:59.332 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T10:52:59.332 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.i.log 2024-02-26T10:52:59.332 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.11.log 2024-02-26T10:52:59.332 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mgr.z.log 2024-02-26T10:52:59.332 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.i.log: /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.8.log 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.5.log 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.f.log 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.f.log: 90.1% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr: 94.8%gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.l.log.gz 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.2.log 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr: 82.4% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T10:52:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.c.log 2024-02-26T10:52:59.338 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-26T10:52:59.338 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T10:52:59.338 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.10.log 2024-02-26T10:52:59.338 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.b.log 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.4.log 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.1.log 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.k.log 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.h.log 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.7.log 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.e.log 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.7.log: 90.0% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mgr.y.log 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.k.log.gz 2024-02-26T10:52:59.339 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T10:52:59.341 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.2.log: 92.5% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mgr.z.log.gz 2024-02-26T10:52:59.342 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T10:52:59.342 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T10:52:59.342 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mon.c.log 2024-02-26T10:52:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mgr.y.log: 92.3% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mgr.y.log.gz 2024-02-26T10:52:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log 2024-02-26T10:52:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T10:52:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T10:52:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log: 88.5% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T10:52:59.346 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mon.b.log 2024-02-26T10:52:59.359 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T10:52:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T10:52:59.368 INFO:teuthology.orchestra.run.smithi060.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-26T10:52:59.368 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T10:52:59.369 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-02-26T10:52:59.370 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.6.log 2024-02-26T10:52:59.371 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mon.a.log 2024-02-26T10:52:59.373 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T10:52:59.373 INFO:teuthology.orchestra.run.smithi060.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-02-26T10:52:59.374 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.g.log 2024-02-26T10:52:59.375 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.j.log 2024-02-26T10:52:59.377 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.3.log 2024-02-26T10:52:59.377 INFO:teuthology.orchestra.run.smithi060.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T10:52:59.377 INFO:teuthology.orchestra.run.smithi060.stderr: 89.9% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T10:52:59.378 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mgr.x.log 2024-02-26T10:52:59.380 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.0.log 2024-02-26T10:52:59.380 INFO:teuthology.orchestra.run.smithi060.stderr: 94.5% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.g.log.gz 2024-02-26T10:52:59.381 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T10:52:59.383 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log 2024-02-26T10:52:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.cephadm.log: 94.6% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.j.log.gz 2024-02-26T10:52:59.384 INFO:teuthology.orchestra.run.smithi060.stderr: 82.6% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T10:52:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T10:52:59.386 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.9.log 2024-02-26T10:52:59.389 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.d.log 2024-02-26T10:52:59.389 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T10:52:59.390 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.a.log 2024-02-26T10:52:59.397 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.d.log: /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.a.log.gz 2024-02-26T10:52:59.402 INFO:teuthology.orchestra.run.smithi060.stderr: 94.2% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.d.log.gz 2024-02-26T10:52:59.404 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T10:52:59.552 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mgr.x.log.gz 2024-02-26T10:52:59.725 INFO:teuthology.orchestra.run.smithi113.stderr: 91.7% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mon.c.log.gz 2024-02-26T10:52:59.736 INFO:teuthology.orchestra.run.smithi100.stderr: 91.7% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mon.b.log.gz 2024-02-26T10:53:00.038 INFO:teuthology.orchestra.run.smithi113.stderr: 94.7% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.f.log.gz 2024-02-26T10:53:00.384 INFO:teuthology.orchestra.run.smithi100.stderr: 94.8% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.e.log.gz 2024-02-26T10:53:00.390 INFO:teuthology.orchestra.run.smithi060.stderr: 90.5% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mon.a.log.gz 2024-02-26T10:53:03.550 INFO:teuthology.orchestra.run.smithi113.stderr: 95.2% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.c.log.gz 2024-02-26T10:53:04.180 INFO:teuthology.orchestra.run.smithi100.stderr: 92.0% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.10.log.gz 2024-02-26T10:53:05.387 INFO:teuthology.orchestra.run.smithi113.stderr: 91.8% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.11.log.gz 2024-02-26T10:53:06.259 INFO:teuthology.orchestra.run.smithi060.stderr: 91.7% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.0.log.gz 2024-02-26T10:53:06.585 INFO:teuthology.orchestra.run.smithi113.stderr: 91.8% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.8.log.gz 2024-02-26T10:53:06.983 INFO:teuthology.orchestra.run.smithi100.stderr: 92.1% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.1.log.gz 2024-02-26T10:53:07.155 INFO:teuthology.orchestra.run.smithi113.stderr: 93.7% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.i.log.gz 2024-02-26T10:53:07.524 INFO:teuthology.orchestra.run.smithi060.stderr: 91.8% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.6.log.gz 2024-02-26T10:53:07.760 INFO:teuthology.orchestra.run.smithi060.stderr: 91.7% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.9.log.gz 2024-02-26T10:53:07.799 INFO:teuthology.orchestra.run.smithi060.stderr: 91.8% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.3.log.gz 2024-02-26T10:53:07.802 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-02-26T10:53:07.802 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m8.471s 2024-02-26T10:53:07.802 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m30.844s 2024-02-26T10:53:07.802 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m2.325s 2024-02-26T10:53:08.853 INFO:teuthology.orchestra.run.smithi100.stderr: 91.6% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.7.log.gz 2024-02-26T10:53:08.939 INFO:teuthology.orchestra.run.smithi113.stderr: 91.8% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.2.log.gz 2024-02-26T10:53:09.918 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.h.log.gz 2024-02-26T10:53:10.859 INFO:teuthology.orchestra.run.smithi100.stderr: 91.8% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.4.log.gz 2024-02-26T10:53:11.575 INFO:teuthology.orchestra.run.smithi113.stderr: 91.6% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-osd.5.log.gz 2024-02-26T10:53:11.577 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-02-26T10:53:11.577 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m12.264s 2024-02-26T10:53:11.577 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m46.868s 2024-02-26T10:53:11.577 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m1.395s 2024-02-26T10:53:32.066 INFO:teuthology.orchestra.run.smithi100.stderr: 93.1% -- replaced with /var/log/ceph/51464d50-d492-11ee-95c3-87774f69a715/ceph-mds.b.log.gz 2024-02-26T10:53:32.067 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-02-26T10:53:32.067 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m32.754s 2024-02-26T10:53:32.067 INFO:teuthology.orchestra.run.smithi100.stderr:user 1m15.956s 2024-02-26T10:53:32.067 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m2.263s 2024-02-26T10:53:32.068 INFO:tasks.cephadm:Archiving logs... 2024-02-26T10:53:32.071 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/remote/smithi060/log 2024-02-26T10:53:32.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T10:53:37.267 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/remote/smithi100/log 2024-02-26T10:53:37.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T10:53:48.307 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/remote/smithi113/log 2024-02-26T10:53:48.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T10:53:55.873 INFO:tasks.cephadm:Removing cluster... 2024-02-26T10:53:55.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51464d50-d492-11ee-95c3-87774f69a715 --force 2024-02-26T10:53:56.221 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:53:57.012 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51464d50-d492-11ee-95c3-87774f69a715 --force 2024-02-26T10:53:57.204 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:53:57.548 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51464d50-d492-11ee-95c3-87774f69a715 --force 2024-02-26T10:53:57.740 INFO:teuthology.orchestra.run.smithi113.stdout:Deleting cluster with fsid: 51464d50-d492-11ee-95c3-87774f69a715 2024-02-26T10:53:58.086 INFO:tasks.cephadm:Removing cephadm ... 2024-02-26T10:53:58.086 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T10:53:58.149 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T10:53:58.166 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T10:53:58.181 INFO:tasks.cephadm:Teardown complete 2024-02-26T10:53:58.181 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-26T10:53:58.192 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-26T10:53:58.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T10:53:58.201 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T10:53:58.208 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T10:53:58.371 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-26T10:53:58.382 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-26T10:53:58.383 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:53:58.391 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:53:58.394 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:53:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:53:58.407 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:53:58.410 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:53:58.410 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-02-26T10:53:58.410 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 49 +1918us[+1918us] +/- 126ms 2024-02-26T10:53:58.410 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv02.front.sepia.ceph.com 3 6 377 50 -2752us[-3045us] +/- 54ms 2024-02-26T10:53:58.410 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +2489us[+2489us] +/- 77ms 2024-02-26T10:53:58.410 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:53:58.412 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:53:58.412 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-02-26T10:53:58.412 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 +1938us[+1938us] +/- 126ms 2024-02-26T10:53:58.412 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -2731us[-2995us] +/- 54ms 2024-02-26T10:53:58.412 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +2496us[+2233us] +/- 77ms 2024-02-26T10:53:58.412 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:53:58.441 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:53:58.452 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:53:58.452 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-02-26T10:53:58.452 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 +1911us[+1911us] +/- 126ms 2024-02-26T10:53:58.452 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv02.front.sepia.ceph.com 3 6 377 50 -2758us[-2964us] +/- 54ms 2024-02-26T10:53:58.453 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 114 +2489us[+2277us] +/- 76ms 2024-02-26T10:53:58.453 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:53:58.454 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-26T10:53:58.465 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-26T10:53:58.465 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-26T10:53:58.487 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T10:53:58.738 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T10:53:58.811 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T10:53:58.883 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:53:58.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:53:58.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:53:59.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:53:59.025 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:53:59.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:53:59.056 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-26T10:53:59.067 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-26T10:53:59.078 INFO:teuthology.task.internal:Duration was 1857.115006 seconds 2024-02-26T10:53:59.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-26T10:53:59.088 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-26T10:53:59.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T10:53:59.097 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T10:53:59.099 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T10:53:59.140 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:53:59.142 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:53:59.270 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:53:59.692 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-26T10:53:59.692 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T10:53:59.692 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T10:53:59.757 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-02-26T10:53:59.758 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T10:53:59.787 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-02-26T10:53:59.787 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T10:53:59.814 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-26T10:53:59.814 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T10:53:59.822 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T10:53:59.830 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T10:53:59.924 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-26T10:53:59.924 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T10:53:59.930 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T10:53:59.932 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T10:54:00.440 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-26T10:54:00.452 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-26T10:54:00.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T10:54:00.557 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T10:54:00.584 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T10:54:00.614 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-26T10:54:00.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:54:00.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:54:00.637 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:54:00.660 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-02-26T10:54:00.681 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-02-26T10:54:00.745 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-02-26T10:54:00.803 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:54:00.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:54:00.857 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:54:00.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:54:00.872 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:54:00.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:54:00.886 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-26T10:54:00.898 INFO:teuthology.task.internal:Transferring archived files... 2024-02-26T10:54:00.899 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/remote/smithi060 2024-02-26T10:54:00.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T10:54:01.025 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/remote/smithi100 2024-02-26T10:54:01.025 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T10:54:01.067 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562/remote/smithi113 2024-02-26T10:54:01.067 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T10:54:01.108 INFO:teuthology.task.internal:Removing archive directory... 2024-02-26T10:54:01.108 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T10:54:01.116 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T10:54:01.118 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T10:54:01.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-26T10:54:01.183 INFO:teuthology.task.internal:Not uploading archives. 2024-02-26T10:54:01.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-26T10:54:01.196 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-26T10:54:01.197 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T10:54:01.205 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T10:54:01.207 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T10:54:01.219 INFO:teuthology.orchestra.run.smithi100.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 10:54 /home/ubuntu/cephtest 2024-02-26T10:54:01.222 INFO:teuthology.orchestra.run.smithi113.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 10:54 /home/ubuntu/cephtest 2024-02-26T10:54:01.258 INFO:teuthology.orchestra.run.smithi060.stdout: 395173 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 10:54 /home/ubuntu/cephtest 2024-02-26T10:54:01.261 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-26T10:54:01.274 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-26T10:54:01.307 INFO:teuthology.nuke:Checking targets against current locks 2024-02-26T10:54:01.334 DEBUG:teuthology.nuke:shortname: smithi060 2024-02-26T10:54:01.334 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T10:54:01.355 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:16.757995', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMWnHpBAj9JGPCdzouGkLMVA2TWfW5K4EzxQ93ol71TVMNevNgAn5p62vIxtrhOkOkbdZR2C9fdH97znyTefiQQ='} 2024-02-26T10:54:01.359 DEBUG:teuthology.nuke:shortname: smithi100 2024-02-26T10:54:01.360 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T10:54:01.381 DEBUG:teuthology.nuke:shortname: smithi113 2024-02-26T10:54:01.381 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T10:54:01.385 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:16.760825', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOKLjzfObQmFIIiBw3Da5e09/RMkGdDH2omX9S+dysBhRwk1YZ3KS+AeXmpi9PUvrlWxqeVFv8SPXmQl5UH/oMs='} 2024-02-26T10:54:01.414 INFO:teuthology.orchestra.console.smithi060:Power off 2024-02-26T10:54:01.415 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T10:54:01.423 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573562', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:16.759496', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM2zRjhcwm9/pB7qnx5uaBvrQYhj7wZUT1vvlGVnFTvZrYnHymXYVUMMMhKUJOJ1dI4ssCDg8PwC34vvqF1gBKU='} 2024-02-26T10:54:01.438 DEBUG:teuthology.orchestra.console.smithi060:power off output: Chassis Power Control: Down/Off 2024-02-26T10:54:01.438 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:01.463 INFO:teuthology.orchestra.console.smithi100:Power off 2024-02-26T10:54:01.463 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T10:54:01.475 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-02-26T10:54:01.485 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-02-26T10:54:01.485 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:01.502 INFO:teuthology.orchestra.console.smithi113:Power off 2024-02-26T10:54:01.502 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T10:54:01.511 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-02-26T10:54:01.525 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2024-02-26T10:54:01.525 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:01.550 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-02-26T10:54:05.476 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:05.512 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:05.550 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:05.587 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-02-26T10:54:05.622 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-02-26T10:54:05.660 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-02-26T10:54:09.588 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:09.623 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:09.662 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:09.699 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2024-02-26T10:54:09.734 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-02-26T10:54:09.773 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-02-26T10:54:09.800 INFO:teuthology.orchestra.console.smithi060:Power off completed 2024-02-26T10:54:13.736 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:13.774 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:54:13.847 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-02-26T10:54:13.884 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2024-02-26T10:54:13.949 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-02-26T10:54:13.985 INFO:teuthology.orchestra.console.smithi113:Power off completed 2024-02-26T10:54:14.091 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsstress}} duration: 1857.1150064468384 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_vshankar@teuthology status: fail success: false 2024-02-26T10:54:14.092 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T10:54:14.176 INFO:teuthology.run:FAIL